STAT3613 Tutorial 04

THE UNIVERSITY OF HONG KONG DEPARTMENT OF STATISTICS AND ACTUARIAL SCIENCE STAT2313/3613 Marketing Engineering Tutorial 4 Review of Factor Analysis (a) Notation • p - number of variables in the data set. • m - number of selected common factors. • F = { F 1 , F 2 , . . . , F m } | - unobservable and random factors. • X = { X 1 , X 2 , . . . , X p } | - continuous random variables. • μ = { μ 1 , μ 2 , . . . , μ p } | - mean of X 1 , . . . , X p . • Σ - p × p sample covariance matrix for X . • s ij = s ji - sample covariance between X i and X j . • Γ - p × p correlation matrix for X . • r ij = r ji - covariance between X i and X j . • L =      l 11 l 12 . . . l 1 m l 21 l 22 . . . l 2 m . . . . . . . . . . . . l p 1 l p 2 . . . l pm      - matrix of factor loadings. • l ij - factor loadings (covariance) of the i th variable and the j th factor. • h 2 i - communality (portion of variance explained by the m factors) of X i . • = { 1 , 2 , . . . , p } | - additional sources of variation/errors/specific factors for the variables X 1 , . . . , X p . • Ψ =      ψ 1 0 . . . 0 0 ψ 2 . . . 0 . . . . . . . . . . . . 0 0 . . . ψ p      - matrix of uniqueness or specific variance. • ψ i - uniqueness or specific variance of the specific factor i for X i . (b) Objective The basic idea is that it may be possible to describe a set of p variables X 1 , X 2 , . . . , X p in terms of a smaller number of unobservable random quantities called factors F 1 , F 2 , . . . , F m , and hence illustrate the relationship between these variables by approximating the covariance or correlation matrix. 1

(c) Factor analysis model (i) Model form: (In matrix, X - μ = LF + ) X 1 - μ 1 = l 11 F 1 + l 12 F 2 + . . . + l 1 m F m + 1 X 2 - μ 2 = l 21 F 1 + l 22 F 2 + . . . + l 2 m F m + 2 . . . X p - μ p = l p 1 F 1 + l p 2 F 2 + . . . + l pm F m + p (ii) Assumptions 1. X is linearly dependent upon F and . 2. E ( F ) = 0 ⇒ E ( F j ) = 0 for j = 1 , . . . , m 3. Cov( F ) = I ⇒ var( F j ) = 1 for j = 1 , . . . , m 4. E ( ) = 0 ⇒ E ( i ) = 0 for i = 1 , . . . , p 5. Cov( ) = Ψ =      ψ 1 0 . . . 0 0 ψ 2 . . . 0 . . . . . . . . . . . . 0 0 . . . ψ p      ⇒ var( i ) = ψ i for i = 1 , . . . , p 6. F and are independent ⇒ Cov( F , ) = 0 (iii) Properties 1. Cov( X ) = Σ = Cov( LF + ) = L Cov( F ) L | + Cov( ) = LL | + Ψ where the ( i, i ) th element is var( X i ) = l 2 i 1 + l 2 i 2 + . . . + l 2 im + ψ i = h 2 i + ψ i and the ( i, j ) th element is Cov( X i , X k ) = l i 1 l k 1 + l i 2 l k 2 + . . . + l im l km , for i 6 = j . 2. Cov( X , F ) = L where the ( i, j ) th element is Cov( X i , F j ) = l ij . Note: L is not unique: • Let T be any m × m (non-random) orthogonal matrix such that T T | = T | T = I • Define a new loading L * and factor F * L * = LT , F * = T | F We have Σ * = L * ( L * ) | + Ψ = LT T | L | + Ψ = LL | + Ψ = Σ E ( F * ) = T | E ( F ) = 0 var( F * ) = T | var( F ) T = T | IT = I 2