STAT3613 Tutorial 04

THE UNIVERSITY OF HONG KONG DEPARTMENT OF STATISTICS AND ACTUARIAL SCIENCE STAT2313/3613 Marketing Engineering Tutorial 4 Review of Factor Analysis (a) Notation • p - number of variables in the data set. • m - number of selected common factors. • F = { F 1 , F 2 , . . . , F m } | - unobservable and random factors. • X = { X 1 , X 2 , . . . , X p } | - continuous random variables. • μ = { μ 1 , μ 2 , . . . , μ p } | - mean of X 1 , . . . , X p . • Σ - p × p sample covariance matrix for X . • s ij = s ji - sample covariance between X i and X j . • Γ - p × p correlation matrix for X . • r ij = r ji - covariance between X i and X j . • L =      l 11 l 12 . . . l 1 m l 21 l 22 . . . l 2 m . . . . . . . . . . . . l p 1 l p 2 . . . l pm      - matrix of factor loadings. • l ij - factor loadings (covariance) of the i th variable and the j th factor. • h 2 i - communality (portion of variance explained by the m factors) of X i . • = { 1 , 2 , . . . , p } | - additional sources of variation/errors/specific factors for the variables X 1 , . . . , X p . • Ψ =      ψ 1 0 . . . 0 0 ψ 2 . . . 0 . . . . . . . . . . . . 0 0 . . . ψ p      - matrix of uniqueness or specific variance. • ψ i - uniqueness or specific variance of the specific factor i for X i . (b) Objective The basic idea is that it may be possible to describe a set of p variables X 1 , X 2 , . . . , X p in terms of a smaller number of unobservable random quantities called factors F 1 , F 2 , . . . , F m , and hence illustrate the relationship between these variables by approximating the covariance or correlation matrix. 1

(c) Factor analysis model (i) Model form: (In matrix, X - μ = LF + ) X 1 - μ 1 = l 11 F 1 + l 12 F 2 + . . . + l 1 m F m + 1 X 2 - μ 2 = l 21 F 1 + l 22 F 2 + . . . + l 2 m F m + 2 . . . X p - μ p = l p 1 F 1 + l p 2 F 2 + . . . + l pm F m + p (ii) Assumptions 1. X is linearly dependent upon F and . 2. E ( F ) = 0 ⇒ E ( F j ) = 0 for j = 1 , . . . , m 3. Cov( F ) = I ⇒ var( F j ) = 1 for j = 1 , . . . , m 4. E ( ) = 0 ⇒ E ( i ) = 0 for i = 1 , . . . , p 5. Cov( ) = Ψ =      ψ 1 0 . . . 0 0 ψ 2 . . . 0 . . . . . . . . . . . . 0 0 . . . ψ p      ⇒ var( i ) = ψ i for i = 1 , . . . , p 6. F and are independent ⇒ Cov( F , ) = 0 (iii) Properties 1. Cov( X ) = Σ = Cov( LF + ) = L Cov( F ) L | + Cov( ) = LL | + Ψ where the ( i, i ) th element is var( X i ) = l 2 i 1 + l 2 i 2 + . . . + l 2 im + ψ i = h 2 i + ψ i and the ( i, j ) th element is Cov( X i , X k ) = l i 1 l k 1 + l i 2 l k 2 + . . . + l im l km , for i 6 = j . 2. Cov( X , F ) = L where the ( i, j ) th element is Cov( X i , F j ) = l ij . Note: L is not unique: • Let T be any m × m (non-random) orthogonal matrix such that T T | = T | T = I • Define a new loading L * and factor F * L * = LT , F * = T | F We have Σ * = L * ( L * ) | + Ψ = LT T | L | + Ψ = LL | + Ψ = Σ E ( F * ) = T | E ( F ) = 0 var( F * ) = T | var( F ) T = T | IT = I 2

It means that: var( Z i ) = l 2 i 1 + ψ i i = 1 , 2 , 3 Cov( Z i , Z j ) = l i 1 l j 1 i = 1 , 2 , 3 Therefore, we have var( Z 1 ) = 0 . 9 2 + 0 . 19 = 1 var( Z 2 ) = 0 . 7 2 + 0 . 51 = 1 var( Z 3 ) = 0 . 5 2 + 0 . 75 = 1 Cov( Z 1 , Z 2 ) = 0 . 9 × 0 . 7 = 0 . 63 Cov( Z 1 , Z 3 ) = 0 . 9 × 0 . 5 = 0 . 45 Cov( Z 2 , Z 3 ) = 0 . 7 × 0 . 5 = 0 . 35 Thus, var( Z ) = LL | + Ψ (b) h 2 1 = l 2 11 = 0 . 9 2 = 0 . 81 h 2 2 = l 2 21 = 0 . 7 2 = 0 . 49 h 2 3 = l 2 31 = 0 . 5 2 = 0 . 25 Proportion of variance of the i th variable Z i explained by the common factor = h 2 i σ ii . Since σ ii = var( Z i ) = 1 , i = 1 , 2 , 3 , Therefore, 81% of var( Z 1 ) is explained by the factor. 49% of var( Z 2 ) is explained by the factor. 25% of var( Z 3 ) is explained by the factor. (c) Corr( Z i , F 1 ) = Cov( Z i , F 1 ) p var( Z i ) p var( F 1 ) = l i 1 √ 1 √ 1 = l i 1 Hence, Corr( Z 1 , F 1 ) = 0 . 9 Corr( Z 2 , F 1 ) = 0 . 7 Corr( Z 3 , F 1 ) = 0 . 5 Since Z 1 has the greatest correlation with F 1 , it carries the greatest weight in the common factor. 2. A marketing researcher did a factor analysis using the sample correlation matrix of rating scores on five attributes of a new product, obtaining the following factor loadings (three of them have been omitted from the table) for two common factors by the ML method: Attribute Before Rotation After Rotation F 1 F 2 F 1 F 2 Taste 0.976 -0.139 0.027 Money’s worth 0.150 0.860 0.873 0.003 Flavour -0.032 0.133 0.971 Good for snack 0.535 0.739 0.404 Nutrition 0.146 0.963 0.973 -0.018 4

(a) Can you construct the common factors as linear combinations of the Attributes using the loadings as coefficients? Why or why not? (b) Find the three missing values in the above table. (c) Roughly speaking, how much of the variability in Flavour cannot be explained by the common factors? (d) Before rotation, how much of the total variability of the Attributes can be explained by the first factor? By the two factors together? (e) Make an attempt to interpret the rotated factors. (f) The determinants of the correlation matrix based on a sample of 150 respondents and of the correlation matrix reproduced by the loadings are respectively 3 . 635 × 10 - 3 and 3 . 721 × 10 - 3 . Perform the likelihood ratio test for the adequacy of the two factor model at the 5% significance level. Solution (a) No. Remember the factor model: X - μ = L F + ( p × 1) ( p × 1) ( p × m )( m × 1) ( p × 1) Normally, p > m and since L is not a square matrix, we do not have L - 1 . F cannot be obtained by F = L - 1 ( X - μ ) (b) The communality h 2 i = l 2 i 1 + . . . + l 2 im does not change with orthogonal rotations. Hence, 0 . 976 2 + ( - 0 . 139) 2 = 0 . 027 2 + ( l * 12 ) 2 ⇒ l * 12 = 0 . 985 l 2 31 + ( - 0 . 032) 2 = 0 . 133 2 + 0 . 971 2 ⇒ l 31 = 0 . 980 0 . 535 2 + 0 . 739 2 = ( l * 41 ) 2 + 0 . 404 ⇒ l * 41 = 0 . 818 (c) From the correlation matrix we have σ 33 = 1 . Specific variance ψ 3 = σ 33 - h 2 3 = 1 - (0 . 133 2 + 0 . 971 2 ) = 0 . 039 Hence, the proportion of variance of Flavour cannot be explained by the common factors = ψ 3 σ 33 = 0 . 039 1 = 0 . 039 . (d) Before rotation: 5