Ch. 12: Principal component analysis
I. Situation
A. For a given set of variables, we want to find a linear
combination of variables which explains the maximum of the
variance.
B. It is a variable-reduction technique.
C. It is an interdependence model (No IV, DV distinction).
A. All extracted principal components are orthogonal to each
other.
II. Model
A. PC1 = z1 = a1y1 + a2y2 + . . + apyp = a’y
PC2 = z2 = b1y1 + b2y2 + . . + bpyp = b’y
B. If a’a =1, then a is said to be normalized. The a can always
be normalized by a pre-multiplication of C where, c =
aa
a
'
.
C. Thus, z = PCi = Ay, where A = orthogonal matrix (C) such
that A’A = I and z’z = (Ay)’(Ay) = y’A’Ay = y’y.
D. The orthogonal matrix, A, transforms y to z which has the
same distance from the origin and the axes are rotated.
E. Thus,
Sz = ASA’ =
2
22
21
...00
0...
00
0.
..0
0
0.
.00
0.
.00
zp
z
z
s
s
s
F. The diagonal elements of Sz (
2
Zp
s
) are eigenvalues (λi) of S,
which are the variances of the principal components of
z = a’y.
III. Procedure
A. Start with a R (correlation matrix) to standardize the
measurement unit (S will give us a different solution from
that of R).
B. Each variable has its own variance (eigenvalue) of 1.
C. The first selection can be made by excluding any PC whose
eigenvalue is smaller than 1.
D. Utilizing the parsimony rule, select a minimum # of PCs with
the maximum variance explained (e.g. 80%).
B. A scree graph of eigenvalues can help in determining the
number of PCs.
Docsity.com
