Download Assignment for Canonical Correlation - Multivariate Analysis | STAT 571 and more Assignments Descriptive statistics in PDF only on Docsity!
0.1. CANONICAL CORRELATIONS 1
0.1 Canonical correlations
0.1.1 Example: Grades data
Return to the grades data. Using factor analysis, we found two main factors: An overall ability factor, and a contrast of HW+Labs versus Exams+Final. Here we lump in inclass assignments with homework and labs, and find the canonical correlations between the sets (HW, Labs, InClass) and (Exams, F inal), so that q 1 = 3 and q 2 = 2. Letting the Y matrix be the residuals from the model with gender, so ν = n − 2 = 105. The estimate S of the covariance matrix is
S =
HW Labs InClass Exams F inal HW 137. 04 144. 89 74. 75 44. 53 48. 21 Labs 144. 89 251. 11 152. 79 55. 64 57. 03 InClass 74. 75 152. 79 525. 59 51. 16 63. 3 Exams 44. 53 55. 64 51. 16 85. 69 57. 29 F inal 48. 21 57. 03 63. 3 57. 29 107. 46
and the correlations between the variables are HW Labs InClass Exams F inal HW 1 0. 78 0. 28 0. 41 0. 40 Labs 0. 78 1 0. 42 0. 38 0. 35 InClass 0. 28 0. 42 1 0. 24 0. 27 Exams 0. 41 0. 38 0. 24 1 0. 60 F inal 0. 40 0. 35 0. 27 0. 60 1
There are q 1 × q 2 = 6 correlations between variables in the two sets. Canonical correlations aim to summarize the overall correlations by the two δi’s. Letting S be the sample covariance matrix, the estimate of the Ξ matrix is given by
Ξ̂ = S− 111 /^2 S 12 S− 221 /^2
In R,
xi <- symsqrtinv(sigma[1:3,1:3])%%sigma[1:3,4:5]%%symsqrtinv(sigma[4:5,4:5])
This matrix is not particularly interpretable, except the larger the elements, the more correlation between the two sets of variables. The singular value decomposition is the goal.
2 CONTENTS
We can use the svd routine in R on Ξ̂ to find
svd(xi) $d [1] 0.48225950 0.
$u [,1] [,2] [1,] -0.7178030 0. [2,] -0.5264093 -0. [3,] -0.4556886 0.
$v [,1] [,2] [1,] -0.7030917 -0. [2,] -0.7110992 0.
That is,
G
H′^ =
The matrices containing the loading vectors are
S− 111 /^2 G =
(^) and S− 221 /^2 H =
The d 1 = 0.48’s, which is fairly high, and d 2 = 0.064, which is practically negligible. (See below.) Thus it is enough to look at the first columns of G and H. We can change signs, and take the first loadings for the first set of variables to be (0. 065 , 0. 007 , 0 .014), which is primarily the homework score. For the second set of variables, the loadings are (0. 062 , 0 .053), essentially a straight sum of exams and final. Thus the correlations among the two sets of variables can be almost totally explained by the correlations between homework and the sum of exams and final. That correlation is 0.45, which is almost the optimum of 0.48.
4 CONTENTS
For the BIC, we need the dimension of the model. The number of parameters for the Σii’s we know to be qi(qi + 1)/2. For Ξ, look at
Ξ = G
∆∗^0
H′, ∆∗^ =
δ 1 0 · · · 0 0 δ 2 · · · 0 .. .
0 0 · · · δK
that is, ∆∗^ has just the positive δk’s. To count the number of parameters in the Ξ, we add up those for G, H, and ∆∗. From G’s point of view, the dimension is as for principal components, where the pattern of λ’s is (1,... , 1 , q 1 − K), since the first K δk’s are distinct (possibly), and the last q 1 − K are equal (to 0). Thus
dim(G) =
(q^21 − (1 + · · · + 1 + (q 1 − K)^2 )) (14)
=
(q^21 − K − (q 1 − K)^2 )) (15)
=
(−K + 2Kq 1 − K^2 ). (16)
Similarly for H, but with q 2 instead of q 1. Finally, obviously ∆∗^ has K parameters. Then
dim(G) + dim(H) + dim(∆∗) =
(−K + 2Kq 1 − K^2 ) +
(−K + 2Kq 2 − K^2 ) + K (17)
= Kq 1 + Kq 2 − K^2 = K(q − K). (18)
Finally, we can take
BIC(MK ) = ν
∑^ K
k=
log(1 − d^2 k) + log(ν)K(q − K) (19)
because the qi(qi + 1)/2 parts are the same for each model. To the example, we have three models: K = 0, 1 , 2. K = 0 means the two sets of variables are independent, which we already know is not true, and K = 2 is the unrestricted model. The calculations, with ν = 105, d 1 = 0.48226 and d 2 = 0.064296:
K ν
log(1 − d^2 k) dim(Ξ) BIC 100 × Prob 0 0 0 0 0. 9939 1 − 27. 7949 4 − 9. 1791 97. 8478 2 − 28. 2299 6 − 0. 3061 1. 1583
Clearly K = 1 is best, which is what we figured above.
