Download Exploratory Factor Analysis - Lecture Notes | STAT 530 and more Study notes Statistics in PDF only on Docsity!
STAT 530/J530 B.Habing Univ. of S.C. (^1)
STAT 530/J
September 4 th^ , 2008
Instructor: Brian Habing Department of Statistics LeConte 203 Telephone: 803-777- E-mail: [email protected]
STAT 530/J530 B.Habing Univ. of S.C. (^2)
Today
- Multivariate Data continued…
- A Hint of Matrices
- What if Some Data is Missing?
- The Multivariate Normal Distribution
STAT 530/J530 B.Habing Univ. of S.C. (^3)
Covariances
It is also necessary to summarize the relationship between the variables, and this can be done with the covariance.
( , )^1
( , ) [( )( )]
1 ∑ =
∧ − − −
n i j k ij j il l
j k j j k k x x x x n Covx x
Covx x E x μ x μ
STAT 530/J530 B.Habing Univ. of S.C. (^4)
Sample Covariance Matrix
And is estimated by the sample covariance matrix:
Where is thei th^ observation.
1
(x x)(x x) S 1 1
21 11
11 12 1 −
− −
⎥⎥
⎥⎥
⎥
⎦
⎤
⎢⎢
⎢⎢
⎢
⎣
⎡
n s s
s s
s s s n i i i T
q qq
q
L L
M O M
M
L
⎥ ⎥ ⎥ ⎦
⎤ ⎢ ⎢ ⎢ ⎣
⎡
in
i i x
x M
1 x
STAT 530/J530 B.Habing Univ. of S.C. (^5)
Correlation Matrix
The Covariance Matrix is commonly rescaled to be the correlation matrix:
Where
⎥⎥
⎥⎥
⎥
⎦
⎤
⎢⎢
⎢⎢
⎢
⎣
⎡ Ρ= q qq
q
ρ ρ
ρ ρ
ρ ρ ρ
L L
M O M
M
L
1
21 22
11 12 1
ii jj
ij
ij σ σ
STAT 530/J530 B.Habing Univ. of S.C. (^6)
Regression in Terms of Matrices
yi = β 0 + β 1 xi + ε i
STAT 530/J530 B.Habing Univ. of S.C. (^10)
(Multivariate) Normal Distribution
− −
f x e x
x 2
2 2
( )
( )^ σ
μ
σ π
1 / 2 21 (^ (x )^1 (x ))
(x) | 2 |
μ μ π − − − Σ− −
T
f e
STAT 530/J530 B.Habing Univ. of S.C. (^11)
What About the Covariance?
Just like σ must be greater than 0 for the normal distribution, the covariance Σ must be positive definite for a multivariate normal.
Σ is positive definite if x TΣx>0 for all vectors x that aren’t all zero.
STAT 530/J530 B.Habing Univ. of S.C. (^12)
Generating MVN Data
library(MASS) mu<-c(5,0,-1) sigma<-matrix(c(1,0.5,-0.2, 0.5,1,0, -.2,0,4), ncol=3,byrow=T) x<-mvrnorm(n=1000,mu,sigma)
STAT 530/J530 B.Habing Univ. of S.C. (^13)
Why?!?
Being positive definite guarantees that:
Σ-1^ exists and is positive definite (x-μ) T^ Σ-1^ (x-μ) is positive |Σ| is positive
1 / 2 21 (^ (x )^1 (x ))
(x) | 2 |
μ μ π − − − Σ− −
T
f e
STAT 530/J530 B.Habing Univ. of S.C. (^14)
Properties of the MVN
Cross Sections: All linear combinations of normal random variables (including the “marginal distributions” are normally distributed. qqnorm(x[,1]) qqline(x[,1]) qqnorm(x%%c(1,2,4)) qqline(x%%c(1,2,4))**
STAT 530/J530 B.Habing Univ. of S.C. (^15)
Geometry of the Multivariate Normal
“Topographic Maps”: The pdf of a multivariate normal makes ellipses of equal probability. plot(x[,1],x[,2])
