Download Matrix Algebra - Basic Statistics for Behavioral Sciences - Lecture Notes and more Study notes Statistics for Psychologists in PDF only on Docsity!
Ch. 2: Matrix Algebra I. Introduction A. Matrix: A set of (real or complex) numbers or variables arranged in a rectangular array (rows and columns). e.g.
2 X 3
A
2 X 2
B
- A square matrix is a special case of a rectangular matrix.
B. Vector: a one-dimension matrix (a vector is a special case of a matrix).
Column vector:
4 X 1
a Row vector: [ 1 2 3 4 ]
1 4
X a
C. Equality of matrices: Two matrices are equal if the dimension and the elements in corresponding positions are equal (p.7). D. Transpose and symmetric matrices
- Transpose: interchanging the rows and columns of a matrix. e.g
2 X 3
A
3 X 2
A
- Symmetric matrix: the transpose of a matrix is identical to the original matrix (square matrix). e.g.
3 X 3
A
3 X 3
A
E. Special matrices
- Diagonal matrix: a square matrix with numbers and variables on the diagonal and zeros on the off- diagonal.
3 X 3
D
- A diagonal matrix can be made by taking the diagonals of a square matrix and replacing the off-diagonals with zeros. e.g.
3 X 3
A ( )
3 3 D Diag A X
- Identity matrix: a diagonal matrix with ones (1s).
3 X 3
I
- Upper and lower triangular matrix a) Upper triangular matrix: a square matrix with zeros below the diagonal. b) Lower triangular matrix: a square matrix with zeros above the diagonal.
3 X 3
U
3 X 3
L
- Unit matrix: a square matrix with 1s.
3 X 3
J
- Null matrix: a matrix with zeros.
3 X 3
O
II. Matrix operations Given:
2 X 2
A
2 X 2
B
2 X 2
C
A. Addition (elementwise): two matrices must have the same dimension.
h) Example
A = B = C =
AB =
ABC = (AB)C
- Sum of product: ( a’b ) a) Given a and b are vectors of nX1,
a = b = a = b =
Then, a’b is called sum of product and is a scalar.
e.g.,
a’b =
= ΣXY
b) ab’ is a matrix, either rectangular or square.
ab’ =
- Sum of squares: (a’a) a’a is sum of squares and is a scalar and aa’ is a square (symmetric) matrix.
a’a =
aa’ =
- Length of vector a = a ' a
- Since a’b is a scalar, it is equal to its transpose.
a’b = (a’b)’ = b’a
Then, (a’b)^2 = (a’b)(a’b) = (a’b)(a’b)’
= (a’b)(b’a) = a’(bb’)a
DA =
AD =
c) DAD =
d) If D = I , then IA = AI = A
E. Matrix partitioning
- Matrix A can be partitioned into submatrices
A =
A =
- Assuming conformability, the partitioned matrices or vectors can be multiplied as usual. Example
Ab =
A = b =
Then, Ab =
- If a matrix is nXp (n<p) and the rank of the matrix is n, then the matrix is to be said a full rank matrix. If not, the matrix is called less than full rank.
- If a square matrix (nXn) is a full rank matrix, then the matrix is called a nonsingular matrix. If it is not, then the matrix is called a singular matrix. G. Inverse
- If a matrix A is square and full rank, it is called nonsingular.
- Only a nonsingular matrix has its inverse (A-1) such that, AA -1^ = A-1A = I e.g.
A = A -1^ =
AA -1^ =
- If A is square and less than full rank, then it is called singular and has no inverse form.
- Computation of an inverse ( A is nonsingular). a) A =
AA -1^ = I =
b) Short-cut for a 2X2 matrix (nonsingular).
A = (^)
21 22
11 12 a a
a a
- Exchange a 11 and a 22.
- Change the sign of a 12 and a 21.
- Compute a 11 a 22 – a 12 a 21 (Determinant of A ).
- Divide each element by Det( A ).
- Example,
H. Determinant
- A scalar of a square matrix (nXn), which describes some characteristics of matrix A , | A |.
- | A | = a 11 if n = 1, n | A | = Σ a 1j| A 1j|(-1)1+j^ if n>1. j=
- Example
- If matrix A is singular, then | A | = 0, and A has no
- Columns are mutually independent.
In order to normalize each column, divided the elements in each column by its length, 3 , 6 , 2.
∴ C =
Columns are mutually independent and normalized.
- Rotation of axes a) We can transform x to z using the orthogonal matrix C. If z = Cx , then z’z = ( Cx )’( Cx ) = x’C’Cx = x’Ix = x’x. b) The distance among elements in z is the same as the distance among elements in x. K. Eigenvalue and eigenvector.
- For every square matrix A , a scalar λ and a non-zero vector x , there exists the following relationship, Ax = λ x.
- Then, λ is called eigenvalue and x is called eigenvector.
- Eigenvalue a) To find λ and x , we start with an equation, ( A – λ I ) x = 0. b) If x is non-zero (eigenvector is not zero), the only solution for this equation is that ( A – λ I ) is singular and thus, | A – λ I | = 0 (determinantal or characteristic equation). c) Solving the equation for λ will give us eigenvalues. d) Example,
A =
| A – λ I | = 0
- Eigenvector a) From Ax = λ x
We can set x 1 as a normalized vector.
x 1 =
With the same method and some computation
x 2 =
- tr( A ) and | A | For any square matrix A with eigenvalues λ 1 , λ 2 ,... λn, tr( A ) = Σλi, and | A | = Πλi. Example,
D =
λ n
2
1
c) A symmetric matrix A can be diagonalized into a CDC’ expression, called spectral decomposition, with orthogonal matrices ( C and C ’) and a diagonal matrix ( D ). d) The diagonal matrix D contains eigenvalues of A. e) The orthogonal matrix C contains normalized eigenvectors of A. N. Square root matrix
- A positive definite matrix A can also be decomposed into A 1/2 A 1/2, where A 1/2^ is a square root matrix of A. A = A1/2A1/2 , and A1/2^ = CD1/2C’ , where
D1/2^ =
λ n
2
1
- With a similar manner we can say, A^2 = CD^2 C’, and A-1^ = CD-1C’ where
D^2 =
2
2 2
2 1
λ n
λ
λ
, and D-1^ =
−
−
−
1
1 2
1 1
λ n
