Download Matrix Multiplication and Inverse Matrices and more Lecture notes Economics in PDF only on Docsity!
1 Matrices: Introduction, Terminology, Notation
1.1 Introduction
Consider the problem of solving the system of equations
x − 2 y = 1 (1) − 2 x + 3y = − 2 (2)
We can do this easily using substitution: use equation (1) to write x = 1 + 2y, and then substitute this expression into equation (2) to get
−2(1 + 2y) + 3y = − 2 ,
which is easily solved to find y = 0. Now substitute y = 0 into the equation x = 1 + 2y to find x = 1. Therefore, the solution to the system of equations is x = 1 and y = 0.
This works fine for this small system of two equations. What about something larger, say
x − 2 y + z = 1 − 2 x + 3 y + z = − 2 5 x − 7 y − 3 z = − 3
We could again use substitution, but the problem is quite a bit harder (and messier) this time— try it! The situation gets worse the more equations we have. We would like a more systematic and organized approach to solving such problems, and fortunately, there is one. The system (3) can be expressed in the form (^)
x y z
Each of the rectangular structures
x y z
(^) and
(^) is called a matrix , and
there are methods of manipulating them to efficiently solve the system of equations. Our main focus will be the application of matrices (the plural of matrix) to solving systems of equations, but it should noted that matrices arise in many areas of science, economics and computing.
1.2 Terminology and Notation
Matrices are denoted using bold uppercase letters. For example, let
A =
Here A is a matrix of 3 rows and 3 columns, and we say A has size 3 × 3. The size of a matrix is always stated as rows×columns, that is, rows first, columns second.
The entries or elements of A are the numbers. These are denoted by their position:
aij = entry in row i and column j of A.
For example, in our matrix A, a 31 = 5 since the entry in row 3 and column 1 is 5. Notice again, when referring to position of entries: rows first, columns second. Similarly, a 23 = 1 is the entry in row 2 column 3. Also notice how we have used the lowercase letter “a” in aij to correspond with our choice of uppercase “A” used to represent the matrix. If instead we used B to represent our matrix, we would use bij to refer to the individual entries.
Now that we have some notation, let’s state the formal definition of a matrix:
Definition: A rectangular array of numbers consisting of m horizontal rows and n vertical columns
a 11 a 12 · · · a 1 n a 21 a 22 · · · a 2 n ... ...... ... am 1 am 2 · · · amn
is called an m × n matrix or matrix of size m × n. For entry aij , i is the row subscript, while j is the column subscript.
A general matrix is sometimes denoted [aij ]m×n.
Example: Let B =
[ 1 6 / 7 4
]
The size of B is 2 × 3. A couple of entries of B are
b 23 = 3 b 21 = 1/ 2.
Example: Let P = [^3 − 4 π ]^. P is called a row matrix or row vector. Here p 11 = 3, p 12 = −4 and p 13 = π.
Example: Let
Q =
e
Q is called a column matrix or column vector. Here q 11 = 0, p 21 = 2 and p 31 = e.
then
AT^ =
Example: Let B =
[ 1 2
] }
size 2 × 3
then
BT^ =
size 3^ ×^2
Using this last example, notice that
(BT)T^ =
[ 1 2
]
= B.
This turns out to be true in general: for any matrix A, (AT)T^ = A.
Example: Let
U =
size 3^ ×^1
then UT^ = [^1 − 1 / 2 √ 2 ]^ }^ size 1 × 3
1.5 Some Special Matrices
Zero Matrix The m × n zero matrix is the m × n matrix with all entries zero. For example, the 2 × 3 zero matrix is (^02) × 3 =
[ 0 0
]
Square Matrix A matrix with the same number n of rows and columns is called a square matrix of order n. For example, A =
Here the entries a 11 = 1, a 22 = 3, and a 33 = −3 (reading from upper-left to lower-right) form the main diagonal of A.
Diagonal Matrix A square matrix [aij ]n×n with all entries not on the main diagonal equal to zero is called a diagonal matrix. That is, aij = 0 if i 6 = j. For example
is a diagonal matrix, however (^)
is not.
Upper Triangular Matrix A square matrix is called upper triangular if all entries below the main diagonal are zero, for example
A =
Lower Triangular Matrix A square matrix is called lower triangular if all entries above the main diagonal are zero, for example
A =
Triangular Matrix A matrix which is either upper or lower triangular.
2 Matrix Addition and Scalar Multiplication
Matrices inherit many of the properties of ordinary real numbers, including some of the operations of arithmetic. Here we define two of these operations: addition of matrices, and the multiplication of a matrix by a number, called scalar multiplication.
2.1 Matrix Addition
Let A = [aij ]m×n and B = [bij ]m×n. Then
A + B = [aij + bij ]m×n.
That is, provided A and B are the same size, the matrix C = A + B is simply the m × n matrix formed by adding the corresponding entries of A and B: cij = aij + bij.
Then
4 A + 2B = 4
[ 1
]
[ − 3 / 2
]
[ 4
]
[ − 3
]
[ 1
]
Like matrix addition, scalar multiplication inherits many of the multiplication rules of the ordinary real numbers. In the following, let k, k 1 and k 2 be scalars, and A and B be matrices of size m × n:
(i) k(A + B) = kA + kB. (ii) (k 1 + k 2 )A = k 1 A + k 2 A. (iii) k 1 (k 2 A) = (k 1 k 2 )A. (iv) 0 A = (^0) m×n. (v) k (^0) m×n = (^0) m×n. (vi) (kA)T^ = kAT^.
2.3 Matrix Subtraction
Now that we have clearly defined matrix addition and scalar multiplication, the operation of sub- traction can be stated simply: if A and B are matrices of size m × n, then we define
A − B = A + (−1)B.
Here are a few examples to put this all together:
Example: Solve 3
[ (^) x y
]
[ 1
]
[ − 1 / 2
]
for
[ (^) x y
]
Solution:
3
[ (^) x y
]
[ 1
]
[ − 1 / 2
]
[ (^) x y
]
[ 1
]
[ − 6
]
[ (^) x y
]
[ 1
]
[ − 6
]
[ (^) x y
]
[ − 5
]
[ (^) x y
]
=^13
[ − 5
]
[ (^) x y
]
[ − 5 / 3
]
Therefore
[ (^) x y
]
[ − 5 / 3
]
Example: Let
A =
[ 3 − 4 5
]
, B =
[ 1 4
]
, C =
[ − 1 1 3
]
Compute 2(3C − A) + 2B.
Solution:
First, notice that A, B and C are all the same size, 2 × 3 in this case, so addition is defined. So we have
2 (3 C − A) + 2 B
[ − 1 1 3
]
[ 3 − 4 5
])
[ 1 4
]
([ − 3 3 9
]
[ − 3 4 − 5
])
[ 2 8
]
[ − 6 7 4
]
[ 2 8
]
[ − 12 14 8
]
[ 2 8
]
[ 10 22
]
Answers to Problems for Sections 1 and 2
(i) size 3 × 3; order 3. (ii) q 21 = 0, q 13 = − 1 /2, q 33 = −1. (iii) QT^ is lower triangular.
[ 1 4
]
- Yes.
- x = y = 0.
- x = −2001.
- (i)
[
]
(ii)
[ 21 29 / 2
]
(iii)
[
]
(iv)
[ 18
]
- x = 90/29, y = − 24 /29.
- x = 6, y = 4/3.
- x = −6, y = −14, z = 1.
3 Matrix Multiplication
3.1 Introduction
So far we have seen two algebraic operations with matrices, addition and scalar multiplication, and we have also seen how the zero matrix plays a role in these operations similar to that of the zero of the real number system. The properties of matrix addition and scalar multiplication are similar to those of the ordinary real numbers, and it is natural to ask how far these similarities extend. In
particular, is it possible the define the notion of multiplication of two matrices. The answer is yes, though matrix multiplication is not quite as straightforward as matrix addition.
3.2 Definition of Matrix Multiplication
Here’s the formal definition of matrix multiplication:
Definition: Let A = [aij ]m×p and B = [bij ]p×n. Then we define the product C = AB as the matrix with entries cij = ai 1 b 1 j + ai 2 b 2 j + ai 3 b 3 j + · · · + aipbpj. Notice:
(i) ai 1 , ai 2 , ai 3 ,... , aip are the elements of row i of A, while b 1 j , b 2 j , b 3 j ,... , bpj are the elements of column j of B. (ii) For matrix multiplication to be defined, the number of columns of A must be the same as the number of rows of B.
Example: Let
A =
,^ B^ =
[ 3
]
Then
A B =
Notice A is size 3 × 2, B is size 2 × 2, and the product AB is size 2 × 2.
The process of multiplying matrices can be described in terms of a certain type of product. Let R and C be the row and column matrices
R = [^ r 11 r 12 · · · r 1 p^ ]^ and C =
c 11 c 21 ... cp 1
^.
In this last example we see that (AB)T^ = BTAT, and this turns out to be true in general.
Example: Let
A =
[ 1
]
I 2 =
[ 1
]
Then AI 2 =
[ 1
] [ 1
]
[ 1
]
= A.
Similarly,
I 2 A =
[ 1
] [ 1
]
[ 1
]
= A.
The matrix I 2 in this last example is called the identity matrix of order 2. More generally:
Definition: The n × n matrix
In =
is called the identity matrix of order n, or the n × n identity matrix. It has the property that [aij ]m×pIp = [aij ]m×p , and Im[aij ]m×p = [aij ]m×p. When the size of the identity matrix is understood it is often written simply as I. The letter I should not be used to represent any other matrix— it always means the identity matrix.
Comparing matrix algebra with that of the ordinary real numbers, notice that In plays the role of the number one.
As with matrix addition, matrix multiplication has many of the properties of ordinary multiplication of real numbers. Let A, B, C be matrices, I the identity, and k a scalar. Then
(i) A(BC) = (AB)C. (ii) A(B + C) = AB + AC. (iii) (A + B)C = AC + BC. (iv) k(AB) = (kA)B = (Ak)B = A(kB) = A(Bk).
(v) AI = IA = A. (vi) (AB)T^ = BTAT^. (vii) If p ≥ 1 is an integer, and A is square, Ap^ = AA︸ ︷︷ · · · A︸ p factors
(viii) A^0 = I.
In the properties listed above it is assumed that the matrices are of the sizes required for the operations to be defined. Notice how the order of the multiplication is preserved in the expansion in (ii) and (iii). This is important since matrix multiplication is not commutative in general. Example: Expand (A + B)(C + D). (A + B)(C + D) = (A + B)C + (A + B)D = AC + BC + AD + BD. Example: Expand (A + B)(A − B). (A + B)(A − B) = (A + B)A − (A + B)B = A^2 + BA − AB − B^2. Notice in this last example, the terms BA and −AB cannot be canceled unless AB = BA, which is not true in general. Example: Let
A =
B^ =
Then
AB =
That is, AB = I. In this last example, B is called the inverse of A, and it is the only matrix with this property. You will find that BA = I in this case also. The inverse of A is written A−^1. We will explore inverses in more detail later. Example: Let A =
[ 1
]
B =
[ 4 − 6
]
- Let
A =
,^ B^ =^
[ 3 − 2 1 ]
Compute (AB)^2.
Solutions to Problems for Section 3
- (i)
AB =
(ii)
BD =
(iii) 3 I +^32 CD =
(iv)
DC(B − A) =
(v)
(A − 2 I)(B + 3I) =
(vi) (CA)T^ − ATCT^ = (^03) × 2
- (^)
x y z
x y z
AB = BA =
ax 0 0 0 by 0 0 0 cz
(AB)^2 =
4 Solving Systems of Equations by Reducing Matrices
4.1 Introduction
One of the main applications of matrix methods is the solution of systems of linear equations. Consider for example solving the system
2 x − 3 y = 2 −x + 2y = 1.
As we observed before, this system can easily be solved using the method of substitution. Another more systematic method is that of reduction of matrices, the solution by a sequence of steps which can be carried out efficiently using matrices.
Before explaining the method in detail, the idea behind it is first illustrated by applying it to solve the simple system above. In the following, we list on the left the steps of the solution applied to the system of equations, and on the right we show how the corresponding steps can be written in matrix form.
First, state the problem in matrix form:
© 1 2 x − 3 y = 2 ©^2 −x + 2y = 1
[ 2 − 3 2
]
Now multiply the first equation by 1/2 to make the coefficient of x one:
© 1 x − 32 y = 1
© 2 −x + 2y = 1
^1
is called the coefficient matrix. The matrix [ (^2) − 3 2 − 1 2 1
]
is called the augmented coefficient matrix.
4.2.2 Elementary Row Operations
The operations performed on the augmented matrix in the first section are called elementary row operations. There are three types of elementary row operations:
(i) Interchanging two rows. The notation for this is Ri ↔ Rj , which means interchange rows i and j. (ii) Multiplying a row by a nonzero constant. Notation: kRi, meaning multiply row i by the constant k. (iii) Adding k times one row to another. Notation: kRi + Rj , meaning add k times row i to row j, leaving row i unchanged. With this notation, the row being changed (row j) is listed second.
Performing elementary row operation on the augmented matrix of a system of equation does not alter the information contained in the augmented matrix. That is, the system represented by the augmented matrix following the elementary row operation has the same solution as that before. This is an important point. If a matrix is obtained from another by one or more elementary row operations, the two matrices are said to be equivalent.
It should be pointed out that the notation for the elementary row operations is not universal, and different books may use variations on the notation given here.
4.2.3 Reduced Form of a Matrix
The final augmented matrix in our problem had the form
[ 1 0
]
; the
[ 1
]
part of the augmented matrix is said to be in reduced form (or reduced row echelon form). Although the reduced form in this case turned out to be the identity matrix of order two, this need not be the case. More generally, a matrix is said to be in reduced form if
(i) The first nonzero entry in a row (if any) is 1, while all other entries of the column containing that 1 are 0; (ii) The first nonzero entry in a row is to the right of the first nonzero entry in each row above; and
(iii) Any rows consisting entirely of zeros are at the bottom of the matrix.
The “first nonzero entry in a row” is called the leading entry of the row.
Example: The matrix
[ 2
]
is not reduced, since the leading entry of the first row is 2, not 1.
Example: The matrix
(^) is reduced since it meets the three conditions of being a reduced
matrix.
4.2.4 Procedure for Reducing a Matrix
We next give the procedure for reducing a matrix, followed by an example. The description of the procedure is slightly technical— it is best to work through several examples to understand how it works.
To Put a Matrix in Reduced Form: Suppose a matrix has n rows.
(i) Set j = 1 (here j is a “row counter”, and so we are starting with the first row). (ii) Rearrange rows j, j + 1,... , n to that the leading entry of row j is positioned as far to the left as possible. (iii) Multiply row j by a nonzero constant to make the leading entry equal 1. (iv) Use this leading entry of 1 to reduce all other entries in its column to 0 using elementary row operations. (v) If any of rows j + 1,... , n contain nonzero terms, increase j by 1 and go to step (ii)
Here’s an example illustrating the steps of this procedure along with the notation for the elementary row operation:
Example: Reduce the matrix (^)
Solution:
