Download 2.5 Inverse Matrices and more Study notes Mathematics in PDF only on Docsity!
2.5. Inverse Matrices 81
2.5 Inverse Matrices
Suppose A is a square matrix. We look for an “ inverse matrix ” A^1 of the same size, such that A^1 times A equals I. Whatever A does, A^1 undoes. Their product is the identity matrix—which does nothing to a vector, so A^1 Ax D x. But A^1 might not exist. What a matrix mostly does is to multiply a vector x. Multiplying Ax D b by A^1 gives A^1 Ax D A^1 b. This is x D A^1 b. The product A^1 A is like multiplying by a number and then dividing by that number. A number has an inverse if it is not zero— matrices are more complicated and more interesting. The matrix A^1 is called “A inverse.”
DEFINITION The matrix A is invertible if there exists a matrix A^1 such that
A^1 A D I and AA^1 D I: (1)
Not all matrices have inverses. This is the first question we ask about a square matrix: Is A invertible? We don’t mean that we immediately calculate A^1. In most problems we never compute it! Here are six “notes” about A^1.
Note 1 The inverse exists if and only if elimination produces n pivots (row exchanges are allowed). Elimination solves Ax D b without explicitly using the matrix A^1.
Note 2 The matrix A cannot have two different inverses. Suppose BA D I and also AC D I. Then B D C , according to this “proof by parentheses”:
B.AC / D .BA/C gives BI D IC or B D C: (2)
This shows that a left-inverse B (multiplying from the left) and a right-inverse C (multi- plying A from the right to give AC D I ) must be the same matrix.
Note 3 If A is invertible, the one and only solution to Ax D b is x D A^1 b:
Multiply Ax D b by A^1 : Then x D A^1 Ax D A^1 b:
Note 4 (Important) Suppose there is a nonzero vector x such that Ax D 0. Then A cannot have an inverse. No matrix can bring 0 back to x.
If A is invertible, then Ax D 0 can only have the zero solution x D A^1 0 D 0.
Note 5 A 2 by 2 matrix is invertible if and only if ad bc is not zero:
2 by 2 Inverse:
a b c d
D
ad bc
d b c a
This number ad bc is the determinant of A. A matrix is invertible if its determinant is not zero (Chapter 5). The test for n pivots is usually decided before the determinant appears.
82 Chapter 2. Solving Linear Equations
Note 6 A diagonal matrix has an inverse provided no diagonal entries are zero:
If A D
d 1 : : : dn
5 then^ A^1 D
1=d 1 : : : 1=dn
Example 1 The 2 by 2 matrix A D
1 2
is not invertible. It fails the test in Note 5, because ad bc equals 2 2 D 0. It fails the test in Note 3, because Ax D 0 when x D .2; 1/. It fails to have two pivots as required by Note 1. Elimination turns the second row of this matrix A into a zero row.
The Inverse of a Product AB
For two nonzero numbers a and b, the sum a C b might or might not be invertible. The numbers a D 3 and b D 3 have inverses 13 and 13. Their sum a C b D 0 has no inverse.
But the product ab D 9 does have an inverse, which is 13 times 13. For two matrices A and B, the situation is similar. It is hard to say much about the invertibility of A C B. But the product AB has an inverse, if and only if the two factors A and B are separately invertible (and the same size). The important point is that A^1 and B^1 come in reverse order :
If A and B are invertible then so is AB. The inverse of a product AB is
.AB/^1 D B^1 A^1 : (4)
To see why the order is reversed, multiply AB times B ^1 A^1. Inside that is BB^1 D I :
Inverse of AB .AB/.B^1 A^1 / D AIA^1 D AA^1 D I:
We moved parentheses to multiply BB ^1 first. Similarly B^1 A^1 times AB equals I. This illustrates a basic rule of mathematics: Inverses come in reverse order. It is also common sense: If you put on socks and then shoes, the first to be taken off are the. The same reverse order applies to three or more matrices:
Reverse order .ABC /^1 D C ^1 B^1 A^1 : (5)
Example 2 Inverse of an elimination matrix. If E subtracts 5 times row 1 from row 2, then E^1 adds 5 times row 1 to row 2:
E D
(^5) and E^1 D
Multiply EE^1 to get the identity matrix I. Also multiply E ^1 E to get I. We are adding and subtracting the same 5 times row 1. Whether we add and then subtract (this is EE ^1 / or subtract and then add (this is E^1 E/, we are back at the start.
84 Chapter 2. Solving Linear Equations
The Gauss-Jordan method computes A^1 by solving all n equations together. Usually the “augmented matrix” ŒA b has one extra column b. Now we have three right sides e 1 ; e 2 ; e 3 (when A is 3 by 3). They are the columns of I , so the augmented matrix is really the block matrix Œ A I . I take this chance to invert my favorite matrix K, with 2’s on the main diagonal and 1 ’s next to the 2’s:
K e 1 e 2 e 3
D
Start Gauss-Jordan on K
5.^1
2 row^^1 C^ row^^2 /
.^23 row 2 C row 3 /
We are halfway to K^1. The matrix in the first three columns is U (upper triangular). The pivots 2; 32 ; 43 are on its diagonal. Gauss would finish by back substitution. The contribution of Jordan is to continue with elimination! He goes all the way to the “reduced echelon form”. Rows are added to rows above them, to produce zeros above the pivots :
Zero above third pivot
5.^3
4 row^^3 C^ row^^2 /
Zero above second pivot
.^23 row 2 C row 1 /
The last Gauss-Jordan step is to divide each row by its pivot. The new pivots are 1. We have reached I in the first half of the matrix, because K is invertible. The three columns of K^1 are in the second half of ΠI K ^1 :
(divide by 2/ (divide by 32 / (divide by 43 /
1 0 0^341214
0 1 0^12 1
D
I x 1 x 2 x 3
D
I K ^1
Starting from the 3 by 6 matrix ΠK I , we ended with ΠI K ^1 . Here is the whole Gauss-Jordan process on one line for any invertible matrix A:
Gauss-Jordan Multiply
A I
by A^1 to get ΠI A^1 :
2.5. Inverse Matrices 85
The elimination steps create the inverse matrix while changing A to I. For large matrices, we probably don’t want A^1 at all. But for small matrices, it can be very worthwhile to know the inverse. We add three observations about this particular K ^1 because it is an important example. We introduce the words symmetric , tridiagonal , and determinant :
1. K is symmetric across its main diagonal. So is K ^1. 2. K is tridiagonal (only three nonzero diagonals). But K ^1 is a dense matrix with no zeros. That is another reason we don’t often compute inverse matrices. The inverse of a band matrix is generally a dense matrix. 3. The product of pivots is 2. 32 /. 43 / D 4. This number 4 is the determinant of K.
K ^1 involves division by the determinant K ^1 D
This is why an invertible matrix cannot have a zero determnant.
Example 4 Find A^1 by Gauss-Jordan elimination starting from A D
4 7
. There are two row operations and then a division to put 1 ’s in the pivots:
A I
D
this is
U L^1
1 0^72 ^32
this is
I A^1
That A^1 involves division by the determinant ad bc D 2 7 3 4 D 2. The code for X D inverse.A/ can use rref , the “row reduced echelon form” from Chapter 3:
I D eye .n/I % Define the n by n identity matrix R D rref .ŒA I /I % Eliminate on the augmented matrix ŒA I X D R. W ; n C 1 W n C n/ % Pick A^1 from the last n columns of R
A must be invertible, or elimination cannot reduce it to I (in the left half of R). Gauss-Jordan shows why A^1 is expensive. We must solve n equations for its n columns.
To solve Ax D b without A^1 , we deal with one column b to find one column x.
In defense of A^1 , we want to say that its cost is not n times the cost of one system Ax D b. Surprisingly, the cost for n columns is only multiplied by 3. This saving is because the n equations Axi D ei all involve the same matrix A. Working with the right sides is relatively cheap, because elimination only has to be done once on A. The complete A^1 needs n^3 elimination steps, where a single x needs n^3 =3. The next section calculates these costs.
2.5. Inverse Matrices 87
Gauss-Jordan on triangular L
5 D
L I
.3 times row 1 from row 2/ .4 times row 1 from row 3/ .then 5 times row 2 from row 3/
5 D
I L^1
L goes to I by a product of elimination matrices E 32 E 31 E 21. So that product is L^1. All pivots are 1’s (a full set). L^1 is lower triangular, with the strange entry “ 11 ”. That 11 does not appear to spoil 3; 4; 5 in the good order E 211 E 31 ^1 E 32 ^1 D L.
REVIEW OF THE KEY IDEAS
1. The inverse matrix gives AA^1 D I and A^1 A D I. 2. A is invertible if and only if it has n pivots (row exchanges allowed). 3. If Ax D 0 for a nonzero vector x, then A has no inverse. 4. The inverse of AB is the reverse product B^1 A^1. And .ABC /^1 D C ^1 B^1 A^1. 5. The Gauss-Jordan method solves AA^1 D I to find the n columns of A^1. The augmented matrix
A I
is row-reduced to
I A^1
WORKED EXAMPLES
2.5 A The inverse of a triangular difference matrix A is a triangular sum matrix S :
A I
D
5 D
I A^1
D
I sum matrix
If I change a 13 to 1 , then all rows of A add to zero. The equation Ax D 0 will now have the nonzero solution x D .1; 1; 1/. A clear signal: This new A can’t be inverted.
88 Chapter 2. Solving Linear Equations
2.5 B Three of these matrices are invertible, and three are singular. Find the inverse when it exists. Give reasons for noninvertibility (zero determinant, too few pivots, nonzero solution to Ax D 0 ) for the other three. The matrices are in the order A; B; C; D; S; E:
4 3 8 6
Solution
B^1 D
C ^1 D
S ^1 D
A is not invertible because its determinant is 4 6 3 8 D 24 24 D 0. D is not invertible because there is only one pivot; the second row becomes zero when the first row is subtracted. E is not invertible because a combination of the columns (the second column minus the first column) is zero—in other words Ex D 0 has the solution x D .1; 1; 0/. Of course all three reasons for noninvertibility would apply to each of A; D; E.
2.5 C Apply the Gauss-Jordan method to invert this triangular “Pascal matrix” L. You see Pascal’s triangle —adding each entry to the entry on its left gives the entry below. The entries of L are “binomial coefficients”. The next row would be 1; 4; 6; 4; 1.
Triangular Pascal matrix L D
5 D^ abs(pascal (4,1))
Solution Gauss-Jordan starts with ΠL I and produces zeros by subtracting row 1 :
ΠL I D
The next stage creates zeros below the second pivot, using multipliers 2 and 3. Then the last stage subtracts 3 times the new row 3 from the new row 4 :
5 D^ ŒI L
All the pivots were 1! So we didn’t need to divide rows by pivots to get I. The inverse matrix L^1 looks like L itself, except odd-numbered diagonals have minus signs. The same pattern continues to n by n Pascal matrices, L^1 has “alternating diagonals”.
90 Chapter 2. Solving Linear Equations
11 (a) Find invertible matrices A and B such that A C B is not invertible.
(b) Find singular matrices A and B such that A C B is invertible.
12 If the product C D AB is invertible .A and B are square), then A itself is invertible. Find a formula for A^1 that involves C ^1 and B.
13 If the product M D ABC of three square matrices is invertible, then B is invertible. (So are A and C .) Find a formula for B^1 that involves M ^1 and A and C.
14 If you add row 1 of A to row 2 to get B, how do you find B ^1 from A^1?
Notice the order. The inverse of B D
A
is :
15 Prove that a matrix with a column of zeros cannot have an inverse.
16 Multiply
(^) a b c d
times
(^) d b c a
. What is the inverse of each matrix if ad ¤ bc?
17 (a) What matrix E has the same effect as these three steps? Subtract row 1 from row 2, subtract row 1 from row 3, then subtract row 2 from row 3. (b) What single matrix L has the same effect as these three reverse steps? Add row 2 to row 3, add row 1 to row 3, then add row 1 to row 2.
18 If B is the inverse of A^2 , show that AB is the inverse of A.
19 Find the numbers a and b that give the inverse of 5 eye(4) – ones(4,4): 2 6 6 4
1
D
a b b b b a b b b b a b b b b a
What are a and b in the inverse of 6 eye(5) – ones(5,5)?
20 Show that A = 4 eye(4) – ones(4,4) is not invertible: Multiply A ones(4,1).
21 There are sixteen 2 by 2 matrices whose entries are 1’s and 0’s. How many of them are invertible?
Questions 22–28 are about the Gauss-Jordan method for calculating A ^1.
22 Change I into A^1 as you reduce A to I (by row operations):
A I
D
and
A I
D
23 Follow the 3 by 3 text example but with plus signs in A. Eliminate above and below the pivots to reduce ΠA I to ΠI A^1 :
A I
D
2.5. Inverse Matrices 91
24 Use Gauss-Jordan elimination on ΠU I to find the upper triangular U ^1 :
U U ^1 D I
1 a b 0 1 c 0 0 1
(^4) x 1 x 2 x 3
5 D
25 Find A^1 and B^1 ( if they exist ) by elimination on ΠA I and ΠB I :
A D
(^5) and B D
26 What three matrices E 21 and E 12 and D^1 reduce A D
2 6
to the identity matrix? Multiply D^1 E 12 E 21 to find A^1.
27 Invert these matrices A by the Gauss-Jordan method starting with ΠA I :
A D
(^5) and A D
28 Exchange rows and continue with Gauss-Jordan to find A^1 :
A I
D
29 True or false (with a counterexample if false and a reason if true):
(a) A 4 by 4 matrix with a row of zeros is not invertible. (b) Every matrix with 1’s down the main diagonal is invertible. (c) If A is invertible then A^1 and A^2 are invertible.
30 For which three numbers c is this matrix not invertible, and why not?
A D
2 c c c c c 8 7 c
31 Prove that A is invertible if a ¤ 0 and a ¤ b (find the pivots or A^1 ):
A D
a b b a a b a a a
2.5. Inverse Matrices 93
42 Direct multiplications 1–4 give MM ^1 D I , and I would recommend doing # 3. M ^1 shows the change in A^1 (useful to know) when a matrix is subtracted from A:
1 M D I uv and M ^1 D I C uv=.1 vu/ .rank 1 change in I / 2 M D A uv and M ^1 D A^1 C A^1 uvA^1 =.1 vA^1 u/ 3 M D I U V and M ^1 D In C U.Im V U /^1 V 4 M D A U W ^1 V and M ^1 D A^1 C A^1 U.W VA^1 U /^1 VA^1
The Woodbury-Morrison formula 4 is the “matrix inversion lemma” in engineering. The Kalman filter for solving block tridiagonal systems uses formula 4 at each step. The four matrices M ^1 are in diagonal blocks when inverting these block matrices (v is 1 by n, u is n by 1 , V is m by n, U is n by m). I u v 1
A u v 1
In U V Im
A U
V W
43 Second difference matrices have beautiful inverses if they start with T 11 D 1 (instead of K 11 D 2 ). Here is the 3 by 3 tridiagonal matrix T and its inverse:
T 11 D 1 T D
5 T ^1 D
One approach is Gauss-Jordan elimination on ΠT I . That seems too mechanical. I would rather write T as the product of first differences L times U. The inverses of L and U in Worked Example 2.5 A are sum matrices , so here are T and T ^1 :
LU D
difference difference
U ^1 L^1 D
sum sum
Question. ( 4 by 4 ) What are the pivots of T? What is its 4 by 4 inverse? The reverse order UL gives what matrix T ? What is the inverse of T ?
44 Here are two more difference matrices, both important. But are they invertible?
Cyclic C D
5 Free ends^ F^ D
One test is elimination—the fourth pivot fails. Another test is the determinant, we don’t want that. The best way is much faster, and independent of matrix size:
Produce x ¤ 0 so that C x D 0. Do the same for F x D 0. Not invertible.
Show how both equations C x D b and F x D b lead to 0 D b 1 C b 2 C C bn. There is no solution for other b.
94 Chapter 2. Solving Linear Equations
45 Elimination for a 2 by 2 block matrix : When you multiply the first block row by CA^1 and subtract from the second row, the “ Schur complement ” S appears: I 0 CA^1 I
A B
C D
D
A B
0 S
A and D are square S D D CA^1 B:
Multiply on the right to subtract A^1 B times block column 1 from block column 2.
A B 0 S
I A^1 B
0 I
D ‹ Find S for
A B
C I
D
The block pivots are A and S. If they are invertible, so is ΠA B I C D .
46 How does the identity A.I C BA/ D .I C AB/A connect the inverses of I C BA and I C AB? Those are both invertible or both singular: not obvious.
