Download LU Decomposition - Numerical Analysis - Solved Exam and more Exams Mathematical Methods for Numerical Analysis and Optimization in PDF only on Docsity!
Chapter 04.
LU Decomposition
After reading this chapter, you should be able to:
- identify when LU decomposition is numerically more efficient than Gaussian
- eliminationdecompose a nonsingular matrix into LU, and,
- show how LU decomposition is used to find the inverse of a matrix. I hear about LU decomposition used as a method to solve a set of simultaneous linear equations. What is it? We already studied two numerical methods of finding the solution to simultaneous linear equations – Naïve Gauss elimination and Gaussian elimination with partial pivoting. Then, why do we need to learn another method? To appreciate why LU decomposition could be a better choice than the Gauss elimination techniques in some cases, let us discuss first what LU decomposition is about.
For a nonsingular matrix [ A ] on which one can successfully conduct the Naïve Gauss
elimination forward elimination steps, one can always write it as
[ A ] =[ ][ L U ]
where
[ ] L = Lower triangular matrix
[ U ]= Upper triangular matrix
Then if one is solving a set of equations [ A ][ X ]= [ C ], then
[ ][ L U ][ X ] = [ C ]as ([ A ]=[ ][ L U ])
Multiplying both sides by [ ] L − 1 ,
[ ] L −^1 [ ][ L U ][ X ] =[ ] L −^1 [ C ]
[ ][ I U ][ X ]= [ ] L − 1 [ C ]as ([ ] L − 1 [ ] L =[ I ])
[ U ][ X ] = [ ] L − 1 [ C ]as ([ ][ I U ]=[ U ])
Let
[ ] L −^1 [ C ] =[ Z ]
04.07.2 Chapter 04.
then
[ ][ L Z ] = [ C ] (1)
and
[ U ][ X ] = [ Z ] (2)
So we can solve Equation (1) first for [ Z ] by using forward substitution and then use
Equation (2) to calculate the solution vector [ X ]by back substitution.
This is all exciting but LU decomposition looks more complicated than Gaussian elimination. Do we use LU decomposition because it is computationally more efficientthan Gaussian elimination to solve a set of n equations given by [ A ][ X ]=[ C ]?
For a square matrix [ A ] of n × n size, the computational time^1 CT | DE to decompose the [ A ] matrix to (^) [ L ][ U ]form is given by CT | DE = (^) ^83 + 4 2 −^203 T n^3 n n , where T = clock cycle time^2.
The computational time CT | FS to solve by forward substitution [ ][ L Z ] = [ C ]is given by
CT | FS = T ( 4 n 2 − 4 n )
The computational time CT | BS to solve by back substitution [ U ][ X ] = [ Z ]is given by
CT | BS = T ( 4 n 2 + 12 n ) So, the total computational time to solve a set of equations by LU decomposition is CT | LU = CT | DE + CT | FS + CT | BS = T (^) ^83 n^3 + 4 n^2 −^203 n + T ( 4 n^2 − 4 n )+ T ( 4 n 2 + 12 n ) = T ^83 n^3 + 12 n^2 +^43 n
Now let us look at the computational time taken by Gaussian elimination. The computational time CT | (^) FE for the forward elimination part, CT | FE = (^) ^83 + 8 2 −^323 T n^3 n n ,
(^1) The time is calculated by first separately calculating the number of additions, subtractions, multiplications, and divisions in a procedure such as back substitution, etc. We then assume4 clock cycles each for an add, subtract, or multiply operation, and 16 clock cycles for a divide operation as is the case for a typical AMD®-K7 chip. http://www.isi.edu/~draper/papers/mwscas07_kwon.pdf 2 As an example, a 1.2 GHz CPU has a clock cycle of 1 /( 1. 2 × 109 )= 0. 833333 ns
04.07.4 Chapter 04.
= T ^83 n^4 + 12 n^3 +^43 n^2
Clearly for large n , CT | inverse (^) GE >> CT | inverse (^) LU as CT | inverse (^) GE has the dominating terms of n^4 and CT | inverse (^) LU has the dominating terms of n^3. For large values of n , Gaussian elimination method would take more computational time (approximately n / 4 times – prove it) than the LU decomposition method. Typical values of the ratio of the computational time for different values of n are given in Table 1. Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination. n (^) 10 100 1000 10000 CT | inverse (^) GE / CT | inverse (^) LU 3.28 25.83 250.8 2501
Are you convinced now that LU decomposition has its place in solving systems of equations?We are now ready to answer other curious questions such as
- How do I find LU matrices for a nonsingular matrix [ A ]?
- How do I conduct forward and back substitution steps of Equations (1) and (2), respectively?
How do I decompose a non-singular matrix [ A ] , that is, how do I find [ A ] = [ ][ L U]?
If forward elimination steps of the Naïve Gauss elimination methods can be applied on a
nonsingular matrix, then [ A ] can be decomposed into LU as
n n nn
n
n
a a a
a a a
a a a A
1 2
21 22 2
11 12 1 [ ]
nn
n
n
n n u
u u
u u u
22 2
11 12 1
1 2
21
The elements of the [ U ]matrix are exactly the same as the coefficient matrix one obtains at
the end of the forward elimination steps in Naïve Gauss elimination.
The lower triangular matrix [ ] L has 1 in its diagonal entries. The non-zero elements on the
non-diagonal elements in [ ] L are multipliers that made the corresponding entries zero in the
upper triangular matrix [ U ]during forward elimination.
Let us look at this using the same example as used in Naïve Gaussian elimination.
LU Decomposition 04.07.
Example 1 Find the LU decomposition of the matrix
[ ]
A
Solution
[ A ] =[ ][ LU ]
33
22 23
11 12 13 31 32
21 0 0
u
u u
u u u
The [ U ]matrix is the same as found at the end of the forward elimination of Naïve Gauss
elimination method, that is
[ ]
U
To find 21 and 31 , find the multiplier that was used to make the a (^) 21 and a (^) 31 elements zero in the first step of forward elimination of the Naïve Gauss elimination method. It was
25
21 =^64
31 =^144
To find 32 , what multiplier was used to make a (^) 32 element zero? Remember a (^) 32 element
was made zero in the second step of forward elimination. The [ A ] matrix at the beginning of
the second step of forward elimination was
So
- 8
32 16.^8
Hence
[ ]
L
Confirm [ ][ L U ] = [ A ].
LU Decomposition 04.07.
z 2 (^) == 177177. (^). 22 −− 22 .. 5656 z × (^1106). 8 =− 96. 208 z 3 (^) = 279. 2 − 5. 76 z 1 − 3. 5 z 2
= 279. 2 − 5. 76 × 106. 8 − 3. 5 × (− 96. 208 )
Hence
[ ]
3
2
1 z
z
z Z
This matrix is same as the right hand side obtained at the end of the forward elimination steps of Naïve Gauss elimination method. Is this a coincidence? Now solve
[ U ][ X ] =[ Z ]
3
2
1 .
a
a
a
25 a 1 + 5 a 2 + a 3 = 106. 8 − 4. 8 a 2 − 1. 56 a 3 =− 96. 208
- 7 a 3 = 0. 76 From the third equation
- 7 a 3 = 0. 76 07
a =. = 1. Substituting the value of a 3 in the second equation, − 4. 8 a 2 − 1. 56 a 3 =− 96. 208
- 8
2 96.^2081.^563
a = − + a
=− + ×
Substituting the value of a 2 and a 3 in the first equation, 25 a 1 + 5 a 2 + a 3 = 106. 8 25 a 1 =^106.^8 −^5 a^2 −^ a^3
04.07.8 Chapter 04.
25
=^106.^8 −^5 ×^19.^691 −^1.^0857
Hence the solution vector is
3
2
1 a
a
a
How do I find the inverse of a square matrix using LU decomposition?
A matrix [ B ]is the inverse of [ A ] if
[ A ][ B ] = [ ] I =[ B ][ A ].
How can we use LU decomposition to find the inverse of the matrix? Assume the first
column of [ B ](the inverse of T [ A ] ) is
[ b 11 b 12 ...... bn 1 ] Then from the above definition of an inverse and the definition of matrix multiplication
[ ]
1
21
11 b n
b
b A
Similarly the second column of [ B ]is given by
[ ]
2
22
12 b n
b
b A
Similarly, all columns of [ B ]can be found by solving n different sets of equations with the
column of the right hand side being the n columns of the identity matrix. Example 3 Use LU decomposition to find the inverse of
[ ]
A
Solution Knowing that
[ A ] =[ ][ LU ]
04.07.10 Chapter 04.
7 b 31 = 3. 2 Backward substitution starting from the third equation gives
7 b 31 =^3.^2 = 4. 571
8
21 2.^561.^5631
b = − + b
- 8
b 11 =^1 −^5 b^21 −^ b^31 25
=^1 −^5 (−^0.^9524 )−^4.^571
Hence the first column of the inverse of [ A ] is
31
21
11 b
b
b
Similarly by solving
32
22
12 b
b
b gives
32
22
12 b
b
b
and solving
33
23
13 b
b
b gives
33
23
13 b
b
b
Hence
[ ]
1 0.^047620.^083330.^03571
A
Can you confirm the following for the above example?
[ A ][ A ]− 1 =[ ] I =[ A ]− 1 [ A ]
Key Terms: LU decomposition Inverse
