Download Numerical methods on solving linear systems and more Study notes Advanced Calculus in PDF only on Docsity!
Ching, Robby Andre July 3 , 2020
Introduction and Overview
The roots for a system of 2 linear equations could easily be obtained by the Graphical method. If
it is a system of 3 linear equations, the Cramer’s rule and the simple Elimination of unknowns could be
used. But for a system of more than 3 linear equations, it would be difficult and tedious to use these two
methods. This section offers algorithm-able solutions for systems with more than 3 linear equations,
making them easier to solve once coded with a computer program [1].
When a system consists of parallel lines (a), there would be no solution for such a system. If the
linear equations of a system are coincident to one another (b), there would be infinite solutions. These
two cases are called Singular Systems. If a system is very close to being singular(c), then it is called an ill-
conditioned system. The determinant of a singular system is equal to 0, while that for an ill-conditioned
system, near 0. Such systems would pose trouble in calculations [1].
𝑎
𝑏
𝑐
Naïve Gauss Elimination
It is primarily based on the Elimination of Unknowns, where two steps are involved: Forward
Elimination and Backward Substitution. Suppose we have a system of 5 linear equations, then the 1
st
Forward Elimination would go like this [1]:
11
12
13
14
15
1
11
21
22
23
24
25
2
𝑖 1
𝑖
𝑖 1
𝑎
11
𝑖 1
31
32
33
34
35
3
41
42
43
44
45
4
51
52
53
54
55
5
*i is the ith row excluding the pivot equation of the current forward elimination phase.
example:
21
21
11
22
21
12
23
21
13
24
21
14
25
21
15
2
21
1
22
′
23
′
24
′
25
′
2
′
The resulting equations would then be the following. For the 2
nd
elimination, we have:
11
12
13
14
15
1
22
′
23
′
24
′
25
′
2
′
22
32
′
33
′
34
′
35
′
3
′
𝑖 2
𝑖
𝑖 2
′
𝑏
22
′
2
42
′
43
′
44
′
45
′
4
′
52
′
53
′
54
′
55
′
5
′
example:
32
′
32
22
′
33
′
32
23
′
34
′
32
24
′
35
′
32
25
′
3
′
32
2
′
33
′′
34
′′
35
′′
3
′′
The resulting equations would then be the following. For the 3rd elimination, we have:
11
12
13
14
15
1
22
′
23
′
24
′
25
′
2
′
33
′′
34
′
35
′′
3
′′
33
43
′′
44
′′
45
′′
4
′′
𝑖 3
𝑖 3
33
3
53
′′
54
′′
55
′′
5
′′
example:
43
′′
43
33
′′
44
′′
43
34
′′
45
′′
43
35
′′
4
′′
43
3
′′
44
′′′
45
′′′
4
′′′
The resulting equations would then be the following. For the 3rd elimination, we have:
11
12
13
14
15
1
22
′
23
′
24
′
25
′
2
′
33
′′
34
′
35
′′
3
′′
44
′′′
45
′′′
4
′′′
44
54
′′′
55
′′′
5
′′′
𝑖 4
𝑖 4
44
4
11
12
13
14
15
1
22
′
23
′
24
′
25
′
2
′
33
′′
34
′
35
′′
3
′′
44
′′′
45
′′′
4
′′′
55
′′′′
5
′′′′
22
′
23
′
24
′
25
′
33
′′
34
′
35
′′
44
′′′
45
′′′
55
′′′′
11
22
′
33
′′
34
′
35
′′
44
′′′
45
′′′
55
′′′′
11
22
′
33
′′
44
′′′
45
′′′
55
′′′′
11
22
′
33
′′
[(𝑑
44
′′′
55
′′′′
45
′′′
∙ 0 )] → 𝑫 = 𝒂
𝟏𝟏
𝟐𝟐
′
𝟑𝟑
′′
𝟒𝟒
′′′
𝟓𝟓
′′′′
*That is, for any upper triangular matrix, its determinant is the product of all the elements of its primary
diagonal line. If the value of D is equal to 0, then the system is singular. If the value of D is near 0, then
the system is ill-conditioned [1].
Weaknesses and Improvements on the Naïve Gauss Elimination
Scaling
Because the determinant value is relative to the value of the coefficients of the variables, the
resulting determinant may become a high positive value even though it is an ill-conditioned system or
singular system. To avoid this, we scale up the coefficients and the constant of the variables when the
coefficients are relatively higher compared to others, such that the highest coefficient is 1. That is, for
each equation, we find the highest coefficient and divide all the coefficients and the constant of that
equation with that coefficient. It also minimizes round-off errors for systems that has equations with
relatively larger coefficients than others [1].
Another benefit for scaling the system in which the largest coefficient would be 1 is to thoroughly
check if a system is an ill-conditioned system. Suppose we have a well-conditioned system (1) and an ill-
conditioned system (2) [1]:
( 1 )
( 2
)
But when we scale these two equations such that the highest coefficient is 1, then we have the
following results:
( 1 )
1
2
2
3
( 2
)
This shows that the determinant of the ill-conditioned system is now closer to 0, which really is
supposed to be near 0.
Partial Pivoting
If the initial (
st
) pivot element is 0 or near 0, there would be an error because it would be a division
by 0. To avoid this, we do partial pivoting, in which the 1
st
equation (
st
row) would be replaced with
another equation(row) in which its 1
st
coefficient is the farthest from 0 (most positive or most negative),
and this equation would be the new 1
st
pivot equation [1].
Use of more decimal places
To have better approximations on the roots or solution for an ill-conditioned system, one could
employ higher significant figures in the whole process of the iterations [1].
Sample #1: Solve the following linear system using Gauss Elimination
st
Forward Elimination:
21
31
Sample #2: Develop, debug, and test a program in either a high-level
language or macro language of your choice to solve a system of
equations with Gauss elimination with partial pivoting.
The problem was solved using Matlab:
Summary of results:
Sample #3: A team of 5 parachutists is connected by a weightless cord while free-falling
at a velocity of v = 9 m/s. Calculate the tension in each section of the cord and the
acceleration of the team, given the following: (g=9.81 m/s
2
Parachutist Mass (m in kg) Drag coefficient
(c in kg/s)
mg vc
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
The problem was solved using Matlab. The summary of the results:
𝟐
Sample #4: Solve the circuit in the figure below for
the currents in each wire. Use the Kirchhoff’s
Current rule
𝑖 = 0 and Voltage rule
(where 𝜉 is electromotive force or emf in V, R is
resistance in ohms, and I is current in amperes). Use
Gauss elimination with pivoting.
*The arrows of the current are all assumed. Upon
solving, if the value turned out to be +, the direction
of assumption is correct. If it is negative, then the
direction of assumption is incorrect. Let the order be
arranged as:
12
25
32
34
47
58
63
65
76
80
89
97
Using the Kirchhoff’s current rule:
12
32
25
12
25
32
63
32
34
32
34
63
34
47
34
47
25
65
58
25
58
65
76
63
65
63
65
76
47
97
76
47
76
97
58
80
89
58
80
89
89
97
89
97
Using the Kirchhoff’s voltage rule:
*If the arrow is in a direction that tends to go clockwise, the sign is negative, if counter-clockwise, positive.
32
25
65
63
25
32
63
65
34
47
76
63
34
47
63
76
58
65
76
89
97
58
65
76
89
97
89
97
47
34
32
25
32
34
47
89
97
*Pivoting was actually done during the Forward Elimination process. The results of partial pivoting were
not displayed to have a more friendly display of results.
Summary of Results:
𝟏𝟐
𝟐𝟓
𝟑𝟐
𝟑𝟒
𝟒𝟕
𝟓𝟖
𝟔𝟑
𝟔𝟓
𝟕𝟔
𝟖𝟎
𝟖𝟗
𝟗𝟕
Gauss-Jordan Elimination
It is a variation of the Naïve Gauss Elimination method. In here, the unknown to be eliminated is
eliminated from all other equations. Also, all rows or equations are normalized by their pivot elements,
and no Backward Substitution is required. The resulting matrix of the last elimination is an Identity matrix ,
called the Row Echelon Form of the initial Augmented Matrix. The result or solution would be exactly the
same with that of the Naïve Gauss Elimination [1].
*The process of dividing all the coefficients of a pivot equation with the pivot element is called
Normalization [1].
11
12
13
1
21
22
23
2
31
32
33
3
Express the system into an augmented matrix:
[
11
12
13
21
22
23
31
32
33
1
2
3
]
Normalize the 1
st
row by dividing it by the 1
st
pivot element 𝑎
11
[
12
11
13
11
21
22
23
31
32
33
1
11
2
3
]
[
12
′
13
′
21
22
23
31
32
33
1
′
2
3
]
Multiply the 1
st
coefficient of each coefficient on each of the element of the 1
st
pivot equation and subtract
them respectively to each of the element in that equation:
[
12
′
13
′
21
21
22
21
12
′
23
21
23
31
31
32
31
12
′
33
31
12
′
1
′
2
21
1
′
3
31
1
′
]
[
12
′
13
′
22
′
23
′
32
′
33
′
1
′
2
′
3
′
]
Sample #1: Solve the given system of linear equations by using Gauss-Jordan Elimination
Normalizing the 1
st
row:
[
]
[
]
[
]
Eliminating the 1
st
variable coefficients:
[
6 − ( 6 )(− 0. 50 )]
[
]
Normalizing the 2
nd
row:
[
]
[
]
Eliminating the 2
nd
variable coefficients:
[
9 − (− 0. 50 )(− 26 ) ]
[
]
Normalizing the 3
rd
row:
[
]
[
]
Eliminating the 3
rd
variable coefficient and finding the Reduced Row Echelon form:
[
]
[
]
Summary:
Sample #2: Use Gauss-Jordan Elimination to solve the given system. Use scaling and if necessary, partial
pivoting.
Summary of results:
Gauss-Seidel Method
This is an iterative and indirect method of finding the solution for a given system of linear
equations. It is the counterpart of Fixed-Point iteration or Method of Successive Substitution in the root
finding of a single linear equation. Initial guesses for all the (n-1) variables present in the system are taken,
usually being a value of 0. These will be the values used in solving the first variable from one of the
equations in the system for the first iteration. For the next variable, the solved first variable and the other
variables with an initial guess of 0 will be used, still under the 1
st
iteration. This procedure will be done
until all the variables are solved. For the second iteration, the variables to be used are the solved variables
from the first iteration. Suppose we have a system [1]:
11
12
13
1
𝟏
𝟏𝟐
𝟏𝟑
𝟏𝟏
21
22
23
2
𝟐
𝟐𝟏
𝟐𝟑
𝟐𝟐
31
32
33
3
𝟑
𝟑𝟏
𝟑𝟐
𝟑𝟑
For the 1
st
iteration with initial guess of 𝑥 = 𝑥
0
0
0
1
1
12
0
13
0
11
1
2
21
1
23
0
22
1
3
31
1
32
1
33
For the 2
nd
iteration:
2
1
12
1
13
1
11
2
2
21
2
23
1
22
1
3
31
2
32
2
33
where
𝑎,𝑖
𝑖
2
𝑖
1
𝑖
2
For the nth iteration:
𝑛
1
12
𝑛− 1
13
𝑛− 1
11
𝑛
2
21
𝑛
23
𝑛− 1
22
1
3
31
𝑛
32
𝑛
33
where
𝑎,𝑖
𝑖
𝑛
𝑖
𝑛− 1
𝑖
𝑛
Termination Criteria and Error Estimates
It has the same termination criteria for the numerical methods on root finding for single linear
equations. The iterations would stop if the desired number of iterations are achieved, or if the
approximate relative error of one of the variables is equal to or less than the desired approximate relative
error [1].
𝑎,𝑖
𝑖
𝑛
𝑖
𝑛− 1
𝑖
𝑛
Convergence criterion
The convergence criterion for the Gauss-Seidel method is the same with that of the Fixed-Point
iteration or MOSS, the derivative of the function must be less than 1 (that is, its slope must be lower than
unity) [1]. The equations can be expressed in this manner such that the variable to be solved is an explicit
function of the rest of the variables:
1
12
13
11
2
21
23
22
3
31
32
33
12
11
13
11
12
11
13
11
𝟏𝟐
𝟏𝟑
𝟏𝟏
