Download 12 Solved Problems on Linear Systems - Homework | MATH 6370 and more Assignments Mathematical Methods for Numerical Analysis and Optimization in PDF only on Docsity!
Department of Mathematics Fall 2007
MATH 6370, Section 10583 MW 5:30-7:00PM
A. Caboussat
Homework Chapter 8
Homework from the text
Chapter Exercises
Additional Homework
Problem 1 15
Let A ∈ MN. Show that the series
∞ k=
A
k converges if and only if ρ(A) < 1, and that, under this condition,
the matrix I − A is invertible and
∞ ∑
k=
A
k = (I − A)
− 1 .
Hint: Use ρ(A) < 1 ⇔ limk→∞ A k = 0.
Problem 2 41
For η ∈ R, we consider the linear system
Aη x = bη, Aη =
1 1 − η 0
1 / 2 1 0
0 0 1
(^) , b η =
3 − η
2
2
The matrix A is nonsingular if η 6 = −1.
2.a) What are the (necessay and sufficient) conditions on η to ensure that the Gauss-Seidel method for the
resolution of (1).
2.b) For which value η
⋆ of η can we apply the conjugate gradient method? For this particular value, find the
coefficient C arising in the error estimation:
x
k+ − x
Aη⋆
6 C
x
k − x
Aη⋆
, k > 0.
2.c) Does the Jacobi method converge when η = η ⋆ ?
Problem 3 17
Let A, M ∈ MN and set N = M − A. To solve numerically the linear system A~x = ~b, we use the iterative
method:
M~xi+1 = N~xi + ~b, i = 0, 1 , 2 ,... , ~x 0 given.
let us assume that both A and M + N
T are symmetric positive definite.
3.a) Show that the matrix M is invertible.
3.b) Show that the iterative method converges, i.e. ρ(M − 1 N ) < 1.
Let A ∈ MN be the matrix defined by
A =
and
b ∈ R
N .
4.a) Show that, if λk = 2(1 − cos
kπ N +
) and ϕ~
k ∈ R
N defined by ϕ~
k j = sin
kjπ m+
, j = 1,... , N , then
A~ϕ
k = λk ~ϕ
K , k = 1, 2 ,... , N.
4.b) We consider the relaxed Jacobi method, with relaxation parameter ω > 0, defined by:
~xn+1 = (1 − ω)~xn + ω
D
− 1 (E + F )~xn + D
b
, n = 0, 1 , 2 ,...
Show that, for the matrix A defined earlier, this method is equivalent to
~xn+1 = ~xn +
ω
b − A~xn
Compute the convergence speed of the method (i.e. the spectral radius of the matrix governing the
iterative method).
Problem 5 22
Let A ∈ Mn×n, A = D −E −F where D, −E and −F are the matrices given by the diagonal elements, the lower
diagonal elements and the upper diagonal elements of A respectively. We assume that A and D are invertible
and we solve Ax = b with the following Gauss-Seidel method, by applying relaxation for each component:
(~y
k )i =
Aii
(~b) i −^
ji
Aij (~x
k )j
(~x
k+ )i = ω(~y
k )i + (1 − ω)(~x
k )i
This method is the SOR (successive overrelaxation) mehod. In the matrix-vector notation, the SOR method
can be written
ω
D − E
~x
n+
1 − ω
ω
D + F
~x
n
b. (3)
The symmetrized SOR relaxation method
ω
D − E
~y
n
1 − ω
ω
D + F
~x
n
ω
D − F
~x
n+
1 − ω
ω
D + E
~y
n
is called SSOR (symmetric successive overrelaxation).
5.a) Show that (2) and (3) are equivalent.
5.b) Show that, if A is symmetric positive definite, then the methods SOR and SSOR are convergent when
0 < ω < 2.
5.c) When A is the tri-diagonal matrix with coefficients 2 in the main diagonal and −1 in the two other
diagonals, find the optimal value of ω for the SOR method.
Let n be a positive integer, A ∈ Mn×n a symmetric definite positive matrix, and ~x
0 ,
b two given vectors in R
n .
For j ∈ N, we note ~x
j the approximation of ~x = A
b after j iterations of the conjugate gradient algorithm and
Vj the j
th Krylov subspace.
We denote by ¯k the smallest integer (¯k 6 n) such that ~x
¯k = ~x.
Show that, if 0 6 j 6 ¯k − 1, we have
~x
j+ = ~x
0
h
j+
where
h
j+ ∈ Vj+1 satisfies
(~x
0
h
j+ , ~η)A = (
b, ~η), ∀~η ∈ Vj+1.
(This means that the conjugate gradient method is a fixed subspaces method without relaxation with respect to
V 1 , V 2 ,... , Vk.)
Hint: Start by proving that, for 0 6 j 6
k − 1, we have for two successive iterations ~x
j+ and ~x
j :
~x
j+ = ~x
j
j+ ,
where ~g
j+ ∈ Vj+1 satisfies
(~x
j
j+ , ~η)A = (~b, ~η), ∀~η ∈ Vj+1.
Use Vj+1 = span{ w~ 1 ,... , ~w j+ }, where the w~ l are the descent directions generated by the conjugate gradient
method.
Problem 9 30
Let A ∈ Mn×n a symmetric positive definite matrix and ~x
0 ,
b two vectors of R
n .
For j ∈ N , we denote by ~x
j the approximation of ~x = A
b obtained after j iterations of the conjugate gradient
algorithm and Vj the j
th Krylov space. We denote by
k the smallest integer such that (
k 6 n) ~x
k = ~x.
9.a) Show that ∣ ∣
∣~xj^ − ~x
A
∣~x
0 − ~x + ~ξ
A
, ∀~ξ ∈ Vj , ∀j 6 ¯k.
9.b) Show that, if P 0 j = {p ∈ Pj : p(0) = 1}, we have
∣~xj^ − ~x
A
6 |||p(A)||| A−^1
∣~x^0 − ~x
A
, ∀p ∈ P
0 j , ∀j 6
k.
Problem 10 31
Let A ∈ Mn×n a symmetric positive definite matrix and ~x 0 , ~b two vectors of R n .
For j ∈ N , we denote by ~x
j the approximation of ~x = A
b obtained after j iterations of the conjugate gradient
algorithm and Vj the j
th Krylov space.
We denote by ¯k the smallest integer such that (¯k 6 n) ~x k = ~x.
10.a) Show that, for all j 6 k¯, we have
∣~xj^ − ~x
A
cond(A) − 1
cond(A) + 1
j ∣ ∣
∣~x^0 − ~x
A
Hint: Set p(x) = αTj (1 − 2
x−λ 1 λN −λ 1 ), where Tj (x) is the j
th Chebychev polynomial. (T 0 (x) = 1, T 1 (x) = x,
Tm+1(x) = 2xTm(x)−Tm− 1 (x), m > 1) and use the inequalities |Tj (x)| 6 1 if − 1 6 x 6 1 and Tj (x) > x j
if x > 1.
Let E ∈ Mn×n a symmetric positive definite matrix. Show that solving the linear system E~x =
b with the
GMRES method is equivalent to solving E
2 ~x = E
b with the conjugate gradient algorithm preconditioned with
C = E.
Problem 12 40
Let us consider the 3 × 3 matrix:
A =
12.a) Study the convergence of the Jacobi and Gauss-Seidel methods.
12.b) Show that:
ρ(H) = ρ(J)
2 .
Problem 13 42
Scaling means the multiplication of quantities of interest by constant scalars. In physical and technical appli-
cations, this typically means the transformation of units. A numerical method for a given problem is said to be
scaling invariant, if its application to the scaled problem gives rise to the same results as in the unscaled case.
Show that the Gauss-Seidel iteration applied to the linear algebraic system Ax = b, A ∈ R
n×n , b ∈ R
n , is
scaling invariant.
Problem 14 43
We consider the tridiagonal matrix A = (aij ) ∈ R n×n given by:
aij =
2 when i = j
− 1 when |i − j| = 1
0 when |i − j| > 2
, 1 6 i, j, 6 n.
14.a) Show that A is positive definite.
14.b) Let D := diag (A) be the diagonal of A and denote by L the lower triangular part so that A = L+D+L
T .
Consider the linear system Ax = b, b ∈ R
n and show that the preconditioned conjugate gradient method
with preconditioner C := EE
T , where E :=
1 2 D + L, converges after at most two steps.
Hint : Show that the convergence properties of the preconditioned conjugate gradient method correspond
to the original conjugate gradient method applied to the transformed matrix A˜ := E − 1 AE −T and use
the fact that the original conjugate gradient method converges in at most p steps, where p is the number
of different eigenvalues of the matrix.
Problem 15 48
The conjugate gradient is applied to a symmetric positive definite matrix A with the results ||e 0 || A = 1 and
||e 10 || A
− 10
. Based on this data:
15.a) What bound can you give on cond(A)?
15.b) What bound can you give on ||e 20 || A
