Scarica Algebra lineare e sistemi lineari e più Dispense in PDF di Analisi Numerica solo su Docsity!
1 Afternotes on the solvability of linear systems and their generalized
2 solutions
Antonio Orlando & Mariela Luege
∗ 3
4 Contents
5 1 Notions on Linear Spaces 1
6 1.1 Basis of Vector Spaces................................... 3
7 1.2 Linear operators...................................... 4
8 1.3 Linear continuous operators................................ 5
9 2 Matrices 6
10 2.1 General concepts...................................... 6
11 2.2 Special Types of Matrices................................. 10
12 2.2.1 Triangular...................................... 10
13 2.2.2 Hessenberg matrix................................. 10
14 2.2.3 Hermitian & Skew-Hermitian matrices...................... 10
15 2.2.4 Unitary matrix................................... 10
16 2.2.5 Permutation.................................... 11
17 2.2.6 Nilpotent & Indepotent.............................. 11
18 2.2.7 Rank-one matrices................................. 11
19 2.3 Matrix norms and inner product............................. 11
20 3 Subspaces of a Linear operator 14
21 3.1 General notions....................................... 14
22 3.2 Kernel and Range..................................... 14
23 3.3 Projectors.......................................... 18
24 4 Linear Systems 18
25 5 Eigenvalues and Eigenvectors 22
26 5.1 General concepts...................................... 22
27 5.2 Eigenvalues and Eigenvectors of a Hermitian Matrix.................. 28
28 5.3 Eigenvalues and Eigenvectors of a Skew–Hermitian Matrix............... 29
29 5.4 Eigenvalues and Eigenvectors of a Unitary Matrix................... 29
30 5.5 Eigenvalues and Eigenvectors of a Projection...................... 29
∗ CONICET-FACET, Universidad Nacional de Tucum´an, Argentina, {aorlando, mluege}@herrera.unt.edu.ar
31 6 Matrix factorizations 29
32 6.1 P AQ = LU –factorization................................. 30
33 6.2 Rank–factorization..................................... 30
34 6.3 QR–factorization...................................... 30
6.4 Singular Value Decomposition (SVD) of A ∈ R
m×n 35................... 31
6.4.1 Rank-one decomposition of A ∈ R
m×n 36...................... 35
37 6.5 Polar decomposition.................................... 38
38 6.5.1 The positive square root of a positive semidefinite matrix............ 41
39 6.5.2 Application..................................... 43
40 6.6 Schur–factorization..................................... 44
41 A Fundamental Theorem of Algebra 44
42 Abstract
43 Objective of this tutorial is to discuss the conditions for the solution of a linear system
44 Ax = b where A ∈ Rm×n, x ∈ Rn^ and b ∈ Rm, thus m is the number of equations (number
45 of rows) and n is the number of unknowns (number of columns). We will consider the question
46 of existence of classical solutions, that is, given b ∈ Rm^ whether we can find x ∈ Rn^ such that
47 Ax = b, and in such a case, whether there is only one solution, and of existence of generalized
solutions when we cannot find any x ∈ R
n 48 such that Ax = b. We will also discuss the issue of
49 stability and discuss how in practice we can identify the different cases and the corresponding
50 methods of solution, by addressing also the case of when m and n are very large, i.e. for
51 very large systems. We will describe finally applications, such as structural systems, hydraulic
52 networks, electrical circuits, where each of the above cases can occur, and their interpretation in
53 terms of classical methods of solution of the above applications such as Cross’s method, Ritter’s
54 method and alike.
55 1 Notions on Linear Spaces
56 In a finite-dimensional vector space, there is only one topology induced by a norm: the Euclidean
57 topology, given that on finite dimensional spaces, all the norms are equivalent. In fact, whether
58 K = R or K = C, we have the following.
Proposition 1.1. In K
n 59 any two norms are equivalent.
60 If a norm derives from an inner product, this does not mean that all the other norms equivalent
61 to it, also derive from an inner product. They define the same topology, that is, the normed spaces
62 equipped with the two equivalent norms have the same convergent sequences to the same limits [6,
63 page 288].
64 There is a characterizing condition about norms which ensures that there exists an inner product
65 such that generates that norm. Let X be a real (complex) vector space with an inner (Hermitian)
product (x, y). Then ‖x‖ =
66 (x, x) is a norm on X. But in general, norms on vector spaces are
67 not induced by inner or hermitian products. We have the following result from [6, page 287].
68 Proposition 1.2. Let ‖·‖ be a norm on a real (respectively, complex) normed space X. A necessary
69 and sufficient condition for the existence of an inner (Hermitian) product (·, ·) such that ‖x‖ = √
70 (x, x) for any x ∈ X, is that the parallelogram law holds,
‖x + y‖
2
2 = 2(‖x‖
2
2 71 ) ∀ x, y ∈ X. (1.1)
92 1.1 Basis of Vector Spaces
93 We start with the definition of linearly indipendent vectors which is a notion that applies to a finite
94 number of vectors [6, p. 5 & p. 43].
95 Definition 1.1. Let V be a vectors space. We say that a finite number of vectors, vi ∈ V , i =
96 1 ,... , p, p ∈ N, are linearly indipendent if their linear compbination is equal to the zero vector only
97 and only if all the coefficients of the linear combination are zero. That is,
98 α 1 v 1 +... + αpvp = 0 ⇒ αi = 0 i = 1,... , p.
99 More generally, we say that a set S ⊆ V of vectors is a set of linearly indipendent vectors
100 whenever any finite list of elements of S is made of by linearly indipendent vectors.
101 Definition 1.2. Given a subset S ⊆ V of the vector space V , we call span of S and denote it by
102 span S, the subset of V of all finite linear combination of elements of S.
103 Proposition 1.4. Assume S ⊆ V , a subset of the vector space V. Then span S is a linear subspace
104 of V and dim span S ≤ dim V.
105 Definition 1.3. Let W be a linear subspace of the vector spsace V and S ⊆ V a subset of vectors
106 of V. If W = span S, we say that the elements of S span W or that S is a set of generators for W.
107 In the particular case that W = V and S is a subset of linearly indipendent vectors of V and
108 there holds span S = V , then S takes on the name of basis of V.
109 Definition 1.4. Let V be a vector space. A set S of linearly indipendent vectors such that span S =
110 V is called a basis of V.
Let xi ∈ R
n 111 , i = 1,... , p with p ≤ n. The set of vectors {x 1 , x 2 ,... , xp} is orthogonal if each
112 pair of vectors in the set is orthogonal, that is, xi · xj = 0 whenever i 6 = j and the set is said to be
113 orthonormal if xi · xi = δij , that is, in addition to be orthogonal each other, the vectors are of unit
114 norm. In this case, we say that the vectors are normalized.
Proposition 1.5. Let {x 1 ,... , xp} ⊆ R
n 115 be a set of orthogonal vectors, none of which is the zero
vector. Then the set of vectors {x 1 ,... , xp} ⊆ R
n 116 is linearly independent.
117 Proof. We have to show that by setting
118 α 1 x 1 +... + αpxp = 0 , (1.4)
119 for αi ∈ R, we get that there must be αi = 0, i = 1,... , p. We start from (1.4) with αi ∈ R. Then
120 we multiply both sides by xj , thus we get that there must be αj = 0, j = 1,... , p.
121 Remark 1.2. The orthogonality of a set of vectors is defined by taking all the pairs of vectors
122 belonging to the set. For the linear independence of the set of vectors, on the other hand, one must
123 consider the whole set. That is, let {x 1 , x 2 ,... , xp} be a set of vectors that are pairwise linearly
124 indipendent, that is,
125 αixi + αj xj = 0 ⇒ αi = αj = 0, i = 1,... , p
126 we cannot conclude that the set of vectors {x 1 , x 2 ,... , xp} is linearly independent. Consider, for
instance, in R
2 127 the following set {(1, 0), (0, 1), (1, 1)} which are paiwrwise linearly indipendent but
128 clearly it is not a set of linearly independent vectors.
129 Proposition 1.6 (Gram-Schmidt process). Let {x 1 ,... , xp} ⊆ V be a finite subset of vectors of
130 V and denote by S the subspace of V spanned by {x 1 ,... , xp}, i.e. S = span{x 1 ,... , xp}. Then
131 dim S ≤ p and starting from {x 1 ,... , xp} we can construct a set of k linearly indipendnet vectors
132 {z 1 ,... , zk}, k ≤ p, which are mutually orthogonal, have unit norm and are such that
133 span{x 1 ,... , xp} = span{z 1 ,... , zk}.
134 In particular, if we apply Proposition 1.6 to a set of n linearly indipendent vectors {x 1 ,... , xn}
135 of V , then we can replace such set by an orthonormal set of n vectors. The following is the form
136 that is generally stated the Gram-Schmidt processes.
137 Corollary 1.1. Given a set of linearly indipendent vectors S = {x 1 ,... , xk} and let X denote
138 the finite dimensional subspace spanned by such vectors, X = span{x 1 ,... , xk}, the set S can be
139 replaced by a set of orthonormal vectors, Z = {z 1 ,... , zk}, which span the same subspace.
140 Proof. The proof of this result is constructive, that is, one gives a process that builds the set Z.
141 One of such processes is the Gram-Schmidt orthonormalization process [10, p. 16].
142 Remark 1.3. The Gram-Schmidt process described above is a theoretical procedure whose numerical
143 implementation is rather unstable, for the orthogonality condition can be lost due to the condition
144 to insure that the inner product is exactly equal to zero [8, p. 231].
145 Proposition 1.7. Every n–dimensional real or complex vector space has an orthonormal basis
146 (that is, a basis consisting of an orthornormal set).
147 1.2 Linear operators
148 Let V and W be vector spaces on the field K (that is, K = C or K = R). A mapping A : V → W
149 is also called an operator of V into W. The operator is said to be linear (or linear transformation)
150 if there holds
151 A(λx + αy) = λA(x) + αA(y) ∀ x, y ∈ V, ∀λ, α ∈ K. (1.5)
152 Definition 1.5. A function that is injective and sujective is called bijective.
153 Remark 1.4. A linear operator is therefore a mapping from one vector space to another that pre-
154 serves the vector space operations, that is, it is a homemorphism between vector spaces. In algebra,
155 an homorphism is a structure preserving map between two algebraic structures of the same type
156 (such as two groups, two rings or two vector spaces). As a result, homomorphism of vector spaces
157 are the linear operators. Isomorphism is used as synonym of linear operators that are bijectives,
158 that is, linear operators that are surjectives (that is, the image of the mapping coincides with the
159 whole space) and invertibles (that is, different vectors have different images).
160 The zero space V = { 0 } is the zero space formed by only one element which must therefore be
161 the zero element because of the definition axioms of vector space. It is a finite dimensional space
162 and its dimension is equal to zero [14, page 50].
163 Definition 1.6. Let V and W be linear spaces. We denote by L(V, W ) the set of all the linear
164 transformations of V into W.
165 The set L(V, W ) is a linear space under the ordinary sum of linear mappings and scalar multi-
166 plication of a mapping by a scalar [14, page 60] and [6, page 44]. If W = R, then L(V, R) is the
167 set of all the linear functionals defined on V. In this case the linear space L(V, R) is called the
168 algebraic dual of V and is sometimes denoted by V +.
207 Proposition 1.12. Let V and W be normed spaces with norm ‖ · ‖V and ‖ · ‖W , respectively.
208 Consider a linear operator A of V into W. The operator A is continuous if and only if it is
209 bounded, that is, the operator A meets (1.7).
210 Linear operators between finite-dimensional normed spaces are always continuous operators [5,
211 page 145], thus they are bounded operators because of Proposition 1.12.
212 Proposition 1.13. Let V and W be normed spaces with finite dimension and A : V → W a linear
213 operator of V into W. Then A is a continuous operator.
214 Remark 1.6. Give examples of unbounded operators which, because of Proposition 1.13, must
215 necessarily be linear operators between infinite dimensional spaces V and W. For instance, check
216 what we can say about the Lapalce operator and other operators. Also, check how you show that
217 a given linear operator is unbounded. Also, give an example of linear operator between infinite
218 dimensional spaces that is bounded.
219 Let V and W be normed vector spaces and assume A : V → W to be a linear operator of V
220 into W. If the linear operator A between normed spaces is bounded then from (1.7) follows that
221 the following set
‖Ax‖W
‖x‖V
, x ∈ V \ { 0 }
223 is bounded from above. If now we consider the vector space L(V, W ) of all the linear bounded (or
224 equivalently continuous) operators of V into W , that is, each of them is linear and meets (1.8), we
225 can define in L(V, W ) the following mapping
ν : A ∈ L(V, W ) → sup x∈V { 0 }
‖Ax‖W
‖x‖V
227 Exercise 1.1. Show that the mapping (1.9) is a norm.
228 Definition 1.8. The norm (1.9) of the bounded operators of V into W is called the consistent
229 norm with the norms of V and W or the operator norm induced by the norm of V and W.
230 Hereafter, given the normed spaces V and W , by L(V, W ) we denote the normed space of
231 all linear and continuous operators of V into W equipped with the norm (1.9). As a result of
232 Proposition 1.13, if V and W have finite dimension, then L(V, W ) will be the vector space of all
233 the linear operators of V into W.
234 2 Matrices
235 2.1 General concepts
For a = (ai) ∈ C
n 236 , we denote by a the complex conjugate of a defined by the complex conjugate
of the components of a, that is, a = (ai), and for A = [aij ] ∈ C
m×n 237 , the matrix A is the complex
238 conjugate of A, i.e. the matrix whose entries are the complex conjugate of the entries of A. We
239 define the dot−product between a and b as follows
a · b =
∑^ n
i=
240 aibi , (2.1)
241 which, for a, b ∈ Rn^ is an inner product in Rn^ but not in Cn, where the standard inner product is
242 defined as
243 〈a, b〉Cn = a · b. (2.2)
244 By the relation between the complex conjugate operation and the operations of sum and product,
245 we have that
a · b = a · b =
∑^ n
i=
246 aibi.
Exercise 2.1. (i) Verify that for u, v ∈ C
n , u · v is not an inner product in C
n 247 , whereas u · v
is an inner product in C
n
(ii) Verify for u ∈ C
n 249 that u · u ≥ 0.
250 An m × n matrix A with entries in K is an ordered table of elements of K arranged in m−rows
251 and n−columns. It is customary to index the rows from top to bottom from 1 to m and the columns
252 from left to right from 1 to n. If {aij } denotes the element of entry in the ith row and jth column,
253 we write [6, page 10]
A =
a 11 a 12... a 1 n
a 21 a 22... a 2 n
............
am 1 am 2... amn
254 , or [A]ij = aji, i = 1,... , m j = 1,... , n. (2.3)
By the definition of product of matrix by vector, for A ∈ C
m×n and a ∈ C
n 255 , we have that
256 Aa = Aa ,
257 where A is the matrix whose entries are the complex conjugate of the entries of A.
Assume that the matrix A ∈ K
m×n 258 with the entries aij ∈ K is given by (2.3). We define the
matrix transpose of A, and denote it by A
T , as the element of K
n×m 259 given by
A
T
a 11 a 21... am 1
a 12 a 22... am 2
............
a 1 n a 2 n... amn
, or [A
T ]ij = [A]ji = aji, i = 1,... , m
j = 1,... , n.
whereas the conjugate transpose (or Hermitian transpose) matrix of A, denoted by A
H 261 , is the
element of K
n×m 262 defined by
A
H
A
)T
AT
a 11 a 21... am 1
a 12 a 22... am 2
............
a 1 n a 2 n... amn
, or [A
H ]ij = [A]ji = aji, i = 1,... , m
j = 1,... , n.
that is, A
H 264 is obtained from A by taking the transpose and then taking the complex conjugate of
265 each entry. We recall that for a, b ∈ R, and i the imaginary unit, the complex conjugate of a + ib
with the column vectors cA i
∈ Rm, i = 1, 2 ,... , n and the row vectors rA i
296 ∈ Rn, given by, respec-
297 tively,
c
A i =
a 1 i
a 2 i
ami
i = 1, 2 ,... , n and r
A (^298) i = [ai 1 ai 2... ain] i = 1, 2 ,... , m.
As a result, Ax, x ∈ R
n , represents the point of R
m 299 obtained by the linear combination of the
300 column vectors of A,
Ax = x 1 c
A 1 +^ x^2 c
A 2 +^...^ +^ xnc
A (^301) n , (2.9)
302 with the coefficients of the linear combination given by the components of the vector x ∈ Rn,
whereas let y ∈ R
m , then A
T y represents the point of R
n 303 obtained by the linear combination of
the column vectors of A
T 304 which are the row vectors of A, that is, we have
A
T y = y 1 c
AT 1 +^ y^2 c
AT 2 +^...^ +^ ymc
AT m
= y 1 (r
A 1 )
T
A 2 )
T +... + ym(r
A m)
T .
Let A ∈ R
m×n and B ∈ R
n×p we are intereste dto show how we can express AB ∈ R
m×p
307 Consider A as a column array of row vectors and B as a row array of column vectors, that is,
Am×n =
a 1
. . .
am
with^ a
T i ∈^ R
n , , i = 1,... , m and
Bn×p =
[
b 1 |... |bp
]
with bi ∈ R
n , i = 1,... , p ,
308
309 then
AB = [ai · bj ]i=1, ..., m j=1, ..., p
a 1 · b 1... a 1 · bp
. . .
am · b 1... am · bp
Let A ∈ K
n×n 311 be a square matrix, the trace of A is denoted by tr(A) and is the sum of the
312 main diagonal entries of A, that is,
tr(A) =
n ∑
i=
313 aii.
Let A, B ∈ K
m×n 314 the trace meets the following property
tr(AB
H ) = tr(A
H B) = tr(B
H A) = tr(BA
H ) =
i=1,...,m j=1,...n
aij bij =
m ∑
i=
315 ai · bi
316 where, for the last equality, we interpret A and B as an ordered set of row vectors, that is
A =
a 1
. . .
am
and^ B^ =
b 1
. . .
bm
318 2.2 Special Types of Matrices
319 We can distinguisgmatrices according to their structure, which makes them easier to perform some
320 operations, and according to some properties.
321 2.2.1 Triangular
322 2.2.2 Hessenberg matrix
323 2.2.3 Hermitian & Skew-Hermitian matrices
324 Proposition 2.4. All entries on the main diagonal of a skew-Hermitian matrix have to be pure
325 imaginary or zero.
326 Proposition 2.5. A is skew-Hermitian if and only if the real part Re(A) is skew-symmetric and
327 the coefficient of the imaginary part Im(A) is symmetric.
328 Proposition 2.6. If A is skew-Hermitian, then the matrix exponential exp(A) is unitary.
329 To be checked the following.
Proposition 2.7. A is skew-Hermitian if and only if Ax · y = −x · Ay for any x, y ∈ K
n
331 2.2.4 Unitary matrix
Definition 2.2. Let A ∈ K
n×n
- The matrix A is called unitary if its column vectors are mutually
333 orthogonal and have unit norm.
Proposition 2.8. Let A ∈ K
n×n
- The matrix A is unitary if and only if there holds
AA
H 335 = I. (2.11)
336 Proof. If we recall the interpretation of the product of two matrices AB, then (2.11) means
AA
H
a 1 · a 1 a 1 · a 2... a 1 · an
a 2 · a 1 a 2 · a 2... a 2 · an
. . .
an · a 1 an · a 2... an · an
= I =
337
338 that is, ai · aj = δij.
From Proposition 2.2, we therefore conclude that it is also A
H 339 A = I and consequently,
A
− 1 = A
H
341 This condition characterizes the unitarty matrices.
342 Remark 2.1. We have given the definition of unitary matrix for square matrices. Assume A ∈
K
m×n 343 with m ≤ n, and the column vectors of A mutually orthogonal. The matrix A is not called
344 a unitary matrix, but we simply say that A is a matrix with its columns vectors that are mutually
345 orthogonal.
375 ‖x‖∞ = max{|xi|, i = 1,... , n}.
The induced or standar norm in K
m×n 376 has the following properties.
377 Proposition 2.10. Let A ∈ Km×n, B ∈ Kn×p, and let us consider Kn^ equipped with the `p norm
and K
m 378 equipped with the `q norm, p, q ≥ 1. Then
‖Ax‖p ≤ ‖A‖p,q‖x‖q ∀x ∈ K
n ,
‖AB‖p,q ≤ ‖A‖p,q‖B‖p,q.
379
Let us consider now the case in which we assume the same norm in K
m and K
n
Definition 2.5. The norm induced by the `p norm in K
m×n 381 , p ≥ 1 , is given by
‖A‖p = sup x 6 = 0
‖Ax‖p
‖x‖p
383 and for p = ∞, it is
‖A‖∞ = sup x 6 = 0
‖Ax‖∞
‖x‖∞
385 For p = 2 we have therefore
‖A‖ 2 = sup x 6 = 0
‖Ax‖ 2
‖x‖ 2
which is called the Euclidean or the spectral norm of K
n×n
Remark 2.2. I think that it is posisble to show the following: Let A ∈ K
m×n
- Consider the
389 mapping
390 p ∈ [1, ∞[ → ‖A‖p ,
391 then
lim p→∞
392 ‖A‖p = ‖A‖∞ ,
393 with ‖A‖∞ defined by (2.15)
394 We have now the following results.
Proposition 2.11. Let A ∈ K
m×n
- Then there holds.
‖A‖ 1 = sup x 6 = 0
‖Ax‖ 1
‖x‖ 1
= max j=1,...,n
i=1,...,m
|aij | ;
‖A‖∞ = sup x 6 = 0
‖Ax‖∞
‖x‖∞
= max i=1,...,m
j=1,...,n
|aij | ;
‖A‖ 2 = sup x 6 = 0
‖Ax‖ 2
‖x‖ 2
λmax(AT^ A) ,
where λmax(A
T 397 A) is the maximum singular value of A, that is, the square root of the maximum
eigenvalue of the symmetric (or Hermitian) matrix A
T A (or A
H 398 A). The norm ‖A‖ 1 is called the
399 maximum absolute column sum norm, whereas the norm ‖A‖∞ is called the maximum absolute row
400 sum norm [2, p. 4].
401 In the vector space Km×n, we can introduce also other norms depending on the applications,
402 finally on the ease that one can prove convergence results. One matrix norm of particular interest
403 is the Frobenius norm.
Proposition 2.12. For A, B ∈ K
m×n 404 , the following mapping
ν : (A, B) ∈ K
m×n × K
m×n 7 → ν(A, B) = tr(B
T 405 A) ∈ C , (2.18)
is a sesquilinear form, and defines an inner product in K
m×n 406 which we call the Frobenius inner
407 product.
408 The mapping ν(A, B) defined by (2.18) is usually denoted as A : B. The norm associated with
409 the inner product (2.18) is then given by
‖A‖F =
A : A =
tr(AH^ A) =
∑^ n
i,j=
aij aij =
∑^ n
i,j=
410 |aij |^2 , (2.19)
411 which is known as the Frobenius norm and is different from the spectral norm ‖A‖ 2 defined in
412 (2.16). The Frobenius norm is not an induced norm, that is a norm of the type (2.13). For
instance, let I ∈ K
n×n 413 n > 1 be the identity matrix, then any induced norm of I is equal to one,
‖I‖p = sup x 6 = 0
‖Ix‖p
‖x‖p
415 whereas the Frobenius norm of I is equal to n, that is,
‖I‖F =
tr (IT^ I) =
tr(I) =
416 n.
417 The Frobenius norm corresponds actually to the standard ` 2 norm of Kn. To see this, let us
identify K
m×n with K
mn , that is given A ∈ K
m×n we associate with A the element a ∈ K
mn 418
419 obtained by orderily stacking into one column the column vectors of A, that is
A =
[
c
A 1 |c
A 2 |^...^ |c
A n
]
∈ K
m×n → a =
c
A 1
c
A 2 . . .
c
A n
with c
A i ∈^ K
m 420 , i = 1,... , n.
Viceversa, given a ∈ K
mn 421 and m, n ∈ N, then we can associate with a the matrix A = [aij ] such
422 that
∈ { 1 ,... , mn} → aij = a with
j = d
` m
e
i = ` − (j − 1)m
424 where dxe is the ceiling function which takes the real number x and returns the least integer greater
425 than or equal to x.
Following this identification, the standard `p norms in K
mn , or any other norm in K
mn 426 defines
also a matrix norm in K
m×n
. Let A ∈ K
m×n denote the matrix defined by a ∈ K
mn 427 by means of
(2.20). For p ≥ 1, the matrix norm correspondent to the standard `p norm of K
mn 428 is given by
454 As a result of (2.9), since R(A) = {Ax ∈ Rm^ : x ∈ Rn}, the range of the matrix A is
455 therefore referred to also as the columns space of A [1, page 97] whereas from (2.10), since since
R(A
T ) = {A
T y ∈ R
n : y ∈ R
m }, the range of the transpose matrix A
T 456 of A is therefore referred
457 to also as the row space of A [1, page 98].
458 The folowing result is fundamental.
Theorem 3.1. Let A ∈ R
m×n
- The number of linearly independent columns of the matrix A is
460 equal to the number of linearly independet rows of the matrix A.
It is easy to constate that let A ∈ R
m×n and B ∈ R
n×p 461 , m, n, p ∈ N so tha the product AB is
462 well defined. Then
AB =
[
Ac
B 1 Ac
B 2...^ Ac
B n
]
464 that is, each column of AB is a linear combination of the column vectors of A. Thus from (3.1)
it follows that Bij = (c
B i )j^ ,^ j^ = 1,... , p^ is the coefficient of the column vector^ c
A (^465) j. It follows
466 therefore that
467 R(AB) ⊆ R(A). (3.2)
468 However, what is more interesting is the reverse result. That is, if C and A are two matrices such
469 that R(C) ⊆ R(A), then there exists a matrix B such that C = AB. This fact is formalized by
470 the following lemma [1, page 98]
Proposition 3.1. Let A ∈ R
m×n and C ∈ R
m×p
- Then
472 R(C) ⊆ R(A)
if and only if C = AB for some matrix B ∈ R
n×p
Proposition 3.2. Let A ∈ C
n×n be a square matrix. If A is nonsingular so is A
H and A
H 474 A is
a symmetric positive definite matrix whereas if A is singular then A
H 475 A is a symmetric positive
476 semidefinite matrix.
Proposition 3.3. Let A, B ∈ C
n×n 477 be square matrices. If AB = I then the matrices A and B
478 are nonsingular, and there also holds BA = I.
479 Let X, Y ⊆ V. We define the set sum X + Y as the subset of V given as follows,
480 X + Y = {z ∈ V : z = x + y, x ∈ X, y ∈ Y }.
481 If X and Y are linear subspaces of V , then also X + Y is a linear subspace of V and so is X ∩ Y.
482 Let X, Y linear subspaces of V. If X ∩ Y = { 0 }, then the sum of the two linear subspaces X
483 and Y is denoted as X ⊕ Y. So by this symbol, we refer to the set X + Y and also that X ∩ Y = { 0 }
484 [6, page 47].
485 Definition 3.1 (Direct Sum). Let V be a vector space, and H 1 and H 2 vector subspaces of V. We
486 say that V is a direct sum of H 1 and H 2 , if
487 V = H 1 ⊕ H 2 ,
488 that is, for any x ∈ V there is one and only one h 1 ∈ H 1 and h 2 ∈ H 2 such that x = h 1 + h 2. In
489 this case, we also say that the subspaces H 1 and H 2 are suplementaries, and each of them is the
490 supplement of the other.
491 We have the following result [6, page 18]
492 Proposition 3.4. Let V be a finite dimensional vector space. Assume V to be the direct sum of
493 H 1 and H 2 , that is, V = H 1 ⊕ H 2. Then there holds
494 dim(V ) = dim(H 1 ) + dim(H 2 ). (3.3)
495 In the Definition 3.1, V is a vector space. Let us consider now the case V is a pre-Hilbert space,
496 that is, V is endowed with the topological structure induced by the inner product of V and denote
497 by x · y or (·, ·)V the inner product in V.
498 Let X a subset of the pre-Hilbert space V. We define the orthogonal set of X given by
X
⊥ 499 = {y ∈ V : y · x = 0 ∀ x ∈ X}. (3.4)
Exercise 3.1. (i) Let V be a pre-Hilbert space. Show that X
⊥ 500 is a linear subspace of V and is
501 a closed set.
(ii) Let V be a pre-Hilbert space and X ⊆ V. What is the relationship between (X
⊥ )
⊥ 502 and X?
503 We have the following definitions of Euclidean and Hermitian space [6, page 79].
504 Definition 3.2. Let V be a pre-Hilbert finite dimensional space over the field K. The space V is
505 called an Euclidean space if K = R whereas V is called an Hermitian space if K = C.
506 Let V be an Euclidean space. If we interpret the elements of V as column vectors or as matrices
507 with only one column, and we consider the row by column product between matrices, we can express
508 the inner product between elements of V as follows
x · y = (x, y)V = x
T y = y
T 509 x.
510 Proposition 3.5 (Orthogonal direct sum). Let V be a finite dimensional pre-Hilbert space (i.e. V
511 can be either an Euclidean or a Hermitian space). Assume H to be a linear subspace of V. Then
512 there holds
V = H ⊕ H
⊥
- (3.5)
514 Proposition 3.6 (Adjoint operator). Let V and W be pre-Hilbert spaces with inner product (·, ·)V
515 and (·, ·)W , respectively. Assume A : V → W to be a linear operator. Then there exists one and
only one operator A
T 516 : W → V which meets the following equation
(y, Ax)W = (A
T 517 y, x)V ∀x ∈ V, ∀y ∈ W. (3.6)
The operator A
T 518 is linear and is called the adjoint of A.
Proposition 3.7. If we identify A with the matrix A, then the adjoint A
T 519 of A is defined by the
transpose matrix A
T
Given A : R
n → R
m (i.e. A ∈ R
m×n and the transpose matrix A
T of A, A
T : R
m → R
n 521 (i.e.
A ∈ R
n×m , while Ax, x ∈ R
n , represents a linear combination of the columns of A, A
T y, y ∈ R
m 522 ,
523 represents a linear combination of the rows of A.
Proposition 3.8 (Alternative Theorem). Let R
n and R
m be Euclidean spaces and A : R
n → R
m 524
a linear operator. Denote by A
T : R
m → R
n 525 the adjoint operator of A. There holds
K(A
T ) ⊥ R(A)
K(A) ⊥ R(A
T ).
556 For full rank matrices, we can then specify A as a full row rank if rank(A) = m or as a full
557 column rank if rank(A) = n [7, page ].
Proposition 3.12. Let A ∈ R
m×n
- Then we have the following characterization.
rank(A) = m ⇔ AA
T is no singular ,
rank(A) = n ⇔ A
T A is no singular.
Proposition 3.13. Let A ∈ R
m×n , b ∈ R
m , x ∈ R
n
- Then there holds
561 b ∈ R(A) ⇔ rank(A) = rank([A|b]) , (3.14)
562 where
[A|b] =
[
c
A 1 c
A 2...^ c
A n b^
]
563
564 Proposition 3.14. Let V and W be linear vector spaces, A : V → W a linear operator of V into
W and A
T 565 : W → V the adjoint of A. We have that the linear operator A is injective if and only
if K(A) = { 0 } and it is surjective if and only if K(A
T 566 ) = { 0 }.
In the case V and W are finite dimensional spaces, and let A ∈ R
m×n 567 be the corresponding
568 matrix. Then we have the following characterizations [14, Theorem 3.1].
K(A) = { 0 } ⇔ rank(A) = n
K(A
T ) = { 0 } ⇔ rank(A) = m.
From (3.15) we have that in the case of square matrices A ∈ R
n×n 570 , i.e. m = n, that is
571 endomorphisms (linear mapping of a vector space V into the same space V ) of finite dimensional
572 spaces, A is invertible if and only if A is surjective [6, page 51].
573 3.3 Projectors
574 4 Linear Systems
Let A ∈ R
m×n , x ∈ R
n and b ∈ R
m 575 , in this section we discuss the system of m equations in n
576 unknowns.
577 Ax = b. (4.1)
If b ∈ R(A) then there exists at least one x ∈ R
n 578 which meets (4.1). We refer to such x as classical
579 solutions and we say that the sytstem (4.1) is compatible. When b 6 ∈ R(A), then clearly there is
no x ∈ R
n that meets (4.1). In such a case, we inquire whether we can define some x
∗ ∈ R
n 580 as a
581 generalised solution of (4.1).
582 We first recall that the vector subspaces of a topological vector space are not necessarily closed
583 sets. However, if the vector subspace has finite dimension, then it is a closed set. We have the
584 following result which is stated for general Hausdorff topological vector spaces, thus it holds in
585 particular also for vector normed spaces [5, page 87].
586 Proposition 4.1. Let V be a topological vecor space and X ⊆ V a finite dimensional subspace of
587 V. Then X is a closed subset of V.
588 As a consequence of Proposition 4.1, we have that for A ∈ Rm×n, since R(A) is a subspace
of R
m , then R(A) is finite dimensional subspace and is therefore a closed subspace of R
m 589 , in
particular, R(A) is a closed convex subset of R
m
591 Let us consider now the problem of best approximation in a normed space. We assume the case
of a finite dimensional space, and without sake of generality, we refer to the space R
m 592 , m ∈ N.
Problem 4.1. Given b ∈ R
m , with R
m 593 equipped with the norm ‖ · ‖, we consider the problem
Find b ∈ R(A) :
minimize ‖y − b‖ ∀y ∈ R(A).
594
595 Problem 4.1 is called best approximation problem in a normed space and it is the problem of
596 the minimum distance of a point to a closed convex set in a normed space. The vectors b ∈ R(A)
which satisfy Problem 4.1 are called best approximation from R(A) to b ∈ R
m
Remark 4.1. In the formulation of Problem 4.1, ‖ · ‖ denotes any type of norm on R
m
- Even
599 though all these norms are equivalent, solutions of Problem 4.1 depend on the norm we choose [13,
600 Fig. 1.4 & Fig 1.5] and we see that, while we are guaranteed about the existence of solutions to
Problem 4.1, since R(A) is a finite-dimensional subspace of the normed space R
m 601 , the uniqueness
602 depends on whether the norm is strictly convex or not.
603 We have the following general theorem about the existence of solutions to Problem 4.1 [13, Thm.
604 1.2].
605 Theorem 4.1. If A is a finite–dimensional linear space in a normed linear space V , then for every
606 b ∈ V , there exists an element of A that is a best approximation from A to b.
607 As a result, Problem 4.1 has at least one solution. We have then the following uniqueness general
608 result [13, Thm. 2.4].
609 Theorem 4.2. Let A be a convex set in a normed linear space V , whose norm is strictly convex.
610 Then, for all b ∈ V , there is at most one best approximation from A to b.
611 In the assumptions of Theorem 4.2 (where A is a convex set, so that we cannot apply The-
612 orem 4.1, thus we are not guaranteed about the existence of solutions) if the problem of best
613 approximation has solution, the solution is unique.
As a consequence of Theorem 4.1 and Theorem 4.2 we can easily conclude that in the case R
m 614
615 is a Hilbert space, since R(A) is a linear subspace and is convex, and the norm deriving by the
616 inner product is strictly convex [13, Sec. 2.4], Problem 4.1 has solution and is unique. This occurs,
for instance, if we consider R
m 617 equipped with the norm ` 2 which derives from the inner product
x · y =
∑m (^618) i=1 xiyi.
619 Proposition 4.2. If Rm^ is a Hilbert space, for any b ∈ Rm^ Problem 4.1 has solution and is unique.
620 Remark 4.2. Check whether it is possible to give the proof using the direct method of the calculus
621 of variations. Note that a strictly convex function on a closed convex set might not have a global
622 minimizer, just consider the function f (x) = exp(x) on R which is closed and convex. A relevant
623 assumption is the coercivity, and V be a Banach space, given that the coercivity allows one to
624 conclude that the infimizing sequence is bounded and the reflexivity of the space allows the extraction
625 of a convergent infimizing subsequence. Check also what are the regularity properties of a strictly
626 convex function in a normed space. I am interested to know if I can say the function is lower
semicontinuous. In the case we can give such proof for R
m 627 a Hilbert space, check what of the
628 arguments fail in the case Rm^ is a Banach space.
