Linear Algebra: Dimension, Isomorphism, Self-Adjoint Maps, and Orthogonal Projections, Lecture notes of Linear Algebra

Download Linear Algebra: Dimension, Isomorphism, Self-Adjoint Maps, and Orthogonal Projections and more Lecture notes Linear Algebra in PDF only on Docsity!

Linear Algebra: Graduate Level Problems and Solutions

Igor Yanovsky

Disclaimer: This handbook is intended to assist graduate students with qualifying examination preparation. Please be aware, however, that the handbook might contain, and almost certainly contains, typos as well as incorrect or inaccurate solutions. I can not be made responsible for any inaccuracies contained in this handbook.

1 Basic Theory

1.1 Linear Maps

Lemma. If A ∈ M atmxn(F) and B ∈ M atnxm(F), then

tr(AB) = tr(BA).

Proof. Note that the (i, i) entry in AB is

∑n ∑m j=1^ αij^ βji, while (j, j) entry in^ BA^ is i=1 βjiαij^.

Thus tr(AB) =

∑^ m

i=

∑^ n

j=

αij βji,

tr(BA) =

∑^ n

j=

∑^ m

i=

βjiαij.

1.2 Linear Maps as Matrices

Example. Let Pn = {α 0 + α 1 t + · · · + αntn^ : α 0 , α 1 ,... , αn ∈ F} be the space of polynomials of degree ≤ n and D : V → V the differential map

D(α 0 + α 1 t + · · · + αntn) = α 1 + · · · + nαntn−^1.

If we use the basis 1 , t,... , tn^ for V then we see that D(tk) = ktk−^1 and thus the (n + 1)x(n + 1) matrix representation is computed via

[D(1) D(t) D(t^2 ) · · · D(tn)] = [0 1 2t · · · ntn−^1 ] = [1 t t^2 · · · tn]

n 0 0 0 · · · 0

1.3 Dimension and Isomorphism

A linear map L : V → W is isomorphism if we can find K : W → V such that LK = IW and KL = IV.

V −−−L−→ W

IV

x  x IW

V

K ←−−−− W

Theorem. V and W are isomorphic ⇔ there is a bijective linear map L : V → W.

Proof. ⇒ If V and W are isomorphic we can find linear maps L : V → W and K : W → V so that LK = IW and KL = IV. Then for any y = IW (y) = L(K(y)) so we can let x = K(y), which means L is onto. If L(x 1 ) = L(x 2 ) then x 1 = IV (x 1 ) = KL(x 1 ) = KL(x 2 ) = IV (x 2 ) = x 2 , which means L is 1 − 1. ⇐ Assume L : V → W is linear and a bijection. Then we have an inverse map L−^1 which satisfies L ◦ L−^1 = IW and L−^1 ◦ L = IV. In order for this inverse map to be allowable as K we need to check that it is linear. Select α 1 , α 2 ∈ F and y 1 , y 2 ∈ W. Let xi = L−^1 (yi) so that L(xi) = yi. Then we have

L−^1 (α 1 y 1 + α 2 y 2 ) = L−^1 (α 1 L(x 1 ) + α 2 L(x 2 )) = L−^1 (L(α 1 x 1 + α 2 x 2 )) = IV (α 1 x 1 + α 2 x 2 ) = α 1 x 1 + α 2 x 2 = α 1 L−^1 (y 1 ) + α 2 L−^1 (y 2 ).

Theorem. If Fm^ and Fn^ are isomorphic over F, then n = m.

Proof. Suppose we have L : Fm^ → Fn^ and K : Fn^ → Fm^ such that LK = IFn^ and KL = IFm. L ∈ M atnxm(F) and K ∈ M atmxn(F). Thus

n = tr(IFn^ ) = tr(LK) = tr(KL) = tr(IFm^ ) = m.

Define the dimension of a vector space V over F as dimF V = n if V is isomorphic to Fn.

Remark. dimC C = 1, dimR C = 2, dimQ R = ∞.

The set of all linear maps {L : V → W } over F is homomorphism, and is denoted by homF(V, W ).

Corollary. If V and W are finite dimensional vector spaces over F, then homF(V, W ) is also finite dimensional and

dimF homF(V, W ) = (dimF W ) · (dimF V )

Proof. By choosing bases for V and W there is a natural mapping

homF(V, W ) → M at(dimF W )×(dimF V )(F) ' F(dimF^ W^ )·(dimF^ V^ )

This map is both 1-1 and onto as the matrix represetation uniquely determines the linear map and every matrix yields a linear map.

1.6 Linear Maps and Subspaces

L : V → W is a linear map over F. The kernel or nullspace of L is

ker(L) = N (L) = {x ∈ V : L(x) = 0}

The image or range of L is

im(L) = R(L) = L(V ) = {L(x) ∈ W : x ∈ V }

Lemma. ker(L) is a subspace of V and im (L) is a subspace of W.

Proof. Assume that α 1 , α 2 ∈ F and that x 1 , x 2 ∈ ker(L), then L(α 1 x 1 + α 2 x 2 ) = α 1 L(x 1 ) + α 2 L(x 2 ) = 0 ⇒ α 1 x 1 + α 2 x 2 ∈ ker(L). Assume α 1 , α 2 ∈ F and x 1 , x 2 ∈ V , then α 1 L(x 1 ) + α 2 L(x 2 ) = L(α 1 x 1 + α 2 x 2 ) ∈ im (L).

Lemma. L is 1-1 ⇔ ker(L) = { 0 }.

Proof. ⇒ We know that L(0·0) = 0·L(0) = 0, so if L is 1−1 we have L(x) = 0 = L(0) implies that x = 0. Hence ker(L) = { 0 }. ⇐ Assume that ker(L) = { 0 }. If L(x 1 ) = L(x 2 ), then linearity of L tells that L(x 1 − x 2 ) = 0. Then ker(L) = { 0 } implies x 1 − x 2 = 0, which shows that x 1 = x 2 as desired.

Lemma. L : V → W , and dim V = dim W. L is 1-1 ⇔ L is onto ⇔ dim im (L) = dim V.

Proof. From the dimension formula, we have

dim V = dim ker(L) + dim im(L).

L is 1-1 ⇔ ker(L) = { 0 } ⇔ dim ker(L) = 0 ⇔ dim im (L) = dim V ⇔ dim im (L) = dim W ⇔ im (L) = W , that is, L is onto.

1.7 Dimension Formula

Theorem. Let V be finite dimensional and L : V → W a linear map, all over F, then im(L) is finite dimensional and

dimF V = dimF ker(L) + dimF im(L)

Proof. We know that dim ker(L) ≤ dim V and that it has a complement M of dimension k = dim V − dim ker(L). Since M

ker(L) = { 0 } the linear map L must be 1-1 when restricted to M. Thus L|M : M → im(L) is an isomorphism, i.e. dim im(L) = dim M = k.

1.8 Matrix Calculations

Change of Basis Matrix. Given the two basis of R^2 , β 1 = {x 1 = (1, 1), x 2 = (1, 0)} and β 2 = {y 1 = (4, 3), y 2 = (3, 2)}, we find the change-of-basis matrix P from β 1 to β 2. Write y 1 as a linear combination of x 1 and x 2 , y 1 = ax 1 + bx 2. (4, 3) = a(1, 1) + b(1, 0) ⇒ a = 3, b = 1 ⇒ y 1 = 3x 1 + x 2. Write y 2 as a linear combination of x 1 and x 2 , y 2 = ax 1 + bx 2. (3, 2) = a(1, 1) + b(1, 0) ⇒ a = 2, b = 1 ⇒ y 2 = 2x 1 + x 2.

Write the coordinates of y 1 and y 2 as columns of P. P =

[

]

1.9 Diagonalizability

Definition. Let T be a linear operator on the finite-dimensional space V. T is diagonalizable if there is a basis for V consisting of eigenvectors of T.

Theorem. Let v 1 ,... , vn be nonzero eigenvectors of distinct eigenvalues λ 1 ,... , λn. Then {v 1 ,... , vn} is linearly independent.

Alternative Statement. If L has n distinct eigenvalues λ 1 ,... , λn, then L is diagonalizable. (Proof is in the exercises).

Definition. Let L be a linear operator on a finite-dimensional vector space V , and let λ be an eigenvalue of L. Define Eλ = {x ∈ V : L(x) = λx} = ker(L − λIV ). The set Eλ is called the eigenspace of L corresponding to the eigenvalue λ. The algebraic multiplicity is defined to be the multiplicity of λ as a root of the characteristic polynomial of L, while the geometric multiplicity of λ is defined to be the dimension of its eigenspace, dim Eλ = dim(ker(L − λIV )). Also,

dim(ker(L − λIV )) ≤ m.

Eigenspaces. A vector v with (A − λI)v = 0 is an eigenvector for λ.

Generalized Eigenspaces. Let λ be an eigenvalue of A with algebraic multiplicity m. A vector v with (A − λI)mv = 0 is a generalised eigenvector for λ.

2 Inner Product Spaces

2.1 Inner Products

The three important properties for complex inner products are:

1. (x|x) = ||x||^2 > 0 unless x = 0.
2. (x|y) = (y|x).
3. For each y ∈ V the map x → (x|y) is linear. The inner product on Cn^ is defined by

(x|y) = xt^ y¯

Consequences: (α 1 x 1 + α 2 x 2 |y) = α 1 (x 1 |y) + α 2 (x 2 |y), (x|β 1 y 1 + β 2 y 2 ) = β¯ 1 (x|y 1 ) + β¯ 2 (x|y 2 ), (αx|αx) = α α¯(x|x) = |α|^2 (x|x).

2.2 Orthonormal Bases

Lemma. Let e 1 ,... , en be orthonormal. Then e 1 ,... , en are linearly independent and any element x ∈ span{e 1 ,... , en} has the expansion

x = (x|e 1 )e 1 + · · · + (x|en)en.

Proof. Note that if x = α 1 e 1 + · · · + αnen, then (x|ei) = (α 1 e 1 +· · ·+αnen|ei) = α 1 (e 1 |ei)+· · ·+αn(en|ei) = α 1 δ 1 i +· · ·+αnδni = αi.

Cauchy-Schwarz Inequality. V complex inner product space.

|(x|y)| ≤ ||x||||y||, x, y ∈ V.

Proof. First show projy(x) ⊥ x − projy(x):

(projy(x)|x − projy(x)) =

( (^) (x|y) ||y||^2

y

∣x^ −^

(x|y) ||y||^2

y

( (^) (x|y) ||y||^2

y

∣x

( (^) (x|y) ||y||^2

y

(x|y) ||y||^2

y

(x|y) ||y||^2

(y|x) − (x|y) ||y||^2

(x|y) ||y||^2

(y|y) = (x|y) ||y||^2

(y|x) − (x|y) ||y||^2

(x|y) = 0.

||x|| ≥ ||projy(x)|| =

(x|y) (y|y)

y

(x|y) (y|y)

∣||y||^ =^

|(x|y)| ||y||

Triangle Inequality. V complex inner product space.

||x + y|| ≤ ||x|| + ||y||.

Proof. ||x + y||^2 = (x + y|x + y) = ||x||^2 + 2Re(x|y) + ||y||^2 ≤ ||x||^2 + 2|(x|y)| + ||y||^2 ≤ ||x||^2 + 2||x||||y|| + ||y||^2 = (||x|| + ||y||)^2.

Let M ⊂ V be a finite dimensional subspace of an innter product space, and e 1 ,... , em an orthonormal basis for M. Using that basis, define E : V → V by

E(x) = (x|e 1 )e 1 + · · · + (x|em)em

Note that E(x) ∈ M and that if x ∈ M , then E(x) = x. Thus E^2 (x) = E(x) implying that E is a projection whose image is M. If x ∈ ker(E), then

0 = E(x) = (x|e 1 )e 1 + · · · + (x|em)em ⇒ (x|e 1 ) = · · · = (x|em) = 0.

This is equivalent to the condition (x|z) = 0 for all z ∈ M. The set of all such vectors is the orthogonal complement to M in V is denoted

M ⊥^ = {x ∈ V : (x|z) = 0 for all z ∈ M }

Theorem. Let V be an inner product space. Assume V = M

M ⊥, then im(projM ) = M and ker(projM ) = M ⊥. If M ⊂ V is finite dimensional then V = M

M ⊥^ and

projM (x) = (x|e 1 )e 1 + · · · + (x|em)em

for any orthonormal basis e 1 ,... , em for M.

Proof. For E defined as above, ker(E) = M ⊥. x = E(x) + (I − E)(x) and (I − E)(x) ∈ ker(E) = M ⊥. Choose z ∈ M.

||x − projM (x)||^2 ≤ ||x − projM (x)||^2 + ||projM (x) − z||^2 = ||x − z||^2 ,

where equality holds when ||projM (x) − z||^2 = 0, i.e., projM (x) is the only closest point to x among the points in M.

Theorem. Let E : V → V be a projection on to M ⊂ V with the property that V = ker(E)

ker(E)⊥. Then the following conditions are equivalent.

1. E = projM.
2. im(E)⊥^ = ker(E).
3. ||E(x)|| ≤ ||x|| for all x ∈ V.

Proof. We have already seen that (1) ⇔ (2). Also (1),(2) ⇒ (3) as x = E(x)+(I −E)(x) is an orthogonal decomposition. So ||x||^2 = ||E(x)||^2 + ||(I − E)(x)||^2 ≥ ||E(x)||^2. Thus, we only need to show that (3) implies that E is orthogonal. Choose x ∈ ker(E)⊥ and observe that E(x) = x − (1 − E)(x) is an orthogonal decomposition. Thus

||x||^2 ≥ ||E(x)||^2 = ||x − (1 − E)(x)||^2 = ||x||^2 + ||(1 − E)(x)||^2 ≥ ||x||^2

This means that (1 − E)(x) = 0 and hence x = E(x) ∈ im(E) ⇒ ker(E)⊥^ ⊂ im(E). Conversely, if z ∈ im(E) = M , then we can write z = x + y ∈ ker(E)

ker(E)⊥. This implies that z = E(z) = E(y) = y, where the last equality follows from ker(E)⊥^ ⊂ im(E). This means that x = 0 and hence z = y ∈ ker(E)⊥.

3 Linear Maps on Inner Product Spaces

3.1 Adjoint Maps

The adjoint of A is the matrix A∗^ such that aij = ¯aji. A : Fm^ → Fn, A∗^ : Fn^ → Fm. (Ax|y) = (Ax)t^ y¯ = xtAt^ ¯y = xt( A¯ty) = (x|A∗y).

dim(M ) + dim(M ⊥) = dim(V ) = dim(V ∗)

Theorem. Suppose S = {v 1 , v 2 ,... , vk} is an orthonormal set in an n-dimensional inner product space V. Then a) S can be extended to an orthonormal basis {v 1 , v 2 ,... , vk, vk+1,... , vn} for V. b) If M = span(S), then S 1 = {vk+1,... , vn} is an orthonormal basis for M ⊥. c) If M is any subspace of V , then dim(V ) = dim(M ) + dim(M ⊥).

Proof. a) Extend S to a basis S′^ = {v 1 , v 2 ,... , vk, wk+1,... , wn} for V. Apply Gram- Schmidt process to S′. The first k vectors resulting from this process are the vectors in S. S′^ spans V. Normalizing the last n − k vectors of this set produces an orthonormal set that spans V. b) Because S 1 is a subset of a basis, it is linearly independent. Since S 1 is clearly a subset of M ⊥, we need only show that it spans M ⊥. For any x ∈ V , we have

x =

∑^ n

i=

(x|vi)vi.

If x ∈ M ⊥, then (x|vi) = 0 for 1 ≤ i ≤ k. Therefore,

x =

∑^ n

i=k+

(x|vi)vi ∈ span(S 1 ).

c) Let M be a subspace of V. It is finite-dimensional inner product space because V is, and so it has an orthonormal basis {v 1 , v 2 ,... , vk}. By (a) and (b), we have dim(V ) = n = k + (n − k) = dim(M ) + dim(M ⊥).

3.2 Self-Adjoint Maps

A linear operator L : V → V is self-adjoint (Hermitian) if L∗^ = L, and skew-adjoint if L∗^ = −L.

Theorem. If L is self-adjoint operator on a finite-dimensional inner product space V , then every eigenvalue of L is real.

Proof. Method I: Suppose L is a self-adjoint operator in V. Let λ be an eigenvalue of L, and let x be a nonzero vector in V such that Lx = λx. Then

λ(x|x) = (λx|x) = (Lx|x) = (x|L∗x) = (x|Lx) = ¯λ(x|x).

Thus λ = ¯λ, which means that λ is real.

Proof. Method II: Suppose that L(x) = λx for x 6 = 0. Because a self-adjoint operator is normal, then if x is an eigenvector of L then x is also an eigenvector of L∗. Thus, λx = L(x) = L∗(x) = λx¯.

Proposition. If L is self- or skew-adjoint, then for each invariant subspace M ⊂ V the orthogonal complement is also invariant, i.e., if L(M ) ⊂ M , then also L(M ⊥) ⊂ M ⊥.

Proof. Assume that L(M ) ⊂ M. If x ∈ M and z ∈ M ⊥, then since L(x) ∈ M we have

0 = (z|L(x)) = (L∗(z)|x) = ±(L(z)|x).

Since this holds ∀x ∈ M , it follows that L(z) ∈ M ⊥.

3.3 Polarization and Isometries

Real inner product on V :

(x + y|x + y) = (x|x) + 2(x|y) + (y|y) ⇒ (x|y) =

((x + y|x + y) − (x|x) − (y|y)) =

(||x + y||^2 − ||x||^2 − ||y||^2 ).

Complex inner products (are only conjugate symmetric) on V :

(x + y|x + y) = (x|x) + 2Re(x|y) + (y|y) ⇒ Re(x|y) =

(||x + y||^2 − ||x||^2 − ||y||^2 ). Re(x|iy) = Re(−i(x|y)) = Im(x|y). In particular, we have Im(x|y) =

(||x + iy||^2 − ||x||^2 − ||iy||^2 ).

We can use these ideas to check when linear operators L : V → V are 0. First note that L = 0 ⇔ (L(x)|y) = 0, ∀x, y ∈ V. To check the ⇐ part, let y = L(x) to see that ||L(x)||^2 = 0, ∀x ∈ V.

Theorem. Let L : V → V be self-adjoint. L = 0 ⇔ (L(x)|x) = 0, ∀x ∈ V.

Proof. ⇒ If L = 0 ⇒ (L(x)|x) = 0, ∀x ∈ V. ⇐ Assume that (L(x)|x) = 0, ∀x ∈ V.

0 = (L(x + y)|x + y) = (L(x)|x) + (L(x)|y) + (L(y)|x) + (L(y)|y) = (L(x)|y) + (y|L∗(x)) = (L(x)|y) + (y|(L(x))) = 2Re(L(x)|y).

Now insert y = L(x) to see that 0 = Re(L(x)|L(x)) = ||L(x)||^2.

Theorem. Let L : V → V be a linear map on a complex inner-product space. Then L = 0 ⇔ (L(x)|x) = 0, ∀x ∈ V.

Proof. ⇒ If L = 0 ⇒ (L(x)|x) = 0, ∀x ∈ V. ⇐ Assume that (L(x)|x) = 0, ∀x ∈ V.

0 = (L(x + y)|x + y) = (L(x)|x) + (L(x)|y) + (L(y)|x) + (L(y)|y) = (L(x)|y) + (L(y)|x)

0 = (L(x + iy)|x + iy) = (L(x)|x) + (L(x)|iy) + (L(iy)|x) + (L(iy)|iy) = −i(L(x)|y) + i(L(y)|x)

⇒

[

−i i

] [

(L(x)|y) (L(y)|x)

]

[

]

Since the columns of the matrix on the left are linearly independent the only solution is the trivial one. In particular (L(x)|y) = 0.

3.4 Unitary and Orthogonal Operators

A linear transformation A is orthogonal is AAT^ = I, and unitary if AA∗^ = I, i.e. A∗^ = A−^1.

Theorem. L : V → W is a linear map between inner product spaces. TFAE:

1. L∗L = IV , (L is unitary)
2. (L(x)|L(y)) = (x|y) ∀x, y ∈ V , (L preserves inner products)
3. ||L(x)|| = ||x|| ∀x ∈ V. (L preserves lengths)

Proof. (1) ⇒ (2). L∗L = IV ⇒ (L(x)|L(y)) = (x|L∗L(y)) = (x|Iy) = (x|y), ∀x ∈ V. Also note: L takes orthonormal sets of vectors to orthonormal sets of vectors. (2) ⇒ (3). (L(x)|L(y)) = (x|y), ∀x, y ∈ V ⇒ ||L(x)|| =

(L(x), L(x)) =

(x, x) = ||x||. (3) ⇒ (1). ||L(x)|| = ||x||, ∀x ∈ V ⇒ (L∗L(x)|x) = (L(x)|L(x)) = (x|x) = (Ix|x) ⇒ ((L∗L − I)(x)|x) = 0, ∀x ∈ V. Since L∗L − I is self-adjoint (check), L∗L = I.

Two inner product spaces V and W over F are isometric, if we can find an isom- etry L : V → W , i.e. an isomorphism such that (L(x)|L(y)) = (x|y).

Theorem. Supposet L is unitary, then L is an isometry on V.

Proof. An isometry on V is a mapping which preserves distances. Since L is unitary, ||L(x) − L(y)|| = ||L(x − y)|| = ||x − y||. Thus L is an isometry.

Spectral Theorem for Normal Operators. Let L : V → V be a normal operator on a complex inner product space. Then there exists an orthonormal basis e 1 ,... , en such that L(e 1 ) = λ 1 e 1 ,... , L(en) = λnen.

Proof. Prove this by induction on dim V. Since L is complex linear, we can use the Fundamental Theorem of Algebra to find λ ∈ C and x ∈ V { 0 }, so that L(x) = λx ⇒ L∗(x) = λx¯. ker(L−λIV ) = ker(L∗−λI¯ V ). Let x⊥^ = {z ∈ V : (z|x) = 0}, an orthogonal complement to x. To get induction, we need to show that x⊥^ is invariant under L, i.e. L(x⊥) ⊂ x⊥. Let z ∈ x⊥^ and show Lz ∈ x⊥.

(L(z)|x) = (z|L∗(x)) = (z|¯λx) = λ(z|x) = 0.

Check that L|x⊥ is normal. Similarly, x⊥^ is invariant under L∗, i.e. L∗^ : x⊥^ → x⊥, since

(L∗(z)|x) = (z|L(x)) = (z|λx) = ¯λ(z|x) = 0.

⇒ L∗|x⊥ = (L|x⊥ )∗^ since (L(z)|y) = (z|L∗y), z, y ∈ x⊥.

3.7 Unitary Equivalence

Two nxn matrices A and B are unitary equivalent if A = U BU ∗, where U is an nxn matrix such that U ∗U = U U ∗^ = IFn.

Corollary. (nxn matrices)

1. A normal matrix is unitary equivalent to a diagonal matrix.
2. A self-adjoint matrix is unitary or orthogonally equivalent to a real diago- nal.
3. A skew-adjoint matrix is unitary equivalent to a purely imaginary diagonal.
4. A unitary matrix is unitary equivalent to a diagonal matrix whose diagonal elements are unit scalars.

3.8 Triangulability

Schur’s Theorem. Let L : V → V be a linear operator on a finite dimensional complex inner product space. Then we can find an orthonormal basis e 1 ,... , en such that the matrix representation [L] is upper triangular in this basis, i.e.,

L = [e 1 · · · en] [L] [e 1 · · · en]∗^ = [e 1 · · · en]

α 11 α 12 · · · α 1 n 0 α 22 · · · α 2 n .. .

0 0 · · · αnn

[e 1 · · · en]∗.

Generalized Schur’s Theorem. Let L : V → V be a linear operator on an n dimen- sional vector space over F. Assume that χL(t) = (t − λ 1 ) · · · (t − λn) for λ 1 ,... , λn ∈ F, then V admits a basis x 1 ,... , xn such that the matrix representation with respect to x 1 ,... , xn is upper triangular.

Proof. The proof is by induction on the dimension n of V. The result is immediate if n = 1. So suppose that the result is true for linear operators on (n − 1)-dimensional inner product spaces whose characteristic polynomials split. We can assume that L∗ has a unit eigenvector z. Suppose that L∗(z) = λz and that W = span({z}). We show that W ⊥^ is L-invariant. If y ∈ W ⊥^ and x = cz ∈ W , then

(L(y)|x) = (L(y)|cz) = (y|L∗(cz)) = (y|cL∗(z)) = (y|cλz) = cλ(y|z) = cλ(0) = 0.

So L(y) ∈ W ⊥. It is easy to show that the characteristic polynomial of LW ⊥ divides the characteristic polynomial of L and hence splits. Thus, dim(W ⊥) = n − 1, so we may apply the induction hypothesis to LW ⊥ and obtain an orthonormal basis γ of W ⊥ such that [LW ⊥ ]γ is upper triangular. Clearly, β = γ

{z} is an orthonormal basis for V such that [L]β is upper triangular.

4 Determinants

4.1 Characteristic Polynomial

The characteristic polynomial of A is defined as χA(t) = tn^ +αn− 1 tn−^1 +... α 1 t+α 0. The characteristic polynomial of L : V → V can be defined by

χL(t) = det(L − tIV ).

Facts: det L−^1 = (^) det^1 L. det A = det AT^. If A is orthogonal, det A = ±1, since det(I) = det(AAT^ ) = det(A) det(AT^ ) = (det A)^2. If U is unitary, | det A| = 1, and all |λi| = 1.

Example. Let β = {x 1 , x 2 } = {(2, 1), (3, 1)} be a basis for R^2. (xi(ξ 1 , ξ 2 )). We find the dual basis of β given by β′^ = {f 1 , f 2 }. To determine formulas for f 1 and f 2 , we seek functionals f 1 (ξ 1 , ξ 2 ) = a 1 ξ 1 + a 2 ξ 2 and f 2 (ξ 1 , ξ 2 ) = b 1 ξ 1 + b 2 ξ 2 such that

f 1 (x 1 ) = 1, f 1 (x 2 ) = 0, f 2 (x 1 ) = 0, f 2 (x 2 ) = 1. Thus { 1 = f 1 (x 1 ) = f (2, 1) = 2a 1 + a 2 0 = f 1 (x 2 ) = f (3, 1) = 3a 1 + a 2

0 = f 2 (x 1 ) = f (2, 1) = 2b 1 + b 2 1 = f 2 (x 2 ) = f (3, 1) = 3b 1 + b 2

The solutions yield a 1 = − 1 , a 2 = 3 and b 1 = 1, b 2 = − 2. Hence f 1 (ξ 1 , ξ 2 ) = −ξ 1 + 3ξ 2 and f 2 (ξ 1 , ξ 2 ) = ξ 1 − 2 ξ 2 , or f 1 = (− 1 , 3), f 2 = (1, −2), form the dual basis.

Example. Let β = {x 1 , x 2 , x 3 } = {(1, 0 , 1), (0, 2 , 0), (− 1 , 0 , 2)} be a basis for R^3. (xi(ξ 1 , ξ 2 , ξ 3 )). We find the dual basis of β given by β′^ = {f 1 , f 2 , f 3 }. To deter- mine formulas for f 1 ,f 2 ,f 3 we seek functionals f 1 (ξ 1 , ξ 2 , ξ 3 ) = a 1 ξ 1 + a 2 ξ 2 + a 3 ξ 3 , f 2 (ξ 1 , ξ 2 , ξ 3 ) = b 1 ξ 1 + b 2 ξ 2 + b 3 ξ 3 , and f 3 (ξ 1 , ξ 2 , ξ 3 ) = c 1 ξ 1 + c 2 ξ 2 + c 3 ξ 3 such that fi(xj ) = δij.  



1 = f 1 (x 1 ) = f (1, 0 , 1) = a 1 + a 3 0 = f 1 (x 2 ) = f (0, 2 , 0) = 2a 2 0 = f 1 (x 3 ) = f (− 1 , 0 , 2) = −a 1 + 2a 3

0 = f 2 (x 1 ) = f (1, 0 , 1) = b 1 + b 3 1 = f 2 (x 2 ) = f (0, 2 , 0) = 2b 2 0 = f 2 (x 3 ) = f (− 1 , 0 , 2) = −b 1 + 2b 3  



0 = f 3 (x 1 ) = f (1, 0 , 1) = c 1 + c 3 Thus a 1 = 23 , a 2 = 0, a 3 = 13 , 0 = f 3 (x 2 ) = f (0, 2 , 0) = 2c 2 b 1 = 0, b 2 = 12 , b 3 = 0, 1 = f 3 (x 3 ) = f (− 1 , 0 , 2) = −c 1 + 2c 3 c 1 = − 13 , c 2 = 0, c 3 = 13.

Hence f 1 (ξ 1 , ξ 2 , ξ 3 ) = 23 ξ 1 + 13 ξ 3 , f 2 (ξ 1 , ξ 2 , ξ 3 ) = 12 ξ 2 , f 3 (ξ 1 , ξ 2 , ξ 3 ) = − 13 ξ 1 + 13 ξ 3 , or f 1 = ( 23 , 0 , 13 ), f 2 = (0, 12 , 0), f 3 = (− 13 , 0 , 13 ), form the dual basis.

Example. Let W is the subspace of R^4 spanned by x 1 = (1, 2 , − 3 , 4) and x 2 = (0, 1 , 4 , −1). We find the basis for W◦, the annihilator of W. If suffices to find a basis of the set of linear functionals f (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = a 1 ξ 1 + a 2 ξ 2 + a 3 ξ 3 + a 4 ξ 4 for which f (x 1 ) = 0 and f (x 2 ) = 0 : f (1, 2 , − 3 , 4) = a 1 + 2a 2 − 3 a 3 + 4a 4 = 0 f (0, 1 , 4 , −1) = a 2 + 4a 3 − a 4 = 0 The system of equations in a 1 , a 2 , a 3 , a 4 is in echelon form with free variables a 3 and a 4. Set a 3 = 1, a 4 = 0 to obtain a 1 = 11, a 2 = − 4 , a 3 = 1, a 4 = 0 ⇒ f 1 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = 11ξ 1 − 4 ξ 2 + ξ 3. Set a 3 = 0, a 4 = − 1 to obtain a 1 = 6, a 2 = − 1 , a 3 = 0, a 4 = − 1 ⇒ f 2 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = 6ξ 1 − ξ 2 − ξ 4. The set of linear functionals {f 1 , f 2 } is a basis of W ◦.

Example. Given the annihilator described by the three linear functionals in R^4 :

f 1 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = ξ 1 + 2ξ 2 + 2ξ 3 + ξ 4 f 2 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = 2 ξ 1 + ξ 4 f 3 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = − 2 ξ 1 − 4 ξ 3 + 3ξ 4

we find the subspace it annihilates. After the row reduction, we find that the functionals below ahhihilate the same subspace

g 1 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = ξ 1 + 2ξ 3 g 2 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = ξ 2 g 3 (ξ 1 , ξ 2 , ξ 3 , ξ 4 ) = ξ 4

The subspace annihilated consists of the vectors with ξ 1 = − 2 ξ 3 , ξ 2 = ξ 4 = 0. Thus the subspace that is annihilated is given by span{(− 2 , 0 , 1 , 0)}.

Proposition. If M ⊂ V is a subspace of a finite dimensional space and x 1 ,... , xn is a basis for V such that M = span{x 1 ,... , xm}, then M ◦^ = span{fm+1,... , fn}, where f 1 ,... , fn is the dual basis. In particular we have

dim(M ) + dim(M ◦) = dim(V ) = dim(V ′).

Proof. Let x 1 ,... , xm be a basis for M ; M = span{x 1 ,... , xm}. Extend to {x 1 ,... , xn}, a basis for V. Construct a dual basis f 1 ,... , fn for V ′, fi(xj ) = δij. We show that fm+1,... , fn is a basis for M ◦. First, show M ◦^ = span{fm+1,... , fn}. Let f ∈ M ◦^ ⊂ V ′. f =

∑n i=1 cifi^ =^

∑n i=1 f^ (xi)fi^ =^

∑m i=1 f^ (xi)fi^ +^

∑n ∑n i=m+1^ f^ (xi)fi^ = i=m+1 f^ (xi)fi^ ∈^ span{fm+1,... , fn}. Second, {fm+1,... , fn} are linearly independent, since {fm+1,... , fn} is a subset of basis for V ′. Thus, dim(M ◦) = n − m = dim(V ) − dim(M ).

Theorem. W 1 and W 2 are subspaces of a finite-dimensional vector space. Then W 1 = W 2 ⇔ W 1 ◦ = W 2 ◦.

Proof. ⇒ If W 1 = W 2 , then of course W 1 ◦ = W 2 ◦. ⇐ If W 1 6 = W 2 , then one of the two subspaces contains a vector which is not in the other. Suppose there is a vector x ∈ W 2 but x /∈ W 1. There is a linear functional f such that f (z) = 0 for all z ∈ W 1 , but f (x) 6 = 0. Then f ∈ W 1 ◦ but f ∈/ W 2 ◦ and W 1 ◦ 6 = W 2 ◦.

Theorem. Let W be a subspace of a finite-dimensional vector space V. Then W = W ◦◦.

Proof. dim W + dim W ◦^ = dim V , dim W ◦^ + dim W ◦◦^ = dim V ′, and since dim V = dim V ′^ we have dim W = dim W ◦◦. Since W ⊂ W ◦◦, we see that W = W ◦◦.