Download Linear Algebra: Vector Spaces, Linear Transformations, and Eigenvalues and more Lecture notes Applied Mathematics in PDF only on Docsity!
Linear Transformations and
Diagonalization
Fields and Vector Spaces
Definition 2.1: Field
A field is a set F together with binary operations on F, called addition and multiplication, such that: - F together with addition is an abelian group (we use the notations a + b, 0, -a) - F{0} together with multiplication is an abelian group (we use the notations a · b, 1, a^-1) - The following distributivity axiom holds: For all a, b, c ∈ F, a(b + c) = ab + ac in F.
Example 2.2: Examples of Fields
(a) The sets Q, R and C with the usual addition and multiplication are fields. (b) The set Z with the usual addition and multiplication is not a field because there is no multiplicative inverse of 2 in Z. (c) The set F2 = {0, 1} with the following operations is a field: + | 0 1 ---|--- 0 | 0 1 1 | 1 0 · | 0 1 ---|--- 0 | 0 0 1 | 0 1 F2 is the smallest field. (d) Let p be a prime. The set Fp = {0, 1, ..., p-1} with addition as in Example 1.5(b) and multiplication defined as x · y := remainder left when xy is divided by p, is a field.
Proposition 2.
Let F be a field and a, b ∈ F. Then: (a) 0a = 0 in F. (b) (-a)b = -(ab) in F.
Definition 2.4: Vector Space
A vector space V over a field F is an abelian group (with binary operation written additively as x + y) together with a map F × V → V (called scalar multiplication, written as (a, x) → ax) such that the following axioms are satisfied: (i) 1st distributivity law: For all a, b ∈ F and x ∈ V, (a + b)x = ax + bx. (ii) 2nd distributivity law: For all a ∈ F and x, y ∈ V, a(x + y) = ax + ay. (iii) For all a, b ∈ F and x ∈ V, (ab)x = a(bx). (iv) For all x ∈ V, 1x = x.
Example 2.5: Examples of Vector Spaces
(a) For every n ∈ N, the set Rn with the usual addition and scalar multiplication is a vector space over R. Similarly, for any field F, the set Fn = {(a1, ..., an) : a1, ..., an ∈ F} is a vector space over F. (b) The additive group of R or C, with the usual multiplication R × V → V as scalar multiplication, is a vector space over Q. (c) The abelian group R, with the scalar multiplication defined by a ⊗ x := a^2 x, does not satisfy the 1st distributivity law, and hence is not a vector space over R.
Fields and Vector Spaces
First Distributivity Law
The first distributivity law does not hold. We need to check whether $a \otimes (x + y) = a \otimes x + a \otimes y$ for all $a \in \mathbb{R}$ and $x, y \in V$. $\text{LHS} = a \otimes (x + y)$ $\text{RHS} = a \otimes x + a \otimes y$ $\text{LHS} \neq \text{RHS}$, so the first distributivity law does not hold.
Second Distributivity Law
The second distributivity law does hold. We need to check whether $a \otimes (b \otimes x) = (ab) \otimes x$ for all $a, b \in \mathbb{R}$ and $x \in V$. $\text{LHS} = a \otimes (b \otimes x) = a^2 (b^2 x)$ $\text{RHS} = (ab) \otimes x = (a^2 b^2) x = a^2 (b^2 x)$ $\text{LHS} = \text{RHS}$, so the second distributivity law does hold.
Axiom 3
Axiom 3 does hold. We have $1 \otimes x = 1^2 x = 1x$ for all $x \in V$.
Axiom 4
Axiom 4 does hold.
Proposition 2.
Let $V$ be a vector space over a field $\mathbb{F}$ and let $a, b \in \mathbb{F}$ and $x, y \in V$. Then we have:
(a) $(a - b)x = ax - bx$ (b) $a(x - y) = ax - ay$ (c) $ax = 0_V \Leftrightarrow a = 0_\mathbb{F}$ or $x = 0_V$ (d) $(-1)x = -x$
Proof: (a) $(a - b)x + bx = ((a - b) + b)x = (a + ((- b) + b))x = (a + 0_\mathbb{F})x = ax$ $\therefore (a - b)x = ax - bx$ (add $-bx$ to both sides). (b) See Coursework. (c) "$\Rightarrow$": See Coursework. "$ \Leftarrow$": Put $a = b$ and $x = y$ in (a) and (b), respectively. (d) Put $a = 0$ and $b = 1$ in (a) and use (c).
Example 2.
The "mother" of almost all vector spaces is the set $\mathbb{F}^S$ of all maps from a set $S$ to a field $\mathbb{F}$. We define addition and scalar multiplication on $\mathbb{F}^S$ as follows:
subsets of the vector space $\mathbb{R}^I$ are subspaces: i. ${f \in \mathbb{R}^I : f(s_0) = 0}$ for any $s_0 \in I$ ii. The set of all continuous functions $f: I \to \mathbb{R}$ iii. The set of all differentiable functions $f: I \to \mathbb{R}$ iv. The set $\mathbb{P}_n$ of polynomial functions $f: I \to \mathbb{R}$ of degree at most $n$ v. The space of solutions of a homogeneous linear differential equation (e) The subset $\mathbb{Z}^n$ of $\mathbb{R}^n$ is closed under addition but not under scalar multiplication.
Proposition 2.
Let $W_1, W_2$ be subspaces of a vector space $V$ over a field $ \mathbb{F}$. Then the intersection $W_1 \cap W_2$ and the sum $W_1 + W_2 := {x_1 + x_2 \in V : x_1 \in W_1, x_2 \in W_2}$ are subspaces of $V$ as well.
Proof: For $W_1 \cap W_2$: - $0_V \in W_1 \cap W_2$ because $0_V \in W_1$ and $0_V \in W_2$. - $x, y \in W_1 \cap W_2 \Rightarrow x, y \in W_1$ and $x, y \in W_2 \Rightarrow x + y \in W_1$ and $x + y \in W_2 \Rightarrow x + y \in W_1 \cap W_2$. - $a \in \mathbb{F}, x \in W_1 \cap W_ \Rightarrow x \in W_1$ and $x \in W_2 \Rightarrow ax \in W_1$ and $ax \in W_2 \Rightarrow ax \in W_1 \cap W_2$.
For $W_1 + W_2$: - $0_V = 0_V + 0_V \in W_1 + W_2$. - $x, y \in W_1 + W_2 \Rightarrow \exists x_1, y_1 \in W_1, x_2, y_2 \in W_2$ such that $x = x_1 + x_2, y = y_1 + y_2 \Rightarrow x + y = (x_1 + x_2) + (y_1 + y_2) \in W_1 + W_2$. - $a \in \mathbb{F}, x \in W_1 + W_2 \Rightarrow \exists x_ \in W_1, x_2 \in W_2$ such that $x = x_1 + x_2 \Rightarrow ax = a(x_1 + x_2) \in W_1 + W_2$.
Example 2.
Let $W_1 = {(a, 0) : a \in \mathbb{R}}$ and $W_2 = {(0, b) : b \in \mathbb{R}}$ as in 2.10(e). Then $W_1 + W_2 = \mathbb{R}^2$.
Bases
Definition 3.
Let $V$ be a vector space over a field $\mathbb{F}$. Let $x_1, \ldots, x_n \in V$.
(a) An element $x \in V$ is a linear combination of $x_1, \ldots, x_n$ if there are $a_1, \ldots, a_n \in \mathbb{F}$ such that $x = a_1 x_1 + \ldots + a_n x_n$. (b) The set of all linear combinations of $x_1, \ldots, x_n$ is called the Span of $x_1, \ldots, x_n$, denoted by $\text{Span}(x_1, \ldots, x_n)$. (c) We say that $x_1, \ldots, x_n$ span (or generate) $V$ if $V = \text{Span}(x_1, \ldots, x_n)$.
Example 3.
(a) Let $V = M_{n \times m}(\mathbb{F})$. For $i \in {1, \ldots, n}$ and $j \in {1, \ldots, m}$, let $E_{ij}$ be the $(n \times m)$-matrix with 1 at position $(i, j)$ and 0 elsewhere. Then the matrices $E_{ij}$ form a spanning set of $V$. (b) Do the vectors $\begin{pmatrix} 1 \ i \end{pmatrix}, \begin{pmatrix} i \ 2 \end{pmatrix} \in \mathbb{C}^2$ span $\mathbb{C}^2$? Yes, they do. (c) $\mathbb{P}_n = \text{Span}(1, t, \ldots, t^n)$.
Proposition 3.
Let $V$ be a vector space over a field $\mathbb{F}$. Let $x_1, \ldots, x_n \in V$. Then $\text{Span}(x_1, \ldots, x_n)$ is the smallest subspace of $V$ that contains $x_1, \ldots, x_n$. In particular:
(a) If $x \in \text{Span}(x_1, \ldots, x_n)$, then $\text{Span}(x_1, \ldots, x_n, x) = \text{Span}(x_1, \ldots, x_n)$. (b) For any $a_2, \ldots, a_n \in \mathbb{F}$, we have $\text{Span}(x_1, \ldots, x_n) = \text{Span}(x_1, x_
- a_2 x_1, \ldots, x_n - a_n x_1)$.
Proof: - $\text{Span}(x_1, \ldots, x_n)$ is a subspace of $V$ because it satisfies the subspace conditions. - $\text{Span}(x_1, \ldots, x_n)$ contains $x_1, \ldots, x_n$ by definition. - $\text{Span}(x_1, \ldots, x_n)$ is the smallest subspace of $V$ containing $x_1, \ldots, x_n$.
Part (a) and (b) follow from the properties of subspaces.
Definition 3.
Let $V$ be a vector space over a field $\mathbb{F}$. The vectors $x_1, \ldots, x_n \in V$ are linearly independent if the only way to write $a_1 x_1 + \ldots + a_n x_n = 0$ is with $a_1 = \ldots = a_n = 0$. Otherwise, they are linearly dependent.
A linear combination $a_1 x_1 + \ldots + a_n x_n$ is trivial if $a_1 = \ldots = a_n = 0$, and non-trivial otherwise.
Example 3.
(further examples omitted)
Linear Transformations
Definition 4.1: Matrices
An (m × n) - matrix A over a field F is an array:
A = [ a11 ... a1n ... ... am1 ... amn ]
any field $F$. The whole statement is actually Lemma 5.3 in Linear Algebra I.
Examples of Linear Transformations
(b) Let $V$ be a vector space over a field $F$. Then the following maps are linear transformations:
$\id : V \to V, x \mapsto x$ (identity); $0 : V \to V, x \mapsto 0_V$ (zero map); the map $V \to V, x \mapsto ax$, for any given $a \in F$ fixed (stretch).
Proof: Easy. (Example 5.4(c),(d) in Linear Algebra I)
(c) Let $L : V \to W$ and $M : W \to Z$ be linear transformations between vector spaces over a field $F$. Then their composition $M \circ L : V \to Z$ is again a linear transformation.
Proof: (See also Section 5.3 of Linear Algebra I)
i. Let $x, y \in V$: $$(M \circ L)(x + y) = M(L(x + y)) = M(L(x) + L(y)) = M(L(x)) + M(L(y)) = (M \circ L)(x) + (M \circ L)(y)$$
ii. Let $x \in V$ and $a \in F$: $$(M \circ L)(ax) = M(L(ax)) = aM(L(x)) = a(M \circ L)(x)$$
(d) Let $V$ be the subspace of $\R^\R$ consisting of all differentiable functions. Then differentiation $D : V \to \R^\R, f \mapsto f'$, is a linear transformation.
Proof:
i. Let $f, g \in V$: $D(f + g) = (f + g)'$
ii. Let $a \in \R$ and $f \in V$: $D(af) = (af)' = af' = a(Df)$
The middle equality in both chains of equalities has been proved in Calculus.
(e) The map $L : \R^2 \to \R, \begin{pmatrix} x_1 \ x_2 \end{pmatrix} \mapsto x_1x_2$, is not a linear transformation.
Proof: Let $a = 2$ and $x = \begin{pmatrix} 1 \ 1 \end{pmatrix} \in \R^2$. Then: $L(ax) = L\begin{pmatrix} 2 \ 2 \end{pmatrix} = 4$, but $aL(x) = 2 \cdot 1 = 2$.
Matrix Representation of Linear Transformations
Proposition 4.4 (Matrix representation I)
Let $F$ be a field. Let $L : F^n \to F^m$ be a linear transformation. Then there exists a unique matrix $A \in M_{m \times n}(F)$ such that $L = L_A$ (as defined in 4.3(a)). In this case we say that $A$ represents $L$ (with
respect to the standard bases of $F^n$ and $F^m$). (See also Theorem 5. of Linear Algebra I)
For example, the map $\R^3 \to \R^2, \begin{pmatrix} c_1 \ c_2 \ c_ \end{pmatrix} \mapsto \begin{pmatrix} 2c_1 + c_3 \ -4c_2 \ c_ \end{pmatrix}$, is represented by $A = \begin{pmatrix} 2 & 0 & 1 \ -4 & 1 & 0 \end{pmatrix} \in M_{2 \times 3}(\R)$.
Proof:
Uniqueness: Suppose $A \in M_{m \times n}(F)$ satisfies $L = L_A$. Then the $j$-th column of $A$ is $Ae_j = L_A(e_j) = L(e_j)$ (for $j = 1, \ldots, n$), so $A$ is the $(m \times n)$-matrix with the column vectors $L(e_1), \ldots, L(e_n)$.
Existence: Let $A$ be defined this way. We want to show $L = L_A$.
Let $c = \begin{pmatrix} c_1 \ \vdots \ c_n \end{pmatrix} \in F^n$. Then $c = c_1e_1 + \ldots + c_ne_n = c_1L_A(e_1) + \ldots + c_nL_A(e_n)$ (because $L$ and $L_A$ are linear transformations), so $L(c) = L_A(c)$ (because $L(e_j) = L_A(e_j)$ for all $j = 1, \ldots, n$).
Kernel and Image of Linear Transformations
Definition 4.
Let $L : V \to W$ be a linear transformation between vector spaces $V, W$ over a field $F$. Then $$\ker(L) := {x \in V : L(x) = 0_W}$$ is called the kernel of $L$ and $$\im(L) := {y \in W : \exists x \in V : y = L(x)}$$ is called the image (or range) of $L$.
Remark 4.
Let $A \in M_{m \times n}(F)$. Then $\N(A) = {c \in F^n : Ac = 0}$ denotes the nullspace of $A$ (see also Section 6.2 of Linear Algebra I) and $\im(L_A) = \Col(A)$, where $\Col(A)$ denotes the column space of $A$; i.e. $\Col(A) = \Span(a_1, \ldots, a_n)$ where $a_1, \ldots, a_n$ denotes the $n$ columns of $A$. (See also Section 6.4 of Linear Algebra I)
Proof: - First assertion: by definition. - Second assertion: follows from 4.9(a) applied to the standard basis of $F^n$.
Lemma 4.
Let $V$ and $W$ be vector spaces over a field $F$ and let $L : V \to W$ be a linear transformation. Then:
(a) $\ker(L)$ is a subspace of $V$. (b) $\im(L)$ is a subspace of $W$.
Proof:
Isomorphisms between Vector Spaces
Definition 4.
Let $V, W$ be vector spaces over a field $F$. A bijective linear transformation $L : V \to W$ is called an isomorphism. The vector spaces $V$ and $W$ are called isomorphic if there exists an isomorphism $L : V \to W$; we then write $V \cong W$.
Example 4.
(a) For any vector space $V$ over a field $F$, the identity $\id : V \to V$ is an isomorphism. (b) If $L : V \to W$ is an isomorphism, then the inverse map $L^{-1} : W \to V$ is an isomorphism as well. (See also Definition 5.21 from Linear Algebra I) (c) If $A \in M_{n \times n}(\R)$ is invertible, then $L_A : \R^n \to \R^n$ is an isomorphism. (d) The map $L : \R^2 \to \C, \begin{pmatrix} a \ b \end{pmatrix} \mapsto a + bi$, is an isomorphism between the vector spaces $\R^2$ and $\C$ over $\R$. (e) For any $n \in \N$, the map $L : \R^{n+1} \to \P_n, \begin{pmatrix} a_0 \ \vdots \ a_n \end{pmatrix} \mapsto a_0 + a_1t + \ldots + a_nt^n$ is an isomorphism between the vector spaces $\R^{n+1}$ and $\P_n$ over $\R$. (f) For any $m, n \in \N$ we have $\R^{m \times n} = M_{m \times n}(\R)$.
Proof: - (b) and (c) see Coursework. - (d) and (e) follow from the following proposition and 3.9(c) and (d), respectively. - (f) (only in the case $m = n = 2$) The map $\R^4 \to M_{2 \times 2}(\R), \begin{pmatrix} a_1 \ a_2 \ a_3
a_4 \end{pmatrix} \mapsto \begin{pmatrix} a_1 & a_2 \ a_3 & a_ \end{pmatrix}$ is clearly an isomorphism.
Proposition 4.
Let $V$ be a vector space over a field $F$ with basis $x_1, \ldots, x_n$. Then the map $L : F^n \to V, \begin{pmatrix} a_1 \ \vdots \ a_n \end{pmatrix} \mapsto a_1x_1 + \ldots + a_nx_n$ is an isomorphism. (We will later use the notation $I_{x_1, \ldots, x_n}$ for the map $L$.)
Proof:
(a) Let $a
Proof of (⋆)
Let's elaborate the text into a Markdown-formatted document suitable for study purposes.
Eigenspaces, Eigenvalues, and Eigenvectors
Definition 6.1 : Let $V$ be a vector space over a field $F$ and let $L: V \to V$ be a linear transformation.
The set $E_\lambda(L) := {x \in V: L(x) = \lambda x}$ is called the eigenspace of $L$ corresponding to $\lambda$. An element $\lambda \in F$ is called an eigenvalue of $L$ if $E_\lambda(L)$ is not the zero space. Any vector $x$ in $E_\lambda(L)$ different from the zero vector is called an eigenvector of $L$ with eigenvalue $\lambda$.
The eigenspaces, eigenvalues, and eigenvectors of a matrix $A \in M_n \times n(F)$ are those of the linear transformation $L_A: F^n \to F^n, x \mapsto Ax$.
Lemma 6.2 :
The eigenspace $E_\lambda(L)$ is a subspace of $V$ for every $ \lambda \in F$.
Proposition 6.3 :
Let $F$ be a field, $A \in M_n \times n(F)$, and $\lambda \in F$. Then $ \lambda$ is an eigenvalue of $A$ if and only if $\det(\lambda I_n - A) = 0$.
The polynomial $p_A(\lambda) := \det(\lambda I_n - A)$ is called the characteristic polynomial of $A$.
Example 6.4 :
Determine the (complex) eigenvalues of the matrix $A = \begin{pmatrix} 5i & 3 \ 2 & -2i \end{pmatrix} \in M_2 \times 2(C)$ and find a basis of the eigenspace of $A$ for each eigenvalue.
Example 6.5 :
Let $V$ be the real vector space of infinitely often differentiable functions from $\mathbb{R}$ to $\mathbb{R}$, and let $D: V \to V, f \mapsto f'$, denote differentiation. For every $\lambda \in \mathbb{R}$, the eigenspace of $D$ with eigenvalue $\lambda$ is of dimension 1 with basis given by the function $\exp_\lambda: \mathbb{R} \to \mathbb{R}, t \mapsto e^{\lambda t}$.
Diagonalizability
Definition 6.6 : A linear transformation $L: V \to V$ is diagonalizable if there exists a basis $x_1, \dots, x_n$ of $V$ such that the matrix $D$ representing $L$ with respect to this basis is a diagonal matrix.
A square matrix $A \in M_n \times n(F)$ is diagonalizable if the linear transformation $L_A: F^n \to F^n, x \mapsto Ax$, is diagonalizable.
Proposition 6.7 :
Diagonalizability
Theorem 6.
A matrix A is diagonalizable if and only if the characteristic polynomial of A splits into linear factors and the algebraic multiplicity equals the geometric multiplicity for each eigenvalue of A.
Example 6.
Determine the diagonalizability of the matrix A = [0 1; -1 0] when viewed as an element of M2×2(R), M2×2(C), and M2×2(F2).
For M2×2(R)
The characteristic polynomial of A is λ^2 + 1 , which does not split into linear factors. Therefore, A is not diagonalizable when viewed as an element of M2×2(R).
For M2×2(C)
The characteristic polynomial of A is λ^2 + 1 = (λ + i)(λ - i). The eigenvalues are +i and -i, and the algebraic and geometric multiplicities are both 1 for each eigenvalue. Therefore, A is diagonalizable when viewed as an element of M2×2(C). An invertible matrix M that diagonalizes A is M = [-i, i; 1, 1].
For M2×2(F2)
The characteristic polynomial of A is λ^2 + 1 = (λ + 1)^2. The eigenvalue is 1 (which is equal to -1 in F2), and the algebraic multiplicity is 2. However, the geometric multiplicity is 1. Therefore, A is not diagonalizable when viewed as an element of M2×2(F2).
Cayley-Hamilton Theorem (Theorem 6.14)
Let F be a field, A be an element of Mn×n(F), and pA be the characteristic polynomial of A. Then pA(A) is the zero matrix.
Example 6.
For the matrix A = [0 1; -1 0], the characteristic polynomial is pA(λ) = λ^2 + 1. Applying the Cayley-Hamilton theorem, we have pA(A) = A^2 + I2 = [0 0; 0 0], which is the zero matrix.
Proof of Theorem 6.
The proof is provided in two cases:
When A is a diagonal matrix. When A is a diagonalizable matrix.
The general case is omitted.
