Gram Schmidt Process-Linear Algebra-Lecture 30 Slides-Mathematics, Slides of Linear Algebra

Download Gram Schmidt Process-Linear Algebra-Lecture 30 Slides-Mathematics and more Slides Linear Algebra in PDF only on Docsity!

MATH 304

Linear Algebra Lecture 30: The Gram-Schmidt process (continued).

Orthogonal sets

Let V be a vector space with an inner product. Definition. Nonzero vectors v 1 , v 2 ,... , vk ∈ V form an orthogonal set if they are orthogonal to each other: 〈vi , vj 〉 = 0 for i 6 = j. If, in addition, all vectors are of unit norm, ‖vi ‖ = 1, then v 1 , v 2 ,... , vk is called an orthonormal set.

Theorem Any orthogonal set is linearly independent.

The Gram-Schmidt orthogonalization process

Let V be a vector space with an inner product. Suppose x 1 , x 2 ,... , xn is a basis for V. Let v 1 = x 1 ,

v 2 = x 2 − 〈x 2 , v 1 〉 〈v 1 , v 1 〉

v 1 ,

v 3 = x 3 − 〈x 3 , v 1 〉 〈v 1 , v 1 〉

v 1 − 〈x 3 , v 2 〉 〈v 2 , v 2 〉

v 2 ,

.................................................

vn = xn −

〈xn, v 1 〉 〈v 1 , v 1 〉 v 1 − · · · −

〈xn, vn− 1 〉 〈vn− 1 , vn− 1 〉 vn− 1.

Then v 1 , v 2 ,... , vn is an orthogonal basis for V.

Any basis x 1 , x 2 ,... , xn

Orthogonal basis v 1 , v 2 ,... , vn

Properties of the Gram-Schmidt process:

• vk = xk − (α 1 x 1 + · · · + αk− 1 xk− 1 ), 1 ≤ k ≤ n;
• the span of v 1 ,... , vk is the same as the span of x 1 ,... , xk ;
• vk is orthogonal to x 1 ,... , xk− 1 ;
• vk = xk − pk , where pk is the orthogonal projection of the vector xk on the subspace spanned by x 1 ,... , xk− 1 ;
• ‖vk ‖ is the distance from xk to the subspace spanned by x 1 ,... , xk− 1.

Problem. Let Π be the plane in R^3 spanned by vectors x 1 = (1, 2 , 2) and x 2 = (− 1 , 0 , 2). (i) Find an orthonormal basis for Π. (ii) Extend it to an orthonormal basis for R^3.

x 1 , x 2 is a basis for the plane Π. We can extend it to a basis for R^3 by adding one vector from the standard basis. For instance, vectors x 1 , x 2 , and x 3 = (0, 0 , 1) form a basis for R^3 because ∣∣ ∣ ∣∣ ∣

Using the Gram-Schmidt process, we orthogonalize the basis x 1 = (1, 2 , 2), x 2 = (− 1 , 0 , 2), x 3 = (0, 0 , 1):

v 1 = x 1 = (1, 2 , 2),

v 2 = x 2 −

〈x 2 , v 1 〉 〈v 1 , v 1 〉

v 1 = (− 1 , 0 , 2) −

v 3 = x 3 −

〈x 3 , v 1 〉 〈v 1 , v 1 〉

v 1 −

〈x 3 , v 2 〉 〈v 2 , v 2 〉

v 2

Problem. Find the distance from the point y = (0, 0 , 0 , 1) to the subspace Π ⊂ R^4 spanned by vectors x 1 = (1, − 1 , 1 , −1), x 2 = (1, 1 , 3 , −1), and x 3 = (− 3 , 7 , 1 , 3).

Let us apply the Gram-Schmidt process to vectors x 1 , x 2 , x 3 , y. We should obtain an orthogonal system v 1 , v 2 , v 3 , v 4. The desired distance will be |v 4 |.

x 1 = (1, − 1 , 1 , −1), x 2 = (1, 1 , 3 , −1), x 3 = (− 3 , 7 , 1 , 3), y = (0, 0 , 0 , 1).

v 1 = x 1 = (1, − 1 , 1 , −1),

v 2 = x 2 −

〈x 2 , v 1 〉 〈v 1 , v 1 〉 v 1 = (1, 1 , 3 , −1)−

v 3 = x 3 −

〈x 3 , v 1 〉 〈v 1 , v 1 〉

v 1 −

〈x 3 , v 2 〉 〈v 2 , v 2 〉

v 2

= (− 3 , 7 , 1 , 3) −

Problem. Find the distance from the point z = (0, 0 , 1 , 0) to the plane Π that passes through the point x 0 = (1, 0 , 0 , 0) and is parallel to the vectors v 1 = (1, − 1 , 1 , −1) and v 2 = (0, 2 , 2 , 0).

The plane Π is not a subspace of R^4 as it does not pass through the origin. Let Π 0 = Span(v 1 , v 2 ). Then Π = Π 0 + x 0.

Hence the distance from the point z to the plane Π is the same as the distance from the point z − x 0 to the plane Π 0.

We shall apply the Gram-Schmidt process to vectors v 1 , v 2 , z − x 0. This will yield an orthogonal system w 1 , w 2 , w 3. The desired distance will be |w 3 |.

v 1 = (1, − 1 , 1 , −1), v 2 = (0, 2 , 2 , 0), z − x 0 = (− 1 , 0 , 1 , 0).

w 1 = v 1 = (1, − 1 , 1 , −1),

w 2 = v 2 −

〈v 2 , w 1 〉 〈w 1 , w 1 〉

w 1 = v 2 = (0, 2 , 2 , 0) as v 2 ⊥ v 1.

w 3 = (z − x 0 ) −

〈z − x 0 , w 1 〉 〈w 1 , w 1 〉 w 1 −

〈z − x 0 , w 2 〉 〈w 2 , w 2 〉 w 2

= (− 1 , 0 , 1 , 0) −

|w 3 | =

Modifications of the Gram-Schmidt process

Another modification is a recursive process which is more stable to roundoff errors than the original process. Suppose x 1 , x 2 ,... , xn is a basis for an inner product space V. Let w 1 = (^) ‖xx^11 ‖ , v 2 = x 2 − 〈x 2 , w 1 〉w 1 , v 3 = x 3 − 〈x 3 , w 1 〉w 1 ,

................................................. vn = xn − 〈xn, w 1 〉w 1. Then w 1 , v 2 ,... , vn is a basis for V , ‖w 1 ‖ = 1, and w 1 is orthogonal to v 2 ,... , vn. Now repeat the process with vectors v 2 ,... , vn, and so on.