Download Linear Algebra: Vector Spaces and Euclidean Distance in Rn and more Study notes Mathematics in PDF only on Docsity!
Chapter 1
The Euclidean Space
The objects of study in advanced calculus are differentiable functions of several variables. To set the stage for the study, the Euclidean space as a vector space endowed with the dot product is defined in Section 1.1. To aid visualizing points in the Euclidean space, the notion of a vector is introduced in Section 1.2. In Section 1.3 Euclidean motions, mappings preserving the Euclidean distance, are briefly discussed. The last Section 1. contains a discussion on the cross product which is only defined for vectors in the three dimensional Euclidean space, that is, our physical space.
1.1 The Dot Product
An n-tuple is given by
x = (x 1 , x 2 , · · · , xn) , xj ∈ R, j = 1, · · · , n.
It is called an ordered pair when n = 2. Denote by Rn^ the collection of all n-tuples. The zero n-tuple, (0, 0 , · · · , 0), will be written as 0 from time to time. There are two algebraic operations defined on Rn, namely, the addition
x + y = (x 1 + y 1 , x 2 + y 2 , · · · , xn + yn) ,
and the scalar multiplication
αx = (αx 1 , αx 2 , · · · , αxn) , α ∈ R ,
where x = (x 1 , x 2 , · · · , xn), y = (y 1 , y 2 , · · · , yn).
Recall that the ordinary multiplication assigns a number as the product of two numbers, so it can be regarded as a map from R × R to R. One may expect a multiplication on Rn
1
2 CHAPTER 1. THE EUCLIDEAN SPACE
assigns an n-tuple to a given pair of n-tuples, that is, it is a map from Rn^ × Rn^ to Rn. But here the scalar multiplication is not such a multiplication, instead it maps R × Rn^ to Rn.
From linear algebra we know that with these two operations, addition and scalar multiplication, Rn^ becomes a vector space of dimension n over the field of real numbers. The so-called canonical basis of Rn^ is given by
e 1 = (1, 0 , 0 , · · · , 0) , e 2 = (0, 1 , 0 , · · · , 0) , · · · , en = (0, 0 , 0 , · · · , 1).
Using this basis every n-tuple can be written as the linear combination of the basis elements in a very simple way,
x = (x 1 , x 2 , · · · , xn)
=
∑^ n
j=
xj ej
= x 1 e 1 + x 2 e 2 + · · · xnen.
For instance, the point (2, − 3 , 6) in R^3 is equal to
(2, − 3 , 6) = 2(1, 0 , 0) − 3(0, 1 , 0) + 6(0, 0 , 1) = 2 e 1 − 3 e 2 + 6e 3.
In lower dimensions n = 2, 3 , the notations i, j, k are used instead of e 1 , e 2 , e 3 in some texts. We will not adopt them here though.
For x, y ∈ Rn, the dot product between x and y is defined by
x · y =
∑^ n
j=
xj yj
= x 1 y 1 + x 2 y 2 + · · · xnyn.
Recall that the axioms for an inner product on a vector space V over reals are: For u, v, w, y, z ∈ V, α, β ∈ R ,
(a) 〈u, u〉 ≥ 0 and equals to 0 iff u = 0 ,
(b) 〈u, v〉 = 〈v, u〉 ,
(c) 〈αu + βv, w〉 = α〈u, w〉 + β〈v, w〉.
Note that (b) and (c) imply
(d) 〈u, αv + βw〉 = α〈u, v〉 + β〈u, w〉.
4 CHAPTER 1. THE EUCLIDEAN SPACE
which forces t 1 xj = yj for all j = 1, · · · , n. So we can take α = t 1 in case c =
j y
(^2) j > 0.
When all xj ’s vanish but not all yj ’s, we exchange x and y to get the same conclusion.
Finally, when all xj ’s and yj ’s vanish, the inequality clearly holds.
Accompanying with the notion of the inner product are those of the norm and the distance. Indeed, the Euclidean norm of an n-tuple is defined to be
|x| = (x · x)^1 /^2
( (^) n ∑
j=
x^2 j
x^21 + x^22 + · · · x^2 n.
The Euclidean distance between x and y is defined by
|x − y| =
(x 1 − y 1 )^2 + (x 2 − y 2 )^2 + · · · + (xn − yn)^2.
In terms of these notions, Cauchy-Schwarz Inequality can be rewritten in a compact form
|x · y| ≤ |x||y|.
In mathematics, a distance is a rule to assign a non-negative number to any pair of elements in a set under consideration. The rule consists of three “axioms”: For a, b, c in this set,
(i) d(a, b) ≥ 0 , and equal to 0 iff a = b ,
(ii) d(a, b) = d(b, a) , and
(iii) d(a, b) ≤ d(a, c) + d(c, b).
Now, taking d(x, y) = |x − y|, we see that it satisfies all these three axioms: x, y, z ∈ Rn,
(a) |x − y| ≥ 0 and equal to 0 if and only if x = y ,
(b) |x − y| = |y − x| ,
(c) |x − y| ≤ |x − z| + |z − y|.
1.1. THE DOT PRODUCT 5
Indeed, (a) and (b) are obvious. To prove (c), write u = x − z, v = z − y to get
|u + v| ≤ |u| + |v| ,
which holds after taking square of both sides and then applying the Cauchy-Schwarz In- equality. I let you work out the details in the exercise. Note that |x| = |x− 0 |, so the norm of x is its distance to the zero n-tuple. In these notes, norm and distance are referred to Euclidean norm and Euclidean distance without further specification.
Note that the same notation |x| stands for the absolute value of x when x is a real number. The norm of x is the same as its absolute value when n = 1. When n ≥ 2, the notation |x| stands for the norm only as there is no such a definition of absolute value for an n-tuple. The notation ‖x‖ is also used to denote the norm of x, but it will not be used here.
Recall that the cosine function cos t is strictly decreasing from 1 to −1 as t goes from 0 to π. Keeping this in mind, we are going to define the angle between two non-zero n-tuples. By Cauchy-Schwarz Inequality, the absolute value of the expression x · y/|x||y| lies in the interval [− 1 , 1]. Therefore, by what we just said, there exists a unique θ ∈ [0, π] satisfying cos θ = x · y/|x||y|, that is,
x · y = |x||y| cos θ.
We define the angle between two non-zero n-tuples x and y to be θ where x, y are arbi- trary n-tuples. The angle between two n-tuples makes no sense when of them is zero. By definition this angle must belong to [0, π]. Moreover, it is symmetric, that is, the angle between x and y is the same as the angle between y and x. At this stage, the notion of an angle is defined purely in an analytical manner and does not bear any geometric meaning. We will link it to geometry in the next section.
Two n-tuples x and y are perpendicular or orthogonal to each other if x · y = 0. In terms of the angle, they are perpendicular if and only if their angle is π/2. The zero n-tuple is perpendicular to all n-tuples. By Cauchy-Schwarz Inequality, we also know that two non-zero x and y satisfy x = cy for some c > 0 when their angle θ = 0 and satisfy x = cy, c < 0 when θ = π.
Example 1.1. Find all n-tuples x that are perpendicular to (1, − 1 , 2) and (− 1 , 0 , 3). These points satisfy (1, − 1 , 2) · x = 0 , (− 1 , 0 , 3) · x = 0 ,
that is, the linear system (^) { x − y + 2z = 0 , −x + 3z = 0.
1.2. VECTOR REPRESENTATION 7
On the other hand, when α < 0, the resulting vector points in the opposite direction of the original vector with size changes equal to |α|.
How about substraction of two vectors? Let w = v − u. Then w can be obtained by first drawing the triangle with vertices at (0, 0), u and v and then translate the side from u to v so that its base is located at the origin. The translated side is w.
We may also find the midpoint of two ordered pairs. For u, v, its midpoint is given by (u + v)/2. Regarding as a vector, this midpoint can be described in the following way. First draw the parallelogram formed by u and v. Then the intersection point of the two diagonal lines of this parallelogram is the tip of the midpoint (vector).
When we regard an n-tuple x as a vector, it is more convenient to call its norm the magnitude of the vector. It is a unit vector if its magnitude is equal to 1. Likewise, the distance between two points may be called the length of the line segment connecting x and y. It is consistent with the classical Pythagoras theorem. In fact, the definition of the Euclidean norm and distance were inspired by this classical theorem.
Next we show that the angle defined in the last section, which purely depends on analytical terms, is the same as the “geometric angle”. To see it, let x = (a, b) and y = (c, d) be two non-zero vectors in the plane. By the Law of Cosines in trigonometry (see Comments at the end of this chapter),
(c − a)^2 + (d − b)^2 = (a^2 + b^2 ) + (c^2 + d^2 ) − 2
c^2 + d^2
a^2 + b^2 cos φ ,
where φ ∈ [0, π] is the “geometric angle” between x and y. Simplifying, we have
−2(ac + db) = − 2
c^2 + d^2
a^2 + b^2 cos φ ,
which is equal to x · y = |x||y| cos φ.
Comparing with the definition of θ, we have cos φ = cos θ so that φ = θ. In other words, the geometric angle coincides with the analytical angle. The same argument works in higher dimensions as we can restrict to the plane containing any two given vectors.
A vector is uniquely determined by its magnitude and direction. To be more precise we fix them in definition. Any vector with unit length is called a direction. Each non-zero vector x can be written as x = |x| ξ,
where |x| is its magnitude and
ξ =
x |x|
8 CHAPTER 1. THE EUCLIDEAN SPACE
its direction. Every direction ξ = (ξ 1 , ξ 2 , · · · , ξn) can further be expressed as
ξ = (cos α 1 , cos α 2 , · · · , cos αn) ,
where αk ∈ [0, π] are called the direction angles of ξ. From ξ ·ek = cos αk we see that αk is the angle between ξ and the ek-axis. These cos αk’s are called the direction cosines of x.
Example 1.2. Find the magnitude and direction of (1, 2 , −7) and determine the vector (2, a, 6) that is perpendicular to (1, 2 , −7). The magnitude of (1, 2 , −7) is
|(1, 2 , −7)| =
12 + 2^2 + (−7)^2 =
and its direction is (1, 2 , −7)/
- By orthogonality,
0 = (1, 2 , −7) · (2, a, 6) = 2 + 2a − 42 = 0 ,
which implies a = 20. The vector (2, 20 , 6) is perpendicular to (1, 2 , −7).
One may also consider the vector from the initial point x to the terminal point y, or the vector based at some point. Unlike a vector, a vector from x to y is an arrow whose base and tip are x and y respectively. Obviously such a “vector” is parallel to the position vector of y − x whose base is now at the origin. The length and direction of a vector from x to y are defined as the respective length and direction of y−x.
Example 1.3. Consider the triangle with vertices at (1, 2), (3, 4), (0, −1). Find the di- rection of the vector pointing at the midpoint of the side connecting (1, 2) and (3, 4) from (0, −1). Well, first we translate (0, −1) to the origin so that the triangle is congruent to the one whose vertices are ((1, 2) − (0, −1), (3, 4) − (0, −1), (0, −1) − (0, −1), that is, (1, 3), (3, 5), (0, 0). The midpoint of the side from (1, 3) and (3, 5) is given by
1 2
and its direction is given by
(2, 4) √ 22 + 4^2
(No need to simplify further.)
Example 1.4. (a) Find the magnitude and direction of the vector from (1, −1) to (− 2 , 5). (b) Find all directions that are perpendicular to the vector in (a).
10 CHAPTER 1. THE EUCLIDEAN SPACE
which yields immediately that
|A(x − y)|^2 = 〈A(x − y), A(x − y)〉 = |x − y|^2 = |x|^2 − 2 〈x, y〉 + |y|^2.
On the other hand, a direct calculation shows that
〈A(x − y), A(x − y)〉 = 〈Ax − Ay, Ax − Ay〉 = |Ax|^2 − 2 〈Ax, Ay〉 + |Ay|^2.
By comparing, we see that for all x, y,
〈A′Ax, y〉 = 〈Ax, Ay〉 = 〈x, y〉 ,
which implies that A′A = I. Thus A is a orthogonal matrix. Finally, this relation also shows that T is a Euclidean motion whenever A is orthogonal.
∑^ Here we have used the following derivation in linear algebra: For a matric (Bx)j^ = k bkj^ xk, 〈x, By〉 = 〈B′x, y〉.
Indeed,
〈x, By〉 =
j
xj
k
bkj yk
k
j
bkj xj yk
k
yk
j
b′ jkxj = 〈B′x, y〉.
Here are some examples of Euclidean motions.
(1) Take A to be the identity and b a nonzero vector. Then T x = x + b is a translation. The origin is moved to b after the motion.
(2) When n = 2, the Euclidean motion
T x =
[
] [
x 1 x 2
]
is the reflection with respect to the x-axis and
T x =
[
] [
x 1 x 2
]
1.3. EUCLIDEAN MOTIONS 11
is the reflection with respect to the y-axis. (In matrix form the vector x is understood as a column vector.) On the other hand, given any plane in R^3 , one may consider the reflection with respect to this plane. For instance,
T x =
x 1 x 2 x 3
is the reflection with respect to the xy-plane in R^3. The reflection with respect to any straight line in R^2 or with respect to any plane in R^3 can be defined similarly.
(3) The (counterclockwise) rotation of angle θ in R^2 is given by the Euclidean motion
T x =
[
cos θ − sin θ sin θ cos θ
] [
x 1 x 2
]
, θ ∈ (0, 2 π).
In R^3 , one can perform a rotation around a fixed axis. For instance, the rotation
T x =
cos θ − sin θ 0 sin θ cos θ 0 0 0 1
x 1 x 2 x 3
takes the z-axis as its axis of rotation.
Let us verify that Euclidean motions are closed under compositions. Let T x = Ax + b and Sx = Bx + c be two Euclidean motions. Its composition is given by
ST x = B(Ax + b) + c = Cx + d , C ≡ BA, d = Bb + c.
As
C′C = (BA)′BA = A′B′BA = A′IA = I,
we conclude that ST is again a Euclidean motion. Furthermore, we claim that each Euclidean motion admits an inverse. Indeed, letting U x = A−^1 x−A−^1 b which is obviously an Euclidean motion, we have
U T x = A−^1 (Ax + b) − A−^1 b = x.
Summing up, the collection of all Euclidean motions forms a group called the Euclidean group of Rn. (It is alright if you have not learned what a group is. You will learn it in MATH2070.)
In the following we study the structure of Euclidean motions for n = 2, 3. Apparently it suffices to look at the orthogonal matrix A.
1.3. EUCLIDEAN MOTIONS 13
Similarly
Lx =
and
Ly =
Theorem 1.4. * In R^3 , every orthogonal matrix can be written as
(a) Rz (α)Rx(β)Rz (γ) , or
(b) Rz (α)Rx(β)Rz (γ)Lz ,
for some α, β, and γ.
Proof. * Let A = (aij ) , i, j = 1, 2 , 3 , be orthogonal. We have
cos θ − sin θ 0 sin θ cos θ 0 0 0 1
a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33
cos θa 11 − sin θa 21 cos θa 12 − sin θa 22 cos θa 13 − sin θa 23 sin θa 11 + cos θa 21 sin θa 12 + cos θa 22 sin θa 13 + cos θa 23 a 31 a 32 a 33
Choose θ so that cos θa 13 − sin θa 23 = 0 and write the resulting matrix as
b 11 b 12 0 b 21 b 22 b 23 b 31 b 32 b 33
We further have (^)
0 cos ϕ − sin ϕ 0 sin ϕ cos ϕ
b 11 b 12 0 b 21 b 22 b 23 b 31 b 32 b 33
b 11 b 12 0 cos ϕb 21 − sin ϕb 31 cos ϕb 22 − sin ϕb 32 cos ϕb 23 − sin ϕb 33 sin ϕb 21 + cos ϕb 31 sin ϕb 22 + cos ϕb 32 sin ϕb 23 + cos ϕb 33
Choose ϕ so that cos ϕb 23 − sin ϕb 33 = 0 and write the resulting matrix as
c 11 c 12 0 c 21 c 22 0 c 31 c 32 c 33
14 CHAPTER 1. THE EUCLIDEAN SPACE
This matrix is the product of three orthogonal matrices, again it is orthogonal. Therefore, c 33 = ±1. Moreover, from
c 11 × 0 + c 21 × 0 + c 31 × c 33 = 0 ,
we deduce c 31 = 0. Similarly, c 32 = 0. The matrix is in fact of the form
c 11 c 12 0 c 21 c 22 0 0 0 ± 1
where the 2 × 2-matrix is orthogonal. It can be written as Rz (γ) or Rz (γ)Lz for some γ according to Proposition 1.3. We conclude that
Rx(ϕ)Rz (θ)A = Rz (γ) ,
or Rx(ϕ)Rz (θ)A = Rz (γ)Lz ,
that is, A = Rz (−θ)Rx(−ϕ)Rz (γ) ,
or A = Rz (−θ)Rx(−ϕ)Rz (γ)Lz.
The desired result follows by taking α = −θ and β = −ϕ.
1.4 The Cross Product in R^3
The cross product assigns a 3-vector to two given 3-vectors. There is no such product in the general dimension. Somehow it shows how special our physical space is. The cross product is important due to its relevance in physics and engineering.
Notations like x, y, u and v are common for vectors. We have used x, y in the previous sections. Now we use u, v in this one.
First the definition. Let u, v ∈ R^3 , define the cross product of u = (u 1 , u 2 , u 3 ) and v = (v 1 , v 2 , v 3 ) to be
u × v = (u 2 v 3 − u 3 v 2 , −(u 1 v 3 − u 3 v 1 ), u 1 v 2 − u 2 v 1 ).
In particular, we have
e 1 × e 2 = e 3 , e 2 × e 3 = e 1 , e 3 × e 1 = e 2.
16 CHAPTER 1. THE EUCLIDEAN SPACE
is where your thumb points to. To see this, one identifies the direction of u with e 1. If v lies in the first or second quadrants, v = αe 1 + βe 2 , β > 0, and u × v points to e 3. If v lies in the third or fourth quadrants, v = αe 1 + βe 2 , β < 0, and u × v points to −e 3. This is the right hand rule.
We have described the direction of the cross product. How about its magnitude? We have
Theorem 1.6. For u, v ∈ R^3 ,
|u × v| = |u||v| sin θ, θ ∈ [0, π] ,
where θ is the angle between u and v.
Proof. The proof depends on the identity
|u × v|^2 = |u|^2 |v|^2 − (u · v)^2.
Indeed, by brute force
|u × v|^2 = (u 2 v 3 − u 3 v 2 )^2 + (u 1 v 3 − u 3 v 1 )^2 + (u 1 v 2 − u 2 v 1 )^2
= u^22 v^23 + u^23 v 22 + u^21 v^23 + u^23 v^21 + u^21 v 22 + u^22 v^21 − 2 u 2 v 3 u 3 v 2 − 2 u 1 v 3 u 3 v 1 − 2 u 1 v 2 u 2 v 1.
On the other hand,
|u|^2 |v|^2 − (u · v)^2 = (u^21 + u^22 + u^23 )(v 12 + v^22 + v^23 ) − (u 1 v 1 + u 2 v 2 + u 3 v 3 )^2 = u^22 v^23 + u^23 v^22 + u^21 v 32 + u^23 v^21 + u^21 v^22 + u^22 v 12 − 2 u 2 v 3 u 3 v 2 − 2 u 1 v 3 u 3 v 1 − 2 u 1 v 2 u 2 v 1 ,
whence the identity holds. Now, by the Cosine Law,
|u × v| =
|u|^2 |v|^2 − |u|^2 |v|^2 cos^2 θ = |u||v|| sin θ| = |u||v| sin θ ,
as sin θ ≥ 0 on [0, π].
In conclusion the magnitude and direction of the decomposition of the cross product is given by u × v = |u||v| sin θ n ,
where n is the unit vector determined by the right hand rule (when u and v are linearly independent, that is, when sin θ 6 = 0).
Corollary 1.7. (a) The area of the parallelogram spanned by u and v is equal to |u × v|.
1.4. THE CROSS PRODUCT IN R^3
(b) The area of the triangle with two sides given by u and v is equal to 1 / 2 |u × v|.
(c) The volume of the parallelepiped spanned by u, v and w is equal to
V = |w · (u × v)|.
Proof. (a) follows immediately from Theorem 1.6 and (b) from (a). To prove (c), we may assume u and v lie on the xy-plane after a rotation. The volume of the parallelepiped is given by the product of the area of the parallelogram spanned by u and v with its height. Now |u × v| is equal to the area of this parallelogram. On the other hand, its height is given by |w · e 3 |. Therefore, letting α be the angle between w and the z-axis,
|w · (u × v)| = |w||u × v|| cos α| = |u × v| |w · e 3 | = V.
Example 1.5. Determine if the four points
(1, 0 , 1), (2, 4 , −6), (3, − 1 , 5), (1, − 9 , 19) ,
lie on the same plane in R^3. Well, they lie on the same plane if and only if the paral- lelepiped formed by these vectors has zero volume. We compute the volume using this corollary after subtracting the first vector from the last three vectors (to make sure that the vectors are based at the origin):
(1, 4 , −7) ·
(2, − 1 , 4) × (0, − 9 , 18)
so they lie on the same plane.
Comments on Chapter 1.
1.1. A helping hand from linear algebra. In several occasions we need to solve homoge- neous systems of linear equations. Let us review it by looking at some examples. First, consider the single equation
x − 2 y + 6z = 0, (x, y, z) ∈ R^3.
To solve this equation means to find all possible (x, y, z) satisfying this relation. Obviously (0, 0 , 0) is a solution, but there are many others, for instance, (− 6 , 0 , 1) and (2, 1 , 0) are also solutions. To find all solutions, we set y = a and z = b. Then x = 2a − 6 b, so (2a − 6 b, a, b) = a(2, 1 , 0) + b(− 6 , 0 , 1) gives all solutions. A solution is obtained whenever
1.4. THE CROSS PRODUCT IN R^3
physics can be also be found in Wikipedia under “cross product”. From the same source you will see how the cross product arises from the Lie algebra of the orthogonal group.
Supplementary Reading
1.1 and 1.2 in chapter 1, [Au].
