Download Inner Product, Lecture Notes - Advanced Calculus and more Study notes Calculus in PDF only on Docsity!
Math 128A Problem Set 6: Inner Products
and Least Squares
Adrian Down
March 09, 2006
Problem 1
Part a
Let { ej } be a basis for Rn. Any vector in Rn^ can be written in terms of these basis vectors. Given any x, y ∈ Rn,
x =
i
xiei y =
j
yj ej
Any inner product must satisfy bilinearity. Hence any inner product defined on Rn^ must take the form of a linear combination of the components of n dimensional vectors,
〈x, y〉 =
i
j
cij xiyj (ei · ej )
where the constants cij are possible weighting factors. This double sum is indicative of matrix multiplication. I define the matrix,
A =
c 11 e 1 · e 1... c 1 n e 1 · en .. .
cn 1 en · e 1... cnn en · en
With this identification, I can write any inner product in terms of matrix multiplication,
〈x, y〉 = xT^ Ay
I take the transpose of x to ensure that the dimensions of the matrices are such that they can be multiplied together. Any inner product must be symmetric, that is
〈x, y〉 = 〈y, x〉 ⇔ xT^ Ay = yT^ Ax
The result of any inner product must be scalar. Taking the transpose of a scalar leaves the scalar unchanged, so I take the transpose of the right side of the above equation. I use the fact that (BC)T^ = CT^ BT^ for any matrices B and C, as well as the fact that (BT^ )T^ = B.
xT^ Ay =
yT^ Ax
)T
= xT^ AT^ (yT^ )T^ = xT^ AT^ y
Since this equality must hold in the case where x and y are not both 0, it must be that A = AT^. Hence A is symmetric. Finally, the inner product must be positive definite, meaning that
〈x, x〉 ≥ 0 and 〈x, x〉 = 0 ⇔ x = 0
In matrix notation, the requirement of positive definiteness becomes
xT^ Ax ≥ 0 and xT^ Ax = 0 ⇔ x = 0
Thus by definition, A is a positive definite matrix.
Part b
I write the inner product using the diagonalized version of A,
〈x, y〉 = xT^ SΩΩΩST^ y
I rewrite the vectors in the product above,
xT^ S =
ST^ x
)T
S−^1 x
)T
ST^ y = S−^1 y
Since S is invertible by assumption, I can make the definitions,
X = S−^1 x ⇔ x = SX XY = S−^1 y ⇔ y = SY
Now for λ = 3, ( 2 1 1 2
v 1 v 2
v 1 v 2
2 v 1 + v 2 = 3v 1 −v 1 + v 2 = 0 ⇒ v 1 = v 2
Hence the normalized eigenvector is
v =
The values of of w 1 and w 2 are 1 and 3, respectively, and the corresponding matrix S is,
S =
I examine the orthogonality of the vector X =
)T
. Forming the dot product with a general matrix y and setting the result equal to 0,
〈x, y〉 =
y 1 y 2
y 1 y 2
⇒ y 1 + 2y 2 = 0 ⇒ y 1 = − 2 y 2
Thus any vector of the form β
)T
where β ∈ R is orthogonal to x. See figure 1.
Problem 2
I consider the minimization of the function
f (t) = |ta − b|^2
−4−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
−
−
−
0
1
2
3
4 x vectors orthogonal to x
Figure 1: Non-standard orthogonality with alternative inner product.
with respect to t on a general vector space with arbitrary vectors a and b. Using the bilinearity and symmetry that any inner product must have,
f (t) = (ta − b) · (ta − b) = t^2 a · a − 2 a · b + b · b
Maximizing with respect to t, I take the derivative and set the result equal to 0. I call the value of t for which this minimum occurs tm,
f ′(tm) = 2tm a · a − 2 a · b = 0
⇒ tm =
a · b a · a To apply this abstract argument to the particular case given in the prob- lem, I need only compute a · a and a · b. I first identify the given vectors,
Again using Euler’s identity, I can write the sine function in terms of complex exponentials,
sin θ =
eıkθ^ − e−ıkθ 2 ı
I recognize this expression in the integral above,
−θ
eıkx^ =
k
sin kθ
Now I consider x as fixed and treat k as a variable. I take the second derivative of the above result with respect to k. I use the fact that the order of differentiation and integration can be interchanged to differentiate the argument of the integral,
−θ
(ıx)^2 eıkx^ =
∂^2
∂k^2
k
sin kθ
−θ
x^2 eıkx^ = −
∂^2
∂k^2
k
sin kθ
∂k
k
sin kθ +
k
θ cos kθ
k^3
sin kθ −
k^2
θ cos kθ −
k^2
θ cos kθ −
k
θ^2 sin kθ
k^3
sin kθ +
k^2
θ cos kθ +
k
θ^2 sin kθ
Returning to the integral at hand, I want to consider cos x, so I take k = 1. The range of integration is between ±π 2 , so I take θ = π 2. With these substitutions, I get,
− π 2
x^2 cos x dx = ℜ
π^2 2
π^2 2
Combining this with a previous result,
a · b =
2 − π 2
π^2 4
− x^2
cos x dx
π^2 2
π^2 2
I can now find the value of t which minimizes the squared deviation in the least squares approximation,
tm =
a · b a · a
π^5
π^5
−2^0 −1.5 −1 −0.5 0 0.5 1 1.5 2
1
Least squares approximation to cos x cos x 120/π^5 *(pi^2 /2 − x^2 )
Problem 3
Part a
In order to achieve the correct form of the squared deviation δ, I determine that the required form of the squared length of an arbitrary function η(x) is of the form,
|η(x)|^2 =
0
η′′(x)^2 dx + (η′(0) − η(0))^2 + (η′(π) + η(π))^2
to the trial function g, defining a new trial function,
Hence g satisfies the boundary conditions, and is thus the unique solution
Part b
I form the normal equations in matrix form,
( a 1 · a 1 a 1 · a 2 a 2 · a 1 a 2 · a 2
u 1 u 2
a 1 · b a 2 · b
The vector b is the actual solution to the least squares problem. In this case, b = T. The vectors a 1 and a 2 are the basis vectors, which in this case are given,
a 1 = 1 a 2 =
x −
π 2
The scalars u 1 and u 2 are the coefficients of the basis vectors in the least squares approximation. In this case,
u 1 = a u 2 = −b
I write the normal equations with these identifications,
( 1 · 1 1 ·
x − π 2
x − π 2
x − π 2
x − π 2
a −b
(^1 ·^ T
x − π 2
· T
I will need some derivatives and boundary terms, which I compute now,
a 1 = 1 a′ 1 = 0 a′′ 1 = 0 a′ 1 (0) − a 1 (0) = 0 − 1 = − 1 a′ 1 (π) + a 1 (π) = 0 + 1 = 1
a 2 =
x −
π 2
a′ 2 = 2
x −
π 2
a′′ 2 = 2
a′ 2 (0) − a 2 (0) = −π −
π^2 4 a′ 2 (π) + a 2 (π) = π +
π^2 4
I use the definition of the inner product given in §3, part b. I first com- pute the inner products on the right involving T. I assume that I have no knowledge of the exact solution for T. However, the boundary condi- tions are sufficient to evaluate these inner products. Since T ′(0) = T (0) and
- To match the boundary conditions satisfied by the desired solution, I add - g(x) = sin x + - g′(x) = cos x g′(0) = cos 0 = 1 g(0) = sin 0 + 1 = Evaluating the boundary conditions, - g′(π) = cos π = − 1 g(π) = sin π + 1 = - T (x) = sin x + to the boundary value problem, - 10 0.5 1 1.5 2 2.5
- Exact solution, T = sin x +
- 10 0.5 1 1.5 2 2.5
- Least squares approximation to sin x +
