Homework Set 2 Problems - Nonlinear Optimization | MATH 408 | Assignments Mathematics

Math 408 Homework Set 2

(1) Let Rm×nbe the set of real m×nmatrices. Let b∈Rmand A∈Rm×nbe given and define

f:Rn→Rby

f(x) = 1

2kAx −bk2

Minimizing a function of this form is called a linear least squares problem.

(a) Show that ∇f(x) = AT(Ax −b). Hint: f(x) = 1

2(Ax −b)T(Ax −b).

(b) Show that ∇2f(x) = ATA.

M∈Rn×nis said to be positive semi-definite if xTM x ≥0 for all x∈Rn.)

(2) Consider the data points (x, y)∈R, (1,1),(2,0),(−1,2), and (0,−1). We wish to determine a

real polynomial of degree 2 that best fits this data. A general real polynomial of degree 2 has the

form p(λ) = x0+x1λ+x2λ2, where x= (x0, x1, x2)T∈R3. Note that there are more data points

that there are unknown coefficients x0, x1, and x2and so it is unlikely that there exists a second

degree polynomial that fits this data precisely. One approach to finding a polynomial that best fits

this data is to chose x0, x1, and x2so as to minimize the sum of squares error, that is to minimize

the function

f(x)=(p(1) −1)2+ (p(2) −0)2+ (p(−1) −2)2+ (p(0) + 1)2

= (x0+x1+x2−1)2+ (x0+ 2x1+ 4x2)2+ (x0−x1+x2−2)2+ (x0+ 1)2.

(a) Show that this function can be written if the form described in problem (1) above by specifying

an appropriate matrix Aand vector b.

(b) Compute ∇f(x) and ∇2f(x) and show that the formulas derived for these objects in problem

(1) are valid.

(3) Let (ui, vi)∈R2, i = 1,2, . . . , m be any collection of data points in R2. We wish to fit a real

polynomial of degree k < m to this data. Since there are more data points than real coefficients, it

is unlikely that we will be able to determine a polynomial that precisely fits this data. We again

approach this problem by minimizing the sum of squares error. Show that this problem can be

posed as a linear least squares problem and describe the corresponding matrix Aand vector b.

Hint: consider p(λ:x) = x0+x1λ+x2λ2+· · · +xkλk.

(4) A function f:Rn→Ris said to be quadratic if there exists α∈R,g∈Rn, and H∈Rn×nsuch

that f(x) = α+gTx+1

2xTHx .

(a) Show that the functions described in problem (1) are quadratic by specifying the values for α,

gand H.

(b) Show that ∇f(x) = 1

2(H+HT)x+gand ∇2f(x) = 1

2(H+HT).

α+gTx+1

2xT(H+HT)xand (H+HT) is symmetric.

(5) Find the global minimizers and maximizers, if they exist, for the following functions.

(a) f(x) = x2

1−4x1+ 2x2

2+ 7

(b) f(x) = e−kxk2

1−2x1x2+1

3x3

2−4x2

(d) f(x) = (2x1−x2)2+ (x2−x3)2+ (x3−1)2

(e) f(x) = x4

1+ 16x1x2+x8

(f) f(x) = Pn−1

j=1 10j(xj−x2

j+1)2(The Rosenbrock function)

Partial preview of the text

Download Homework Set 2 Problems - Nonlinear Optimization | MATH 408 and more Assignments Mathematics in PDF only on Docsity!

Math 408 Homework Set 2

(1) Let Rm×n^ be the set of real m × n matrices. Let b ∈ Rm^ and A ∈ Rm×n^ be given and define f : Rn^ → R by f (x) =

‖Ax − b‖^22. Minimizing a function of this form is called a linear least squares problem. (a) Show that ∇f (x) = AT^ (Ax − b). Hint: f (x) = 12 (Ax − b)T^ (Ax − b). (b) Show that ∇^2 f (x) = AT^ A. (c) Show that AT^ A is always a symmetric positive semi-definite matrix. (Recall that a matrix M ∈ Rn×n^ is said to be positive semi-definite if xT^ M x ≥ 0 for all x ∈ Rn.) (2) Consider the data points (x, y) ∈ R, (1, 1), (2, 0), (− 1 , 2), and (0, −1). We wish to determine a real polynomial of degree 2 that best fits this data. A general real polynomial of degree 2 has the form p(λ) = x 0 + x 1 λ + x 2 λ^2 , where x = (x 0 , x 1 , x 2 )T^ ∈ R^3. Note that there are more data points that there are unknown coefficients x 0 , x 1 , and x 2 and so it is unlikely that there exists a second degree polynomial that fits this data precisely. One approach to finding a polynomial that best fits this data is to chose x 0 , x 1 , and x 2 so as to minimize the sum of squares error, that is to minimize the function f (x) = (p(1) − 1)^2 + (p(2) − 0)^2 + (p(−1) − 2)^2 + (p(0) + 1)^2 = (x 0 + x 1 + x 2 − 1)^2 + (x 0 + 2x 1 + 4x 2 )^2 + (x 0 − x 1 + x 2 − 2)^2 + (x 0 + 1)^2. (a) Show that this function can be written if the form described in problem (1) above by specifying an appropriate matrix A and vector b. (b) Compute ∇f (x) and ∇^2 f (x) and show that the formulas derived for these objects in problem (1) are valid. (3) Let (ui, vi) ∈ R^2 , i = 1, 2 ,... , m be any collection of data points in R^2. We wish to fit a real polynomial of degree k < m to this data. Since there are more data points than real coefficients, it is unlikely that we will be able to determine a polynomial that precisely fits this data. We again approach this problem by minimizing the sum of squares error. Show that this problem can be posed as a linear least squares problem and describe the corresponding matrix A and vector b. Hint: consider p(λ : x) = x 0 + x 1 λ + x 2 λ^2 + · · · + xkλk. (4) A function f : Rn^ → R is said to be quadratic if there exists α ∈ R, g ∈ Rn, and H ∈ Rn×n^ such that f (x) = α + gT^ x + 12 xT^ Hx. (a) Show that the functions described in problem (1) are quadratic by specifying the values for α, g and H. (b) Show that ∇f (x) = 12 (H + HT^ )x + g and ∇^2 f (x) = 12 (H + HT^ ). (c) Show that we may as well assume that H is a symmetric matrix by showing that f (x) = α + gT^ x + 12 xT^ (H + HT^ )x and (H + HT^ ) is symmetric. (5) Find the global minimizers and maximizers, if they exist, for the following functions. (a) f (x) = x^21 − 4 x 1 + 2x^22 + 7 (b) f (x) = e−‖x‖ 2 (c) f (x) = x^21 − 2 x 1 x 2 + 13 x^32 − 4 x 2 (d) f (x) = (2x 1 − x 2 )^2 + (x 2 − x 3 )^2 + (x 3 − 1)^2 (e) f (x) = x^41 + 16x 1 x 2 + x^82 (f) f (x) =

∑n− 1 j=1 10

j (^) (xj − x 2 j+1) (^2) (The Rosenbrock function)

Homework Set 2 Problems - Nonlinear Optimization | MATH 408, Assignments of Mathematics

Related documents

Partial preview of the text

Download Homework Set 2 Problems - Nonlinear Optimization | MATH 408 and more Assignments Mathematics in PDF only on Docsity!

Math 408 Homework Set 2