Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
These are the Lecture Slides of Nonlinear Programming which includes Convex Cost, Linear Constraints, Duality Theorem, Linear Programming Duality, Quadratic Programming Duality, Linear Inequality, Constrained Problem, Minimize, Feasible etc.Key important points are: Gradient Methods, Quadratic Unconstrained Problems, Existence of Optimal Solutions, Iterative Computational Methods, Gradient Methods -Motivation, Principal Gradient Methods, Gradient Methods, Choices of Direction, Symmetric, Neces
Typology: Slides
1 / 8
This page cannot be seen from the preview
Don't miss anything!
min f (x) = 12 x′Qx − b′x, x∈n† where Q is n × n symmetric, and b ∈ n.
∇f (x∗) = Qx∗^ − b = 0,
∇^2 f (x∗) = Q ≥ 0 : positive semidefinite.
− Q : not ≥ 0 ⇒ f has no local minima − If Q > 0 (and hence invertible), x∗^ = Q−^1 b is the unique global minimum. − If Q ≥ 0 but not invertible, either no solution or ∞ number of solutions
Consider min f (x) x∈X
Two possibilities:
− A global minimum exists if f is continuous and X is compact (Weierstrass theorem) − A global minimum exists if X is closed, and f is coercive, that is, f (x) → ∞ when ‖x‖ → ∞
f(x) = (^1)
f(x) = 2 < c 1
f(x) = 3 < c 2
x c c
c x - δ∇f(x)
xα = x + αd
x + δd d
f(x) = (^1) f(x) = 2 < c 1
f(x) = 3 < c 2
x
∇f(x)
c c
c
∇f(x)
xα = x - α∇f(x)
If ∇f (x) = 0, there is an interval (0, δ) of stepsizes such that
f x†− α∇f (x) < f †(x)
for all α†∈ (0, δ).
If d† makes an angle with ∇f (x) that is greater than 90 degrees,
∇f†(x)�d < † 0 ,†
there is an interval (0, δ) of stepsizes such that f†(x+ αd) < f (x†) for all α† ∈ (0, δ).
x^0
x^0 x^1 x^2
f(x) = (^1)
f(x) = 3 < c 2
f(x) = 2 < c 1 .
.
.
Quadratic Approximation of f at x 0
Quadratic Approximation of f at x 1
c
c
c
Slow convergence of steep est descent
Fast convergence of New- ton’s method w/ αk†^ = 1. Given xk†, the method ob tains xk+1^ as the minimum of a quadratic approxima tion of f† based on a sec ond order Taylor expansion around xk†.
Dk^ = Diagonal approximation to
∇^2 f (xk)
Dk^ = (∇^2 f (x^0 ))−^1 , k = 0, 1 ,... ,
Dk^ =
H(xk) , k = 0, 1 ,... ,
where H(xk) is a finite-difference based approxi- mation of ∇^2 f (xk),
)− 1 Dk^ = ∇g(xk)∇g(xk)′^ , k = 0, 1 ,...
