Local and Global Minima - Nonlinear Programming - Lecture Slides, Slides of Computer Science

Download Local and Global Minima - Nonlinear Programming - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

LOCAL AND GLOBAL MINIMA

f(x)

x Strict LocalMinimum Local Minima (^) MinimumStrict Global

Unconstrained local and global minima in one dimension.

NECESSARY CONDITIONS FOR A LOCAL MIN

• Zero slope at a local minimum x†∗

∇f (x†∗) = 0

• Nonnegative curvature at a local minimum x†∗

∇^2 f (x†∗) : Positive Semidefinite

x* = 0 x

f(x) = |x|^3 (convex)

x

f(x) = x^3 f(x) = - |x|^3

x* = 0 x* = 0 x

First and second order necessary optimality conditions for functions of one variable.

SUFFICIENT CONDITIONS FOR A LOCAL MIN

• Zero slope ∇f (x†∗) = 0
• Positive curvature

∇^2 f (x†∗) : Positive Definite

• Proof: Let λ > † 0 be the smallest eigenvalue of ∇^2 f (x†∗). Using a second order Taylor expansion, we have for all d†

1 f (x†∗^ + d) − f (x†∗) = ∇f (x†∗)′d†+ 2

d′∇^2 f (x†∗)d†

• o(‖d‖^2 )

≥

λ† 2

‖d‖^2 + o(‖d‖^2 )

λ o(‖d‖^2 ) = ‖d‖^2. 2

‖d‖^2

For ‖d‖ small enough, o(‖d‖^2 )/‖d‖^2 is negligible relative to λ/ 2.

αx + (1 - α)y, 0 < α < 1

CONVEXITY

x

x

y

x y y

x (^) y

Convex Sets Nonconvex Sets

Convex and nonconvex sets.

αf(x) + (1 - α)f(y)

x z

f(z) y C

A convex function.

OTHER PROPERTIES OF CONVEX FUNCTIONS

• f†is convex if and only if the linear approximation at a point x†∗^ based on the gradient, that is,

f (x) ≥ f (x†∗) + ∇f (x†∗)′(x†− x†∗),† ∀ x†

f(z) f(z) + (z - x)'∇f(x)

x z

− Implication:

∇f (x†∗) = 0 ⇒ x†∗^ is a global minimum

• f† is convex if and only if ∇^2 f (x) is positive semidefinite for all x†