Linear Programming Non-Linear Optimization, Lecture Notes - Mathematics, Study notes of Linear Programming

Linear Programming Non-Linear Optimization, Lecture Notes - Mathematics - Prof. J Vanderbei.pdf, Prof. J Vanderbei, Mathematics, Linear Programming, Non-Linear Optimization, The Interior-Point Algorithm, Reduced KKT System, Convex Optimization Models, FIR Filter Design, [Woofer, Midrange, Tweeter], Celestial Mechanics, Goddard Rocket Problem

Typology: Study notes

2010/2011

Uploaded on 11/02/2011

aeinstein
aeinstein 🇺🇸

4.6

(22)

259 documents

1 / 33

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Home Page
Title Page
Contents
JJ II
JI
Page 1of 33
Go Back
Full Screen
Close
Quit
Nonlinear Optimization:
Algorithms and Models
Robert J. Vanderbei
December 12, 2005
ORF 522
Operations Research and Financial Engineering, Princeton University
http://www.princeton.edu/rvdb
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21

Partial preview of the text

Download Linear Programming Non-Linear Optimization, Lecture Notes - Mathematics and more Study notes Linear Programming in PDF only on Docsity!

Home Page

Title Page

Contents

JJ II

J I

Page 1 of 33

Go Back

Full Screen

Close

Quit

Nonlinear Optimization:

Algorithms and Models

Robert J. Vanderbei

December 12, 2005 ORF 522

Operations Research and Financial Engineering, Princeton University http://www.princeton.edu/∼rvdb

Home Page

Title Page

Contents

JJ II

J I

Page 2 of 33

Go Back

Full Screen

Close

. Outline

  • Algorithm
    • Basic Paradigm
    • Step-Length Control
    • Diagonal Perturbation
  • Convex Problems
    • Minimal Surfaces
    • Digital Audio Filters
  • Nonconvex Problems
    • Celestial Mechanics
    • Putting on an Uneven Green
    • Goddard Rocket Problem

Home Page

Title Page

Contents

JJ II

J I

Page 4 of 33

Go Back

Full Screen

Close

. Introduce Slack Variables

  • Start with an optimization problem—for now, the simplest NLP:

minimize f (x) subject to hi(x) ≥ 0 , i = 1,... , m

  • Introduce slack variables to make all inequality constraints into nonnegativities:

minimize f (x) subject to h(x) − w = 0, w ≥ 0

Home Page

Title Page

Contents

JJ II

J I

Page 5 of 33

Go Back

Full Screen

Close

. Associated Log-Barrier Problem

  • Replace nonnegativity constraints with logarithmic barrier terms in the objective:

minimize f (x) − μ

∑^ m

i=

log(wi)

subject to h(x) − w = 0

Home Page

Title Page

Contents

JJ II

J I

Page 7 of 33

Go Back

Full Screen

Close

. Symmetrize Complementarity Condi-

tions

  • Rewrite system:

∇f (x) − ∇h(x)T^ y = 0 W Y e = μe h(x) − w = 0

Home Page

Title Page

Contents

JJ II

J I

Page 8 of 33

Go Back

Full Screen

Close

. Apply Newton’s Method

  • Apply Newton’s method to compute search directions, ∆x, ∆w, ∆y:

H(x, y) 0 −A(x)T 0 Y W A(x) −I 0

∆x ∆w ∆y

−∇f (x) + A(x)T^ y μe − W Y e −h(x) + w

Here,

H(x, y) = ∇^2 f (x) −

∑^ m

i=

yi∇^2 hi(x)

and A(x) = ∇h(x)

  • Note: H(x, y) is positive semidefinite if f is convex, each hi is concave, and each yi ≥ 0.

Home Page

Title Page

Contents

JJ II

J I

Page 10 of 33

Go Back

Full Screen

Close

. Convex vs. Nonconvex Optimization

Probs

Nonlinear Programming (NLP)

minimize f (x) subject to hi(x) = 0, i ∈ E, hi(x) ≥ 0 , i ∈ I. NLP is convex if

  • hi’s in equality constraints are affine;
  • hi’s in inequality constraints are concave;
  • f is convex; NLP is smooth if
  • All are twice continuously differentiable.

Home Page

Title Page

Contents

JJ II

J I

Page 11 of 33

Go Back

Full Screen

Close

. Modifications for Convex Optimiza-

tion

For convex nonquadratic optimization, it does not suffice to choose the steplength α simply to maintain positivity of nonnegative vari- ables.

  • Consider, e.g., minimizing

f (x) = (1 + x^2 )^1 /^2.

  • The iterates can be computed explicitly:

x(k+1)^ = −(x(k))^3

  • Converges if and only if |x| ≤ 1.
  • Reason: away from 0 , function is too linear.

Home Page

Title Page

Contents

JJ II

J I

Page 13 of 33

Go Back

Full Screen

Close

. Nonconvex Optimization: Diagonal

Perturbation

  • If H(x, y) is not positive semidefinite then N (x, y, w) might fail to be positive definite.
  • In such a case, we lose the descent properties given in previous theorem.
  • To regain those properties, we perturb the Hessian: H˜(x, y) = H(x, y) + λI.
  • And compute search directions using H˜ instead of H.

Notation: let N˜ denote the dual normal matrix associated with H˜.

Theorem If N˜ is positive definite, then (∆x, ∆w, ∆y) is a descent direction for

  1. the primal infeasibility, ‖h(x) − w‖;
  2. the noncomplementarity, wT^ y.

Home Page

Title Page

Contents

JJ II

J I

Page 14 of 33

Go Back

Full Screen

Close

. Notes:

  • Not necessarily a descent direction for dual infeasibility.
  • A line search is performed to find a value of λ within a factor of 2 of the smallest permissible value.

Home Page

Title Page

Contents

JJ II

J I

Page 16 of 33

Go Back

Full Screen

Close

. Modifications for General Problem

Formulations

  • Bounds, ranges, and free variables are all treated implicitly as described in Linear Programming: Foundations and Extensions (LP:F&E).
  • Net result is following reduced KKT system: [ −(H(x, y) + D) AT^ (x) A(x) E

] [

∆x ∆y

]

[

]

  • Here, D and E are positive definite diagonal matrices.
  • Note that D helps reduce frequency of diagonal perturbation.
  • Choice of barrier parameter μ and initial solution, if none is provided, is described in the paper.
  • Stopping rules, matrix reordering heuristics, etc. are as described in LP:F&E.

Home Page

Title Page

Contents

JJ II

J I

Page 17 of 33

Go Back

Full Screen

Close

Examples: Convex Optimization Models

Home Page

Title Page

Contents

JJ II

J I

Page 19 of 33

Go Back

Full Screen

Close

. Specific Example

Scherk.mod with D discretized into a 64 × 64 grid gives the following results: constraints 0 variables 3844 time (secs) loqo 5. lancelot 4. snopt *

Home Page

Title Page

Contents

JJ II

J I

Page 20 of 33

Go Back

Full Screen

Close

. Finite Impulse Response (FIR) Filter

Design

  • Audio is stored digitally in a computer as a stream of short integers: uk, k ∈ Z.
  • When the music is played, these integers are used to drive the displacement of the speaker from its resting position.
  • For CD quality sound, 44100 short integers get played per second per channel.

0 - 1 - 2 - 3 - 4 - 5 - 6 - 7 -