Gradient Methods: A Comprehensive Guide to Optimization Techniques, Schemes and Mind Maps of Mathematics

Download Gradient Methods: A Comprehensive Guide to Optimization Techniques and more Schemes and Mind Maps Mathematics in PDF only on Docsity!

Gradient Methods

April 2004

Preview



Background



Steepest Descent



Conjugate Gradient

Background



Motivation



The gradient notion



The Wolfe Theorems

Motivation



The min(max) problem:



But we learned in calculus how to solve that

kind of question!

min f ( x )

x

Motivation- “real world” problem



Connectivity shapes (isenburg,gumhold,gotsman)



What do we get only from C without geometry?

mesh  { C ( V E , ), geometry }

Motivation- “real world” problem



First we introduce error functionals and then try

to minimize them:

 

2

3

( , )

n

s i j

i j E

E x x x







( , )

1

( )

i j i

i j E

i

L x x x

d



 



3 2

1

( ) ( )

n

n

r i

i

E x L x





 





Motivation- “real world” problem



Changing the parameter:

 

3

( , ) arg min 1 ( ) ( )

n

s r

x

E C   E x  E x







Motivation



General problem: find global min(max)



This lecture will concentrate on finding local

minimum.

f :=



( x , y )

















cos

1

2

x

















cos

1

2

y x

Directional Derivatives:

first, the one dimension

derivative:



v

f x y



 ( , )

2

v  R

v  1

Directional Derivatives :

In general direction…

Directional

Derivatives

x

f x y



 ( , )

y

f x y



 ( , )

























 

n

n

x

f

x

f

f ( x ,..., x ) : ,...,

1

1

f R R

n

: 

The Gradient: Definition

The Gradient Properties



The gradient defines (hyper) plane

approximating the function infinitesimally

y

y

f

x

x

f

z 





 





 