Foundations-Artificial Intelligence-Lecture Handout, Exercises of Artificial Intelligence

This assignment is for Artificial Intelligence course. It was assigned by Madam Amrita Ahuja at Central University of Jammu and Kashmir. Its main points are: Fortunate, Choices, Street, Groups, Strategy, Lagrange, Multipliers, Numerical, Methods, Kernel, Gutters

Typology: Exercises

2011/2012

Uploaded on 07/31/2012

shaina_44kin
shaina_44kin 🇮🇳

3.9

(9)

64 documents

1 / 22

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Foundations: fortunate choices
Unusual choice of separation strategy:
> Maximize “street” between groups
Attack maximization problem:
> Lagrange multipliers + hairy mathematics
New problem is a quadratic minimization:
> Susceptible to fancy numerical methods
Result depends on dot products only
> Enables use of kernel methods.
docsity.com
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16

Partial preview of the text

Download Foundations-Artificial Intelligence-Lecture Handout and more Exercises Artificial Intelligence in PDF only on Docsity!

Foundations: fortunate choices

  • Unusual choice of separation strategy:

> Maximize “street” between groups

  • Attack maximization problem:

> Lagrange multipliers + hairy mathematics

  • New problem is a quadratic minimization:

> Susceptible to fancy numerical methods

  • Result depends on dot products only

> Enables use of kernel methods.

Key idea: find widest separating “street”

-^

The constraints require:

-^

So, subtracting:

-^

Dividing by the length of

w

produces the distance betweenthe lines:

x^1

2

x

1 2

b^ b

x w

x w

1

⋅^

x

(x w

Distance between street’s gutters

w

x

(x w w

1

From maximizing to minimizing…

-^

So, to maximize the width of the street, you needto “wiggle” w until the length of w is minimum, while still honoring constraints on gutter values

-^

One possible approach to finding the minimum isto use the method devised by Lagrange. Workingthrough the method reveals how the sample datafigures into the classification formula.

separation

2

w

…while honoring constraints

  • Remember, the minimization is constrained • You can write the constraints as: Where y

i^

is 1 for plusses and –1 for minuses.

1

)

(^

⋅^

b

y

i

i^

x

w

Dependence on dot products

•^

Using LaGrange’s method, and working through somemathematics, you get to the following problem.

When

solved for the alphas, you then have what you need for theclassification formula.

b

y a

b

f

a

y a

y y a a

a

l ji

i i i

i

l i

i

i i

l ji

j i j i j i

l i

i

  • ⋅ = + ⋅ =

=

=

=

=

)

(

) (

of

sign

check

Then

0

and 0

to

Subject

1 2

Maximize

1 ,

1 ,

1

u x

u w

u

x x

Example

Another example

What you need

•^

To get

x

1

into the high-dimensional space, you use

•^

To optimize, you need

-^

To use, you need

-^

)( x (^1) So, all you need is a way to compute dot products in high-dimensional space as a function of vectors in originalspace! Φ

) ( ) (^

2

1

x

x^

Φ⋅

Φ

) ( ) 1 (^

u

x^

Φ⋅

Φ

What you don’t need

  • Suppose dot products are supplied by• Then, all you need is• Evidently, you don’t need to know what

Ф

is; having K is enough!

) 2 , 1 ( ) 2 ( ) 1 (^

x x

x

x^

K

Φ⋅

Φ

) 2 , 1 (^

x x K

Polynomial Kernel

Radial-basis kernel

Aside: about the hairy mathematics

  • Step 1: Apply method of Lagrange

multipliers

s

derivative

zero has

where

places

Find

s

constraint

subject to

minimize To

1

2

2

∑=

l i

i i i

i i b

y a

L

b

y w x

w

w x

w

docsity.com

Aside: about the hairy mathematics

Step 2: remember how to

differentiate vectors

Step 3: find derivatives of the Lagrangian L

x

w

w

x

w

w w

and

2

=

=

l i

i i

l i

i i i

y

a

L b

y

a

L

1

1

x

w

w