Nonlinear Functions - Introduction to Pattern Recognition - Lecture Slides, Slides of Advanced Algorithms

Download Nonlinear Functions - Introduction to Pattern Recognition - Lecture Slides and more Slides Advanced Algorithms in PDF only on Docsity!

• We have considered multilayer feedforward networksfor classification and regression. PR NPTEL course – p.1/

• We have considered multilayer feedforward networksfor classification and regression. - We can learn general nonlinear functions with such arepresentation. PR NPTEL course – p.2/

• We have considered multilayer feedforward networksfor classification and regression. - We can learn general nonlinear functions with such arepresentation. - There are other neural network models which canalso approximate any continuous function. - We consider the so called Radial Basis functio (RBF)networks now. PR NPTEL course – p.4/

• Consider a 3-layer feedforward network with m input nodes, one output node and p hidden nodes. PR NPTEL course – p.5/

• Consider a 3-layer feedforward network with m input nodes, one output node and p hidden nodes. - In the multilayer networks we considered, each nodecomputes a weighted sum of its inputs and passes itthrough a sigmoid to calculate its output. - Then we can write the output of the network asfollows. y

p ∑^ j = β j f

m ∑ i = w ij x i

b j

PR NPTEL course – p.7/

• We can rewrite this as y

p ∑^ j = β j φ

X, θ j

PR NPTEL course – p.8/

• We can rewrite this as y

p ∑^ j = β j φ

X, θ j

• This also represents the output of a 3-layer networkwhere the output of j th hidden node is given by φ

X, θ j

.

• We consider such networks now. PR NPTEL course – p.10/

• We take the function φ to be radially symmetric. PR NPTEL course – p.11/

• We take the function φ to be radially symmetric. - That is, we take the parameter vector, θ j to be of the same dimension as

X

and take the function to be φ

X

θ j

.

• Now we write the output of the network as y

p ∑^ j = β j φ

X

θ j

PR NPTEL course – p.13/

• We take the function φ to be radially symmetric. - That is, we take the parameter vector, θ j to be of the same dimension as

X

and take the function to be φ

X

θ j

.

• Now we write the output of the network as y

p ∑^ j = β j φ

X

θ j

• The motivation comes from techniques forinterpolation. PR NPTEL course – p.14/

• Suppose we have data

X

i , d i

, i

, N

where

X

i

m , d i

.

• We want a smooth function to interpolate this data. PR NPTEL course – p.16/

• Suppose we have data

X

i , d i

, i

, N

where

X

i

m , d i

.

• We want a smooth function to interpolate this data. - In our context, the question is: do there exist some β j , θ j and a φ so that the function h

X

p ∑^ j = β j φ

X

θ j

satisfies h

X

i

d i for i

, N

. PR NPTEL course – p.17/

• It turns out that if we take p

N

and θ j

X

j , then there are many functions φ which can achieve such perfect interpolation.

• That is, we take h

X

N ∑^ j = w j φ

X

X

j

)^ PR NPTEL course – p.19/

• It turns out that if we take p

N

and θ j

X

j , then there are many functions φ which can achieve such perfect interpolation.

• That is, we take h

X

N ∑^ j = w j φ

X

X

j

• We want, h

X

i

d i for i

, N

. PR NPTEL course – p.20/