Continuous Function - Introduction to Pattern Recognition - Lecture Slides, Slides of Design and Analysis of Algorithms

Download Continuous Function - Introduction to Pattern Recognition - Lecture Slides and more Slides Design and Analysis of Algorithms in PDF only on Docsity!

• We have been considering RBF networks. PR NPTEL course – p.1/

• We have been considering RBF networks. - These are 3-layer networks that can approximate anycontinuous function through a basis functionexpansion. PR NPTEL course – p.2/

• We have been considering RBF networks. - These are 3-layer networks that can approximate anycontinuous function through a basis functionexpansion. - The basis functions here (which are data dependentas earlier) exhibit some radial symmetry. - These networks have the so called perfectinterpolation property. PR NPTEL course – p.4/

• The function represented by an RBF network with p hidden nodes can be written as y

p ∑^ j = w j φ

X

θ j

PR NPTEL course – p.5/

• The function represented by an RBF network with p hidden nodes can be written as y

p ∑^ j = w j φ

X

θ j

where

X

is the input to the network.

• w j is weight from j th hidden node to the output. PR NPTEL course – p.7/

• The function represented by an RBF network with p hidden nodes can be written as y

p ∑^ j = w j φ

X

θ j

where

X

is the input to the network.

• w j is weight from j th hidden node to the output.

φ

X

θ j

is the output of the j th hidden node and θ j is the parameter vector associated with j th hidden node, j

, p . PR NPTEL course – p.8/

• A very popular model is the Gaussian RBF network. - Here the output is written as y

p ∑^ j = w j exp

X

θ j

2 2 σ 2

PR NPTEL course – p.10/

• A very popular model is the Gaussian RBF network. - Here the output is written as y

p ∑^ j = w j exp

X

θ j

2 2 σ 2

• The θ j is called the center of the j th hidden or RBF node and σ is called the width. PR NPTEL course – p.11/

• We next consider learning the parameters of a RBFnetwork from training samples. PR NPTEL course – p.13/

• We next consider learning the parameters of a RBFnetwork from training samples. - Let

X

i , d i

, i

, N

be the training set. PR NPTEL course – p.14/

• We next consider learning the parameters of a RBFnetwork from training samples. - Let

X

i , d i

, i

, N

be the training set.

• Suppose we are using the Gaussian RBF. - Then we need to learn the centers ( θ j ) and widths ( σ ) of the hidden nodes and the weights into the outputnode ( w j ). PR NPTEL course – p.16/

• Like earlier, we can find parameters to minimizeempirical risk under squared error loss function. PR NPTEL course – p.17/

• Like earlier, we can find parameters to minimizeempirical risk under squared error loss function. - Same as minimizing sum of squares of errors. Let

J

N ∑ i =

p ∑^ j = w j exp

X

i

θ j

2 2 σ 2

d i

2

J

is a function of σ , w j , θ j , j

, p . PR NPTEL course – p.19/

• We can find the weights/parameters of the network tominimize

J

. PR NPTEL course – p.20/