Cost Function-Machine Learning and Artificial Intelligence-Lecture Slides, Slides of Machine Learning

Download Cost Function-Machine Learning and Artificial Intelligence-Lecture Slides and more Slides Machine Learning in PDF only on Docsity!

Neural Networks:

Learning

Cost func5on

Machine Learning

Andrew Ng

Neural Network (Classifica2on)

Binary classifica5on

1 output unit

Layer 1 Layer 2 Layer 3 Layer 4

Mul5-­‐class classifica5on (K classes)

K output units

total no. of layers in network no. of units (not coun5ng bias unit) in layer pedestrian car motorcycle truck E.g. , , ,

Neural Networks: Learning Backpropaga5on algorithm

Machine Learning

Gradient computa2on

Need code to compute:

Gradient computa2on: Backpropaga2on algorithm

Intui5on: “error” of node in layer.

Layer 1 Layer 2 Layer 3 Layer 4

For each output unit (layer L = 4)

Backpropaga2on algorithm

Training set Set (for all ).

For

Set

Perform forward propaga5on to compute for

Using , compute

Compute

Forward Propaga2on

Andrew Ng

Forward Propaga2on

Andrew Ng

Forward Propaga2on

“error” of cost for (unit in layer ).

Formally, (for ), where

Neural Networks: Learning Implementa5on note: Unrolling parameters

Machine Learning

Andrew Ng

Example

thetaVec = [ Theta1(:); Theta2(:); Theta3(:)]; DVec = [D1(:); D2(:); D3(:)]; Theta1 = reshape(thetaVec(1:110),10,11); Theta2 = reshape(thetaVec(111:220),10,11); Theta3 = reshape(thetaVec(221:231),1,11);

Andrew Ng Have ini5al parameters. Unroll to get initialTheta to pass to fminunc(@costFunction, initialTheta, options)

Learning Algorithm

function [jval, gradientVec] = costFunction(thetaVec)

From thetaVec, get.

Use forward prop/back prop to compute

and.

Unroll to get gradientVec.

Andrew Ng

Numerical es2ma2on of gradients

Implement: gradApprox = (J(theta + EPSILON) – J(theta –

EPSILON)) /(2EPSILON)*

Andrew Ng

Parameter vector

(E.g. is “unrolled” version of )