AdaBoost Algorithm - Introduction to Pattern Recognition - Lecture Slides, Slides of Design and Analysis of Algorithms

Download AdaBoost Algorithm - Introduction to Pattern Recognition - Lecture Slides and more Slides Design and Analysis of Algorithms in PDF only on Docsity!

• We have been discussing methods to combinemultiple classifiers.

• We have been discussing methods to combinemultiple classifiers. - Boosting is a general approach to designing suchclassifier ensembles.

• We have been discussing methods to combinemultiple classifiers. - Boosting is a general approach to designing suchclassifier ensembles. - We considered AdaBoost, a very popular method todesign classifier ensembles. - Let us briefly review the algorithm again.

AdaBoost Algorithm

1. Initialize: w i

(^1) n

i .

AdaBoost Algorithm

1. Initialize: w i

(^1) n

i .

2. For m=1 to M do a. Generate a training set by sampling with

w i

m

. PR NPTEL course – p.7/

AdaBoost Algorithm

1. Initialize: w i

(^1) n

i .

2. For m=1 to M do a. Generate a training set by sampling with

w i

m

. b. Learn classifier h m using this training set. PR NPTEL course – p.8/

AdaBoost Algorithm

1. Initialize: w i

(^1) n

i .

2. For m=1 to M do a. Generate a training set by sampling with

w i

m

. b. Learn classifier h m using this training set. c. Let ξ m

n ∑ i = w i

m

I

[y i 6 = h m ( X i )] where

I

A is indicator of

A

. We assume ( ξ m

). d. Set α m = ln

1 − ξ m ξ m

. (We have α m

). PR NPTEL course – p.10/

Algorithm contd.

e. Update the weights by w ′ i

m

w i

m ) exp

α m

I

[ y i 6 = h m ( X i )]

w i

m

w ′ i

m

i w ′ i

m

PR NPTEL course – p.11/

• This algorithm allows one to learn a set of classifiersfor a problem.

• This algorithm allows one to learn a set of classifiersfor a problem. -

M

is a parameter. Due to the sampling with weights, we can continue the procedure for arbitrary number ofiterations.

• This algorithm allows one to learn a set of classifiersfor a problem. -

M

is a parameter. Due to the sampling with weights, we can continue the procedure for arbitrary number ofiterations.

• After learning each classifier, we update the weights. - The updating is such thatif h m

X

i

y i , then w i

m

w i

m

.

• This algorithm allows one to learn a set of classifiersfor a problem. -

M

is a parameter. Due to the sampling with weights, we can continue the procedure for arbitrary number ofiterations.

• After learning each classifier, we update the weights. - The updating is such thatif h m

X

i

y i , then w i

m

w i

m

.

• If the current classifier misclassifies a pattern, itsweight for the next iteration is increased.

• We thus ensure that higher weightage is given topreviously misclassified examples while sampling forthe training set of next classifier. - However, this does not mean we choose only suchexamples next time.

• We thus ensure that higher weightage is given topreviously misclassified examples while sampling forthe training set of next classifier. - However, this does not mean we choose only suchexamples next time. - We also want to maintain proper variability in thesuccessive training sets generated.