Maximum Likelihood Estimation - Introduction to Pattern Recognition - Lecture Slides, Slides of Advanced Algorithms

Download Maximum Likelihood Estimation - Introduction to Pattern Recognition - Lecture Slides and more Slides Advanced Algorithms in PDF only on Docsity!

Recap^ •^ To implement Bayes Classifier we need classconditional densities.

Recap^ •^ To implement Bayes Classifier we need classconditional densities.^ •^ We have considered maximum likelihood estimationfor parameters of a density and seen many examples.

Recap^ •^ To implement Bayes Classifier we need classconditional densities.^ •^ We have considered maximum likelihood estimationfor parameters of a density and seen many examples.^ •^ We have briefly looked at Bayesian estimation ofparameters.^ •^ In this class we discuss Bayesian estimation in moredetail.

Bayesian Estimation (Recap)^ •^ We think of the parameter as a random variable.

Bayesian Estimation (Recap)^ •^ We think of the parameter as a random variable.^ •^ We capture our lack of knowledge about the value ofa parameter through a probability density over theparameter space.^ •^ We call this the^ prior

density of the parameter.

Bayesian Estimation (Recap)^ •^ We think of the parameter as a random variable.^ •^ We capture our lack of knowledge about the value ofa parameter through a probability density over theparameter space.^ •^ We call this the^ prior

density of the parameter.

-^ Any information we may have about the value ofparameter can be incorporated into this.

Bayesian Parameter Estimation^ •^ As earlier, let^ θ^ be the parameter and let

D^ be the

data

Bayesian Parameter Estimation^ •^ As earlier, let^ θ^ be the parameter and let

D^ be the

data • Recall that^ D^ =

{x,^ · · ·^ , x}^1 n

is the set of^ iid^ data and each

xhas density^ f^ (x|^ θi^ i^

(which is the assumed model).

PR NPTEL course – p.11/

-^ Now, using Bayes theorem we get^ f^ (θ^ | D) =

f^ (D |^ θ)f^ (θ) ∫ f^ (D |^ θ)f^ (θ)^ dθ

∏ where f (D | θ) = i^

f^ (x|^ θ)^ is the data likelihoodi^

that we considered earlier.

-^ A form for the prior density, that results in the sameform of density for the posterior is called

conjugate prior.

-^ A form for the prior density, that results in the sameform of density for the posterior is called

conjugate

prior. • Posterior density depends on product of prior anddata likelihood. • The form of data likelihood depends on the formassumed for^ f^ (x^ |^ θ).

-^ A form for the prior density, that results in the sameform of density for the posterior is called

conjugate

prior. • Posterior density depends on product of prior anddata likelihood. • The form of data likelihood depends on the formassumed for^ f^ (x^ |^ θ). • Hence the conjugate prior is determined by the theform of^ f^ (x^ |^ θ)^ (and hence that of data likelihood).

-^ When we use conjugate prior, the prior and posteriorwould belong to the same class of densities. •^ Hence calculating posterior would be like updatingparameter values.

-^ When we use conjugate prior, the prior and posteriorwould belong to the same class of densities. •^ Hence calculating posterior would be like updatingparameter values. •^ We consider a few examples of Bayesian estimationnow.