Conditional Densities - Introduction to Pattern Recognition - Lecture Slides, Slides of Design and Analysis of Algorithms

Download Conditional Densities - Introduction to Pattern Recognition - Lecture Slides and more Slides Design and Analysis of Algorithms in PDF only on Docsity!

Recap

To implement Bayes Classifier we need classconditional densities.

Recap

To implement Bayes Classifier we need classconditional densities.

Two main approaches to estimating densities –Parametric and non-parametric

Recap

To implement Bayes Classifier we need classconditional densities.

Two main approaches to estimating densities –Parametric and non-parametric

In the parametric method we assume that the form ofthe density is known and estimate the parameters.

Maximum likelihood method is a general procedurefor obtaining consistent estimators for parameters.

Recap

Maximum Likelihood (ML) estimate is the maximizerof the likelihood (or log likelihhod) function.

Recap

Maximum Likelihood (ML) estimate is the maximizerof the likelihood (or log likelihhod) function.

For most standard density models, one cananalytically derive ML estimates.

We have seen some examples of obtaining MLestimates.

Recap

Maximum Likelihood (ML) estimate is the maximizerof the likelihood (or log likelihhod) function.

For most standard density models, one cananalytically derive ML estimates.

We have seen some examples of obtaining MLestimates.

We now see more examples of ML estimates.

Example

Suppose the assumed density for

x

is exponential

f

x

λ

λ

exp(

λx

, x

Example

Suppose the assumed density for

x

is exponential

f

x

λ

λ

exp(

λx

, x

Given

iid

data,

D

x

1

, x

n

, we need to

estimate

λ

.

Example

The log likelihood function is

l

λ

| D

n

i

=

(ln(

λ

λx

i

Example

The log likelihood function is

l

λ

| D

n

i

=

(ln(

λ

λx

i

Differentiating w.r.t.

λ

and equating to zero, we get

n λ

n

i

=

x

i

This gives us the final ML estimate as

λ

n

n i

=

x

i

The final estimate is intuitively clear.(Note that

Ex

1 λ

).

Another Example

Consider the multidimensional Gaussian density f

x

θ

π

d

exp

x

μ

T

−

1

x

μ

where

x

d

and

θ

μ,

are the parameters.

PR NPTEL course – p.17/

To find the ML estimate for the parameters, we haveto maximise the log likelihood.

To find the ML estimate for the parameters, we haveto maximise the log likelihood.

Recall that the log likelihood function is defined by

l

θ

| D

n

i

=

f

x

i

θ

where

D

x

1

, x

n

constitutes the

iid

data from

which we are estimating the parameters of the density.

PR NPTEL course – p.20/