Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
This lecture was delivered by Dr. Asad Raza at Pakistan Institute of Engineering and Applied Sciences, Islamabad (PIEAS) for Pattern Classification and Recognition. It includes: Pattern, Recognition, Bayes, Functions, Distributions, Probability, Bayesian, Expectation, Maximistation, Nonparpametric, Parzen, Classification
Typology: Slides
1 / 71
This page cannot be seen from the preview
Don't miss anything!
2
^
Typical application areas^
Machine vision Character recognition (OCR) Computer aided diagnosis Speech recognition Face recognition Biometrics Image Data Base retrieval Data mining Bionformatics ^
The task
:^ Assign unknown objects – patterns – into the correct
class.
This is known as classification.
3
Features:
These are measurable quantities obtained from
the patterns, and the classification task is based on theirrespective values.
Feature vectors
:^ A number of features
constitute the feature vectorFeature vectors are treated as random vectors.
,
,..., 1
xl
x^ ^
^
l
T l
x
x x^
5
^
The classifier consists of a set of functions, whose values,computed at
, determine the class to which the
corresponding pattern belongs ^
Classification system overview
sensor featuregeneration featureselection classifierdesign systemevaluation
Patterns
6
Supervised – unsupervised pattern recognition:The two major directions^
Supervised
:^
Patterns whose class is known a-priori
are used for training. ^
Unsupervised
:^
The number of classes is (in general)
unknown and no training patterns are available.
8
Computation of a-posteriori probabilities^
Assume known• a-priori probabilities• This is
also known as the likelihood of
) ( )..., ( ), (^
2
1
M P
P
P
M
i x p^
i^
,..., (^2) , 1 ,) (^
i to r w x
9
^
2 1
) ( ) ( ) (
) (
) ( ) ( ) (
) ( ) ( ) ( ) (
i
i i i i
i
i i
i
P x p
x p
x p
P x p x P
P x p x P x
p^
^
The Bayes rule (
Μ =2)
where
11
)
(
)
(^
2
2
1
1
R
R
and
12
Equivalently in words:
Divide space in two regions
Probability of error^
Total shaded area
Bayesian classifier is OPTIMAL with respect tominimizing the classification error probability!!!!
2
2
1
1
in
If
in
If
x
R x
x
R x
0
0
1
2
x
x
e^
14
The Bayes classification rule for many (M>2) classes:^
Given
classify it to
if:
^
Such a choice also minimizes the classification errorprobability
Minimizing the average risk^
For each wrong decision, a penalty term is assigned sincesome decisions are more sensitive than others
i j x
P x
P^
j
i^
x^
i
15
^
For
M
=
penalty term for deciding class
,
although the pattern belongs to
,^
etc.
^
Risk with respect to
)
(
22 21
12 11
L
^12
R
R
1
12
1
11 1
2
1
^
^
2
17
Choose
and
so that
r^
is minimized
Then assign
to
if
Equivalently:assign
in
if
:^ likelihood ratio
R^^1
R^2 x^
i ) (^
2
11 12
22 21 2 1
1 2
12
x p
x p
2
2
22
1
1
12 2
2
2
21
1
1
11 1
18
y
probabilit
error
tion
classifica
Minimum
if
if
if
and 1 2 ) ( ) (
12
21
12 21 1
2
2
(^2112) 2
1
1
22
11
2
1
x P
x P
x
x P
x P
x
If
20
^
Then the threshold value is: ^
Threshold
for minimum
r
1 2
)) 1 (
exp( )
exp(:
:
minimum for
0
2
2
0 0
x
x
x
x
P
x^
e
1 2
2
) 2
(^1) ( ˆ
)) 1 ( ( exp 2 ) ( exp: ˆ
0
2
2
0
n
x
x
x
x
ˆ x^0
21
Thus
moves to the left of
(WHY?)
ˆ x^^0
0
1 2
x
