Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Material Type: Notes; Class: PROBAB MTH,ELEC ENG; Subject: STATISTICS; University: Iowa State University; Term: Unknown 1989;
Typology: Study notes
1 / 16
This page cannot be seen from the preview
Don't miss anything!
Motivation for the estimation problem
Nonlinear MMSE estimation
Estimation based on several measurement
Linear MMSE estimation
Orthogonality Principle
Reading: Bertsekas & Tsitsiklis, 4. EE/STAT 322, #
There are many examples and applications of the estimation problem:
RADAR: estimation the location/speed of the target
Communications: estimation of signal frequency
Speech: recovering of the voice signal from the noisy signal
Weather reporting: estimation of tomorrow’s temperature
EE/STAT 322, #
Estimating the value of a signal in noise:
are random variables.
is a random variable uniformly distributed between (0, 1)
is Gaussian RV with zero mean and variance
σ
W 2
and
are independent
We can observe
, want to find a good estimate
X = g ( Y )
of
could be the signal picked up by a microphone in a classroom with
background noise. EE/STAT 322, #
Many different criteria: least-squares, minimum mean-squared error, ...
All useful depending on the application/complexity ...
We will minimize the
mean squared error
2 ]
We assume that we know the distribution of
and
and hence the
joint distribution of
and
EE/STAT 322, #
SETUP: There is a RV
whose value we don’t know.
There is another
, whose value we
know
Suppose we know the distribution of
What is the best guess of
, given that
y
, in the MMSE sense?
That is, want to find a function
X = g ( y )
such that we can minimize
g
( y
))
2 | Y = y ]
Any function of
y
is call an
estimator
. Some are good, some are bad. The
estimator
g
( y
)
that minimizes
is called the MMSE estimator.
The same reasoning as in the previous slide leads to
∂g
y
)
g
( y
) =
X | Y = y ]
EE/STAT 322, #
7
SETUP: Given several measurements
1 ,
2 ,... , Y
n
,
want to find the
MMSE estimate of another RV
That is, find a function
g
( Y
1 , Y
2 ,... , Y
n
)
to minimize
X − g ( Y 1
2 ,... , Y
n
))
2 | Y
1
y
1 , Y
2
y
2 ,... , Y
n
y
n
]
and hence
X − g ( Y 1
2 ,... , Y
n
))
2 ]
The answer is similar: the MMSE estimator is given by
g
( Y
1 , Y
2 ,... , Y
n
) =
2 ,... , Y
n
]
EE/STAT 322, #
well, but The nonlinear MMSE estimator based on several measurements performs
It requires the conditional distribution of
given
i ’s
may be difficult to compute (or is in an ugly form)Even if we know the conditional distribution, the conditional expectation
Solution: Lower our standard and look for something simpler EE/STAT 322, #
form of The same idea as in general MMSE, except that we restrict the functional
g
(
...
to be linear; that is,
a i Y i + b )
, and we want to
minimize
n
a i Y i + b
2
The solution for
a
i ’s and
b
can be found out from the system of equations
∂a
j
, j
,... , n
, and
∂b ∂J
∂a
j
∂a
i ( X − ( n
a i Y i + b
2
= E [ 2 ( X − ( n
a i Y i + b
j ]
EE/STAT 322, #
11
The equations above can be expressed in a compact form, called
Yule-Walker
equations.
a
1 ,... , a
n ] R Y + b ¯
XY
a
1 ,... , a
n
]
¯
b
where
Y
T
is the correlation matrix of vector
1 ,... , Y
n
] T
,^
R given by
Y
Y
Y
Y
n
Y
Y
Y
n
Y
n
Y
Y
XY
XY
XY
n
EE/STAT 322, #
13
Homogeneous linear estimate of
aY
Nonhomogeneous linear estimate of
aY
b .
the estimation error
If
is the LMMSE estimate of
, then
, i.e.,
(and
) are uncorrelated with (orthogonal to)
Proof:
Consider a homogeneous case, and let
aY
da^ dJ
aY
We can also show
aY
EE/STAT 322, #
Estimation error
is uncorrelated with estimate
Estimation error
is uncorrelated with observation
i :
i ] = 0
var
var
var
where var
σ
x 2
, var
ρ
XY 2
σ
x 2
, and
var
2 ] = (
ρ
XY 2
σ
x 2
.
EE/STAT 322, #
