Download Solutions to HW4 in ECE561: Detection and Estimation Theory, Spring 2009 - Prof. Olgica Mi and more Assignments Electrical and Electronics Engineering in PDF only on Docsity!
ECE561: Detection and Estimation Theory
HW4 Solutions
Professor Olgica Milenkovic Spring 2009
Problem 1 (20 points)
Refer to Levy, p250-
Problem 2 (20 points)
The MMSE, MAE, MAP estimators are associated with the mean, median, and mode re-
spectively. It was shown in class that if the conditional density is symmetric around the
mode, the loss function is even and convex, then the three estimators are equal. However,
if the distribution is skewed, then the estimators will not be equal. If the distribution is
right-skewed, then
MMSE>MAE>MAP
If the distribution is left-skewed, then the reverse is true, i.e., MAP>MAE>MMSE.
Problem 3 (20 points)
Consider the estimator for the general case
X =
A
Y +
b
where
A is a m × n matrix with full rank.
Let
K =
A
T ˜
A. Then,
K is a symmetric positive-definite matrix. Using Cholesky decom-
position,
K could be decomposed into a product of a lower and upper triangular matrices,
.i.e.,
K = LL
T
where L is a n × n lower triangular matrix. Note that since
K is symmetric,
K =
K
T and so
K could also be expressed as
K = U U
T where U = L
T is an upper triangular
matrix.
A
T ˜
X =
K
Y +
A
T ˜
b
= LL
T ˜ Y +
A
T ˜
b
Let,
X =
A
T ˜ X, Y = L
T ˜ Y , b =
A
T ˜ b, A = L. Then,
X = AY + b, where A is a triangular matrix. So,
X =
A
T ˜
X is the desired estimator with A
restricted to a triangular matrix.
When does the LLS estimator allow for A to be triangular?
A = K
XY
K
− 1
Y
is triangular if K XY
and K
− 1
Y
are triangular. If the observations are Y are
uncorrelated, then K Y
is a diagonal matrix which is also triangular. K XY
will also be trian-
gular of X and Y are uncorrelated.
Problem 4 (20 points)
a) M =
A B
C D
Let G =
I 0
−D
− 1
I
Then, M G =
A − BD
− 1 C B
0 D
Since det(G) = 1, we have det(M G) = det(M ).
det(M G) = det([A − BD
− 1 C]D)
= det(A − BD
− 1
C) × det(D)
= det(D/M ) × det(D)
So, det(M ) = det(D) × det(D/M )
b) Let G =
I −BD
− 1
0 I
Then,
h = number of heads appearing in n flips of a coin,
t = number of tails appearing in n flips of a coin,
p: probability that a head appears.
Since h is a sufficient statistic, the Fisher information is:
J(h) = −E
[
2
∂h
2
log p(h|p)
]
= −E
[
2
∂h
2
log
(
p
h
(1 − p)
t
n!
h!t!
)]
= −E
[
2
∂h
2
(h log(p) + t log(1 − p))
]
= −E
[
h
p
2
t
(1 − p)
2
]
n
p
n
1 − p
n
p(1 − p)
