Download Cramer-Rao Example 2.3 Detailed Deduction and more Schemes and Mind Maps Telecommunications Engineering in PDF only on Docsity!
Cramer-Rao Example 2.3 Detailed Deduction
Chaoran Xiong
October 2023
1 Theory
Example 2.3: In this example we will show the usefulness of the Cram´er-Rao inequality for parameter estimation. Suppose we wish to estimate a nonlinear appearing parameter, a > 0, of the following exponential model:
y˜k = Beatk^ + vk, k = 1, 2... , m
where vk is a zero-mean Gaussian white-noise process with variance given by σ^2. We can choose to employ nonlinear least squares to iteratively determine the parameter a, given measurements yk and a known B > 0 coefficient. We take the nonlinear least squares approach, the problem should be modeled as followed: y˜ = f (ˆx) + e
The problem should be linearized by a first-order Taylor series expansion as
f (ˆx) ≈ f (xc) + H∆x
where
H ≡ ∂f ∂x (^) xc
Bt 1 eat^1 Bt 2 eat^2 · · · Btmeatm
T
If we take the least squares approach, the linearized model should follow the equations below ey = Hx + v
x b = M ye =
H⊤R−^1 H
H⊤R−^1 ˜y M H = I
where E{v} = 0, E{vvT^ } = R, so the covariance of the estimate error should be calculated as follow
E
(ˆx − x)(ˆx − x)⊤^ = E
(M y˜ − x)(M ˜y − x)⊤
= E
(M (Hx + v) − x)(M (Hx + v) − x)⊤
If the problem is considered unbiased, then M H = I, so the covariance should be
E
(M v)(M v)⊤^ = E
M vv⊤M ⊤^ = M E
vv⊤^ M ⊤^ = M RM ⊤
= H−^1 R(H−^1 )⊤^ =
HT^ R−^1 H
where R = σ^2 I, so the covariance of the estimate error is given by
P = σ^2 (H⊤H)−^1 =
HT^ R−^1 H
= F −^1
The matrix P is also equivalent to the Cramer-Rao lower bound. Suppose instead we wish to simplify the estimation process by defining ˜zk ≡ ln ˜yk, using the change of variables approach shown in Table 1.1. Then, linear squares can be applied to determine a. But how optimal is this solution? It is desired to study the effects of applying this linear approach because the logarithmic function also affects the Gaussian noise. The linearization process is shown as follow ln ˜yk = ln
Beatk^ + vk
We apply the Lagrange Mean Value Theorem to perform a first-order expansion of the equation at the point where vk = 0.
ln ˜yk ≈ ln
Beatk
∂ ln (Beatk^ + vk) ∂vk (^) vk =ε
vk
ln ˜yk ≈ ln
Beatk
vk ε + Beatk
where ε ∈ [0, vk], here we take ε = 12 vk to approximate the expansion. Thus the linearized function should be written as follow
ln ˜yk ≈ ln
Beatk
vk 1 2 vk^ +^ Be atk
ln ˜yk − ln B ≈ atk +
2 vk vk + 2Beatk
The linear least squares ” H matrix,” denoted by H, is now simply given by
H =
t 1 t 2 · · · tm
T
However, the new measurement noise will certainly not be Gaussian anymore. We now use the binomial series expansion:
(a+x)n^ = an+nan−^1 x+
n(n − 1) 2! an−^2 x^2 +
n(n − 1)(n − 2) 3! an−^3 x^3 +· · · , x^2 < a^2
A first-order expansion using the binomial series of the new measurement noise is given by
εk ≡ 2 vk
2 Beatk^ + vk
≈ 2 vk
2 Beatk
vk (2Beatk^ )^2
2 Numerical Results
In the Theory section, we conclude that the Log linearization is not valid if σ^4 /
2 B^4 e^4 atk
is not negligible. In this section we show the numerical results from matlab to verify the theory. In this experiment, we take B = 0.25, t ∈ [0, 1] and different values of σ to demonstrate the Cramer-Rao bound effectiveness for non-linear and Log Linearized least square. Fig. 1 shows when the σ^2 is small(0.01 in this case), σ^4 /
2 B^4 e^4 atk
is approximately negligible. The non-linear approach was not so effective because of the first order Taylor Expansion was not that accurate, while Log Linearized Estimation performed well and its covariance Plog was nearly equivalent to its Cramer-Rao bound P (^) logCR. We can also conclude The non-linear Cramer-Rao
bound P (^) nonCR was much larger than the Log linearized Cramer-Rao bound P (^) logCR , so in this case, the Log linearized was very effective.
Figure 1: Non-linear VS Linearized Least square when σ^2 = 0. 01
Fig. 2 shows when the σ^2 is large(0.1 in this case), σ^4 /
2 B^4 e^4 atk
cannot be neglected, both non-linear and Log Linearized approach did not perform well. The covariance Plog was much larger than Cramer-Rao bound P (^) logCR. We
can also conclude the non-linear Cramer-Rao bound P (^) nonCR was nearly equivalent to the Log linearized actual covariance Plog , so in this case the Log linearized proved to be not so effective.
Figure 2: Non-linear VS Linearized Least square when σ^2 = 0. 1
Next, we analyzed the Log linearized actual covariance Plog and its Cramer- Rao bound P (^) logCR .As depicted in Fig. 3, when the σ got larger, Plog become more and more larger than P (^) logCR , so the Log Linearized approach was not so effective.
Figure 3: Plog and P (^) logCR over σ^2
