Stat 312: Lecture 03
Minimum Variance Unbiased Estimator
Moo K. Chung
September 9, 2004
1. Population of interest is a collection of measur-
able objects we are studying. Let X1,· · · , Xnbe
a random sample from the population. Then sam-
ple mean ¯
Xand sample variance S2are unbiased
estimators of population mean µand population
variance σ2respectively.
Proof. Note that
S2=1
n−1X
i=1
(Xi−¯
X)2=1
n−1h
n
X
i=1
X2
i−n¯
X2i.
Then using the fact E(¯
X)2=V¯
X+ (E¯
X)2=
σ2/n +µ2, it can be shown that ES2=σ2.
2. There may be many unbiased estimators of θ.
Given two unbiased estimators ˆ
θ1and ˆ
θ2of θ. We
choose one that gives less variance. If V(ˆ
θ1)≤
V(ˆ
θ2),ˆ
θ1is called more efficient than ˆ
θ2. An effi-
cient estimator has less variability so we are more
likely to make an estimate close to the true param-
eter value. The following coin flipping example
clearly demonstrate this.
> a<-rbinom(1000,1,0.5)
> a
[1]00100101100...
> mean(a)
[1] 0.517
> mean(a[1:3])
[1] 0.3333333
> mean(a[1:9])
[1] 0.4444444
> mean(a[1:11])
[1] 0.3636364
3. Among all unbiased estimators, we choose the
most efficient estimator called the minimum vari-
ance unbiased estimator (MVUE). The MVUE is
an unbiased estimator with the smallest variance.
MVUE is the most efficient estimator. An effi-
cient estimator ˆ
θwill produce an estimate closer
to the true parameter θ.¯
Xis MVUE for µ(we
will not prove this statement).
4. Given a random sample X1,· · · , Xn, a linear es-
timator of parameter θis an estimator of form
ˆ
θ=
n
X
i=1
ciXi.
Then it can be shown that ¯
Xis the MVUE for
population mean µamong all possible linear esti-
mators.
Proof. Case n= 2 will be proved. The general
statement follows inductively. Consider linear es-
timators
ˆµ=c1X1+c2X2.
To be unbiased, c1+c2= 1. To be most efficient
among all unbiased linear estimators, the variance
has to be minimized. The variance is
Vˆµ=c2
1VX1+c2
2VX2=£c2
1+ (1 −c1)2¤σ2
The quadratic term in the bracket 2c2
1−2c1+ 1
is minimized when c1= 1/2.
Review Problems. You are not required to do these
problems but these are problems you should be able to
answer after each lecture. What is an unbiased estima-
tor of population parameter µ2? Exercise 6.3.