Gauss-Markov Theorem: Proof and Explanation, Study notes of Statistics

Download Gauss-Markov Theorem: Proof and Explanation and more Study notes Statistics in PDF only on Docsity!

Gauss Markov Theorem

Dr. Frank Wood

Digression : Gauss-Markov Theorem

In a regression model where E{ i

} = 0 and variance

σ

2 { i

} = σ

2 < ∞ and  i

and  j

are uncorrelated for all i and j the

least squares estimators b 0

and b 1

are unbiased and have minimum

variance among all unbiased linear estimators.

Remember

b 1

(X

i

X )(Y

i

Y )

(Xi −

X )

2

k i

Y

i

, k i

(X

i

X )

(Xi −

X )

2

b 0 =

Y − b 1

X

σ

2 {b 1

} = σ

2 {

k i

Y

i

k

2

i

σ

2 {Y i

= σ

2

(Xi −

X )

2

Proof

I (^) Given these constraints

β 0

c i

• β 1

c i

X

i

= β 1

clearly it must be the case that

ci = 0 and

ci Xi = 1

I (^) The variance of this estimator is

σ

2

{

β 1 } =

c

2

i

σ

2

{Yi } = σ

2

c

2

i

I (^) This also places a kind of constraint on the c i

’s

Proof cont.

Now define c i

= k i

• d i

where the k i

are the constants we already

defined and the d i

are arbitrary constants. Let’s look at the

variance of the estimator

σ

2 {

β 1

c

2

i

σ

2 {Y i

} = σ

2

(k i

• d i

2

= σ

2

(

k

2

i

d

2

i

k i

d i

Note we just demonstrated that

σ

2

k

2

i

= σ

2

{b 1

So σ

2 {

β 1

} is related to σ

2 {b 1

} plus some extra stuff.

Proof end

So we are left with

σ

2 {

β 1

} = σ

2 (

k

2

i

d

2

i

= σ

2

(b 1 ) + σ

2

(

d

2

i

which is minimized when the d i

= 0 ∀ i.

If d i

= 0 then c i

= k i

This means that the least squares estimator b 1 has minimum

variance among all unbiased linear estimators.