Download The t-Test: Statistical Inference with Unequal Variances and more Slides Economic statistics in PDF only on Docsity!
The t Test
April 3-8, 2008
Again, we begin with independent normal observations X 1.... , Xn with unknown mean μ and unknown
variance σ^2. The likelihood function
L(μ, σ 2 |x) =
(2πσ^2 )n/^2
exp −
2 σ^2
∑^ n
i=
(xi − μ) 2
ln L(μ, σ 2 |x) = −
n
2
(ln 2π + ln σ 2 ) −
2 σ^2
∑^ n
i=
(xi − μ) 2
∂μ
ln L(μ, σ 2 |x) = −
σ^2
∑^ n
i=
(xi − μ)
Thus, ˆμ = ¯x.
∂σ^2
ln L(μ, σ 2 |x) = −
n
2 σ^2
2(σ^2 )^2
∑^ n
i=
(xi − μ) 2 .
Thus,
ˆσ 2 =
n
∑^ n
i=
(xi − x¯) 2 .
For the hypothesis H 0 : μ = μ 0 versus H 1 : μ 6 = μ 0 ,
the likelihood ratio test
Λ(x) =
L(μ 0 , σˆ 2 0 |x) L(ˆμ, ˆσ^2 |x)
where the value
ˆσ 2 0 =
n
∑^ n
i=
(xi − μ 0 ) 2
gives the maximum likelihood on the set μ = μ 0.
L(μ 0 , σˆ 2 0 |x) =^
(2πˆσ 02 )n/^2
exp −
2ˆσ^20
∑^ n
i=
(xi − μ 0 ) 2 =
(2πσˆ^20 )n/^2
exp −
n
L(ˆμ, ˆσ 2 |x)) =
(2πσˆ^2 )n/^2
exp −
2ˆσ^2
∑^ n
i=
(xi − ¯x) 2 =
(2πσˆ^2 )n/^2
, exp −
n
and
Λ(x) =
ˆσ 2
ˆσ^20
)n/ 2
=
( ∑n i=1(xi^ −^ μ^0 )
2 ∑n i=1(xi^ −^ ¯x)
2
)−n/ 2
The critical region λ)x) ≤ λ 0 is equivalent to
c ≤
∑n i=1(xi^ −^ μ^0 )
2 ∑n i=1(xi^ −^ x¯)
2
n(¯x − μ 0 ) 2 ∑n i=1(xi^ −^ ¯x)
2
or
T (x) 2 ≥ (c − 1)(n − 1)
where
T (x) =
¯x − μ 0
s/
n
and we write s for the square root of the unbiased estimator of the variance
s 2 =
n − 1
∑^ n
i=
(xi − x¯) 2 .
1 Deriving the t distribution
For the hypothesis test, our next goal is to understand the distribution of T (X).
Step 1.
n( X¯ − μ 0 )/σ is a standard normal random variable.
For this, notice that X¯ is a normal random variable with mean μ 0 and standard deviation σ/
n
Step 2. For each i, Xi − X¯ and X¯ are independent.
For normal random variables, uncorrelated random variables are independent. Thus, it suffices to show
that the covariance is 0. To that end, note that
Cov(Xi − X,¯ X¯) = Cov(Xi, X¯) − Cov( X,¯ X¯).
For the first term, use the fact that Cov(X 1 , Xj ) = 0 if i 6 = j and Cov(X 1 , Xi) = Var(Xi) = σ 2
. Then,
Cov(Xi, X¯) =
n
∑^ n
j=
Cov(Xi, Xj ) =
n
σ
2 .
From Step 1, we know that
Cov( X,¯ X¯) = Var( X¯) =
n
σ 2 .
Now combine to see that Cov(Xi − X,¯ X¯) = 0.
Step 3.
∑n i=1(Xi^ −^ X¯)^2 /σ^2 is a χ-square random variable with n − 1 degrees of freedom.
the joint density
fT,V (t, v) = fZ,U (
t
v √ n − 1
, v)|J(t, v)|.
where |J(z, u)| is the absolute value of the Jacobian of the inverse transformation.
In this case,
J(t, v) = det
[
∂z/∂t ∂z/∂v ∂u/∂t ∂u/∂v
]
= det
[ √
v/
n − 1 t/(
v(n − 1) 0 1
]
v √ n − 1
Then,
fT,V (t, v) =
2 π 2 (n−1)/^2 Γ((n − 1)/2)
v n/ 2 − 3 / 2 exp
v
2
t 2
n − 1
v √ n − 1
Finally, to find the marginal density for T , we integrate with respect to v to obtain
fT (t) =
2 π 2 (n−1)/^2 Γ((n − 1)/2)
n − 1
0
v n/ 2 − 1 exp
v
2
t 2
n − 1
dv.
Change variables by setting w = v(1 + t 2 /(n − 1))/2.
fT (t) =
2 π 2 (n−1)/^2 Γ((n − 1)/2)
n − 1
0
2 w
1 + t^2 /(n − 1)
)n/ 2 − 1
e −w
1 + t^2 /(n − 1)
dw
π(n − 1)Γ((n − 1)/2)
t^2
n − 1
)−n/ 2 ∫ (^) ∞
0
w n/ 2 − 1 e −w dw
Γ(n/2) √ π(n − 1)Γ((n − 1)/2)
t^2
n − 1
)−n/ 2
.
Note that (
1 +
t^2
n − 1
)−n/ 2
→ exp −
t^2
2
and n → ∞. Thus, for large n, the t density is very close to the density of a standard normal.
2 Tests and confidence intervals using t distribution
We can now return to our original hypothesis. The critical region for an alpha level test is determined by
the extreme values of the t statistic.
C = {x; |T (x)| ≥ tα/ 2 }
where
P {T > tα/ 2 } =
α
2
Correspondingly, the γ-level confidence interval is obtained by choosing α = 1 − γ and setting
¯x ±
s √ n
tα/ 2.
For a one-sided hypothesis H 0 : μ ≤ μ 0 versus H 1 : μ ≥ μ 0 ,
the critical region becomes
C = {x; T (x) ≥ tα}
3 Two sample t test
We now have a three dimensional parameter space Θ = {(μ 1 , μ 2 , σ^2 ); μ 1 ∈ R, μ 2 ∈ R, σ^2 ∈ R}..
The two-sided test is H 0 : μ 1 = μ 2 versus H 1 : μ 1 6 = μ 2 ,
The data X 1 ,j ,... , Xnj ,j are independent N (μj , σ^2 ) random variables, j = 1, 2. The likelihood function is
L(μ 1 , μ 2 , σ
2 |x 1 , x 2 ) =
(2πσ^2 )(n^1 +n^2 )/^2
exp −
2 σ^2
∑n^1
i=
(xi, 1 − μ 1 )
2
∑^ n^2
i=
(xi, 2 − μ 2 )
2
Then, the likelihood ratio,
Λ(x 1 , x 2 )) =
L(ˆμ, μ,ˆ σˆ^212 |x 1 , x 2 )
L(ˆμ 1 , ˆμ 2 , ˆσ^2 |x 1 , x 2 )
Write the unbiased estimators for the mean and the variance
x¯j =
nj
nj ∑
i=
xi,j and s 2 j =^
nj − 1
nj ∑
i=
(xi,j − x¯j ) 2 , j = 1, 2.
Then, the overall maximum likelihood estimators are
μˆ 1 = ¯x 1 μˆ 2 = ¯x 2 , σˆ 2 =
(n 1 − 1)s^21 + (n 2 − 1)s^22
n 1 + n 2
The maximum likelihood estimators in the numerator takes place on the set μ 1 = μ 2.
μˆ =
n 1 x¯ 1 + n 2 ¯x 2
n 1 + n 2
, σˆ
2 12 =^
n 1 + n 2
∑n^1
i=
(xi, 1 − ˆμ)
2
∑^ n^2
i=
(xi, 2 − μˆ)
2
The yields the test statistic
T (x 1 , x 2 ) =
¯x 1 − x¯ 2 √ ( 1 n 1 +^
1 n 2
(n 1 −1)s^21 +(n 2 −1)s^22 n 1 +n 2 − 2
from the fact that the likelihood ratio
Λ(x 1 , x 2 ) =
n 1 + n 2 − 2
n 1 + n 2 − 2 + T (x 1 , x 2 )^2
) 2 /(n 1 +n 2 )
.
