Stat 312: Lecture 12
Errors in hypothesis testing
Moo K. Chung
Oct 20, 2004
Type I error: you reject H0when H0is true.
P(Type I error) = P(reject H0|H0true) = α. The
resulting αis called the significance level of the test
and the corresponding test is called a level αtest.
We will use test procedures that give αless than a
specified level (0.05 or 0.01 usually).
I believe that dogs are as smart as people. Assume
IQ of a dog follows Xi∼N(µ, 102). IQ of 10 dogs
are measured: 30, 25, 70, 110, 40, 80, 50, 60, 100,
60. We want to test if dogs are as smart as people by
testing
H0:µ= 100 vs. H1:µ < 100.
(1) Let ¯
Xbe a test statistic and R= (−∞,90) to
be a rejection region. Let’s compute the probabil-
ity of making Type I error based on this testing
procedure. Under the assumption H0is true,
Xi∼N(100,102).
Under this condition, ¯
X∼N(100,10) and
α=P(¯
X < 90).
>help(pnorm)
...
Usage:
... pnorm(q, mean=0, sd=1,...)
...
> pnorm(90,100,sqrt(10))
[1] 0.0007827011
By using this test procedure, it is highly unlikely
to make Type I error.
(2) Let’s see what happens when we change the test
precedure. When R= (−∞,95) with the same
test statistic ¯
X,
α=P(¯
X < 95).
> pnorm(95,100,sqrt(10))
[1] 0.05692315
70 90 110
0.0 0.4 0.8
a
pnorm(a, 100, sqrt(10))
70 90 110
0.0 0.4 0.8
a
1 − pnorm(a, 90, sqrt(10))
Figure 1: Errors in performing hypothesis testing
based on test procedure if ¯
X < 90, reject H0:µ=
100. Left figure: type I error, right figure: type II error.
(3) When R= (−∞,99),α=P(¯
X < 99).
> pnorm(99,100,sqrt(10))
[1] 0.3759148
The test procedure based on rejecting H0if ¯
X <
99 is producing large Type I error. Why?
Type II error: you do not reject H0when H0is
false.
β=P(Type II error)
=P(do not reject H0|H0false )
= 1 −P(reject H0|H1true)
With the fixed test procedure of rejecting H0if ¯
X <
90, let us compute β. When µ= 90 is true,
> 1-pnorm(90,90,sqrt(10))
[1] 0.5
> 1-pnorm(90,95,sqrt(10))
