Download Normal Approximation to the Binomial Distribution and more Summaries Advanced Calculus in PDF only on Docsity!
CHAPTER 9
Normal approximation to the binomial
A special case of the central limit theorem is the following statement.
Theorem 9.1 (Normal approximation to the binomial distribution)
If Sn is a binomial variable with parameters n and p, Binom (n, p), then
P
a 6
Sn − np √ np(1 − p)
6 b
n→∞ P(a 6 Z 6 b),
as n → ∞, where Z ∼ N (0, 1).
This approximation is good if np(1 − p) > 10 and gets better the larger this quantity gets. This means that if either p or 1 − p is small, then this is valid for large n. Recall that by Proposition 6.1 np is the same as ESn and np(1 − p) is the same as Var Sn. So the ratio is equal to (Sn − ESn)/
Var Sn, and this ratio has mean 0 and variance 1, the same as a standard N (0, 1).
Note that here p stays xed as n → ∞, unlike in the case of the Poisson approximation, as we described in Proposition 6.3.
Sketch of the proof. This is usually not covered in this course, so we only explain one (of many) ways to show why this holds. We would like to compare the distribution of
Sn with the distribution of the normal variable X ∼ N
np,
np (1 − p)
. The random
variable X has the density
2 πnp (1 − p)
e−^
(x−np)^2 2 np(1−p) (^).
The idea behind this proof is that we are interested in approximating the binomial dis- tribution by the normal distribution in the region where the binomial distribution diers signi cantly from zero, that is, in the region around the mean np. We consider P (Sn = k), and we assume that k does not deviate too much from np. We measure deviations by some small number of standard deviations, which is
np (1 − p). Therefore we see that k − np should be of order
n. This is not much of a restriction since once k deviates from np by many standard deviations, P (Sn = k) becomes very small and can be approximated by zero. In what follows we assume that k and n − k of order n.
We use Stirling's formula is the following form
m! ∼
2 πme−mmm, 121
where by ∼ we mean that the two quantities are asymptotically equal, that is,their ratio tends to 1 as m → ∞. Then for large n, k and n − k
P (Sn = k) =
n! k! (n − k)!
pk^ (1 − p)n−k
2 πne−nnn √ 2 πke−kkk
2 π (n − k)e−(n−k)^ (n − k)n−k^
pk^ (1 − p)n−k
( (^) p k
)k (^1 − p n − k
)n−k nn
n 2 πk (n − k)
(np k
)k (n (1 − p) n − k
)n−k √ n 2 πk (n − k)
Now we can use identities
ln
(np
k
= − ln
k − np np
ln
n (1 − p) n − k
= − ln
k − np n (1 − p)
Then we can use ln (1 + y) ∼ y − y
2 2 +^
y^3 3 , y^ →^0 to see that
ln
(np
k
)k (n (1 − p)
n − k
)n−k) = k ln
(np
k
n (1 − p) n − k
∼ k
k − np np
k − np np
k − np np
k − np n (1 − p)
k − np n (1 − p)
k − np n (1 − p)
(k − np)^2 2 np (1 − p)
Thus (np k
)k (n (1 − p) n − k
)n−k ∼ e−^
(k−np)^2 2 np(1−p) (^).
Now we use our assumption that k − np should be of order
n to see that
k − np ≈
n, n − k ≈ n (1 − p) −
n, k (n − k) ≈ n^2 p (1 − p) ,
so
In this case we again have
p = 0. 5 , μ = np = 50, σ^2 = np(1 − p) = 25, σ =
np(1 − p) = 5.
The normal approximation can be done in three dierent ways:
P(49 6 Sn 6 51) ≈ P(49 6 50 + 5Z 6 51) = Φ(0.2) − Φ(− 0 .2) = 2Φ(0.2) − 1 ≈ 0. 15852
or
P(48 < Sn < 52) ≈ P(48 < 50 + 5Z < 52) = Φ(0.4) − Φ(− 0 .4) = 2Φ(0.4) − 1 ≈ 0. 31084
or
P(48. 5 < Sn < 51 .5) ≈ P(48. 5 < 50+5Z < 51 .5) = Φ(0.3)−Φ(− 0 .3) = 2Φ(0.3)− 1 ≈ 0. 23582
Here all three answers are approximate, and the third one, 0.23582, is the most accurate among these three. We also can compute the precise answer using the binomial formula:
P(49 6 Sn 6 51) =
∑^51
k=
k
In addition we can obtain the following normal approximations
P(Sn = 49) ≈ P(48. 5 6 50 + 5Z 6 49 .5) = Φ(− 0 .1) − Φ(− 0 .3) = Φ(0.3) − Φ(0.1) ≈ 0. 07808
P(Sn = 50) ≈ P(49. 5 6 50 + 5Z 6 50 .5) = Φ(0.1) − Φ(− 0 .1) = 2Φ(0.1) − 1 ≈ 0. 07966
P(Sn = 51) ≈ P(50. 5 6 50 + 5Z 6 51 .5) = Φ(0.3) − Φ(0.1) ≈ 0. 07808
Finally, notice that 0 .07808 + 0.07966 + 0.07808 = 0. 23582
which is the approximate value for P(49 6 Sn 6 51) ≈ P(48. 5 < 50 + 5Z < 51 .5).
Continuity correction
If a continuous distribution such as the normal distribution is used to approximate a discrete one such as the binomial distribution, a continuity correction should be used.
For example, if X is a binomial random variable that represents the number of successes in n independent trials with the probability of success in any trial p, and Y is a normal random variable with the same mean and the same variance as X. Then for any integer k we have that P (X 6 k) is well approximated by P (Y 6 k) if np (1 − p) is not too small. It is better approximated by P (Y 6 k + 1/2) as explained at the end of this section. The role of 1 / 2 is clear if we start by looking at the normal distribution rst, and seeing how we use it to approximate the binomial distribution.
The fact that this approximation is better based on a couple of considerations. One is that a discrete random variable can only take on only discrete values such as integers, while a continuous random variable used to approximate it can take on any values within an interval around these speci ed values. Hence, when using the normal distribution to approximate the binomial, more accurate approximations are likely to be obtained if a continuity correction is used.
The second reason is that a continuous distribution such as the normal, the probability of taking on a particular value of a random variable is zero. On the other hand, when the normal approximation is used to approximate a discrete distribution, a continuity correction can be employed so that we can approximate the probability of a speci c value of the discrete distribution.
For example, if we want to approximate P (3 6 X 6 5) = P (X = 3 or X = 4 or X = 5) by a normal distribution, it would be a bad approximation to use P (Y = 3 or Y = 4 or Y = 5) as the probability of Y taking on 3 , 4 and 5 is 0. We can use continuity correction to see that
P (3 6 X 6 5) = P (2. 5 6 X 6 5 .5)
and then use the normal approximation by P (2. 5 6 Y 6 5 .5).
Below is a table on how to use the continuity correction for normal approximation to a binomial.
Binomial Normal
If P (X = n) use P (n − 0. 5 < X < n + 0.5)
If P (X > n) use P (X > n + 0.5)
If P (X 6 n) use P (X < n + 0.5)
If P (X < n) use P (X < n − 0 .5)
If P (X > n) use P (X > n − 0 .5)
9.2. SELECTED SOLUTIONS 127
9.2. Selected solutions
Solution to Exercise 9.1(A): The probability of rolling two ves in a particular roll is 361 , so the probability that we roll two ves no more than once in 180 rolls is
p =
Solution to Exercise 9.1(B): we are interested in the number of successes to be 0 or 1, that is, P (0 6 S 180 6 1). Since the binomial is integer-valued, we apply the continuity correction and calculate P (− 0. 5 6 S 180 6 1 .5) instead. We nd that the expected value is
μ = 180 · p = 5 and the standard deviation is σ =
180 p(1 − p) ≈ 2. 205. Now, as always, we convert this question to a question about the standard normal random variable Z,
P (− 0. 5 6 S 180 6 1 .5) = P
6 Z 6
= P (− 2. 49 < Z < − 1 .59)
Solution to Exercise 9.1(C): We use λ = np = 5 (note that we found this already in (b)!). Now we see that
P(E) ≈ e−^5
Solution to Exercise 9.2: Let X denote the number of left-handed students in the class. We use Theorem 9.1 with X ∼ Binom(150, 0 .1) below. Note that np = 15.
(a) P(X > 25) = P
X√− 15
- 5 >^ √^10
- 5
≈ 1 −Φ(2.72) ≈ 0. 00364. Note that in this approximation
we implicitly use that the tail of this probability distribution is small, and so instead of a two-sided interval we just used a one-sided interval.
We can see that the result is really close to the two-sided estimates as follows.
P(150 > X > 25) = P
X − 15
Finally, with the continuity correction the solution is
P(150 > X > 25) = P(150. 5 > X > 24 .5)
= P
X − 15
(2) Similarly to the rst part
P(15 6 X 6 20) = P(14. 5 < X < 20 .5)
Solution to Exercise 9.3: Let X denote the number of black markers. Since X ∼ Binom(50, 0 .6) we have
P(X > N ) ≈ 1 − Φ
N − 30
- 2 and P(X > N + 1) ≈ 1 − Φ
N − 29
From this we deduce that N 6 32. 909 and N > 31. 944 so that N = 32.
