Download The Normal Approximation to the Binomial Distribution: Properties and Approximation and more Summaries Physics in PDF only on Docsity!
Physics 116C Fall 2012
The Normal Approximation to the Binomial Distribution
- Properties of the binomial distribution
Consider a the binomial distribution,
f (x) = C(n, x)pxqn−x^ ,
where
C(n, x) ≡
n! x!(n − x)!
The function f (x) represents the probability of exactly x successes in n Bernoulli trials (cf. pp. 756–758 of Boas), where a given trial has two possible outcomes: a “success” with probability p and a “failure” with probability q = 1 − p. Each repeated trial is an independent event. The expectation value of the binomial distribution can be computed using the follow- ing trick. Consider the binomial expansion
(p + q)n^ =
∑^ n
k=
C(n, k)pkqn−k^.
Then if we take a derivative with respect to p and then multiply by p we obtain
p
d dp
(p + q)n^ =
∑^ n
k=
kC(n, k)pkqn−k^.
Evaluating the left hand side of the above equation then yields
np(p + q)n−^1 =
∑^ n
k=
kC(n, k)pkqn−k^.
The above result is true for any p and q. If we apply it to the case where q = 1 − p, then we find
np =
∑^ n
k=
kf (k) = x ,
where we recognize
∑n k=0 kf^ (k) as the expectation value (or mean) of the binomial dis- tribution. Hence, we conclude that x = np.
By a similar trick, we may compute the variance of the binomial distribution. In this case, we evaluate
p^2
d^2 dp^2
(p + q)n^ =
∑^ n
k=
k(k − 1)C(n, k)pkqn−k^.
Evaluating the left hand side of the above equation then yields
n(n − 1)p^2 (p + q)n−^2 =
∑^ n
k=
k(k − 1)C(n, k)pkqn−k^.
The above result is true for any p and q. If we apply it to the case where q = 1 − p, then we find
n(n − 1)p^2 =
∑^ n
k=
k^2 f (k) −
∑^ n
k=
kf (k) = x^2 − x ,
after recognizing
∑n k=0 k
(^2) f (k) as the average value of x (^2) for the binomial distribution.
Since x = np, we conclude that
x^2 = n(n − 1)p^2 + np.
Hence, the variance is given by
Var(x) = x^2 − (x)^2 = n(n − 1)p^2 + np − n^2 p^2 = np(1 − p).
Since q = 1 − p, one can also write this result as
σ^2 ≡ Var(x) = npq ,
where σ is the standard deviation.
- The normal approximation to the binomial distribution
Remarkably, when n, np and nq are large, then the binomial distribution is well approximated by the normal distribution. According to eq. (8.3) on p.762 of Boas,
f (x) = C(n, x)pxqn−x^ ∼
2 πnpq
e−(x−np)
(^2) / 2 npq .
In these notes, we will prove this result and establish the size of the correction. We start with the explicit form for the binomial distribution,
f (x) =
n! x!(n − x)!
pxqn−x^ ,
where q = 1 − p. By assumption n, np and nq are large.^1 We are interested in ap- proximating the binomial distribution by the normal distribution in the region where the
(^1) As long as p is not too close to either 0 or 1, it follows that np and nq are both of O(n) as n is taken
large.
Exponentiating the above result, it follows that the product of the first two terms in eq. (1) can be written as
(np x
)x (^ nq n − x
)n−x = e−δ
(^2) / 2 npq
[
1 + O
δ^3 n^2
)]
Moreover, the square root factor in eq. (1) can be approximated by
√ n 2 πx(n − x)
n 2 π(np + δ)(nq − δ)
2 πnpq
[
1 + O
δ n
)]
At the beginning of this section, I argued that x should differ from the mean μ = np by a small number of standard deviations, σ =
npq. In particular this number should be of O(1) as n is taken large. Since x = np + δ, this means that at worst, δ ∼ O(
n) for large values of n. In this case, both O(δ^3 /n^2 ) and O(δ/n) in eq. (2) and eq. (3) behave as O(1/
n) as n → ∞. Hence, the binomial probability function can been written as
f (x) =
2 πnpq
e−(x−np)
(^2) / 2 npq
[
1 + O
n
)]
which is the normal distribution with parameters μ = np and σ^2 = npq, up to corrections that vanish as n → ∞. Indeed, the mean value μ and the standard deviation σ of the normal approximation are identical to the mean value and the standard deviation of the original binomial distribution, respectively. That is, for
φ(x) =
2 πnpq
e−(x−np)
(^2) / 2 npq ,
where q = 1 − p, one can easily check that
E(x) =
−∞
xφ(x) dx = np ,
and
Var(x) = E(x^2 ) − [E(x)]^2 =
−∞
x^2 φ(x) dx −
−∞
xφ(x) dx
= npq ,
by performing the explicit integrations. The normal approximation to the binomial distribution holds for values of x within some number of standard deviations of the average value np, where this number is of O(1) as n → ∞, which corresponds to the central part of the bell curve. As previously noted, f (x) is small anyway in other parts of the domain, so that we can ignore the fact that our approximation may not be good there. Eq. (4) also reveals the size of the first correction to the normal approximation to the binomial distribution. Note that the O(1/n) term in eq. (1) has been dropped as this term is much smaller than the O(1/
n) correction term that appears in eq. (4).
