Download Cumulative Distribution - Mathematics and Statistics - Study Notes and more Study notes Mathematical Statistics in PDF only on Docsity!
1
Appendix 12: Cumulative Distribution,
Percentile Functions, and
Random Numbers
The functions described in this appendix are used in more than one procedure. They are grouped into the following categories:
- Special Functions. Gamma function, beta function, incomplete gamma function (ratio), incomplete beta function (ratio), and standard normal function
- Continuous Distributions. Beta, Cauchy, chi-square, exponential, F , gamma, Laplace, logistic, lognormal, normal, Pareto, Student’s t , uniform, and Weibull
- Discrete Distributions. Bernoulli, binomial, geometric, hypergeometric, negative binomial, and Poisson
- Noncentral Continuous Distributions. Noncentral beta, noncentral chi-square, noncentral F , and noncentral Student’s t
Notation
The following notation is used throughout this appendix unless otherwise stated: f (x) Density function of continuous random variable X F(x) Cumulative distribution function of X x (^) p 100p th percentile such that F x 3 8 (^) p = p , for 0 ≤ p ≤ 1 Prob (X = x) Probability density (or mass) function of discrete random variable X Prob 0 X ≤ x 5 Cumulative probability density function of X
Special Functions
Gamma Function
Γ 0 5 a = x a^ − e xdx a >
∞ (^) −
I
1 0 0
Properties
• Γ^ 0 5^1 =^1
• Γ^01 /^25 = π
• Γ 0 5 0 a = a − 1 5 0 Γ a − 15 a > 1
• Γ 0 5 0 a = a − 1! 5 when a is a positive integer
Note. Since Γ( a ) can be very large even for a moderate value of a , the (natural) logarithm of Γ( a ) is computed instead. References. The ln(Γ( a )) function: CACM 291 (1966) and AS 245 (1989).
Beta Function
B( , a b ) = I x a^ −^1 − x b −^1 dx a > , b >
0
1
Properties
• B 0 5 a , 1 = 1 / a
• B 3 12 , 128 =π
• B 0 a b , 5 = Γ0 5 0 5 a Γ b /Γ 0 a + b 5
• B 0 b a , 5 = B 0 a b , 5
Properties
• Using the transformation t = sin 2 θ , we get IB 3 x ; 12 , 12 8 = π^2 sin−^1 x
• IB ( ; , x a b ) = 1 − IB 01 − x b a ; , 5
- Using integration by parts, we get, for b > 1,
IB 0 x a b ; , 5 = ΓΓ0 5 0 5 0 aa^ +Γ^ bb^5 x a^ 01 − x 5 b −^1 + IB 0 x a ; + 1 , b − 15
• Using the fact that dx d x a^ 01 − x 5 b^ = ax a^ −^1 01 − x 5 b^ −^1 − 0 a + b x 5 0 a^ 1 − x 5 b −^1
we have
IB 0 x a b ; , 5 = Γ 0 Γ a^0 + a^^ 1 +5 0 5 Γ b^5 b x a^ 0 1 − x 5 b +IB 0 x a ; + , 1 b 5
References. AS 63 (1973); Inverse: AS 64 (1973), AS 109 (1977).
Standard Normal Function
x e du x x p x p
x u
p
− −∞ −
I
(^2 )
1
π
/
For Φ −^1 , the Abramowitz and Stegun method is used. References. AS 66 (1973); Inverse: AS 111 (1977) and AS 241 (1988). See Patel and Read (1982) for related distributions.
Continuous Distributions
Beta 0 x a b ; , 5. 0 ≤ x ≤ 1 , a > 0 and b > 0
f x (^) a b x x F x x a b x p a b p
a b
p
0 5 0 5 0 5 0 5 0 5 0 5
− −
−
1 1
1
B
IB
IB
Note. Uniform(0,1) is a special case of beta for a = 1 and b = 1.
Random Number
Special case
- Generate U from Uniform(0,1).
- If a = 1, set X = 1 − 01 − U 5 1/ b.
- If b = 1, set X = U 1/ a^.
- If both a = 1 and b = 1, set X = U. Algorithm BN due to Ahrens and Dieter (1974) for a > 1 and b > 1
- Set e = a − 1 , f = b − 1 , c = e + f , g = c ln 0 5 c , u = e / c ,and s = 0.5 / c.
- Generate Y from N(0,1) and set X = sY + u.
- If X < 0 or X > 1 , go to step 2.
- Generate U from Uniform(0,1).
- If ln 0 5 U ≤ 4 e ln 0 X / e 5 + f ln 10 1 − X 5 / f (^) 6 + g +0 5. Y 2 9 , finish; otherwise go to step 2. References. CDF: AS 63 (1973); ICDF: AS 64 (1973) and AS 109 (1977); RV: AS 134 (1979), Ahrens and Dieter (1974), and Cheng (1978).
Exponential 0 x a ; 5. x ≥ 0 , a > 0
f x ae F x e x (^) a p p
ax ax
p
0 5 0 5 0 5
− 1 − (^1) ln 1 0 1
Note. This is the Gamma( x ; 1, a ) distribution.
Random Number
Inverse CDF algorithm Generate U from Uniform(0,1); X = − ln 01 − U 5 / a.
F 0 x a b ; , 5. x ≥ 0 and b > 0
f x (^) a b ab x ab x
F x (^) b axax a^ b
a (^) a a b 0 5 (^0 )
0 5
= ^0 5 0
+
= (^) +
1 − −^ +
2 2 1 2 B IB
/ (^) / /
x ba p a^ b p a b p = p
≤^ ≤
IB
IB
1 1
1 6 1 6
Random Number Using the chi-square distribution
- Generate Y and Z independently from chi-square( a ) and chi-square( b ), respectively.
- Set X = 0 Y / a (^) 5 0/ Z / b 5. References. CDF: CACM 332 (1968). ICDF: use inverse of incomplete beta function.
Gamma 0 x a b ; , 5. x ≥ 0 , a > 0 , and b > 0
f x ba x e F x bx a x (^) b p a p
a (^) a bx
p
0 5 (^) 0 5 0 5 (^1 ) 1 6
− −
−
1
IG
IG
Note. The Erlang distribution is a gamma distribution with integers a and b = 1.
Random Number
Special case If a = 1 and b > 0, generate X from an exponential distribution with parameter b. Algorithm GS due to Ahrens and Dieter (1974) for 0 < a < 1 and b = 1
- Generate U from Uniform(0,1). Set c = 0 e + a (^) 5 / e , where e = exp 10 5.
- Set P = cU. If P > 1 , go to step 4.
- 0 P ≤ 15 Set X = P 1/ a^. Generate V from Uniform(0,1). If V > exp0 5− x , go to step 1; otherwise finish.
- ( P > 1 ) Set X = − ln 10 c − P 5 / a 6. If X < 0 , go to step 1; otherwise go to step 5.
- Generate V from Uniform(0,1). If V > X a −^1 , go to step 1; otherwise finish. Algorithm due to Fishman (1976) for a > 1 and b = 1
- Generate Y from Exponential (1).
- Generate U from Uniform(0,1).
- If ln U ≤ 0 a − 1 1 50 − Y +ln Y 5 , X = aY ; otherwise go to Step 1. References. CDF: AS 32 (1970) and AS 239 (1988); ICDF: Use the relationship between gamma and chi-square distributions. RV: Ahrens and Dieter (1974), Fishman (1976), and Tadikamalla (1978).
Lognormal 0 x a b ; , 5. x ≥ 0 , a >0 and b > 0,
f x (^) xb e
F x (^) b xa
x ae p
x a b
p b^ p
0 5
0 5
1 0 5 6 4 9
0 5
=
= ≤ ≤
−
−
(^2 )
1
2 π
ln / /
Φ ln Φ
Random Number Inverse CDF algorithm
- Generate Z from N(0,1).
- Set X = a exp0 5 bZ.
Normal 0 x a b ; , 5. − ∞ < x < ∞ − ∞ <, a < ∞, and b > 0
f x (^) b e
F x x^ ba x a b p p
x a b
p
0 5
0 5 0 5
=^0 5 4^9
= − = + ≤ ≤
− −
−
2 22
1
π
/
For Φ and Φ −^1 , see “Standard Normal Function” on p. 5.
Random Number Kinderman and Ramage (1976) method
- Generate as X^ = a^ + bz , where^ z^ is an^ N ( , )0 1^ random number. References. CDF: AS 66 (1973); ICDF: AS 111 (1977) and AS 241 (1988); RV: CACM 488 (1974) and Kinderman and Ramage (1976).
Pareto 0 x a b ; , 5. x ≥ a > 0 and b > 0
f x ba ax
F x ax x a p p
b
b
p b
0 5
0 5 0 5
=
= − = − ≤ ≤
−
1
1/
Random Number Inverse CDF
- Generate U from Uniform(0,1).
- Set X = a 01 − U 5 − 1/ b.
Student’s t 0 x a ; 5. − ∞ < x < ∞ and a > 0
f x a
x a
F x
a a x
a a a x
a (^) x
a 0 5 (^0 )
0 5
0 5 = + ^
^
^ ≤ ^
^ >
% &
KK
'
K K
1 −^ +
2 1 2
2 2
B a / 2,1 / 2 IB
x 0 1-^12 IB (^) +
/
x
a p a^ p
a p p
p a
− − ^
^
% &
KK
'
K K
−
−
(^1 )
1 2 12 12
IB
4 4 IB^3 0
When a = 1, this is the Cauchy distribution (Cauchy( x ; 0, 1)).
Random Number
Inverse CDF algorithm
- Generate U from Uniform(0,1).
- Set X = a (^) 1 − ln 1 0 − U (^) 56 1/ b.
Discrete Distributions
Bernoulli 0 x a ; 5. x = 0 1 and 0, ≤ a ≤ 1
Prob Prob
X x a a X x a^ xx
= = x^ − x ≤ = %&−^ == '
0 5 0 5 − 0 5
1
Random Number
Special case If a = 0 , X = 0. If a = 1 , X = 1. Direct algorithm for 0 < a < 1
- Generate U from Uniform(0,1).
- Set X = 1 if U ≤ a (a success) and X = 0 if U > a (a failure).
Binomial 0 x a b ; , 5. x = 0 1, ,..., a a ; = 1 2, ,..., and 0 ≤ b ≤ 1
Prob
Prob IB
X x ax b b
X x b x^ a^ x^ xx^ a a
= = x^ a^ x
^ − ≤ = %&−^ +^ −^ ==^ − '
0 5 0 5 −
0 5 0 5
Random Number
Special case If b = 0, X = 0. If b = 1, X = a. Algorithm BB due to Ahrens and Dieter (1974) for 0 < b < 1
- Set c = a d , = b k , = 0 , y = 0 , and h = 1.
- If c < 40 , generate J from Binomial( c , d ) using algorithm BU. X = k + J.
- If c is odd, go to step 4. If c is even, set c = c − 1 and generate U from Uniform(0,1). If U ≤ d , set k = k + 1.
- Set s = 0 c + 1 5 / 2 and generate S from Beta( s , s ). Set G = hS and Z = y + G.
- If Z ≤ b , set y = Z h , = h − d d , = 0 b − Z 5 / h ,and k = k + s ; otherwise set h = G and d = 0 b − y 5 / h.
- Set c = s − 1 and go to step 2. Computation time for algorithm BB is O(log a ). References. RV: Ahrens and Dieter (1974).
Geometric 0 x a ; 5. x = 1 2, ,... and 0 < a ≤ 1
Prob Prob
X x a a X x a
x x
0 5 0 5 − 0 5 0 5
1
Note. Geometric is a special case of the negative binomial ( x ; 1, a ).
Random Number
Special case If a = 1 , X = 1. Direct algorithm for 0 < a < 1
- Set X = 1.
- Generate U from Uniform(0,1).
- If U > a , set X = X + 1 and go to step 2; otherwise finish.
Negative Binomial 0 x a b ; , 5. x = a a , + 1 , K; a = 1 2, , K and 0 < b ≤ 1
Prob Prob IB
X x xa b b X x b a x a
= = − a^ x^ a −
^
^ − ≤ = − +
0 5 0 5 − 0 5 0 5
Random Number
Special case If b = 1 , X = a. Direct algorithm
- Generate G from Gamma 1 a b , / 1 0 − b 56.
- If G = 0 , go to step 1. Otherwise generate P from Poisson ( G ).
- Compute X = P + a.
Poisson 0 x a ; 5. x = 0 1 2, , , K and a > 0
Prob Prob IG ; +
X x ax e X x a x
x (^) a = = ≤ = −
0 5 − 0 5 0 5
Random Number
Algorithm PG due to Ahrens and Dieter (1974)
1. (Initialization) Set X = 0 and w = a.
- If w > 16, go to step 6.
- Set c = exp 0 − w 5 and p = 1.
- Generate U from Uniform(0,1). Set p = pU.
- If p < c , continue with step 6; otherwise set X = X + 1 and go to step 4.
- Set n = 7 w / 8. Generate G from Gamma( n , 1).
- If G > w, generate Y from Binomial 0 n −1, w / G 5 , set X = X + Y.
- If G ≤ w, set X = X + n w , = w − G , and go to step 2.
Notes. [y] means the integer part of y. Steps 3 to 5 of Algorithm PG are in fact the direct algorithm. References. RV: Ahrens and Dieter (1974).
Noncentral Continuous Distributions
Noncentral Beta 0 x a b c ; , , 5. 0 ≤ x ≤ 1 , a > 0 , b > 0 ,and c ≥ 0
f x (^) j c^ e x^ a j bx
F x (^) j c^ e x a j b
j (^) c a j b j j j
c
0 5 (^0 0 )
0 5 0 5
= (^) +−
= =
− + −^ −
∞
=
∞ −
∑
∑
2 1 1 0
0
2
! ;^ ,
/
/
B
IB
Note. c is the noncentrality parameter. If c = 0 , F ( x ) is the (central) beta distribution function. References. CDF: Abramowitz and Stegun (1965, Chapter 26), AS 226 (1987), and AS R84 (1990).
Noncentral Chi-Square 0 x a c ; , (^) 5. x ≥ 0 , a > 0 , and c ≥ 0
f x (^) j c^ e x^ e a j
F x (^) j c^ e x^ a^ j
j (^) c a j x a j j j j
c
0 5 0 5
0 5
=
= +
− (^) ++ −^ −
∞
=
∞ −
∑
∑
2 22 1 2 0
0
2
/ (^) //^ /
/
IG
The noncentral chi-square random variable is generated as the sum of squares of a independent normal random variates each with mean μ i and variance 1. Then c = (^) ∑μ (^) i^2_._ Note. c is the noncentrality parameter. If c = 0 , F ( x ) is the (central) chi-square distribution function. References. CDF: Abramowitz and Stegun (1965, Chapter 26), AS 170 (1981), AS 231 (1987). Density: AS 275 (1992).
Note. c is the noncentrality parameter. If c = 0 , F ( x ) is the (central) Student’s t distribution function. References. CDF: Abramowitz and Stegun (1965, Chapter 26), AS 5 (1968), AS 76 (1974), and AS 243 (1989).
Algorithm Index
AS 3: Cooper (1968a) AS 5: Cooper (1968b) AS 27: Taylor (1970) AS 32: Bhattacharjee (1970) AS 63: Majumder and Bhattacharjee (1973a) AS 64: Majumder and Bhattacharjee (1973b) AS 66: Hill (1973) AS 76: Young and Minder (1974) AS 91: Best and Roberts (1975) AS 109: Cran, Martin, and Thomas (1977) AS 111: Beasley and Springer (1977) AS 134: Atkinson and Whittaker (1979) AS 147: Lau (1980) AS 152: Lund (1980) AS 170: Narula and Desu (1981) AS 226: Lenth (1987) AS 231: Farebrother (1987) AS 239: Shea (1988) AS 241: Wichura (1988) AS 243: Lenth (1989) AS 245: Macleod (1989) AS 275: Ding (1992) AS R85: Shea (1991) CACM 291: Pike and Hill (1966)
CACM 299: Hill and Pike (1967) CACM 332: Dorrer (1968) CACM 395: Hill (1970a) CACM 396: Hill (1970b) CACM 451: Goldstein (1973) CACM 488: Brent (1974)
References
Abramowitz, M., and Stegun, I. A. eds. 1970. Handbook of mathematical functions. New York: Dover Publications. Ahrens, J. H., and Dieter, U. 1974. Computer methods for sampling from gamma, beta, Poisson and binomial distributions. Computing , 12: 223–246. Atkinson, A. C., and Whittaker, J. 1979. Algorithm AS 134: The generation of beta random variables with one parameter greater than and one parameter less than
- Applied Statistics , 28: 90–93. Beasley, J. D., and Springer, S. G. 1977. Algorithm AS 111: The percentage points of the normal distribution. Applied Statistics , 26: 118–121. Berger, R. L. 1991. AS R86: A remark on algorithm AS 152. Applied Statistics , 40: 374–375. Best, D. J., and Roberts, D. E. 1975. Algorithm AS 91: The percentage points of the c2 distribution. Applied Statistics , 24: 385–388. Bhattacharjee, G. P. 1970. Algorithm AS 32: The incomplete gamma integral. Applied Statistics , 19: 285–287. Box, G. E. P., and Muller, M. E. 1958. A note on the generation of random normal deviates. Annals of Mathematical Statistics , 29: 610–611. Bratley, P., Fox, B. L., and Schrage, L. E. 1987. A guide to simulation. New York: Springer-Verlag. Brent, R. P. 1974. Algorithm 488: A Gaussian pseudo–random number generator. Communications of the ACM, 17: 704–706.
