Download Simulating Systems: Random-Number Generation Properties, Generation, & Testing (80 charact and more Lecture notes Applied Mathematics in PDF only on Docsity!
Systems Simulation
Chapter 7: Random-Number Generation
Fatih Cavdur [email protected]
April 22, 2014
Systems Simulation Chapter 7: Random-Number Generation Introduction
Introduction
Random Numbers (RNs) are a necessary basic ingredient in the simulation of almost all discrete systems. Most computer languages have a subroutine, object or function that generates a RN. Similarly, simulation languages generate RNs that are used to generate event times and other random variables. We will look at the generation of RNs and some randomness tests in this chapter. Next chapter will show how we can use them to generate RVs.
Properties of RNs
Properties of RNs
A sequence of RNs, R 1 , R 2 ,.. ., must have two important statistical properties: uniformity and independence. Each RN, Ri must be an independent sample drawn from a continuous uniform distribution between 0 and 1.
f (r ) =
1 , 0 ≤ r ≤ 1 0 , otherwise
E (R) =
0
rdr =
V (R) = E (R^2 ) − [E (R)]^2 =
Systems Simulation Chapter 7: Random-Number Generation Properties of RNs
Properties of RNs
Some Consequences of Uniformity and Independence
If the interval [0, 1] is divided into n classes (sub-intervals) of equal length, the expected number of observations in each interval is N/n, where N is the total number of observations. The probability of observing a value in a particular interval is independent of the previous values drawn.
Techniques for RN Generation Linear Congruential Method
Linear Congruential Method
The linear congruential method (LCM) produces a sequence of integers, X 1 , X 2 ,... between 0 and m − 1 by following a recursive relationship. Xi+1 = (aXi + c) mod m, i = 0, 1 , 2 ,...
Ri = Xi m
, i = 1, 2 ,...
The initial value X 0 is called the seed, a is called the multiplier, c is the increment and m is the modulus. If c = 0, it is known as the multiplicative congruential method, and if c 6 = 0, it is called as the mixed congruential method.
Systems Simulation Chapter 7: Random-Number Generation Techniques for RN Generation Linear Congruential Method
Linear Congruential Method
Example
Use the LGM to generate a sequence of RNs with X 0 = 27, a = 17, c = 43 and m = 100.
X 0 = 27
X 1 = (17 × 27 + 43) mod 100 = 2 ⇒ R 1 =
X 2 = (17 × 2 + 43) mod 100 = 77 ⇒ R 2 =
X 3 = (17 × 77 + 43) mod 100 = 52 ⇒ R 3 =
Techniques for RN Generation Linear Congruential Method
Linear Congruential Method
Properties to Consider
Generated numbers must be approximately uniform and independent. Moreover, other properties, such as maximum density and maximum period must be considered. By maximum density is meant that the values assumed by Ri , i = 1, 2 ,.. ., leave no large gaps on [0, 1]. In many simulation languages, values such as m = 2^31 − 1 and m = 2^48 are in common use in generators. To help achieve maximum density and to avoid cycling, the generator should have the largest possible period.
Systems Simulation Chapter 7: Random-Number Generation Techniques for RN Generation Linear Congruential Method
Linear Congruential Method
Properties to Consider
(^1) For m a power of 2, say m = 2b, and c 6 = 0, the longest possible period is P = m = 2b, which is achieved whenever c is relatively prime to m (the greatest common factor of c and m is 1) and a = 1 + 4k, where k is an integer. (^2) For m a power of 2, say m = 2b, and c = 0, the longest possible period is P = m/4 = 2b−^2 , which is achieved if the seed X 0 is odd and if the multiplier a, is given by a = 3 + 8k or a = 5 + 8k, for some k = 0, 1 ,.. .. (^3) For m a prime number and c = 0, the longest possible period is P = m − 1, which is achieved whenever the multiplier, a, has the property that the smallest integer k such that ak^ − 1 is divisible by m is k = m − 1.
Techniques for RN Generation Linear Congruential Method
Linear Congruential Method
Properties to Consider-Example 3
With a = 3 , c = 0, prime number m = 17 and X 0 = 1, we have the following sequence with the max period of P = m − 1 = 16 when k = 16 is the smallest integer such that ak^ − 1 = 3^16 − 1 (which equals to 43,046,720) is divisible by k = m − 1 = 16 (verify that for k < 16, ak^ − 1 is not divisible by k = m − 1):
Table : Max Period i Xi i Xi 1 3 9 14 2 9 10 8 3 10 11 7 45 135 1213 124 6 15 14 2 7 11 15 6 8 16 16 1
Systems Simulation Chapter 7: Random-Number Generation Techniques for RN Generation Combined Linear Congruential Method
Combined Linear Congruential Generators
A RNG with a period of 2^31 − 1 ≈ 2 × 109 is no longer adequate due to the increasing complexity. So, combine two or more multiplicative congruential generators in such a way that the combined generator has good statistical properties and a longer period. If Wi 1 , Wi 2 ,... , Wik are any independent, discrete-valued RVs (not necessarily identically distributed), but one of them, say Wi 1 , is uniform on the integers from 0 to m 1 − 2, then, the following is uniform on the integers from 0 to m 1 − 2.
Wi =
∑^ k
j=
Wij
(^) mod m 1 − 1
Techniques for RN Generation Combined Linear Congruential Method
Combined Linear Congruential Generators
Let Xi 1 , Xi 2 ,... Xik be the ith output from k different multiplicative congruential generators.
Xi =
∑^ k
j=
(−1)j−^1 Xij
(^) mod m 1 − 1
Ri =
Xi m m 1 ,^ Xi^ >^0 1 −^1 m 1 ,^ Xi^ = 0
The maximum period is given by
P = (m 1 − 1)(m 2 − 1)... (mk − 1) 2 k−^1
Systems Simulation Chapter 7: Random-Number Generation Techniques for RN Generation Combined Linear Congruential Method
Combined Linear Congruential Generators
Algorithm by L’Ecuyer (1998)
Step (1) Select seed X 1 , 0 in the range [1, 2 , 147 , 483 , 562] for the first generator, and seed X 2 , 0 in the range [1, 2 , 147 , 483 , 398] for the second. Set j = 0. Step (2) Evaluate each individual generator.
X 1 ,j+1 = 40 , 014 X 1 ,j mod 2, 147 , 483 , 563 X 2 ,j+1 = 40 , 692 X 2 ,j mod 2, 147 , 483 , 399
Step (3) Set Xj+1 = (X 1 ,j+1 − X 2 ,j+1) mod 2, 147 , 483 , 562
Tests for RNs
Tests for RNs
To check on whether the desirable properties of uniformity and independence, a number of tests can be performed. The tests can be placed in two categories, according to the properties of interest: uniformity and independence. Frequency Test: Uses the Kolmogorov-Smirnov or the chi-square test o compare the distribution of the set of numbers generated to a uniform distribution. Autocorrelation Test: Tests the correlation between numbers and compares the sample compares the sample correlation to the expected correlation, zero.
Systems Simulation Chapter 7: Random-Number Generation Tests for RNs
Tests for RNs
In testing for uniformity, the hypotheses are as follows:
H 0 : Ri ∼ U[0, 1] H 1 : Ri ≁ U[0, 1]
In testing for uniformity, the hypotheses are as follows:
H 0 : Ri ∼ independently H 1 : Ri ≁ independently
Tests for RNs Frequency Tests
Frequency Tests
Kolmogorov-Smirnov (K-S) Test
This test compared the continuous CDF, F (x), of the uniform distribution with the empirical CDF, SN (x). We have F (x) = x, 0 ≤ x ≤ 1
The empirical CDF SN (x) defined by
SN (x) =
number of R 1 , R 2 ,... , RN which are ≤ x N
K-S test is based on the largest absolute deviation between D = max |F (x) − SN (x)|
Systems Simulation Chapter 7: Random-Number Generation Tests for RNs Frequency Tests
Frequency Tests
K-S Test
Step (1) Rank the data from smallest to largest. Let R(i), denote the ith smallest observation. Step (2) Compute
D+^ = max
i N − R(i)
D−^ = max
R(i) − i − 1 N
Tests for RNs Frequency Tests
Frequency Tests
K-S Test Example
Table : Calculations for K-S Test R(i) 0.05 0.14 0.44 0.81 0. i/N 0.20 0.40 0.60 0.80 1. i/N − R(i) 0.15 0.26 0.16 - 0. R(i) − (i − 1)/N 0.05 - 0.04 0.21 0.
Systems Simulation Chapter 7: Random-Number Generation Tests for RNs Frequency Tests
Frequency Tests
K-S Test Example
(^0) 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 x
R (1) R (2) R (3) R (4) R (5)
F ( x ) SN ( x ) Cumulative probability
Figure : Comparison of F (x) and SN (x)
Tests for RNs Frequency Tests
Frequency Tests
Chi-Square (C-S) Test
The C-S test uses the sample statistic
χ^20 =
∑^ n
i=
(Oi − Ei )^2 Ei
Oi and Ei are the observed and expected number in class i. For equally spaced classes,
Ei =
N
n
It can be shown that χ^20 is approximately chi-squared distributed with n − 1 degrees of freedom.
Systems Simulation Chapter 7: Random-Number Generation Tests for RNs Frequency Tests
Frequency Tests
C-S Test Example (Example 7.7 in DESS)
Considering the given data the following computations are done. Since χ^20 = 3. 4 < χ^20. 05 , 9 = 16.9, the null hypothesis is not rejected.
Table : Calculations for C-S Test Interval Oi Ei Oi − Ei (Oi − Ei )^2 (Oi^ −Ei^ )
2 Ei 1 8 10 -2 4 0. 2 8 10 -2 4 0. 3 10 10 0 0 0. ... ... ... ... ... ... 8 14 10 4 16 1. 9 10 10 0 0 0. 10 11 10 1 1 0. 100 100 0 3.
Tests for RNs Autocorrelation Tests
Autocorrelation Tests
Autocorrelation Test Example (Example 7.8 in DESS)
Considering the data in the text, we test for whether the 3rd, 8th, 13th and so on, numbers are autocorrelated using α = 0.05. Here, i = 3, m = 5, N = 30 and M = 4 (largest integer st 3 + (M + 1)5 ≤ 30). Then,
̂ ρim = 1 M + 1
( (^) M ∑ k=
[Ri+km]
[ Ri+(k+1)m
]
) − 0. 25
= 1 4 + 1 (.23(.28) + .28(.33) + .33(.27) + .27(.05) + .05(.36)) − 0. 25 = − 0. 1945
Systems Simulation Chapter 7: Random-Number Generation Tests for RNs Autocorrelation Tests
Autocorrelation Tests
Autocorrelation Test Example (Example 7.8 in DESS)
σ̂ (^) ρim =
√ 13 M + 7 12(M + 1) =
√ 13(4) + 7 12(4 + 1) = 0. 1280
Z 0 = ̂ ρim σ̂ (^) ρim = −
- 1945
- 1280 = − 1. 516
Since −z 0. 025 = − 1. 96 ≤ Z 0 ≤ 1 .96 = z 0. 025 , we cannot reject the null hypothesis.
Summary
Summary
Reading HW: Chapter 7. Chapter 7 Exercises.
