Download Empirical Distributions and more Schemes and Mind Maps Algebra in PDF only on Docsity!
Empirical Distributions
An empirical distribution is one for which each possible event is assigned a probability derived from experimental observation. It is assumed that the events are independent and the sum of the probabilities is 1.
An empirical distribution may represent either a continuous or a discrete distribution. If it represents a discrete distribution, then sampling is done “on step”. If it represents a continuous distribution, then sampling is done via “interpolation”. The way the table is described usually determines if an empirical distribution is to be handled discretely or continuously; e.g.,
discrete description continuous description value probability value probability 10 .1 0 – 10 -^. 20 .15 10 – 20 -^. 35 .4 20 – 35 -^. 40 .3 35 – 40 -^. 60 .05 40 – 60 -^.
To use linear interpolation for continuous sampling, the discrete points on the end of each step need to be connected by line segments. This is represented in the graph below by the green line segments. The steps are represented in blue:
In the discrete case, sampling on step is accomplished by accumulating probabilities from the original table; e.g., for x = 0.4, accumulate probabilities until the cumulative probability exceeds 0.4; rsample is the event value at the point this happens (i.e., the cumulative probability 0.1+0.15+0.4 is the first to exceed 0.4, so the rsample value is 35).
In the continuous case, the end points of the probability accumulation are needed, in this case x=0.25 and x=0.65 which represent the points (.25,20) and (.65,35) on the graph. From basic college algebra, the slope of the line segment is (35-20)/(.65-.25) = 15/.4 = 37.5. Then slope = 37.5 = (35- rsample )/(.65-.4) so rsample = 35 - (37.5×.25) = 35 – 9.375 = 25.625.
rsample
0 .5 1
60 50 40 30 20 10 (^0) x
Discrete Distributions
To put a little historical perspective behind the names used with these distributions, James Bernoulli (1654-1705) was a Swiss mathematician whose book Ars Conjectandi (published posthumously in 1713) was the first significant book on probability; it gathered together the ideas on counting, and among other things provided a proof of the binomial theorem. Siméon-Denis Poisson (1781-
- was a professor of mathematics at the Faculté des Sciences whose 1837 text Recherchés sur la probabilité des jugements en matière criminelle et en matière civile introduced the discrete distribution now called the Poisson distribution. Keep in mind that scholars such as these evolved their theories with the objective of providing sophisticated abstract models of real-world phenomena (an effort which, among other things, gave birth to the calculus as a major modeling tool).
I. Bernoulli Distribution
A Bernoulli event is one for which the probability the event occurs is p and the probability the event does not occur is 1-p; i.e., the event is has two possible outcomes (usually viewed as success or failure) occurring with probability p and 1-p, respectively. A Bernoulli trial is an instantiation of a Bernoulli event. So long as the probability of success or failure remains the same from trial to trial (i.e., each trial is independent of the others), a sequence of Bernoulli trials is called a Bernoulli process. Among other conclusions that could be reached, this means that for n trials, the probability of n successes is p n^.
A Bernoulli distribution is the pair of probabilities of a Bernoulli event, which is too simple to be interesting. However, it is implicitly used in “yes- no” decision processes where the choice occurs with the same probability from trial to trial (e.g., the customer chooses to go down aisle 1 with probability p) and can be cast in the same kind of mathematical notation used to describe more complex distributions: p z^ (1-p) 1-z^ for z = 0, p(z) = 0 otherwise
While this is notational overkill for such a simple distribution, it’s construction in this form will be useful for understanding other distributions.
p(z) E(X) = (1 - p)⋅ 0 +p⋅ 1 =p
The expected value of the distribution is given by
The standard deviation is given by
σ = (1-p)(0-p)^2 +p(1-p)^2 = p⋅(1 - p)
z
1-p
p
II. Binomial Distribution
The Bernoulli distribution represents the success or failure of a single Bernoulli trial. The Binomial Distribution represents the number of successes and failures in n independent Bernoulli trials for some given value of n. For example, if a manufactured item is defective with probability p, then the binomial distribution represents the number of successes and failures in a lot of n items. In particular, sampling from this distribution gives a count of the number of defective items in a sample lot. Another example is the number of heads obtained in tossing a coin n times.
The binomial distribution gets its name from the binomial theorem which states that the binomial
It is worth pointing out that if a = b = 1, this becomes
Yet another viewpoint is that if S is a set of size n, the number of k element subsets of S is given by
This formula is the result of a simple counting analysis: there are
ordered ways to select k elements from n (n ways to choose the 1 st^ item, (n-1) the 2 nd^ , and so on). Any given selection is a permutation of its k elements, so the underlying subset is counted k! times. Dividing by k! eliminates the duplicates.
Note that the expression for 2n^ counts the total number of subsets of an n-element set.
For n independent Bernoulli trials the pdf of the binomial distribution is given by
p(z) =
0 otherwise
Note that
k!(n-k)!
n! k
n a b where k
n a b k n-k
n
0
n (^) = ⎟⎟ ⎠
k
n 1 1 2
n
0
n n
k
n k!(n-k)!
n!
(n-k)!
n! n ⋅(n-1)⋅...⋅(n-k+1)=
p (1 p) forz 0,1,..., n z
n (^) z n z ⎟⎟ − = ⎠
by the binomialtheorem, p(z) (p (1-p))n 1 ,verifyingthatp(z)isa pdf.
n
0
∑ = +^ =
When choosing z items from among n items, the term
represents the probability that z are defective (and concomitantly that (n-z) are not defective).
The binomial theorem is also the key for determining the expected value E(X) for the random variable X for the distribution. E(X) is given by
(the expected value is just the sum of the discrete items weighted by their probabilities, which corresponds to a sample’s mean value; this is an extension of the simple average value obtained by dividing by n, which corresponds to a weighted sum with each item having probability 1/n).
For the binomial distribution the calculation of E(X) is accomplished by
This gives the result that E(X) = np for a binomial distribution on n items where probability of success is p. It can be similarly shown that the standard deviation is
The binomial distribution with n=10 and p=0.7 appears as follows:
pz^ (1 p)nz z
n (^) − ⎟⎟ − ⎠
i i
n
1
E(X) = ∑p(z)⋅z
p (1 p) z z!(n-z)!
n! p (1 p) z z
n E(X)
n
1
z n-z z n-z n
0
⎟⎟ − ⋅ = −^ ⋅
p (1-p) np(p 1 - p) np z- 1
n- 1 p (1-p) np np z- 1
n - 1 n z- 1 n-z n- 1 1
z- 1 n-z n
1
⎟⎟ = +^ =
term in common
present in every summand
p(z)
0
0 1 2 3 4 5 6 7 8 9 10
z
apply the binomial theorem to this (note that n-z = (n-1) - (z-1) )
np ⋅( 1 −p )
mean
III. Poisson Distribution (values z = 0, 1, 2,.. .)
The Poisson distribution is the limiting case of the binomial distribution where p=λ/n → 0 and n → ∞.
The expected value E(X) = λ. The standard deviation is. The pdf is given by
This distribution dates back to Poisson's 1837 text regarding civil and criminal matters, in effect scotching the tale that its first use was for modeling deaths from the kicks of horses in the Prussian army. In addition to modeling the number of arrivals over some interval of time (recall the relationship to the exponential distribution; a Poisson process has exponentially distributed interarrival times), the distribution has also been used to model the number of defects on a manufactured article. In general the Poisson distribution is used for situations where the probability of an event occurring is very small, but the number of trials is very large (so the event is expected to actually occur a few times).
Graphically, with λ = 2, it appears as:
The sampling function looks like:
0
10 9 8 7 6 5 4 3 2 1 0
z!
λ e p(z)
z (^) ⋅ − λ
z
p(z)
0 1 2 3 4 5 6 7 8 9...
0 1
x
rsample
mean
λ
0
1 2 3 4 5 6 7 8 9 10
z
p(z)
IV. Geometric Distribution
The geometric distribution gets its name from the geometric series:
The pdf for the geometric distribution is given by
p(z) = 0 otherwise
The geometric distribution is the discrete analog of the exponential distribution. Like the exponential distribution, it is "memoryless"; i.e., P(X > a+b | X > a) = P(X > b) (the geometric distribution is the only discrete distribution with this property just as the exponential distribution is the only continuous one behaving in this manner).
Its expected value is given by
(by applying the 3 rd^ form of the geometric series).
The standard deviation is given by p
1 − p .
A plot of the geometric distribution with p = 0.3 is given by
various flavors of the geometric series
2 0
n 2 0
n 0
n (1-r)
, (n 1) r (1-r)
r , n r 1 - r
for r< 1,∑ r = ∑ ⋅ = ∑ + ⋅ =
∞ ∞ ∞
(1 −p)z^ -^1 ⋅p forz=1,2,...
p
(1- 1 p)
E(X) z(1-p) p p 2 1
z - (^1) =
∞
mean
