Download A Refresher on Probability and Statistics for Simulation and more Study notes Engineering in PDF only on Docsity!
Appendix C -- Handout 1
Appendix C – A Refresher on Probability and Statistics
Slide 1 of 27
Simulation with Arena, 3
rd^ ed.
Appendix C^ By Lian Qi
A Refresher onProbability and
Statistics
Appendix C – A Refresher on Probability and Statistics
Slide 2 of 27
Simulation with Arena, 3
rd^ ed.
What We’ll Do ...
•^ Review of probability and statistics necessary todo and understand simulation •^ Outline
Probability – basic ideas, terminology Random variables Sampling Statistical inference – point estimation, confidence intervals
Appendix C – A Refresher on Probability and Statistics
Slide 3 of 27
Simulation with Arena, 3
Probability Basics rd ed.
•^ Experiment
- activity with uncertain outcome
Flip coins, throw dice, pick cards, draw balls from urn, … Operate
a (
real
) call center – Number of calls? Average
customer hold time? Number of customers getting busysignal? Simulate
a call center – same questions as above
•^ Sample space
- complete list of all possible
individual outcomes of an experiment^ ^
Could be easy or hard to characterize
Appendix C – A Refresher on Probability and Statistics
Slide 4 of 27
Simulation with Arena, 3
Probability Basics rd^ ed.
(cont’d.)
•^ Event
- a subset of the sample space Union, intersection, complementation operations
•^ Probability
of an event is the relative likelihood
that it will occur when you do the experiment^ ^
A real number between 0 and 1 (inclusively) Denote by
P (
E ),
P (
E^ ∩
F ), etc.
Interpretation – proportion of time the event occurs in manyindependent repetitions (replications) of the experiment
Appendix C -- Handout 2
Appendix C – A Refresher on Probability and Statistics
Slide 5 of 27
Simulation with Arena, 3
Probability Basics rd^ ed.
(cont’d.)
-^ Some properties of probabilities
If^ S
is the sample space, then
P (
S ) = 1
Can have event
E^ ≠
S^ with
P ( E
) = 1
If Ø is the empty event (empty set), then
P (Ø) = 0
Can have event
E^ ≠
Ø with
P ( E
) = 0
If^ E
C^ is the complement of
E , then
P (
CE
P (
E )
P ( E
∪^
F ) =
P (
E ) +
P (
F ) –
P (
E^ ∩
F )
If^ E
and
F^
are mutually exclusive (i.e.,
E^ ∩
F =
Ø), then
P ( E
∪^
F ) =
P (
E ) +
P (
F )
If^ E
is a subset of
F^ (i.e., the occurrence of
E^
implies the
occurrence of
F ), then
P (
E )^
≤^ P
( F )
If^ o
,^ o 1
, … are the individual outcomes in the sample space, 2 then
Appendix C – A Refresher on Probability and Statistics
Slide 6 of 27
Simulation with Arena, 3
Probability Basics rd^ ed.
(cont’d.)
-^ Conditional
probability
Knowing that an event
F^
occurred might affect the
probability that another event
E^
also occurred
Reduce the effective sample space from
S^
to^ F
, then
measure “size” of
E^
relative to its overlap (if any) in
F ,
rather than relative to
S
Definition (assuming
P (
F )^
≠^ 0):
-^
E^ and
F^
are
independent
if^
P (
E^ ∩∩∩∩
F ) =
P (
E )
P (
F )
Implies
P (
E | F
) =^
P ( E
) and
P (
F | E
) =^
P ( F
), i.e., knowing that
one event occurs tells you nothing about the other If^ E
and
F^
are mutually exclusive, are they independent?
Appendix C – A Refresher on Probability and Statistics
Slide 7 of 27
Simulation with Arena, 3
Random Variables rd ed.
-^ A
random variable
(RV) is a number whose value
is determined by the outcome of an experiment^ ^
Technically, a function or mapping from the sample spaceto the real numbers, but can usually define and work with aRV without going all the way back to the sample space Think: RV is a number whose value we don’t know for surebut we’ll usually know something about what it can be or islikely to be Usually denoted as capital letters:
X ,
Y ,
W^1
,^ W
, etc. 2
-^ Probabilistic behavior described by
distribution
function
Appendix C – A Refresher on Probability and Statistics
Slide 8 of 27
Simulation with Arena, 3
rd^ ed. Discrete vs. Continuous RVs
-^ Discrete - can take on only certain separated
values^ ^
Number of possible values could be finite or infinite
-^ Continuous - can take on any real value in some
range^ ^
Number of possible values is always infinite Range could be bounded on both sides, just one side, orneither
Appendix C -- Handout 4
Appendix C – A Refresher on Probability and Statistics
Slide 13 of 27
Simulation with Arena, 3
Discrete Variances andStandard Deviations rd^ ed.
-^ Data set has measures of “dispersion” –
Sample variance Sample standard deviation
-^ RVs have corresponding measures
Other common notation: Weighted average of squared deviations of the possiblevalues
xi^
from the mean
Standard deviation of
X^
is
Appendix C – A Refresher on Probability and Statistics
Slide 14 of 27
Simulation with Arena, 3
rd^ ed. Continuous Distributions
-^ Now let
X^
be a continuous RV
Possibly limited to a range bounded on left or right or both No matter how small the range, the number of possiblevalues for
X^
is always (uncountably) infinite
Not sensible to ask about
P (
X^ =
x ) even if x is in the
possible range Technically,
P (
X^ =
x ) is always 0
Instead, describe behavior of X in terms of its falling^ between
two values
Appendix C – A Refresher on Probability and Statistics
Slide 15 of 27
Simulation with Arena, 3
rd^ ed. Continuous Distributions
(cont’d.)
-^ Probability density function
(PDF) is a function
f ( x
) with the following three properties: f ( x )^ ≥
0 for all real values
x
The total area under
f ( x
) is 1:
For any fixed
a^ and
b^ with
a^ ≤
b , the probability that
X^
will fall
between
a^ and
b^ is the area under
f ( x
) between
a^ and
b :
-^ Fun facts about PDFs
Observed
X ’s are denser in regions where
f ( x
) is high
The height of a density,
f ( x
), is not the probability of
anything – it can even be > 1 With continuous RVs, you can be sloppy with weak vs.strong inequalities and endpoints
Appendix C – A Refresher on Probability and Statistics
Slide 16 of 27
Simulation with Arena, 3
rd^ ed. Continuous Distributions
(cont’d.)
-^ Cumulative distribution function
(CDF) -
probability that the RV will be
≤≤≤≤^
a fixed value
x :
-^ Properties of continuous CDFs
F (
x )^ ≤
1 for all
x
As^
x^ →
,^ F (
x )^ →
As^
x^ →
,^ F (
x )^ →
F ( x
) is nondecreasing in
x
F ( x
) is a continuous function with slope equal to the PDF:
f ( x ) =
F '(
These four propertiesare also true ofdiscrete CDFsx )
F(x) may ormay not havea closed-formformula
Appendix C -- Handout 5
Appendix C – A Refresher on Probability and Statistics
Slide 17 of 27
Simulation with Arena, 3
rd^ ed.
Continuous Expected Values, Variances, and Standard Deviations
-^ Expectation or mean of X is
Roughly, a weighted “continuous” average of possiblevalues for X Same interpretation as in discrete case: average of a largenumber (infinite) of observations on the RV X
-^ Variance of X is •^ Standard deviation of X is
Appendix C – A Refresher on Probability and Statistics
Slide 18 of 27
Simulation with Arena, 3
rd^ ed.
Independent RVs
-^ X
and 1
X^2
are
independent
if their joint CDF
factors into the product of their marginal CDFs:^ ^
Equivalent to use PMF or PDF instead of CDF
-^ Properties of independent RVs:
They have nothing (linearly) to do with each other Important in probability – factorization simplifies greatly
-^ Independence in simulation
Input: Usually assume separate inputs are indep. – valid? Output: Standard statistics assumes indep. – valid?!?!?!?
Appendix C – A Refresher on Probability and Statistics
Slide 19 of 27
Simulation with Arena, 3
Additional Knowledge rd^ ed.
-^ E
(aX)=a
E (X)
-^ E
(aX+bY)=a
E (X)+b
E (Y)
-^ Var
(aX+b)=a
2 Var
(X)
-^ Var
(aX+bY )=a
2 Var
(X)+b
2 Var
(Y)+2ab
Cov
(X,Y)
-^ Covariance of two RVs (measure linear relation)^ Cov
(X,Y)=
E ((X-
E (X))(Y-
E (Y)))=
E (XY)-
E (X)
E (Y)
-^ If two RVs, X and Y, are independent,
Cov
(X,Y)=
Appendix C – A Refresher on Probability and Statistics
Slide 20 of 27
Simulation with Arena, 3
rd^ ed.
Sampling
-^ Statistical analysis - estimate or infer something
about a
population
or
process
based on only a
sample
from it
Think of a RV with a distribution governing the population Random sample
is a set of
independent and identically
distributed
(IID) observations
X^1
,^ X
Xn
on this RV
In simulation, sampling is making some runs of the modeland collecting the output data Don’t know
parameters
of population (or distribution) and
want to estimate them or infer something about them basedon the sample
Appendix C -- Handout 7
Appendix C – A Refresher on Probability and Statistics
Slide 25 of 27
Simulation with Arena, 3
Confidence Intervals rd^ ed.
-^ A point estimator is just a single number, withsome uncertainty or variability associated with it •^ Confidence interval
quantifies the likely
imprecision in a point estimator^ ^
An interval that contains (
covers
) the unknown population
parameter with specified (high) probability 1 –
α
Called a 100 (1 –
α)% confidence interval for the parameter
Appendix C – A Refresher on Probability and Statistics
Slide 26 of 27
Simulation with Arena, 3
rd^ ed. Confidence Intervals
(cont’d.)
-^ Confidence interval for the population mean
μμμμ :
If we know the population variance
If we don’t know the population variance
-^ CIs for some other parameters – in text
tn -1,1-
α/^
is point below which is area 1 –
α/2 in Student’s
t^ distribution with
n^ – 1 degrees of freedom
s n
t
X^
n^
(^2) / (^1) , 1 α−−
z 1-α
is point below which is area/ 1 –
α/2 in standard normal distribution
n
z
X
α^2 / 1 −
Appendix C – A Refresher on Probability and Statistics
Slide 27 of 27
Simulation with Arena, 3
rd^ ed.
Confidence Intervals in Simulation
-^ Run simulations, get results •^ View each replication of the simulation as a datapoint •^ Random input
random output
-^ Form a confidence interval •^ Brackets (with probability 1 –
αααα ) the “true”
expected output (what you’d get by averaging aninfinite number of replications)
