Probability Distributions - Lecture Notes | STAT 3000, Study notes of Statistics

Download Probability Distributions - Lecture Notes | STAT 3000 and more Study notes Statistics in PDF only on Docsity!

Unit 4:

Probability distributions

5.0 Theoretical Probability Distribution

So far we have learned how to:

− Construct a probability mass function (for discrete rv); − Construct a probability density function (for continuous rv); − Construct a cumulative distribution function; − Calculate various probabilities (using pmf, pdf, or cdf) − Calculate expected value, variance, etc. of a rv.

But if we do a data analysis, obtaining all these things would be difficult, even impossible (because we have a sample, i.e., a limited number of observations and thus we do not know the exact distribution in the population).

However, we can assume that our data follow a certain THEORETICAL PROBABILITY DISTRIBUTION.

Often, observations generated by different statistical experiments (or sampling) have the same type of behavior ‡ they can be described by the same probability distribution.

Therefore, we (usually) do not have to calculate complicated integrals etc., because for theoretical probability distributions exist simple formulas and/or tables that we can use to calculate different probabilities, expected values, variances, etc.

A theoretical probability distribution is described by numbers called PARAMETERS.

Expectations and variances of a theoretical distribution (or a random variable that follows that distribution) are functions of these parameters.

In this course we will learn about the following theoretical probability distributions:

v Discrete probability distributions:

• Binomial distribution

v Continuous probability distributions:

• Normal distribution − t - distribution − χ^2 - distribution − F – distribution

Let X be a random variable indicating gender of a baby, with the values:

− 1 if the baby is a boy ("success") and − 0 if the baby is a girl ("failure") X ~ Bern ( p )

p ‡ probability of having a boy (probability of success) 0 ≤ p ≤ 1

p ‡ parameter of the Bernoulli distribution

Assume that the probability of having a boy is 0.49, i.e., p = 0.49. Then,

X ~ Bern (0.49)

• What is the probability distribution of X?
• What is the expected value of X?
• What is the variance of X?

Now, let's assume that we have more children, i.e., we repeat this random experiment (having a baby) n times …

Then, in this situation:

− The gender of each baby is either boy or girl (success or failure); − The probability p of having a boy is same in every trial; − All trials are independent: the gender of one baby does not influence the gender of other babies (i.e., if your first baby is a boy, it will not change the probability that your second baby is a boy too).

Therefore, we have n Bernoulli trials:

− All n trials are independent; − Each trial: success or failure only Binomial setting − Probability p of success is same in each trial (binomial trial)

Now, we are interested in number of boys among n children.

Let X be a random variable indicating number of boys (i.e., number of successes) among n children.

X ~ B ( n , p )

X has a binomial distribution described by two parameters:

− n - total number of trials − p - probability of success in each trial.

X is called binomial random variable.

5.1.2 Calculating binomial probabilities

Imagine you plan to have 5 children. What is the probability distribution of number of boys among your 5 children?

n = 5 X ‡ number of boys (out of 5 children) p = 0.49 X ~ B (5, 0.49)

Possible values of X are : ……………………………………………………

P(X = x) =?

• What is the probability of having not more than 3 boys out of 5 children?
• What is the probability of having between 2 and 4 boys out of 5 children?

5.1.2.1 Computer tools for calculating binomial probabilities:

Microsoft Excel:

• Statistical Function BINOMDIST ( x , n , cumulative

Internet tools (Java applets):

• Distribution/Density Calculators, Plotters, and RNG's: http://www.stat.ucla.edu/calculators/cdf/
• Probability calculators (within Hyperstat): http://playfair.stanford.edu/~naras/jsm/examplebot.html

0 - no (pmf will be calculated: P ( X = x )) 1 - yes (cdf will be calculated: P ( X ≤ x ))

5.1.3 Expected value and variance of the binomial distribution

Recall binomial setting:

− n independent events (trials); − Each trial with only 2 outcomes: success or failure; − Probability p of success is same in each trial.

X ~ B ( n , p ) ‡ binomial rv X that counts number of successes

Binomial rv X is a sum of n Bernoulli rv Xi , i = 1, …, n.

X = X 1 + X 2 +.......+ X n E ( Xi )= p

Var ( Xi )= p ( 1 − p )

Therefore, the expected value of X is:

And the variance of X is:

• What is the expected value and the variance of proportion of boys in a family with 5 children?

X ‡ number of boys X ~ B (5, .49)

• Which value of p would maximize this variance?

5.2 Normal distribution

− The normal distribution is the most important continuous probability distribution.

− The normal distribution is a natural distribution of many naturally occurring phenomena (physical measurements, errors in scientific measurements …)

− It is used as a base for many statistical inference methods

− It is used to model sampling distributions, error distributions …

− Many other important distributions have been derived from the normal distribution - and, on the other hand, the normal distribution is widely used to approximate other distributions.

− Normal distribution - "mother of all other distributions".

Normal distribution is called Gaussian or "bell-shaped" curve ‡ NORMAL CURVE.

A random variable X that has a normal distribution is called normal random variable.

X ~ N ( μ , σ^2 ) μ ‡ mean σ^2 ‡ variance

μ and σ^2 are parameters of the normal distribution.

The probability density function of the normal distribution is:

2 2

( )^2

2

( ) σ

μ

σ π

−^ − = ⋅

x f x e for −∞ ≤ x ≤ ∞

Plot of the probability density function of a normal distribution:

5.2.2 Standard normal distribution

Standard normal distribution is a normal distribution with μ = 0 and σ^2 = 1.

If rv X follows a standard normal distribution, it is called standard normal rv :

X ~ N (0,1)

Standard normal distribution is a very special distribution ‡ because of that the probability density function of the standard normal distribution is denoted by φ ( x ) (greek letter "phi").

The formula of the pdf of a standard normal distribution is

2

2

2

x x e

− = ⋅ π

φ for −∞ ≤ x ≤ ∞

The cumulative distribution function of the standard normal distribution is denoted by Φ( x ).

∫ −∞

x ( x ) φ ( y ) dy fl this integral does not have closed form solution

To calculate Φ( x ) we use tables.

If X ~ N (0,1), then Φ( x ) = P ( X ≤ x )

CDF of a standard normal distribution is a "S -shaped" line.

Φ(0) = 0.5 ‡ median = 0, because N (0,1) is symmetric about 0.

At x = 0 is also the point of inflection of the cdf curve.

If x → 0, Φ( x ) → 0

If x → ∞, Φ( x ) → 1

5.2.3 Probability calculation using the normal distribution

Example 1:

Suppose that Z ~ N (0,1). Find:

a) P ( Z ≤ 1.15).

P ( Z ≤ 1.15) = Φ (1.15)

To find Φ (1.15) we use Table I in the Hayter book (pp. 907 - 908). This table contains values of the cumulative distribution function (i.e., the area under the pdf) of the standard normal distribution.

e) P (| Z | ≤ 1.15).

f) The value of x for which P ( Z ≤ x ) = 0.75.

g) The value of x for which P (| Z | ≤ x ) = 0.75.

Example 2: "Real life example":

In pig production, average daily gain (ADG) of pigs is one of the main factors that influence efficiency of production. High ADG is desirable and pig breeders try to improve it through selection of pigs with highest ADG for further breeding.

Jim K. is a pig breeder. He has a large herd of pigs in which ADG is normally distributed with mean 800 g/day and standard deviation 75 g/day. He is interested in improving ADG in his herd and asks you for help. Here are his questions:

a) "If I keep for further breeding only pigs with ADG of 900g/day and higher, what proportion of pigs will be kept?"

First step in solving this problem is to standardize the normal random variable of interest.

If X ~ N ( μ , σ^2 ) and σ

− μ

X

Z , then Z ~ N (0, 1) and we can use Table I to

calculate the probabilities.

5.2.4 Critical points of the standard normal distribution

In the previous examples we saw how we can use Table I to determine quantiles (or percentiles ) of the standard normal distribution.

• z = −1.28 ‡ 10 th^ percentile (or 0.1 quantile)
• z = 1.28 ‡ 90 th^ percentile (or 0.9 quantile)

Some of the percentiles of the standard normal distribution are used frequently in statistical testing.

They are called CRITICAL POINTS and have their special notation − zα

α is the probability that the standard normal random variable takes values as large as or larger than zα.

Φ ( zα ) = 1 − α

The most important critical points are (shown in Table I):

z .05 = 1.

z .01 = 2.

When you conduct a statistical test (e.g., z -test) you calculate a test statistic (from the sample) in order to test your hypothesis … If the calculated test statistic is larger than the critical point zα, for a chosen α , you will reject your hypothesis at α significance level (more about it in Chapter 8!).

5.2.5 Linear combination of normal random variables

5.2.5.1 Linear function of a normal rv

X ~ N ( μ , σ^2 ) E ( X ) = μ Var ( X ) = σ^2 Y = aX + b E ( Y ) = aE ( X ) + b = a μ + b Var(Y) = a^2 Var(X) = a^2 σ^2 Y ~ N (a μ + b , a^2 σ^2 )

5.2.5.2 Sum of two normal rv

X (^) 1 ~ N ( μ 1 , σ^21 ) E ( X 1 )= μ 1 Var ( X 1 )= σ^21 X 1 and X 2

X (^) 2 ~ N ( μ 2 , σ 22 ) E ( X 2 )= μ 2 Var ( X 2 )= σ^22 ‡ independent

Y = X 1 + X 2 E ( Y )= E ( X 1 )+ E ( X 2 ) = μ (^) 1 + μ 2 2 2

2 Var ( Y )= Var ( X 1 )+ Var ( X 2 ) = σ (^) 1 + σ

Y ~ N ( μ (^) 1 + μ 2 , σ^21 + σ^22 ) σ (^) Y = σ 12 + σ^22

5.2.5.3 Sum of n normal rv

Population: X ~ N ( μ , σ^2 )

Sample of size n : X 1 , X 2 , X 3 , ……., X n ‡ n iid normal rv's

Each Xi ~ N ( μ , σ^2 ); E ( Xi ) = μ ; Var ( Xi ) = σ^2

Sum of n iid normal rv's: Y = X 1 + X 2 + X 3 + …….+ X n

E(Y) = E ( X 1 + X 2 + X 3 + …….+ X n) = E ( X 1 ) + E ( X 2 ) + ….. + E ( Xn )

= μ + μ + μ + …… + μ = nμ

Var(Y) = Var ( X 1 + X 2 + X 3 + …….+ X n) = Var ( X 1 ) + Var ( X 2 ) + ….. + Var ( Xn )

= σ^2 + σ^2 + σ^2 + …… + σ^2 = nσ^2

Y ~ N ( nμ , nσ^2 )