Download Moment Generating Functions 7.7 Properties of Normal ... and more Schemes and Mind Maps Statistics in PDF only on Docsity!
Moment Generating Functions 7.
Properties of Normal Random Variables 7.
Luwei Yang
November 30, 2020
1 Moment Generating Functions (Ross 7.7)
Moment Generating Function M (t) is
M (t) = E[e
tX ]
x
e
tx p(x) if X is discrete with mass function p(x)
e
tx f (x) dx if X is continuous with density f (x)
where t is a number, X is a random variable, and f (x) is a probability density function of X.
(Note: The function M (t) for t 6 = 0 might not exist)
We can think of the moment generating function M X (t) as a map from the probability density function
f (x) to a new function with e over the region where f (x) is not zero.
If two functions have the same moment generating function, then they must have the same distribution.
We call M (t) the moment generating function because all of the moments of X can be obtained by succes-
sively differentiating M (t) and then evaluating the result at t = 0.
For example:
M (0) = E[e
0 ] = E[1] = 1
The 1
st derivative of M(t) is
M
′ (t) =
∂t
e
tx f (x) dx
∂t
e
tx f (x) dx
xe
tx f (x) dx
Thus,
M
′ (0) =
xe
0 f (x) dx =
xf (x) dx = E[X]
The 2
nd derivative of M(t) is
M
′′ (t) =
∂t
M
′ (t)
∂t
E[Xe
tX ]
= E[
∂t
(Xe
tX )]
= E[X
2 e
tX ]
Thus,
M
′′ (0) = E[X
2 ]
In general, the n
th derivative of M(t) is
M
(n) (t) = E[X
n e
tX ] n ≥ 1
The n
th moment of X is
M
(n) (0) = E[X
n ]
1.1 Example 7d Standard Normal Distribution
Let Z be a standard normal random variable with parameters 0 and 1, we have
M (t) = E[e
tZ ] =
∞
−∞
e
tz
2 π
e
− z 2
(^2) dz = e
t 2
(^2) (steps in textbook)
M
′ (t) =
2 t
e
t 2
2 = te
t 2
2 M
′ (0) = 0 = M ean(E[Z])
M
′′ (t) = e
t 2
(^2) + t
2 e
t 2
(^2) M
′′ (0) = 1 = V ar[Z] = E[Z
2 ]
For an arbitrary normal random variable X = μ + σZ with parameters μ and σ
2 ,
M
X (t) = E[e
tX ] = E[e
t(μ+σZ) ] = e
tμ E[e
tσZ ] = e
tμ e
t 2 σ 2
(^2) = e
μt+ t 2 σ 2
2
If X, Y are independent, then
M (t) = E[e
t(X+Y ) ] = E[e
tX e
tY ] = E[e
tX ]E[e
tY ] = MX (t)MY (t)
If M (t) < ∞ and exists in some region about t = 0, then this uniquely defines the probability distribution.
2 Joint Moment Generating Function
If X 1 ,...,Xn have joint probability density function f (x 1 , ..., xn), then
M (t 1 , t 2 , ..., t n
e
t 1 x 1 +t 2 x 2 +...+tnxn f (x 1 , ..., x n ) d~x = E[e
t 1 x 1 +t 2 x 2 +...+tnxn ]
The individual moment generating functions can be obtained from M (t 1 , ..., t n ) by letting all but one of the
ti’s be 0. That is,
M
Xi (t) = M (0, ..., 0 , t i
where the ti is in the i
th place.
Since M determines the probability density function, and M depends only on μi, Cij , then the joint proba-
bility density function must only depend on μi, Cij.
f (x 1 , ..., xm) =
(2π)
m 2
|C|
e
−
1 2 ( ~ X−~μ)
T C
− 1 ( ~ X−~μ)
C =
C
11
· · · C
1 m
Cm 1 · · · Cmm
4.1 Proposition 8.
If X 1
, ..., X
n are independent and identically distributed normal random variables with mean μ and variance
σ
2 , then the sample mean X and the sample variance S
2 are independent. X is a normal random variable
with mean μ and variance
σ
2
n
(n−1)S
2
σ 2 is a chi-squared random variable with n − 1 degrees of freedom.
5 Additional Examples
5.1 Example 7h
Show that if X and Y are independent normal random variables with respective parameters (μ 1 , σ 1
2 ) and
(μ 2 , σ 2
2 ), then X + Y is normal with mean μ 1
2
2 .
Solution:
M
X+Y (t) = M X (t)M Y (t)
= (e
σ 1
2 t 2
2 +μ 1 t )(e
σ 2
2 t 2
2 +μ 2 t )
= e
(
(σ 1 2 +σ 2 2 )t 2
2 +(μ 1 +μ 2 )t)
which is the moment generating function of a normal random variable with mean μ 1
σ 1
2
2
. The desired result then follows because the moment generating function uniquely determines the
distribution.
5.2 Example 7L
Let X and Y be independent normal random variables, each with mean μ and variance σ
2
. Show that X + Y
and X − Y are independent by computing their joint moment generating function:
Solution:
E[e
t(X+Y )+s(X−Y ) ] = E[e
(t+s)X+(t−s)Y ]
= E[e
(t+s)X ]E[e
(t−s)Y ]
= (e
μ(t+s)+σ 2 (t+s) 2
2 )(e
μ(t−s)+σ 2 (t−s) 2
2 )
= (e
2 μt+σ
2 t
2
)(e
σ
2 s
2
)
We recognize the preceding as the joint moment generating function of the sum of a normal random variable
with mean 2μ and variance 2σ
2 (which is X + Y ) and an independent normal random variable with mean 0
and variance 2σ
2 (which is X − Y ). Because the joint moment generating function uniquely determines the
joint distribution, it follows that X + Y and X − Y are independent normal random variables.
5.3 Exercise 1
Suppose X has the moment generating function
M
X (t) = (1 − 2 t)
− 1 (^2) for t <
Find the first and second moments of X.
Solution:
We have
M
′
X
(t) = −
(1 − 2 t)
−
3 2 (−2) = (1 − 2 t)
−
3 2
M
′′
X
(t) = −
(1 − 2 t)
−
5 2 (−2) = 3(1 − 2 t)
−
5 2
So that
E[X] = M
′ X
−
3 2 = 1
E[X
2 ] = M
′′
X
−
5 (^2) = 3
5.4 Exercise 2
Suppose that you have a fair 4-sided die, and let X be the random variable representing the value of the
number rolled.
(a) Write down the moment generating function of X.
(b) Use this moment generating function to compute the first and second moments of X.
Solution:
(a):
MX (t) = E[e
tX ]
= e
1 ·t
2 ·t
3 ·t
4 ·t
(e
1 ·t
2 ·t
3 ·t
4 ·t )
(b): We have
M
′
X
(t) =
(e
1 ·t
2 ·t
3 ·t
4 ·t )
M
′′
X
(t) =
(e
1 ·t
2 ·t
3 ·t
4 ·t )
so
E[X] = M
′
X
and
E[X
2 ] = M
′′ X
