Download Moment Generating Functions - Lecture Notes | MATH 331 and more Study notes Mathematics in PDF only on Docsity!
Section 11.1 lecture notes Math331, Fall 2008 Instructor: David Anderson
Section 11.1: Moment Generating Functions
Hw, pgs. 465 - 466, #’s 1, 3, 6, 7, 9, 17.
The moments of a random variable tell you things. Ex: first moment gives mean, second gives spread, third gives skewness, etc. Also, if
E(|X|k) < ∞,
then
P (|X| > n) ≤
E[|X|k] nk^
This says that P (|X| > n) approaches zero at least as fast as 1/nk^ as n → ∞.
Proof:
E[|X|k] = E[|X|k (^1) |X|>n] + E[|X|k (^1) |X|≤n] ≥ E[|X|k (^1) |X|>n] ≥ nkE[1|X|>n] = nkP {|X| > n}
=⇒ P {|X| > n} ≤
E[|X|k] nk^
Moment generating functions have two major properties:
- They allow us to calculate moments of RV.
- No two different RVs have same moment generating function. Thus, to prove a RV has a certain distribution, you really only need moment gen. function.
Definition: For a random variable X, the moment generating function of X is
MX (t) = E
[
etX^
]
Therefore, for discrete, continuous RV we have
MX (t) =
x∈R(X)
etxpX (x) discrete case
MX (t) =
−∞
etxf (x)dx continuous case.
Example 1. Let X be a Bernoulli RV with parameter p. Then:
MX (t) = E
[
etX^
]
= et∗^0 ∗ P {X = 0} + et∗^1 ∗ P {X = 1} = (1 − p)et∗^0 + pet∗^1 = (1 − p) + pet.
Example 2. Let X be binomial(n, p). Then, letting q = 1 − p, we have
MX (t) = E
[
etX^
]
∑^ n
x=
etx
n x
pxqn−x^ =
∑^ n
x=
n x
(pet)xqn−x^ =
pet^ + q
)n .
Theorem 1. Let X be a RV with moment genenrating function MX (t). Let M X(n )(0) be nth derivative evaluated at x = 0. Then
E [Xn] = M X(n )(0).
Proof. Will do for discrete case.
M X′ (t) =
d dt
E
[
etX^
]
d dt
x∈R(X)
etxpX (x) =
x∈R(X)
xetxpX (x)
=⇒ M X′ (0) =
x∈R(X)
xe^0 ∗xpX (x) =
x∈R(X)
xpX (x) = E[X].
M X(n )(t) =
d dt
)(n) (^) ∑
x∈R(X)
etxpX (x) =
x∈R(X)
xnetxpX (x)
=⇒ M X(n )(0) =
x∈R(X)
xne^0 ∗xpX (x) =
x∈R(X)
xnpX (x) = E[Xn].
Back to examples.
Example 3. Let X be Bernoulli with parameter p. Then MX (t) = (1 − p) + pet. Therefore,
M X(n )(t) = pet^ for all n > 0. Thus, E[Xn] = M X(n )(0) = p.
Example 4. Let X be binomial(n, p). Then MX (t) = (pet^ + q)n^ Therefore,
M X′ (t) = npet(pet^ + q)n−^1 M X′′ (t) = npet(pet^ + q)n−^1 + n(n − 1)(pet)^2 (pet^ + q)n−^2.
Thus,
E[X] = M X′ (0) = np E[X^2 ] = M X′′ (0) = np + n(n − 1)p^2 =⇒ V ar(X) = E[X^2 ] − E[X]^2 = np + n(n − 1)p^2 − n^2 p^2 = np − np^2 = np(1 − p).
Theorem 2. Let X and Y be RVs with MX (t) = MY (t) for all t. Then, X and Y have the same distribution.
Many times, a moment generating function can be computed or is known, but distribution function is not. Knowing which MGF’s go with which distributions would therefore be very helpful.
Example 5. Let X be a RV with range R(X) = {a 1 ,... , an} and probability mass function
pX (x) =
pi, x = ai 0 , else
