Download Moment Generating Function - Lecture Slides | STAT 322 and more Study notes Statistics in PDF only on Docsity!
MOMENT GENERATING FUNCTION
OUTLINE
- Definition of transform
- Using transform to evaluate moments
- MGF of Sum of Independent RVs
- Convolution
Reading: Bertsekas & Tsitsiklis 4.1, 4.
Introduction to Probability, by Dimitri P. Bertsekas and John N. Tsitsiklis,
ISBN: 1-886529-40-X.
EE/STAT 322, #22 1
PMF, PDF, AND CDF
DISCRETE RV X
pX (x) = P ({X = x})
x pX (x) = 1
CONTINUOUS RV X
fX (x) : P (X ∈ B) =
B
fX (x)dx
−∞
fX (x)dx = 1
CDF:
FX (x) = P (X ≤ x) =
k:k≤x pX (k) discrete ∫ (^) x
−∞
fX (x)dx continuous
DEFINITION OF TRANSFORM
- The transform associated with a random variable X is
MX (s) = E[e
sX ] =
x e sX px(x) discrete ∫ (^) ∞
−∞
e
sx fX (x)dx continuous
- Closely related to Laplace transform, z-transform, and Fourier transform
Laplace transform of a function: L{f (x)} =
−∞
f (x)e
−sx dx
Fourier transform of a function: F{f (x)} =
∞ −∞
f (x)e
−jωx dx
Z transform of a series: Z{fn} =
n fnz
−n
Fourier transform of a series: F{fn} =
n fne −jωn
Here Z = e
s , s = jω.
When X is a continuous RV, then MX (−s) = L{fX (x)}
EE/STAT 322, #22 3
FACTS ABOUT TRANSFORMS
- The transform is a function of s; the function is also called moment
generating function (MGF), or characteristic function.
- Transform is defined for a random variable, but it is essentially a transform
on the PMF or PDF
pX (x) → MX (s) or fX (x) → MX (s)
- The transform is invertible (therefore information lossless)
pX (x) ← MX (s) or fX (x) ← MX (s)
- Motivation: transform is often convenient for calculation of moments
(will show) and proving theorems.
ANOTHER WAY OF GETTING THE SAME THING
d n
ds n
MX (s) =
d n
ds n
E[e
sX ] = E
[
d n
ds n
e
sX
]
= E[X
n e
sX ]
We can exchange the order of differentiation and expectaion because
expectation is a linear operator. (Recall that differentiation is the limit of
some difference quotient.)
First two moments:
E[X] =
d
ds
MX (s)
s=
E[X
2 ] =
d
2
ds 2
MX (s)
s=
EE/STAT 322, #22 7
EXAMPLES
1 2
, pX (3) =
1 6
, px(5) =
1 3
MX (s) =
x
p(x)e
sx
e
2 s
e
3 s
e
5 s
E[X] =
d
ds
MX (s)
s=
2 e
2 s
3 e
3 s
5 e
5 s
s=
λ λ−s
E[X] =
d
ds
MX (s)
s=
λ
(λ − s) 2
s=
λ
E[X
2 ] =
d 2
ds 2
MX (s)
s=
2 λ
(λ − s) 3
s=
λ 2
INDEPENDENCE
Two RV are called independent if
fX,Y (x, y) = fX (x)fY (y)
For independent RVs X and Y , we have
E[g(X)h(Y )] = E[g(X)]E[h(Y )]
The MGF of Z = X + Y is
MZ (s) = E[e
s(X+Y ) ] = E[e
sX ]E[e
sY ] = MX (s)MY (s)
EE/STAT 322, #22 9
EXTENSION TO MORE THAN 2 RVS
Xi, i = 1, 2 ,... , n are said to be independent if
fX 1 ,...,Xn (x 1 ,... , xn) =
n ∏
i=
fX i (xi)
For independent RVs Xi, i = 1, 2 ,... , n, we have
E
[
∏n
i=
gi(Xi)
]
∏^ n
i=
E[gi(Xi)]
The MGF of Z =
∑n
i= Xi is
MZ (s) =
n ∏
i=
MX
i (s)
SUM OF TWO INDEPENDENT NORMAL RVS
2 x), and^ Y^ ∼^ N^ (μy, σ
2 y)
MX (s) = e
μxs+
σ x^2 2 s
2 and MY (s) = e
μys+
σ 2 y 2 s
2
- MGF of their sum Z = X + Y
MZ (s) = e
(μx+μy)s+
(σ^2 x+σ^2 y) 2 s
2
- Therefore, Z is a normal RV with mean μx + μy and variance σ
2 x +^ σ
2 y.
EE/STAT 322, #22 13
CONVOLUTION
Basic result: the PDF of the sum Z = X + Y of two independent RVs X
and Y is the convolution of the PDFs of X and Y.
The discrete case
pZ (z) = P (X + Y = z) =
(x,y):x+y=z
P (X = x, Y = y)
x
P (X = x, Y = z − x) =
x
pX (x)pY (z − x)
The operation is called convolution.
- A sum over x
- the two arguments sum up to z
THE CONTINUOUS CASES Z = X + Y
fZ (z) =
−∞
fX,Z (x, z)dx =
−∞
fX (x)fY (z − x)dx
This is the continuous-“time” convolution.
- We know that the sum Z = X + Y has MGF
MZ (s) = MX (s)MY (s)
⇒fZ (z) = L
− 1 {MZ (s)} =
c+j∞ c−j∞
1 2 πj MZ (s)e
sz ds.
- The convolution property of Laplace transform
fZ (z) =
∞
−∞
fX (x)fY (z − x)dx =
c+j∞
c−j∞
2 πj
MX (s)MY (s)e
sz ds
