Moment Generating Functions and Weak Laws of Large Numbers, Slides of Mathematics

Download Moment Generating Functions and Weak Laws of Large Numbers and more Slides Mathematics in PDF only on Docsity!

18.175: Lecture 8

Weak laws and

moment-generating/characteristic functions

Scott Sheffield

MIT

Outline

Moment generating functions

Weak law of large numbers: Markov/Chebyshev approach

Weak law of large numbers: characteristic function approach

18.175 Lecture 8

Moment generating functions

I (^) Let X be a random variable.

Moment generating functions

I (^) Let X be a random variable. I (^) The moment generating function of X is defined by M(t) = MX (t) := E [etX^ ].

Moment generating functions

I (^) Let X be a random variable. I (^) The moment generating function of X is defined by M(t) = MX (t) := E [etX^ ]. I (^) When X is discrete, can write M(t) =

x e tx (^) pX (x). So M(t) is a weighted average of countably many exponential functions.

Moment generating functions

I (^) Let X be a random variable. I (^) The moment generating function of X is defined by M(t) = MX (t) := E [etX^ ]. I (^) When X is discrete, can write M(t) =

x e tx (^) pX (x). So M(t) is a weighted average of countably many exponential functions. I (^) When X is continuous, can write M(t) =

−∞ e

tx (^) f (x)dx. So M(t) is a weighted average of a continuum of exponential functions.

Moment generating functions

I (^) Let X be a random variable. I (^) The moment generating function of X is defined by M(t) = MX (t) := E [etX^ ]. I (^) When X is discrete, can write M(t) =

x e tx (^) pX (x). So M(t) is a weighted average of countably many exponential functions. I (^) When X is continuous, can write M(t) =

−∞ e

tx (^) f (x)dx. So M(t) is a weighted average of a continuum of exponential functions. I (^) We always have M(0) = 1. I (^) If b > 0 and t > 0 then E [etX^ ] ≥ E [et^ min{X^ ,b}] ≥ P{X ≥ b}etb.

Moment generating functions

I (^) Let X be a random variable. I (^) The moment generating function of X is defined by M(t) = MX (t) := E [etX^ ]. I (^) When X is discrete, can write M(t) =

x e tx (^) pX (x). So M(t) is a weighted average of countably many exponential functions. I (^) When X is continuous, can write M(t) =

−∞ e

tx (^) f (x)dx. So M(t) is a weighted average of a continuum of exponential functions. I (^) We always have M(0) = 1. I (^) If b > 0 and t > 0 then E [etX^ ] ≥ E [et^ min{X^ ,b}] ≥ P{X ≥ b}etb. I (^) If X takes both positive and negative values with positive probability then M(t) grows at least exponentially fast in |t| as |t| → ∞.

Moment generating functions actually generate moments

I (^) Let X be a random variable and M(t) = E [etX^ ]. I (^) Then M′(t) = (^) dtd E [etX^ ] = E [ (^) d dt (e

tX (^) )]^ = E [XetX (^) ].

18.175 Lecture 8

Moment generating functions actually generate moments

I (^) Let X be a random variable and M(t) = E [etX^ ]. I (^) Then M′(t) = (^) dtd E [etX^ ] = E [ (^) d dt (e

tX (^) )]^ = E [XetX (^) ]. I (^) in particular, M′(0) = E [X ].

18.175 Lecture 8

Moment generating functions actually generate moments

I (^) Let X be a random variable and M(t) = E [etX^ ]. I (^) Then M′(t) = (^) dtd E [etX^ ] = E [ (^) d dt (e

tX (^) )]^ = E [XetX (^) ]. I (^) in particular, M′(0) = E [X ]. I (^) Also M′′(t) = (^) dtd M′(t) = (^) dtd E [XetX^ ] = E [X 2 etX^ ]. I (^) So M′′(0) = E [X 2 ]. Same argument gives that nth derivative of M at zero is E [X n].

18.175 Lecture 8

Moment generating functions actually generate moments

I (^) Let X be a random variable and M(t) = E [etX^ ]. I (^) Then M′(t) = (^) dtd E [etX^ ] = E [ (^) d dt (e

tX (^) )]^ = E [XetX (^) ]. I (^) in particular, M′(0) = E [X ]. I (^) Also M′′(t) = (^) dtd M′(t) = (^) dtd E [XetX^ ] = E [X 2 etX^ ]. I (^) So M′′(0) = E [X 2 ]. Same argument gives that nth derivative of M at zero is E [X n]. I (^) Interesting: knowing all of the derivatives of M at a single point tells you the moments E [X k^ ] for all integer k ≥ 0.

18.175 Lecture 8

Moment generating functions actually generate moments

I (^) Let X be a random variable and M(t) = E [etX^ ]. I (^) Then M′(t) = (^) dtd E [etX^ ] = E [ (^) d dt (e

tX (^) )]^ = E [XetX (^) ]. I (^) in particular, M′(0) = E [X ]. I (^) Also M′′(t) = (^) dtd M′(t) = (^) dtd E [XetX^ ] = E [X 2 etX^ ]. I (^) So M′′(0) = E [X 2 ]. Same argument gives that nth derivative of M at zero is E [X n]. I (^) Interesting: knowing all of the derivatives of M at a single point tells you the moments E [X k^ ] for all integer k ≥ 0. I (^) Another way to think of this: write etX^ = 1 + tX + t (^2) X 2 2! +^

t^3 X 3 3! +^.. .. I (^) Taking expectations gives E [etX^ ] = 1 + tm 1 + t

(^2) m 2 2! +^

t^3 m 3 3! +^.. ., where^ mk^ is the^ kth moment. The kth derivative at zero is mk.

18.175 Lecture 8

Moment generating functions for independent sums

I (^) Let X and Y be independent random variables and Z = X + Y.