Transforms in Probability Theory: Moment Generating Functions and Inverse Transforms - Pro, Study notes of Electrical and Electronics Engineering

Download Transforms in Probability Theory: Moment Generating Functions and Inverse Transforms - Pro and more Study notes Electrical and Electronics Engineering in PDF only on Docsity!

ELEC 303 – Random Signals

Lecture 13 – Transforms

Dr. Farinaz Koushanfar ECE Dept., Rice University Oct 8, 2008

Lecture outline

• Reading: 4.4-4.
• Definition and usage of transform
• Moment generating property
• Inversion property
• Examples
• Sum of independent random variables

Transforms

• The transform for a RV X (a.k.a moment generating function ) is a function MX(s) with parameter s defined by MX(s)=E[esX]

Why transforms?

• New representation
• Has usages in
  • Calculations, e.g., moment generation
  • Theorem proving
  • Analytical derivations

Example(s)

• Find the transforms associated with
  • Poisson random variable
  • Exponential random variable
  • Normal random variable
  • Linear function of a random variable

Moment generating property

Example

• Use the moment generation property to find the mean and variance of exponential RV

Inverse of transforms

• Transform MX(s) is invertible – important

The transform MX(s) associated with a RV X uniquely determines the CDF of X, assuming that MX(s) is finite for all s in some interval [-a,a], a>

• Explicit formulas that recover PDF/PMF from the associated transform are difficult to use
• In practice, transforms are often converted by pattern matching

Inverse transform – example 1

• The transform associated with a RV X is
• M(s) =1/4 e-s^ + 1/2 + 1/8 e4s^ + 1/8 e5s
• We can compare this with the general formula
• M(s) = ∑x esx^ pX(x)
• The values of X: -1,0,4,
• The probability of each value is its coefficient
• PMF: P(X=-1)=1/4; P(X=0)=1/2;

P(X=4)=1/8; P(X=5)=1/8;

Inverse transform – example 2

Mixture of two distributions

• Example: fX(x)=2/3. 6e-6x^ + 1/3. 4e-4x, x≥ 0
• More generally, let X 1 ,…,Xn be continuous RV with PDFs fX1, fX2,…,fXn
• Values of RV Y are generated as follows
  • Index i is chosen with corresponding prob pi
  • The value y is taken to be equal to Xi fY(y)= p 1 fX1(y) + p 2 fX2(y) + … + pn fXn(y) MY(s)= p 1 MX1(s) + p 2 MX2(s) + … + pn MXn(s)
• The steps in the problem can be reversed then

Sum of independent RVs

• X and Y independent RVs, Z=X+Y
• MZ(s) = E[esZ] = E[es(X+Y)] = E[esXesY]

= E[esX]E[esY] = MX(s) MY(s)

• Similarly, for Z=X 1 +X 2 +…+Xn,

MZ(s) = MX1(s) MX2(s) … MXn(s)

Sum of independent RVs – example 1

• X 1 ,X 2 ,…,Xn independent Bernouli RVs with parameter p
• Find the transform of Z=X 1 +X 2 +…+Xn

Sum of independent RVs – example 2

• X and Y independent Poisson RVs with means λ and μ respectively
• Find the transform for Z=X+Y
• Distribution of Z?

Sum of independent RVs – example 3

• X and Y independent normal RVs
• X ~ N(μx,σx^2 ), and Y ~ N(μy,σy^2 )
• Find the transform for Z=X+Y
• Distribution of Z?

Review of transforms so far

Bookstore example (1)

Sum of random number of

independent RVs

Bookstore example (2)

Transform of random sum

Bookstore example (3)

More examples…

• A village with 3 gas station, each open daily with an independent probability 0.
• The amount of gas in each is ~U[1,1000]
• Characterize the probability law of the total amount of available gas in the village

More examples…

• Let N be Geometric with parameter p
• Let Xi’s Geometric with common parameter q
• Find the distribution of Y = X 1 + X 2 + …+ Xn