Download Independence of Random Variables: Discrete and Continuous Cases - Prof. Galen Shorack and more Study notes Probability and Statistics in PDF only on Docsity!

Lecture 10: Jan 28. Independent random variables Ross 6.

X and Y are independent if for any subsets A and B of ℜ, P (X ∈ A ∩ Y ∈ B) = P (X ∈ A) × P ((Y ∈ B).

10.1 Independence of two r.vs: discrete case Discrete case: this is equivalent to pX,Y (x, y) = P (X = x, Y = y) = P (X = x).P (Y = y) = pX (x)pY (y).

Clearly this is necessary: take A = {x} and B = {y}. Conversely, if pX,Y (x, y) = pX (x)pY (y), then for any A, B:

P (X ∈ A, Y ∈ B) =

∑ x∈A

∑ y∈B

pX,Y (x, y) =

∑ x∈A

∑ y∈B

pX (x)pY (y)

=

∑ x∈A

pX (x)

∑ y∈B

pY (y) = P (X ∈ A) P (Y ∈ B)

Note this must hold for all x, y. Thus the ranges of the r.v.s cannot depend on each other. Example: X is value on first die, Y on second: these are independent. But if we throw out doubles (i.e. points with x = y) they are no longer independent: if X = 4 we know Y 6 = 4.

10.2 Independence of two r.vs: continuous case With A = (−∞, x) and B = (−∞, y) we see FX,Y (x, y) = FX (x)FY (y). As with the 1-dimensional case, this is also sufficient: X and Y are independent if and only if FX,Y (x, y) = FX (x)FY (y) for all x, y.

Differentiating, we see this means fX,Y (x, y) = fX (x)fY (y), and conversely integrating we see these are equivalent. Note again it must hold for all x, y: the ranges of the r.vs cannot depend on each other. 10.3 Buffon’s needle: a classic example of estimating π.

Parallel lines distance D apart; needle length L, with L ≤ D. Let X be distance from needle midpoint to nearest line; θ be angle of needle to X. X is U (0, D/2), θ is U (0, π/2) and they are independent. We want the probability

P (X ≤ (L/2) cos(θ)) =

∫ ∫ 2 x<L cos(y)

fX (x)fθ(y) dx dy = (^) π^2 D^2

∫ (^) π/ 2 0

∫ (^) L/2 cos(y) 0

dx dy

= 4 π D

∫ (^) π/ 2 0

L

cos(y) dy = 2 L π D [sin(y)]π/ 0 2 = 2 L/πD

10.4 Examples of independence and dependence

X and Y are independent if and only if fX,Y = fX (x)fY (y) for all x, y. Also, if fX,Y = g 1 (x)g 2 (y) for all X, Y , then X and Y are independent, and fX (x) ∝ g 1 (x), fY (y) ∝ g 2 (y).

(i) Example: fX,Y (x, y) = x + y on 0 < x < 1, 0 < y < 1. (Ch 6: # 22) X and Y are not independent: why not?

(ii) Example: fX,Y (x, y) = exp(−(x + y)) on 0 < x < ∞, 0 < y < ∞. X and Y are independent: why? What is fX (x)?

(iii) Example: fX,Y (x, y) = 2 exp(−(4x + 12 y)) on 0 < x < ∞, 0 < y < ∞. X and Y are independent: why? What is fX (x)?

(iv) Example: fX,Y (x, y) = 1/6 on 0 < x < 2, 0 < y < 3, X and Y are independent: why? What is fX (x)?

(v) Example: fX,Y (x, y) = 2 on 0 < x, 0 < y, x + y < 1 X and Y are not independent: why not?

Lecture 11: Jan 30. Examples of computing probabilities

11.1 Computing marginal densities

(i) Example: f (x, y) = x + y on 0 < x < 1, 0 < y < 1. (Ch 6: # 22)

What is fX (x)?

(ii) Example: fX,Y (x, y) = 2 on 0 < x, 0 < y, x + y < 1 X and Y are not independent: why not?

What is fX (x)? (ii) Example: fX,Y (x, y) = 2 exp(−(4x + 12 y)) on 0 < x < ∞, 0 < y < ∞. X and Y are independent: why? What is fX (x)? How would you compute fX (x) if you did not see X and Y are independent?

11.2 Computing probabilities with joint pdf ’s

Recall: P ((X, Y ) ∈ A) = ∫ ∫ A fX,Y^ (x, y)^ dx dy. (i) Example: f (x, y) = x + y on 0 < x < 1, 0 < y < 1. (Ch 6: # 22)

P (X + Y < 1) =

∫ (^1) y=

∫ (^1) −y x=

(x + y) dx dy =

∫ (^1) y=

[^1

x^2 + yx]^10 − ydy

=

∫ (^1) y=

(^1

(1 − y)^2 + y(1 − y))dy =

∫ (^1) y=

(1 − y^2 )dy = 1 2

(1 − (1/3)) = 1 / 3.

(ii) Example: f (x, y) = 2 on 0 < x, 0 < y, x + y < 1

P (X < 3 Y ) =

∫ (^1) y=

∫ (^) min(1−y, 3 y) x=

2 dx dy =

∫ (^1) / 4 y=

6 y dy +

∫ (^1) y=1/ 4

2(1 − y) dy

= [3y^2 ]^10 / 4 + [−(1 − y)^2 ]^11 / 4 = 3 /16 + 9/ 16 = 3 / 4.

11.3: Sum of two independent U(0,1)

Let X ∼ U (0, 1) and Y ∼ U (0, 1) with X and Y independent. So fX,Y (x, y) = 1 on 0 < x < 1, 0 < y < 1.

(a) What is the range of W = X + Y?

(b) Compute P (X + Y < w) where 0 < w < 1. Or, draw a picture. (Answer: w^2 /2)

(c) Hence show fW (w) = w if 0 < w < 1. (This is only part of fW (w).) (c) Note (2 − W ) = (1 − X) + (1 − Y ). Why does this tell me that fW (w) is symmetric about w = 1? (d) Because of its shape, fW (w) is known as a triangular density. Do these densities form a location/scale family?

Lecture 12: Feb 2: Conditional pmf (discrete) and pdf (continuous): two examples

12.1 Convolution of probability mass functions

Let X and Y be independent discrete r.vs with pmf pX () and pY ().

P (W ≡ (X + Y ) = w) =

∑ x

P (W = w ∩ X = x) =

∑ x

P (Y = w − x ∩ X = x) =

∑ x

pY (w − x)pX (x)

Example: sum of independent Poissons; recall the Poisson process number of events. Let X be Po(μ) and Y be Po(ν), and X,Y independent.

P (X + Y = k) =

∑^ k x=

P (Y = (k − x))P (X = x) =

∑^ k x=

exp(−ν)νk−x (k − x)!

exp(−μ)μx x!

= exp(−(μ^ +^ ν)) k!

∑^ k x=

k! (k − x)!x! μx^ νk−x^ = exp(−(μ^ +^ ν)) k! (μ + ν)k

(Remember the binomial theorem.)

12.2 Conditional pmf (Ross 6.4): P (X = x|W = w) = pX,W (x, w)/pW (w) Example: Let X be Po(μ) and Y be Po(ν), and X,Y independent, and W = X +Y. We know W ∼ Po(μ+ν).

P (X = x | W = w) = P (X = x, Y = w − x)/P (W = w) = pX (x)pY (w − x)/P (W = w)

= exp(−μ)μ

x x!

exp(−ν)νw−x (w − x)!

w! exp(−(μ + ν))(μ + ν)w^

= ( w

x

)

( (^) μ μ + ν

)x ( (^) ν μ + ν

)w−x

i.e. X | W = w is Binomial (w, μ/(μ + nu)).

12.3 Convolution of a probability density function (Ross 6.3) X and Y are independent, with fX,Y (x, y) = fX (x)fY (y).

FX+Y (a) =

∫ ∫ x+ylea

fX (x) fY (y) dx dy =

∫ (^) ∞ y=−∞

fY (y)

(∫ (^) a−y x=−∞

fX (x) dx

) dy =

∫ (^) ∞ −∞

FX (a − y) fY (y) dy

fX+Y (a) = d da

(∫ (^) ∞ −∞

FX (a − y) fY (y) dy

)

∫ (^) ∞ −∞

fx(a − y) fY (y) dy

We could use this to show that if X ∼ G(m, λ), Y ∼ G(n, λ) then X + Y ∼ G(m + n, λ) (see Ross P.281), but we know this anyhow from the Poisson process, times to mth^ then nth^ events.

12.4 Conditional probability density function: define fX|Y (x|y) = fX,Y (x, y)/fY (y). (Ross 6.5)

P (x < X ≤ x + δx | y < Y ≤ y + δy) = P^ (x < X^ ≤^ x^ +^ δx^ ∩^ y < Y^ ≤^ +δy) P (y < Y ≤ y + δy) ≈ fX,Y^ (x, y)^ δx δy fY (y) δy

Example: X ∼ G(m, λ), Y ∼ G(n, λ) , independent, so W ≡ X + Y ∼ G(m + n, λ)

fX|W (x|w) = fX,W (x, w)/fW (w) = fX (x) fY (w − x)/fW (w) = λ

mxm− (^1) exp(−λx) Γ(m)

λn(w − x)n−^1 exp(−λ(w − x)) Γ(n)

Γ(m + n) λm+nwm+n−^1 exp(−λw) = Γ(m^ +^ n) Γ(m)Γ(n) w−^1 (x/w)m−^1 (1 − (x/w))n−^1 on 0 ≤ x ≤ w and 0 otherwise

Note λ is gone, but w is a scale parameter. In fact, if V = X/W , fV |W (v|w) = Γ(m + n)vm−^1 (1 − v))n−^1 /Γ(m)Γ(n) on 0 ≤ v ≤ 1, which does not depend on w.