Download Inequalities and Conditional Distributions and more Study notes Probability and Statistics in PDF only on Docsity!
Lecture 13
1. Inequalities
Let us start with an inequality.
Lemma 13.1. If h is a nonnegative function, then for all λ > 0 ,
P{h(X) ≥ λ} ≤ E[h(X)] λ
Proof. We know already that
E[h(X)] =
x
h(x)f (x) ≥
x: h(x)≥λ
h(x)f (x).
If x is such that h(x) ≥ λ, then h(x)f (x) ≥ λf (x), obviously. Therefore,
E[h(X)] ≥ λ
x: h(x)≥λ
f (x) = λP{h(X) ≥ λ}.
Divide by λ to finish.!
Thus, for example,
P {|X| ≥ λ} ≤
E(|X|)
λ “Markov’s^ inequality.”
P {|X − EX| ≥ λ} ≤ Var(X) λ^2
“Chebyshev’s inequality.”
To get Markov’s inequality, apply Lemma 13.1 with h(x) = |x|. To get Chebyshev’s inequality, first note that |X − EX| ≥ λ if and only if |X − EX|^2 ≥ λ^2. Then, apply Lemma 13. 1 to find that
P {|X − EX| ≥ λ} ≤
E
|X − EX|^2
λ^2. Then, recall that the numerator is Var(X).
45
In words:
- If E(|X|) < ∞, then the probability that |X| is large is small.
- If Var(X) is small, then with high probability X ≈ EX.
2. Conditional distributions
If X is a random variable with mass function f , then {X = x} is an event. Therefore, if B is also an event, and if P(B) > 0, then
P(X = x | B) = P({X^ =^ x}^ ∩^ B) P(B)
As we vary the variable x, we note that {X = x}∩B are disjoint. Therefore,
∑
x
P(X = x | B) =
P({X = x} ∩ B) P(B) =
P (∪x {X = x} ∩ B) P(B) = 1.
Thus,
f (x | B) = P(X = x | B)
defines a mass function also. This is called the conditional mass function of X given B.
Example 13.2. Let X be distributed uniformly on { 1 ,... , n}, where n is a fixed positive integer. Recall that this means that
f (x) =
n
if x = 1,... , n, 0 otherwise.
Choose and fix two integers a and b such that 1 ≤ a ≤ b ≤ n. Then,
P{a ≤ X ≤ b} =
∑^ b
x=a
n
= b^ −^ a^ + 1 n
Therefore,
f (x | a ≤ X ≤ b) =
b − a + 1 if^ x^ =^ a,... , b, 0 otherwise.
Remark 13.5. The more general version of Bayes’s formula works too here: Suppose B 1 , B 2 ,... are disjoint and ∪∞ i=1 Bi = Ω; i.e., “one of the Bi ’s hap- pens.” Then,
EX =
∑^ ∞
i=
E(X | Bi )P(Bi ).
Example 13.6. Suppose you play a fair game repeatedly. At time 0, before you start playing the game, your fortune is zero. In each play, you win or lose with probability 1/2. Let T 1 be the first time your fortune becomes +1. Compute E(T 1 ). More generally, let T (^) x denote the first time to win x dollars, where T 0 = 0. Let W denote the event that you win the first round. Then, P(W ) = P(W c^ ) = 1/2, and so
E(T (^) x ) =^1 2
E(T (^) x | W ) +^1 2
E(T (^) x | W c^ ). (11)
Suppose x (= 0. Given W , T (^) x is one plus the first time to make x − 1 more dollars. Given W c^ , T (^) x is one plus the first time to make x + 1 more dollars. Therefore,
E(T (^) x ) =^1 2
[
1 + E(T (^) x− 1 )
]
+^1
[
1 + E(T (^) x+1 )
]
E(T (^) x− 1 ) + E(T (^) x+1 )
Also E(T 0 ) = 0.
Let g(x) = E(T (^) x ). This shows that g(0) = 0 and
g(x) = 1 + g(x^ + 1) +^ g(x^ −^ 1) 2
for x = ± 1 , ± 2 ,....
Because g(x) = (g(x) + g(x))/2,
g(x) + g(x) = 2 + g(x + 1) + g(x − 1) for x = ± 1 , ± 2 ,....
Solve to find that for all integers x ≥ 1,
g(x + 1) − g(x) = −2 + g(x) − g(x − 1).
Lecture 14
Example 14.1 (St.-Petersbourg paradox, continued). We continued with our discussion of the St.-Petersbourg paradox, and note that for all integers N ≥ 1,
g(N ) = g(1) +
∑^ N
k=
g(k) − g(k − 1)
= g(1) +
N∑ − 1
k=
g(k + 1) − g(k)
= g(1) +
N∑ − 1
k=
− 2 + g(k) − g(k − 1)
= g(1) − 2(N − 1) +
∑^ N
k=
g(k) − g(k − 1)
= g(1) − 2(N − 1) + g(N ).
If g(1) < ∞, then g(1) = 2(N − 1). But N is arbitrary. Therefore, g(1) cannot be finite; i.e., E(T 1 ) = ∞. This shows also that E(Tx) = ∞ for all x ≥ 1, because for example T 2 ≥ 1 + T 1! By symmetry, E(Tx) = ∞ if x is a negative integer as well.
