EECS 501 CONVERGENCE OF SEQUENCES OF RVs Fall 2001
Recall: lim
n→∞ xn=x⇔For any ² > 0,∃Nsuch that |xn−x|< ² ∀n > N.
Given: Asequence of random variables {x1, x2. . .}. Need not be iidrv.
DEF: xn→xin probability ⇔lim
n→∞ P r[|xn−x|> ²] = 0 ⇔stochastic
convergence .
EX1: If {xn}iidrv, then sample mean ˆ
Mn=1
nPn
i=1 xi→E[x] in prob.
Proof: See Estimators handout. Requires both E[x] and σ2
xto be finite.
Note: This is weak law of large numbers, since convergence in prob. is weak.
EX2: If {xn}iidrv, fxi=1
A,0< X < A, then max[x1. . . xn]→Ain prob.
Note: Each of these shows consistency of an estimator (#1 of prob. set #7).
DEF: xn→xin mean square ⇔lim
n→∞ E[(xn−x)2] = 0 ⇔L.I.M.
n→∞ xn=x.
EX: If {xn}iidrv, then ˆ
Mn→E[x] in mean square=in quadratic mean.
Note: This is Mean Ergodic Thm.:L.I.M.
n→∞
ˆ
Mn=E[x]. Use ˆ
Mnunbiased:
Proof: E[( ˆ
Mn−E[x])2] = E[( ˆ
Mn−E[ˆ
Mn])2] = σ2
ˆ
Mn=σ2
n→0 if σ2<∞.
DEF: xn→xwith prob. one ⇔P r[{ω∈Ω : lim
n→∞ xn(ω) = x(ω)}] = 1.
Huh? Pr[set of sample functions that converge to sample point of x]=1.
Aliases: Convergence a.s. (almost surely), converg. a.e. (almost everywhere).
How? To show convergence with prob. one, usually use Thm. 3 below.
EX: If {xn}iidrv, then ˆ
Mn→E[x] a.s. (strong law of large numbers).
Thm. 1: Convergence in mean square→convergence in probability.
Proof: Suppose L.I.M.
n→∞ xn=x. Use Markov inequality: As n→ ∞,
P r[|xn−x|> ²] = P r[(xn−x)2> ²2]≤E[(xn−x)2]
²2→0. QED.
Thm. 2: Convergence with probability one→convergence in probability.
Proof: Let An={ω∈Ω : |xn(ω)−x(ω)|> ²}and Fn=∪∞
i=nAi(so limsup).
Huh? An=set of ωs.t. xn(ω) not yet converged within ²at time n.
Fn=set of ωs.t. xn(ω) not yet converged within ²at any time ≥n.
Note: Convergence in probability ⇔lim
n→∞ P r[An] = 0 (see above).
Then: xn→xa.e. ⇔0 = P r[{ω:lim
n→∞ |xn(ω)−x(ω)|> ²}] = P r[lim
n→∞ Fn]
=lim
n→∞ P r[Fn] using cont. of prob. since {Fn}is decreasing sequence.
But: (lim
n→∞ P r[Fn] = 0) →(lim
n→∞ P r[An] = 0) →convergence in prob. QED.
