Electrical Engineering 126: Probability & Random Processes
Final Cheat Sheet
Spring 2018
1 Distributions
•X∼Bernoulli(p), p∈[0,1].
PMF: pX(x) = px(1 −p)1−x,x∈ {0,1}.
MGF: MX(s)=1−p+pexp s.
Moments: E[X] = p, var X=p(1 −p).
•X∼Binomial(n, p), n∈Z+,p∈[0,1].
PMF: pX(x) = n
xpx(1 −p)n−x,x∈ {0, . . . , n}.
MGF: MX(s) = (1 −p+pexp s)n.
Moments: E[X] = np, var X=np(1 −p).
•X∼Geometric(p), p∈(0,1).
PMF: pX(x) = pqx−1,x∈Z+,q= 1 −p.
MGF: MX(s)=(pexp s)/(1 −qexp s), s < ln(1/q).
Moments: E[X] = p−1, var X=q/p2.
•X∼Poisson(λ), λ > 0.
PMF: pX(x) = λxexp(−λ)/x!, x∈N.
MGF: MX(s) = exp(λ(exp s−1)).
Moments: E[X] = λ, var X=λ.
X, Y
independent,
X∼Poisson
(
λ
),
Y∼Poisson
(
µ
) =
⇒
X+Y∼Poisson(λ+µ).
•X∼Uniform[a, b], a<b.
PDF: fX(x) = (b−a)−1,x∈[a, b].
MGF: MX(s) = (exp(sb)−exp(sa))/(s(b−a)).
Moments: E[X] = (a+b)/2, var X= (b−a)2/12.
•X∼Exponential(λ), λ > 0.
PDF: fX(x) = λexp(−λx), x > 0.
CDF: FX(x) = (1 −exp(−λx))
1
{x≥0}.
MGF: MX(s) = λ/(λ−s), s < λ.
Moments: E[X] = λ−1, var X=λ−2.
•X∼ N(µ, σ2), µ∈R,σ2>0.
PDF: fX(x)=(√2πσ)−1exp{−(x−µ)2/(2σ2)}.
CDF: FX(x) = Φ(x).
MGF: MX(s) = exp(µs +σ2s2/2).
Moments: E[X] = µ, var X=σ2.
X, Y
independent,
X∼ N
(
µ1, σ2
1
),
Y∼ N
(
µ2, σ2
2
) =
⇒
X+Y∼ N(µ1+µ2, σ2
1+σ2
2).
Continued:
•X∼Pascal(k, p), k∈Z+,p∈(0,1).
Sum of ki.i.d. Geometric(p).
PMF: pX(x) = x−1
k−1pk(1 −p)x−k,x=k, k + 1, k + 2, . . . .
•X∼Erlang(k, λ), k∈Z+,λ > 0.
Sum of ki.i.d. Exponential(λ).
PDF: fX(x) = λkxk−1exp(−λx)/(k−1)!, x≥0.
•X∼ Nn(µ, Σ), n∈Z+(joint Gaussian).
PDF (assuming Σ invertible): For x∈Rn,
fX(x) = [(2π)ndet Σ]−1/2exp{−(x−µ)TΣ−1(x−µ)/2}.
MGF: MX(s) = E[exp(sTX)] = exp(µTs+sTΣs/2).
Moments: E[X] = µ, cov X= Σ.
2 Definitions & Equations
Tail Sum: For X≥0, E[X] = R∞
0P(X≥x) dx.
Variance
:
var X
=
E
[(
X−E
[
X
])
2
] =
E
[
X2
]
−E
[
X
]
2
.
Sum
:
var Pn
i=1 Xi=Pn
i=1 var Xi+Pi6=jcov(Xi,Xj).
Covariance
:
cov
(
X, Y
) =
E
[
XY
]
−E
[
X
]
E
[
Y
].
Matrix
: If
X
=
(X1, . . . , Xn), (cov X)i,j = cov(Xi, Xj).
Correlation: ρ(X, Y ) = cov(X, Y )/p(var X)(var Y).
Entropy: H(X) = −Px∈X p(x) log2p(x) = −E[log2p(X)].
Order Statistics: fX(i)(x) = nn−1
i−1f(x)F(x)i−1(1 −F(x))n−i.
MGF: MX(s) = E[exp(sX)].
Markov: For X≥0, x > 0, P(X≥x)≤E[X]/x.
Chebyshev: For x > 0, P(|X−E[X]| ≥ x)≤(var X)/x2.
LLSE: L[X|Y]−E[X] = [cov(X, Y )/(var Y)](Y−E[Y]).
1
