Summary of basic probability theory, part 1
D. Joyce, Clark University
Math 218, Mathematical Statistics, Jan 2008
Sample space. Asample space consists of a un-
derlying set S, whose elements are called outcomes,
a collection of subsets of Scalled events, and a
function Pon the set of events, called a probability
function, satisfying the following axioms.
1. The probability of any event is a number in
the interval [0,1].
2. The entire set Sis an event with probability
P(S) = 1.
3. The union and intersection of any finite or
countably infinite set of events are events, and the
complement of an event is an event.
4. The probability of a disjoint union of a finite
or countably infinite set of events is the sum of the
probabilities of those events,
P([
i
Ei) = X
i
P(Ei).
From these axioms a number of other properties
can be derived including these.
5. The the complement E=S−Eof an event
Eis an event, and
P(E) = 1 −P(E).
6. The empty set is an event with probability
P(∅) = 0.
7. For any two events Eand F,
P(E∪F) = P(E) + P(F)−P(E∩F),
therefore
P(E∪F)≤P(E) + P(F).
8. For any two events Eand F,
P(E) = P(E∩F) + P(E∩F).
9. If event Eis a subset of event F, then P(E)≤
P(F).
10. Statement 7 above is called the principle of
inclusion and exclusion. It generalizes to more than
two events.
P(
n
[
r=1
Er) =
n
X
i=1
P(Ei)−X
i<j
P(Ei∩Ej)
+X
i<j<k
P(Ei∩Ej∩Ek)− · · ·
+ (−1)n−1P(E1∩E2∩ · ·· ∩ En)
In words, to find the probability of a union of
nevents, first sum their individual probabilities,
then subtract the sum of the probabilities of all
their pairwise intersections, then add back the sum
of the probabilities of all their 3-way interections,
then subtract the 4-way intersections, and continue
adding and subtracting k-way intersections until
you finally stop with the probability of the n-way
intersection.
Random variables notation. In order to de-
scribe a sample space, we frequently introduce a
symbol Xcalled a random variable for the sam-
ple space. With this notation, we can replace
the probability of an event, P(E), by the notation
P(X∈E), which, by itself, doesn’t do much. But
many events are built from the set operations of
complement, union, and intersection, and with the
random variable notation, we can replace those by
logical operations for ‘not’, ‘or’, and ‘and’. For in-
stance, the probability P(E∪F) can be written as
P(X∈Ebut X /∈F).
Also, probabilities of finite events can be writ-
ten in terms of equality. For instance, the prob-
1
