Understanding Conditional Probability, Independence, and Bayes' Rule, Slides of Java Programming

Download Understanding Conditional Probability, Independence, and Bayes' Rule and more Slides Java Programming in PDF only on Docsity!

Today

• Examples of conditional probability
• Independence and conditional independence
• The Total Probability Theorem
• Bayes’ Rule

Conditional Probability

Suppose we flip two fair coins.

• What is the probability that the first coin lands on tails given that the second coin lands on tails?

Conditional Probability

Suppose we flip two fair coins.

• What is the probability that the first coin lands on tails given that the second coin lands on tails?
• What is the probability that the first coin lands on tails given that at least one coin lands on tails? Suppose I purchase a cookie with probability 0.5, a brownie with probability 0.25, and an apple with probability 0.25. What is the probability I purchase a cookie given that I purchase a baked good?

The Multiplication Rule

We can express P(A 1

∩A

2 ) in terms of a conditional probability: P(A 1

∩A

2

) = P(A

1

) P(A

2

| A

1

Independence

Let’s go back to the two-coin example again… What is the probability that the first coin lands on tails given that the second coin lands on tails?

Independence

Let’s go back to the two-coin example again… What is the probability that the first coin lands on tails given that the second coin lands on tails? Knowing whether or not the first coin lands on tails gives us no new information about how likely it is that the second coin lands on tails – these events are independent

Independence

• Formally, events A and B are independent if and only if P(A ∩ B) = P(A) P(B)
• If P(B) > 0, this condition is equivalent to P(A | B) = P(A)

Independence

• Formally, events A and B are independent if and only if P(A ∩ B) = P(A) P(B)
• If P(B) > 0, this condition is equivalent to P(A | B) = P(A)
• If P(A) > 0, it is equivalent to P(B | A) = P(B)

Independence

• Consider an event A. Are A and A c independent?

Independence

• Consider an event A. Are A and A c independent?
• Consider events A and B such that B A. Are A and B independent?

Conditional Independence

• Events A and B are conditionally independent given C if P(A ∩ B | C) = P(A | C) P(B | C)
• Or equivalently (if all probabilities are nonzero), if P(A | B ∩ C) = P(A | C)

Conditional Independence

Back to the two-coin example again… T 1 = {TH, TT} “1st toss is a tail” T 2 = {HT, TT} “2nd toss is a tail” D = {HT, TH} “the tosses have different results” We already know that T 1 and T 2 are independent. Are they conditionally independent given D?

Total Probability Theorem

Let A 1

, …, A

n be pairwise disjoint events with nonzero probability such that P(A 1

… A

n ) = 1. Then for any event B,

P ( B ) = P ( A

i i = 1 n

∑ ) P ( B^ |^ Ai )

Total Probability Theorem

Let A 1

, …, A

n be pairwise disjoint events with nonzero probability such that P(A 1

… A

n ) = 1. Then for any event B, Exercise: Try to prove this at home using the definition of conditional probability and the additivity axiom

P ( B ) = P ( A

i i = 1 n

∑ ) P ( B^ |^ Ai )