Probability Theory: Subjective Probabilities and Basic Probability Axioms, Study notes of Computer Science

Download Probability Theory: Subjective Probabilities and Basic Probability Axioms and more Study notes Computer Science in PDF only on Docsity!

• Bayesian Networks and Decision Graphs Chapter
  • Chapter 1 – p. 1/

Two perspectives on probability theory In many domains, the probability of an outcome is interpreted as a

relative frequency:

-^ The probability of getting a three by throwing a six-sided die is

1 /^6.

However, we often talk about the probability of an event without being able to specify afrequency for it:^ •^ What is the probability that Denmark wins the world cup in 2010?^ Such probabilities are called

subjective probabilities

Chapter 1 – p. 2/

Basic probability axioms The set of possible outcomes of an “experiment” is called the

sample space^ S:

-^ Throwing a six sided die:^ {^1 ,^2 ,^3

,^4 ,^5 ,^6 }.

-^ Will Denmark win the world cup:

{yes,no}.

-^ The values in a deck of cards:^ {^2

,^3 ,^4 ,^5 ,^6 ,^7 ,^8 ,^9 ,^10 , J, Q, K, A}.

Chapter 1 – p. 3/

Basic probability axioms The set of possible outcomes of an “experiment” is called the

sample space^ S:

-^ Throwing a six sided die:^ {^1 ,^2 ,^3

,^4 ,^5 ,^6 }.

-^ Will Denmark win the world cup:

{yes,no}.

-^ The values in a deck of cards:^ {^2

,^3 ,^4 ,^5 ,^6 ,^7 ,^8 ,^9 ,^10 , J, Q, K, A}.

An^ event^ E^ is a subset of the sample space:^ •^ The event that we will get an even number when throwing a die:

{^2 ,^4 ,^6 }.

-^ The event that Denmark wins:^ {yes

-^ The event that we will get a^6 or below when drawing a card:

{^2 ,^3 ,^4 ,^5 ,^6 }.^ Chapter 1 – p. 3/

Conditional probabilities Every probability is conditioned on a^ context. For example, if we throw a dice:^11 “P ({six}) =^ ” = “P^ (six|symmetric dice) =^6

In general, if^ A^ and^ B^ are events and

P^ (A|B) =^ x, then:“In the context of B we have that^ P^ (A) =^ x” Note:^ It is not^ “whenever^ B^ we have

P^ (A) =^ x”, but rather: if^ B^ and everything else known is irrelevant to^ A, then^ P^ (A) =^ x. Definition:^ For two events^ A^ and^ B^

we have:P^ (A^ ∩^ B)^ P^ (A|B) =^ P^ (B) Example:^ P^ (A^ =^ {^4 }|B^ =^ {^2

P^ (A^ ∩^ B^ =^ {^4 })^1 /^6 , 4 , 6 }) = = P^ (B^ =^ {^2 ,^4 ,^6 })^3 /^6

(^1) =. 3 Chapter 1 – p. 4/

Basic probability calculus: the fundamental rule Let^ A,^ B^ and^ C^ be events. The fundamental rule:^ P^ (A^ ∩^ B) =^ P^ (

A|B)P^ (B).

The fundamental rule, conditioned:^

P^ (A^ ∩^ B|C) =^ P^ (A|B^ ∩^ C)P^ (B|C).

Proof:^ Derived directly from the definition of conditional probability.

Chapter 1 – p. 5/

Basic probability calculus Let A, B and C be events.P^ (A∩B) Conditional probability: P^ (A|B) =^ P^ (B) The fundamental rule: P^ (A^ ∩^ B) =^ P^ (A|B)P^ (B). The conditional fundamental rule:^ P^ (A^ ∩^ B|C) =^ P^ (A|B^ ∩

C)P^ (B|C).^ Chapter 1 – p. 7/

Basic probability calculus Let A, B and C be events.P^ (A∩B) Conditional probability: P^ (A|B) =^ P^ (B) The fundamental rule: P^ (A^ ∩^ B) =^ P^ (A|B)P^ (B). The conditional fundamental rule:^ P^ (A^ ∩^ B|C) =^ P^ (A|B^ ∩

C)P^ (B|C).

P^ (A|B)P^ (B) Bayes rule: P (B|A) = P^ (A)^

Proof:^ P^ (B|A)P^ (A) =^ P^ (B^ ∩^ A) =

P^ (A|B)P^ (B).

Bayes rule, conditioned:^ P^ (B|A^ ∩^ C

P^ (A|B∩C)P^ (B|C)) =.^ P^ (A|C)^

Conditional independence:^ If^ P^ (A|B

∩^ C) =^ P^ (A|C)^ then^ P^ (A^ ∩^ B|C) =

P^ (A|C)^ ·^ P^ (B|C).^ Chapter 1 – p. 7/

The fundamental rule for variables P (A|B)P (B): n^ ×^ m^ multiplications^ P^ (a|b)P^ (bij^ j

) =^ P^ (a,^ b) ij^ bbb^1 2 3 bb^1 2 a 0.^4 0.^3 0.^61 a 0.^6 0.^7 0.^42

bb^1 2 b 3 = 0. 4 0. 4 0. 2 b^3 a 0. 16 0. 12 0.^121 a 0. 24 0. 28 0.^082 P^ (A|B)^ P^ (B)^

P^ (A,^ B)^ Chapter 1 – p. 9/

The fundamental rule for variables P (A|B)P (B): n^ ×^ m^ multiplications^ P^ (a|b)P^ (bij^ j

) =^ P^ (a,^ b) ij^ bbb^1 2 3 bb^1 2 a 0.^4 0.^3 0.^61 a 0.^6 0.^7 0.^42

bb^1 2 b 3 = 0. 4 0. 4 0. 2 b^3 a 0. 16 0. 12 0.^121 a 0. 24 0. 28 0.^082 P^ (A|B)^ P^ (B)^

P^ (A,^ B)

A^ is independent of^ B^ given^ C^ if^ P^

(A|B,^ C) =^ P^ (A|C). bbb 1 2 3 = c( 0. 4 , 0. 6 ) ( 0. 4 ,^0.^6 )^ (^0.^4 ,^0.^6 ) (^1) c( 0. 7 , 0. 3 ) ( 0. 7 ,^0.^3 )^ (^0.^7 ,^0.^3 ) 2

aa^1 2 c 0.^4 0.^61 c 0.^7 0.^32 P^ (A|B,^ C)^

P^ (A|C)^ Chapter 1 – p. 9/

Notation

A^ potential^ φ^ is a table of real numbers over a set of variables,

dom(φ). A table of probabilities is a probability potential.

Chapter 1 – p. 11/

Notation

A^ potential^ φ^ is a table of real numbers over a set of variables,

dom(φ). A table of probabilities is a probability potential.

Multiplication bb^ bb^ bb 1 21 21 2 = a 2 1 a^1 2 a (^1 1 1) a 3 4 a^5 6 a 2 2 2

Chapter 1 – p. 11/

Notation

A^ potential^ φ^ is a table of real numbers over a set of variables,

dom(φ). A table of probabilities is a probability potential.

Multiplication bb^ bb^ bb 1 21 21 2 = a 2 1 a^1 2 a^2 21 1 1 a 3 4 a^5 6 a 2 2 2

Chapter 1 – p. 11/

Notation

A^ potential^ φ^ is a table of real numbers over a set of variables,

dom(φ). A table of probabilities is a probability potential.

Multiplication bb^ bb^ bb 1 21 21 = a 2 1 a^1 21 1 a 3 4 a^5 62

(^2) a (^2 21) a 152 Chapter 1 – p. 11/