Download Modeling Uncertainty in Artificial Intelligence using Probability Theory and more Lab Reports Computer Science in PDF only on Docsity!
CS 2710 Foundations of AI
CS 2710 Foundations of AI
Lecture 15
Milos Hauskrecht [email protected] 5329 Sennott Square
Modeling uncertainty using
probabilities
KB systems. Medical example.
We want to build a KB system for the diagnosis of pneumonia.
Problem description:
- Disease: pneumonia
- Patient symptoms (findings, lab tests) :
- Fever, Cough, Paleness, WBC (white blood cells) count, Chest pain, etc.
Representation of a patient case:
- Statements that hold (are true) for the patient.
E.g:
Diagnostic task: we want to decide whether the patient suffers from the pneumonia or not given the symptoms
Fever = True Cough = False WBCcount= High
CS 2710 Foundations of AI
Uncertainty
To make diagnostic inference possible we need to represent knowledge (axioms) that relate symptoms and diagnosis
Problem: disease/symptoms relations are not deterministic
- They are uncertain (or stochastic) and vary from patient to patient
Pneumonia
Paleness Fever Cough WBC count
Uncertainty
Two types of uncertainty:
- Disease Symptoms uncertainty
- A patient suffering from pneumonia may not have fever all the times, may or may not have a cough, white blood cell test can be in a normal range.
- Symptoms Disease uncertainty
- High fever is typical for many diseases (e.g. bacterial diseases) and does not point specifically to pneumonia
- Fever, cough, paleness, high WBC count combined do not always point to pneumonia
CS 2710 Foundations of AI
Methods for representing uncertainty
Probability theory
- A well defined theory for modeling and reasoning in the presence of uncertainty
- A natural choice to replace certainty factors
Facts (propositional statements)
- Are represented via random variables with two or more values
Example: is a random variable values: True and False
- Each value can be achieved with some probability:
P ( Pneumonia = True )= 0. 001 P ( WBCcount = high )= 0. 005
Pneumonia
Probability theory
- Well-defined theory for representing and manipulating statements with uncertainty
- Axioms of probability:
For any two propositions A, B.
0 ≤ P ( A )≤ 1
P ( True )= 1 and P ( False )= 0 P ( A ∨ B )= P ( A )+ P ( B )− P ( A ∧ B )
CS 2710 Foundations of AI
Modeling uncertainty with probabilities
Probabilistic extension of propositional logic.
- Propositions:
- statements about the world
- Represented by the assignment of values to random variables
- Random variables:
- Boolean
- Multi-valued
- Continuous
Pneumonia iseither True , False
Pain is oneof { Nopain , Mild , Moderate , Severe }
Random variable Values
Random variable Values
HeartRate isavaluein < 0 ; 250 > Random variable (^) Values
Probabilities
Unconditional probabilities (prior probabilities)
Probability distribution
- Defines probabilities for all possible value assignments to a random variable
- Values are mutually exclusive
P ( Pneumonia )= 0. 001 P ( Pneumonia = True )= 0. 001
P ( WBCcount = high )= 0. 005
or
P ( Pneumonia = True )= 0. 001 P ( Pneumonia = False )= 0. 999
Pneumonia P ( Pneumonia ) True False
P ( Pneumonia = False )= 0. 999
CS 2710 Foundations of AI
Joint probabilities
Marginalization
- reduces the dimension of the joint distribution
- Sums variables out
P ( WBCcount )
0. 005 0.^9930.^002
P ( pneumonia , WBCcount )
high normal^ low
Pneumonia True False
WBCcount
P ( Pneumonia )
Marginalization (here summing of columns or rows)
2 × 3 matrix
Full joint distribution
- the joint distribution for all variables in the problem
- It defines the complete probability model for the problem
Example: pneumonia diagnosis Variables: Pneumonia, Fever, Paleness, WBCcount, Cough Full joint defines the probability for all possible assignments of values to Pneumonia, Fever, Paleness, WBCcount, Cough
P ( Pneumonia = T , WBCcount = High , Fever = T , Cough = T , Paleness = F )
P ( Pneumonia = T , WBCcount = High , Fever = T , Cough = F , Paleness = T )
K etc
P ( Pneumonia = T , WBCcount = High , Fever = T , Cough = T , Paleness = T )
CS 2710 Foundations of AI
Conditional probabilities
Conditional probability distribution
- Defines probabilities for all possible assignments, given a fixed assignment to some other variable values
PPneumonia falseWBCcount high
P Pneumonia true WBCcount high
1. 0 1.^01.^0
P ( Pneumonia | WBCcount )
high normal^ low
Pneumonia True False
WBCcount
3 element vector of 2 elements
P ( Pneumonia = true | WBCcount = high )
Conditional probabilities
Conditional probability
- Is defined in terms of the joint probability:
- Example:
P ( pneumonia = true | WBCcount = high ) =
s.t. ( ) 0 ( )
( | )= PB ≠
PB
P AB
P A B
PWBCcount high
P pneumonia trueWBCcount high
P ( pneumonia = false | WBCcount = high )=
PWBCcount high
P pneumonia falseWBCcount high
CS 2710 Foundations of AI
Bayes rule
Assume a variable A with multiple values Bayes rule can be rewritten as:
( )
( | ) ( ) ( | ) PB b
P B b A a P A a P A aj B b j j =
= = = = = =
a 1 , a 2 ,K a k
( | ) ( )
( | ) ( )
∑ (^) = 1 = = =
= = = = (^) k i j j
j j PB b A a P A a
P B b A a PA a
P ( A | B = b ) for all values of a 1^ ,^ a 2 ,K ak
Used in practice when we want to compute:
Bayes Rule in a simple diagnostic inference.
- Device (equipment) operating normally or malfunctionin g.
- Operation of the device sensed indirectly via a sensor
- Sensor reading is either high or lo w
Device status
Sensor reading
P (Device status)
0.9 0.
normal malfunctioning
Device\Sensor high low normal 0.1 0. malfunctioning 0.6 0.
P (Sensor reading| Device status)
CS 2710 Foundations of AI
Bayes Rule in a simple diagnostic inference.
- Diagnostic inference: compute the probability of device operating normally or malfunctioning given a sensor reading
- Note that typically the opposite conditional probabilities are given to us: they are much easier to estimate
- Solution: apply Bayes rule to reverse the conditioning variables
P ( Device status |Sensor reading = high )=?
= =
= =
(Devicestatus |Sensorreading )
(Devicestatus |Sensorreading ) P malfunctioning high
P normal high
Probabilistic inference
Various inference tasks :
- Diagnostic task. (from effect to cause)
- Prediction task. (from cause to effect)
- Other probabilistic queries ( queries on joint distributions).
P ( Pneumonia | Fever = T )
P ( Fever | Pneumonia = T )
P ( Fever )
P ( Fever , ChestPain )
CS 2710 Foundations of AI
Modeling uncertainty with probabilities
- Defining the full joint distribution makes it possible to represent and reason with uncertainty in a uniform way
- We are able to handle an arbitrary inference problem
Problems:
- Space complexity. To store a full joint distribution we need to remember numbers. n – number of random variables, d – number of values
- Inference (time) complexity. To compute some queries requires. steps.
- Acquisition problem. Who is going to define all of the probability entries?
O (d n)
O (d n )
Medical diagnosis example.
- Space complexity.
- Pneumonia (2 values: T,F), Fever (2: T,F), Cough (2: T,F), WBCcount (3: high, normal, low), paleness (2: T,F)
- Number of assignments: 22232=
- We need to define at least 47 probabilities.
- Time complexity.
- Assume we need to compute the marginal of Pneumonia=T from the full joint
- Sum over: 223*2=24 combinations
P ( Pneumonia = T ) =
∈ ∈ = ∈
i TFj TFk hnluTF
P Fever iCough jWBCcount k Pale u , , , , ,
CS 2710 Foundations of AI
Modeling uncertainty with probabilities
- Knowledge based system era (70s – early 80’s)
- Extensional non-probabilistic models
- Solve the space, time and acquisition bottlenecks in probability-based models
- froze the development and advancement of KB systems and contributed to the slow-down of AI in 80s in general
- Breakthrough (late 80s, beginning of 90s)
- Bayesian belief networks
- Give solutions to the space, acquisition bottlenecks
- Partial solutions for time complexities
- Bayesian belief network
Bayesian belief networks (BBNs)
Bayesian belief networks.
- Represent the full joint distribution over the variables more compactly with a smaller number of parameters.
- Take advantage of conditional and marginal independences among random variables
- A and B are independent
- A and B are conditionally independent given C
P ( A , B )= P ( A ) P ( B )
P ( A , B | C )= P ( A | C ) P ( B | C )
P ( A | C , B )= P ( A | C )
