Probability Background - Artificial Intelligence - Lecture Slides, Slides of Artificial Intelligence

Download Probability Background - Artificial Intelligence - Lecture Slides and more Slides Artificial Intelligence in PDF only on Docsity!

Lecture 25

Probability Background

and Hour Exam 1 Review

Making Decisions under Uncertainty

Bayes’s Theorem [2]

• Theorem
• P ( h )  Prior Probability of Assertion (Hypothesis) h
  • Measures initial beliefs (BK) before any information is obtained (hence prior)
• P ( D )  Prior Probability of Data (Observations) D
  • Measures probability of obtaining sample D (i.e., expresses D)
• P ( h | D )  Probability of h Given D
  • | denotes conditioning - hence P(h | D) is a conditional ( aka posterior) probability
• P ( D | h )  Probability of D Given h
  • Measures probability of observing D given that h is correct (“generative” model)
• P ( h  D )  Joint Probability of h and D
  • Measures probability of observing D and of h being correct

PD 

P h D

P D

P D|hP h P h|D

Bayesian Inference:

Query Answering (QA)

• Answering User Queries
  • Suppose we want to perform intelligent inferences over a database DB
    • Scenario 1: DB contains records (instances), some “labeled” with answers
    • Scenario 2: DB contains probabilities (annotations) over propositions
  • QA: an application of probabilistic inference
• QA Using Prior and Conditional Probabilities: Example
  • Query: Does patient have cancer or not?
  • Suppose: patient takes a lab test and result comes back positive
    • Correct + result in only 98% of the cases in which disease is actually present
    • Correct - result in only 97% of the cases in which disease is not present
    • Only 0.008 of the entire population has this cancer
  •   P (false negative for H 0  Cancer ) = 0.02 ( NB : for 1-point sample)
  •   P (false positive for H 0  Cancer ) = 0.03 ( NB : for 1-point sample)
  • P(+ | H 0 ) P( H 0 ) = 0.0078, P(+ | HA ) P( HA ) = 0.0298  hMAP = HA  Cancer

P | Cancer

P |Cancer  

P | Cancer

  P | Cancer

P Cancer

PCancer

Terminology

• Introduction to Reasoning under Uncertainty
  • Probability foundations
  • Definitions: subjectivist, frequentist, logicist
  • (3) Kolmogorov axioms
• Bayes’s Theorem
  • Prior probability of an event
  • Joint probability of an event
  • Conditional (posterior) probability of an event
• Maximum A Posteriori (MAP) and Maximum Likelihood (ML) Hypotheses
  • MAP hypothesis: highest conditional probability given observations (data)
  • ML: highest likelihood of generating the observed data
  • ML estimation (MLE): estimating parameters to find ML hypothesis
• Bayesian Inference: Computing Conditional Probabilities (CPs) in A Model
• Bayesian Learning: Searching Model (Hypothesis) Space using CPs

Summary Points

• Introduction to Probabilistic Reasoning
  • Framework: using probabilistic criteria to search H
  • Probability foundations
    • Definitions: subjectivist, objectivist; Bayesian, frequentist, logicist
    • Kolmogorov axioms
• Bayes’s Theorem
  • Definition of conditional (posterior) probability
  • Product rule
• Maximum A Posteriori (MAP) and Maximum Likelihood (ML) Hypotheses
  • Bayes’s Rule and MAP
  • Uniform priors: allow use of MLE to generate MAP hypotheses
  • Relation to version spaces, candidate elimination
• Next Week: Chapter 15, Russell and Norvig
  • Later: Bayesian learning: MDL, BOC, Gibbs, Simple (Naïve) Bayes
  • Categorizing text and documents, other applications