Generative vs Discriminative Approaches in Probabilistic Reasoning & Graphical Models, Study notes of Computer Numerical Control

Download Generative vs Discriminative Approaches in Probabilistic Reasoning & Graphical Models and more Study notes Computer Numerical Control in PDF only on Docsity!

CS181 Lecture 12:

Prob. Reasoning & Graphical

Models

David C. Parkes

Outline

• Generative vs. Discriminative Models
• Uses of probabilistic methods
• Bayes Nets

Generative can be more

demanding to learn

4

generative would need to learn this

(Bishop)

discriminative only needs to learn this

Generative vs Discriminative

Models

• Generative : learn P( X 1 ,…, X m , Y ) (often via P(X|Y) and P(Y) ) (…use this to compute P(Y|X) )
• Discriminative : learn P( Y | X 1 ,…, X m ) directly
• Still, generative models often preferred:
  • introduce latent variables, obtain natural models
  • especially preferred for unsupervised data

Uses of Probabilistic Models:

1. Prediction

• Reasoning from causes to effects
• E.g. given that someone has a particular illness, what is the prognosis?
• Learn model; then compute P(Effect | Cause)
• Bayesian networks will allow us to do this reasoning

Uses of Probabilistic Models:

2. Diagnosis

• Given that someone has certain symptoms, what illness might they have?
• Learn a model; then compute P(Cause | Effect)
• Use Bayes‟ rule for this:
  • learn P(Cause)
  • learn P(Effect | Cause)
• Bayesian networks will allow us to do this reasoning

Uses of Probabilistic Models:

3. Temporal Reasoning

• Given current state of a system or process, what is the likely state in the next time period?
• Learn model. Then compute P( Statet+1 | Statet) P( Statet+1 | Observation 1 ,… Observationt) …
• Hidden Markov models will allows us to do this reasoning

Uses of Probabilistic Models:

4. Making Decisions

• What action will maximize expected utility in current state
• Learn model P(Statet+1 | Statet, Actiont)
• Learn model Utility (State, Action)
• Compute action to maxa Utility(State,A=a)
• Planning and reinforcement learning will do this

Uses of Probabilistic Models:

6. Classification with Missing

Data

• What if attributes x 1 ,…, xj are known but xj +1,…, xm are missing?
• Learn probability P ( X 1 ,…, X m , Y )
• Can compute P( X 1 ,…, X j, Y ), and thus classify, by “marginalizing out” over X j+1,…, X m

Uses of Probabilistic Models:

7. Learning with Missing Data

• The training data itself may be incomplete (missing attributes, classes)
• Can use the Expectation- Maximization (EM) algorithm

Outline

• Generative vs. Discriminative Models
• Uses of probabilistic methods
• Bayes Nets

Radio doesn’t work. Won’t start. Q: prob fan belt broken?

(Guestrin)