Tractable Representations Inference Probabilistic Learning Models Applications, Lecture notes of Artificial Intelligence

Tractable representations, inference, and probabilistic learning models and their applications. It covers various models such as AoGs, PDGs, NBs, fully factorized sd-DNNF, PSDDs, trees, LTMs, DNNFs, OBDDs, CNets, SPNs, NADEs, thin junction trees, NNF, FBDDs, BDDs, ACs, VAEs, polytrees, d-NNFs, ADDs, SDDs, TACs, GANs, NFs, mixtures, XADDs, XSDDs, MNs, BNs, and FGs. The document also discusses why tractable inference is important and how probabilistic circuits provide a unified framework for tractable models. It also covers building circuits, learning them from data, and compiling other models. The document also discusses why probabilistic inference is important and how it can be used to answer probabilistic queries on a probabilistic model of the world. It also covers approximate inference and its guarantees.

Typology: Lecture notes

2021/2022

Uploaded on 05/11/2023

damyen
damyen 🇺🇸

4.4

(27)

274 documents

1 / 151

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Tractable
Probabilistic
Models
Representations
Inference
Learning
Applications
Guy Van den Broeck
University of California, Los Angeles
based on a joint UAI-19 tutorial with
Antonio Vergari
University of California, Los Angeles
Nicola Di Mauro
University of Bari
September 23, 2019 - 35th International Conference on Logic Programming (ICLP 2019) Las Cruces, New Mexico, USA
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21
pf22
pf23
pf24
pf25
pf26
pf27
pf28
pf29
pf2a
pf2b
pf2c
pf2d
pf2e
pf2f
pf30
pf31
pf32
pf33
pf34
pf35
pf36
pf37
pf38
pf39
pf3a
pf3b
pf3c
pf3d
pf3e
pf3f
pf40
pf41
pf42
pf43
pf44
pf45
pf46
pf47
pf48
pf49
pf4a
pf4b
pf4c
pf4d
pf4e
pf4f
pf50
pf51
pf52
pf53
pf54
pf55
pf56
pf57
pf58
pf59
pf5a
pf5b
pf5c
pf5d
pf5e
pf5f
pf60
pf61
pf62
pf63
pf64

Partial preview of the text

Download Tractable Representations Inference Probabilistic Learning Models Applications and more Lecture notes Artificial Intelligence in PDF only on Docsity!

Tractable

Probabilistic

Models

Representations

Inference

Learning

Applications

Guy Van den Broeck University of California, Los Angeles

based on a joint UAI-19 tutorial with Antonio Vergari University of California, Los Angeles Nicola Di Mauro University of Bari

September 23, 2019 - 35th International Conference on Logic Programming (ICLP 2019) Las Cruces, New Mexico, USA

AoGs PDGs NBs Fully factorized sd-DNNF PSDDs Trees LTMs DNNFs OBDDs CNets SPNs NADEs Thin Junction Trees NNF FBDDs BDDs ACs VAEs Polytrees d-NNFs ADDs SDDs TACs GANs NFs Mixtures XADDs XSDDs MNs BNs FGs

The Alphabet Soup of models in AI

AoGs PDGs NBs Fully factorized sd-DNNF PSDDs Trees LTMs DNNFs OBDDs CNets SPNs NADEs Thin Junction Trees NNF FBDDs BDDs ACs VAEs Polytrees d-NNFs ADDs SDDs TACs GANs NFs Mixtures XADDs XSDDs MNs BNs FGs

Tractable and Intractable

probabilistic models

AoGs PDGs NBs Fully factorized sd-DNNF PSDDs Trees LTMs DNNFs OBDDs CNets SPNs NADEs Thin Junction Trees NNF FBDDs BDDs ACs VAEs Polytrees d-NNFs ADDs SDDs TACs GANs NFs Mixtures XADDs XSDDs MNs BNs FGs

Expressive models without compromises

Why tractable inference?

or expressiveness vs tractability

Probabilistic circuits

a unified framework for tractable models

Building circuits

learning them from data and compiling other models

Applications

what are circuits useful for

Tractable Probabilistic Circuits @ ICLP?

Logical roots of probabilistic circuits

Probabilistic circuits bridge between logic and deep learning

Bring back world models!

Powerful general reasoning tool

⇒ for example in probabilistic logic programming

Elegant knowledge representation formalism

Why probabilistic inference?

q 1 : What is the probability that today is a Monday and there is a traffic jam on Herzl Str.? q 2 : Which day is most likely to have a traffic jam on my route to work?

pinterest.com/pin/190417890473268205/

Why probabilistic inference?

q 1 : What is the probability that today is a Monday and there is a traffic jam on Herzl Str.? q 2 : Which day is most likely to have a traffic jam on my route to work?

⇒ fitting a predictive model!

pinterest.com/pin/190417890473268205/

Why probabilistic inference?

q 1 : What is the probability that today is a Monday and there is a traffic jam on Herzl Str.?

X = {Day, Time, JamStr1, JamStr2,... , JamStrN}

q 1 (m) = pm(Day = Mon, JamHerzl = 1)

pinterest.com/pin/190417890473268205/

Why probabilistic inference?

q 1 : What is the probability that today is a Monday and there is a traffic jam on Herzl Str.?

X = {Day, Time, JamStr1, JamStr2,... , JamStrN}

q 1 (m) = pm(Day = Mon, JamHerzl = 1)

⇒ marginals

pinterest.com/pin/190417890473268205/

Why probabilistic inference?

q 2 : Which day is most likely to have a traffic jam on my route to work?

X = {Day, Time, JamStr1, JamStr2,... , JamStrN}

q 2 (m) = argmaxd pm(Day = d ∧ ∨ i∈route JamStr i)

⇒ marginals + MAP + logical events

pinterest.com/pin/190417890473268205/

Tractable Probabilistic Inference

A class of queries Q is tractable on a family of probabilistic models M iff for any query q ∈ Q and model m ∈ M exactly computing q(m) runs in time O(poly(|q| · |m|)).

Tractable Probabilistic Inference

A class of queries Q is tractable on a family of probabilistic models M iff for any query q ∈ Q and model m ∈ M exactly computing q(m) runs in time O(poly(|q| · |m|)).

⇒ often poly will in fact be linear!

Note: if M and Q are compact in the number of random variables X, that is, |m|, |q| ∈ O(poly(|X|)), then query time is O(poly(|X|)).

What about approximate inference?

Why approximate when we can do exact?

⇒ and do we lose something in terms of expressiveness?

Approximations can be intractable as well [Dagum et al. 1993; Roth 1996]

⇒ But sometimes approximate inference comes with guarantees

Approximate inference by exact inference in approximate model

[Dechter et al. 2002; Choi et al. 2010; Lowd et al. 2010; Sontag et al. 2011; Friedman et al. 2018]

Approximate inference (even with guarantees) can mislead learners

[Kulesza et al. 2007] ⇒ Chaining approximations is flying with a blindfold on