Lecture 2: Tutorial on probability and estimation
II
TTIC 31020: Introduction to Machine Learning
Instructor: Greg Shakhnarovich,
Lecture by Dhruv Batra
TTI–Chicago
September 29, 2010
Lecture 2: Tutorial on probability and estimation II TTIC 31020
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Tutorial on Probability and Estimation-Introduction to Machine Learning-Lecture 02-Computer Science, Lecture notes of Introduction to Machine Learning
Tutorial on Probability and Estimation, Bernoulli Distribution, MAP Estimates, Beta Conjugate Prior, Multivariate Gaussian Distributions, Gaussians, Central Limit Theorem, Univariate Gaussian Distribution, Moments, Multivariate Gaussian, Matrix, Mean of the Gaussian, Covariance, Correlation Vs. Covariance, Covariance Matrix, Covariance of the Gaussian, Geometry, Density Contours, Linear Functions, Gaussian RV, Conditional Marginal, ML for Parameters, Covariance Matrices, Greg Shakhnarovich, Lect
Typology: Lecture notes
2011/2012
1 / 76
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Tutorial on Probability and Estimation-Introduction to Machine Learning-Lecture 02-Computer Science and more Lecture notes Introduction to Machine Learning in PDF only on Docsity!
Lecture 2: Tutorial on probability and estimation
II
TTIC 31020: Introduction to Machine Learning
Instructor: Greg Shakhnarovich, Lecture by Dhruv Batra
TTI–Chicago
September 29, 2010
Roadmap
Last time: Bernoulli distribution, ML and MAP estimates, beta conjugate prior.
Today: uni- and multivariate Gaussian distributions
Background
Things you should have seen before
- (^) Univariate / Multi-variate Gaussians
Background
Things you should have seen before
- (^) Univariate / Multi-variate Gaussians
- Random Vectors: Mean, Variance, Covariance, Correlation
Background
Things you should have seen before
- (^) Univariate / Multi-variate Gaussians
- Random Vectors: Mean, Variance, Covariance, Correlation
- Eigenvectors of Gaussian Covariance matrix
This refresher WILL revise these topics.
Why Gaussians?
Gaussian distributions are widely used in machine learning:
Why Gaussians?
Gaussian distributions are widely used in machine learning:
- (^) Central Limit Theorem!
X¯n = X 1 + X 2 + · · · + Xn
Why Gaussians?
Gaussian distributions are widely used in machine learning:
- (^) Central Limit Theorem!
X¯n = X 1 + X 2 + · · · + Xn √ n X¯n −→ Nd
x; μ, σ^2
Why Gaussians?
Gaussian distributions are widely used in machine learning:
- (^) Central Limit Theorem!
Why Gaussians?
Gaussian distributions are widely used in machine learning:
- (^) Central Limit Theorem!
- Gaussians are convenient computationally;
Why Gaussians?
Gaussian distributions are widely used in machine learning:
- (^) Central Limit Theorem!
- Gaussians are convenient computationally;
- (^) Mixtures of Gaussians (studied later in the class) are sufficient to approximate a wide range of distributions;
- (^) Closely related to squared loss (will study it later), an important error measure in statistics.
Reminder: univariate Gaussian distribution
N (x; μ, σ^2 ) =
(2πσ^2 )^1 /^2
exp
2 σ^2 (x − μ)^2
mean μ determines location variance σ^2 ; standard deviation
σ^2 determines the spread around μ
N (x|μ, σ^2 )
x
2 σ
μ
Moments
Reminder: expectation of a RV x is E [x] ,
xp(x)dx, so
E [x] =
−∞
xN (x; μ, σ^2 )dx = μ
Moments
Reminder: expectation of a RV x is E [x] ,
xp(x)dx, so
E [x] =
−∞
xN (x; μ, σ^2 )dx = μ
Variance of x is var x , E
[
(x − E [x])^2
]
, and