Chapter 3 Linear Models for Regression
Main concepts: page numbers refer to PDF, not actual printed number
3.1 Linear basis functions models
linear regression
goal of linear regression
basis functions
bias parameter
why do we need basis functions
eg1. polynomial models: limitation and solution (p159)
eg2. examples of other basis functions (p159)
Gaussian
Sigmoidal
Fourier
Wavelets
3.1.1
Questions about G noise and G distribution (p160-161)
(i prefer to checking out youtube videos over reading textbook)
ALGORITHM: Gaussian distribution
3.1.2 Can be skipped
3.1.3 Sequential Learning P163
Used for large dataset; Or, appropriate for real- time applications in
which the data observations are arriving in a continuous stream, and
predictions must be made before all of the data points are seen.
Technique: sequential gradient descent - update the parameter vector
w ater each interation
ALGORITHM: Least-mean-square (aka. LMS Algorithms) P164
3.1.4 Regularization of least squares
Regularization allows complex models to be trained on datasets of
limited size without severe over-fitting, essentially by limiting the effective
model complexity.
3.1.5 Predicting multiple target variables
Extension from general gaussian noise distribution
General gaussian noise distribution and log likelihood function
//TO DO
3.2 The Bias Variance Decomposition
