Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
A portion of a lecture note from colorado state university's cs545: linear models for classification course, focusing on linear least squares for classification, indicator variables, masking problem, generative models for classification using qda and lda, and their applications in data fitting and overfitting. The document also includes examples and code snippets.
Typology: Study notes
1 / 92
This page cannot be seen from the preview
Don't miss anything!
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
To classify a sample as being a member of 1 of 3
different classes, we could use integers 1, 2, and 3 as
target outputs.
Class
1
2
3
x
Linear Model
Linear function of x seems to match data fairly well.
Why is this not a good idea?
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
We must convert the continuous y-axis value to discrete
integers 1, 2, or 3. Without adding more parameters,
we are forced to use the general solution of splitting at
1.5 and 2.5.
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
We must convert the continuous y-axis value to discrete
integers 1, 2, or 3. Without adding more parameters,
we are forced to use the general solution of splitting at
1.5 and 2.5.
Class
1
2
3
x
Class 1 Class 2 Class 3
Rats! Boundaries are not where we want them.
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Can plot the three output components on three separate
graphs. What linear functions will each one learn?
Indicator variable 1
1
0
Indicator variable 2
1
0
Indicator variable 3
1
0
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Can plot the three output components on three separate
graphs. What linear functions will each one learn?
Indicator variable 1
1
0
Indicator variable 2
1
0
Indicator variable 3
1
0
Indicator variable 1
1
0
Indicator variable 2
1
0
Indicator variable 3
1
0
Overlay them to see which one is the maximum for each
x value.
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Indicator variable 1
1
0
Indicator variable 2
1
0
Indicator variable 3
1
0
Indicator variable 1
1
0
Indicator variable 2
1
0
Indicator variable 3
1
0
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
See any potential problems?
Indicator
variables
(1,0,0)
x
(0,1,0)
(0,0,1)
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
See any potential problems?
Indicator
variables
(1,0,0)
x
(0,1,0)
(0,0,1)
What if the green line is too low?
Indicator
variables
(1,0,0)
x
(0,1,0)
(0,0,1)
What could cause this?
Indicator Variables Masking Problem Example
Fitting the Generative Distributions to Data Overfitting LDA Example
Too few samples from Class 2.
1
0
x
1
0
x
1
0
x
