Classification Algorithms
Docsity.com
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Classification Algorithms: Generating Rules and Linear Models, Slides of Computer Fundamentals
An overview of classification algorithms, focusing on rule generation and linear models. It covers various methods for generating rules from decision trees, including covering algorithms and simple covering algorithms. The document also discusses linear regression and logistic regression as linear models for classification. It is a useful resource for students and researchers in machine learning and data mining.
Typology: Slides
2012/2013
1 / 38
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Classification Algorithms: Generating Rules and Linear Models and more Slides Computer Fundamentals in PDF only on Docsity!
Classification Algorithms
Docsity.com
Outline
- Rules
- Linear Models (Regression)
- Instance-based (Nearest-neighbor)
2 Docsity.com
Covering algorithms
- Strategy for generating a rule set directly: for
each class in turn find rule set that covers all
instances in it (excluding instances not in the
class)
- This approach is called a covering approach
because at each stage a rule is identified that
covers some of the instances
4 Docsity.com
Example: generating a rule
5
y
x
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) aa
a a
If true then class = a
Docsity.com
Example: generating a rule, III
7
y
x
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) aa
a a
y
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) aa
a a
1·2 x
y
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) a
a a a
1·2 x
2·
If x > 1.2 then class = a
If true then class = a If x > 1.2 and y > 2.6 then class = a
Docsity.com
Example: generating a rule, IV
- Possible rule set for class “b”:
- More rules could be added for “perfect” rule set
8
y
x
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) aa
a a
y
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) aa
a a
1·2 x
y
a
b b
b
b
b
b b
b
b b^ b b b b
a (^) a
a a a
1·2 x
2·
If x > 1.2 then class = a
If true then class = a If x > 1.2 and y > 2.6 then class = a
If x ≤ 1.2 then class = b
If x > 1.2 and y ≤ 2.6 then class = b
Docsity.com
A simple covering algorithm
- Generates a rule by adding tests that maximize
rule’s accuracy
- Similar to situation in decision trees: problem
of selecting an attribute to split on
- But: decision tree inducer maximizes overall purity
- Each new test reduces
rule’s coverage:
10
space of examples
rule so far
rule after adding new term
witten&eibe Docsity.com
Selecting a test
- Goal: maximize accuracy
- t total number of instances covered by rule
- p positive examples of the class covered by rule
- t – p number of errors made by rule
⇒ Select test that maximizes the ratio p/t
- We are finished when p/t = 1 or the set of instances can’t be
split any further
11 witten&eibe Docsity.com
Example: contact lens data, 2
- Rule we seek:
- Possible tests:
13
Age = Young 2/
Age = Pre-presbyopic 1/ Age = Presbyopic 1/ Spectacle prescription = Myope 3/ Spectacle prescription = Hypermetrope 1/ Astigmatism = no 0/ Astigmatism = yes 4/ Tear production rate = Reduced 0/ Tear production rate = Normal 4/
If?
then recommendation = hard
witten&eibe Docsity.com
Modified rule and resulting data
- Rule with best test added:
- Instances covered by modified rule:
14
Age Spectacle prescription Astigmatism Tear production rate Recommended lenses Young Myope Yes Reduced None Young Myope Yes Normal Hard Young Hypermetrope Yes Reduced None Young Hypermetrope Yes Normal hard Pre-presbyopic Myope Yes Reduced None Pre-presbyopic Myope Yes Normal Hard Pre-presbyopic Hypermetrope Yes Reduced None Pre-presbyopic Hypermetrope Yes Normal None Presbyopic Myope Yes Reduced None Presbyopic Myope Yes Normal Hard Presbyopic Hypermetrope Yes Reduced None Presbyopic Hypermetrope Yes Normal None
If astigmatism = yes then recommendation = hard
witten&eibe Docsity.com
Further refinement, 2
- Current state:
- Possible tests:
16
Age = Young 2/ Age = Pre-presbyopic 1/ Age = Presbyopic 1/ Spectacle prescription = Myope 3/ Spectacle prescription = Hypermetrope 1/ Tear production rate = Reduced 0/ Tear production rate = Normal 4/
If astigmatism = yes
and?
then recommendation = hard
witten&eibe Docsity.com
Modified rule and resulting data
- Rule with best test added:
- Instances covered by modified rule:
17
Age Spectacle prescription Astigmatism Tear production rate Recommended lenses Young Myope Yes Normal Hard Young Hypermetrope Yes Normal hard Pre-presbyopic Myope Yes Normal Hard Pre-presbyopic Hypermetrope Yes Normal None Presbyopic Myope Yes Normal Hard Presbyopic Hypermetrope Yes Normal None
If astigmatism = yes and tear production rate = normal then recommendation = hard
witten&eibe Docsity.com
Further refinement, 4
- Current state:
- Possible tests:
- Tie between the first and the fourth test
- We choose the one with greater coverage
19
Age = Young 2/ Age = Pre-presbyopic 1/ Age = Presbyopic 1/ Spectacle prescription = Myope 3/ Spectacle prescription = Hypermetrope 1/
If astigmatism = yes
and tear production rate = normal
and?
then recommendation = hard
witten&eibe Docsity.com
The result
- Final rule:
- Second rule for recommending “hard lenses”:
(built from instances not covered by first rule)
- These two rules cover all “hard lenses”:
- Process is repeated with other two classes
20
If astigmatism = yes and tear production rate = normal and spectacle prescription = myope then recommendation = hard
If age = young and astigmatism = yes and tear production rate = normal then recommendation = hard
witten&eibe Docsity.com