Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Various techniques for learning rules from positive examples, including decision trees to rules, learning rules with sequential covering, and learning rules bottom-up. The document also compares rule learning to knowledge engineering and introduces inductive logic programming. Examples and explanations of algorithms such as foil.
Typology: Study notes
1 / 102
This page cannot be seen from the preview
Don't miss anything!
Learning Rules
CS346-Spring
are a standard knowledge representation that has proven
useful in building expert systems^ if (Outlook = overcast)
then Play_Tennis = YES
if (Outlook = sunny)
(Humidity = high) then Play_Tennis = No
Learning Rules
CS346-Spring
are a standard knowledge representation that has proven
useful in building expert systems^ if (Outlook = overcast)
then Play_Tennis = YES
if (Outlook = sunny)
(Humidity = high) then Play_Tennis = No
relational representations
(first-order-representations; inductive logic programming)
Learning Rules
CS346-Spring
4
Inductive Logic Programming
Learning Rules
CS346-Spring
-^ Set of Rules:
2 k 2 1
1 m 2 1
C Y ... Y or.. Y
C X ... X X
→ ∧ ∧ ∧
→ ∧ ∧ ∧
-^ Disjunctive Rules:DNF:
Disjunction of all rules with
as a consequent
-^ Ordered set of Rules:Decision Lists:
If (Condition-1)
then C
Else if (Condition-2)
then D
…… Else^
Learning Rules
CS346-Spring
-^ Itemset X={x
, …, x 1
}k^
-^ Find all the rules
Y^ with min
confidence and support– support,
s , fraction of examples that contain both X and Y– confidence,
c,^ fraction of examples that contain X that also contain Y. Let min_support = 50%,min_conf = 50%:A^
Æ^ C (s,c) =
A (s,c) =
Customerbuys diaper Customerbuys both Transaction-id Customerbuys beer
Items bought 10
A, B, C
20
A, C
30
A, D
40
B, E, F
Learning Rules
CS346-Spring
What does it mean?
Learning Rules
CS346-Spring
ColorBlue Green Red
Yes Shape
No
Triangle
Circle^ No^
Yes Square^ Yes
For each path in the decision tree create a rule
Learning Rules
CS346-Spring
Circle Red^
→ ∧
ColorBlue Green Red
Yes Shape
No
Triangle
Circle^ No^
Yes Square^ Yes
Yes Square Red^
→ ∧
Yes
Triangle Red^
→
∧
Yes Blue
→
No Green
→
Learning Rules
CS346-Spring
ColorBlue Green Red
Shape
Triangle
Circle^ A^
Square^ B
In the general case:
A Circle Red^
→ ∧^
B
Square Red^
→ ∧
C
Triangle Red^
→
∧
B Blue
→
C Green
→
Learning Rules
CS346-Spring
14
ColorBlue Green Red
Shape
Triangle
Circle^ A^
Square^ B
(unlike Decision Trees
In the general case:
A Circle Red^
→ ∧^
B
Square Red^
→ ∧
C
Triangle Red^
→
∧
B Blue
→
C Green
→
Learning Rules
CS346-Spring
ColorBlue Green Red
Yes Shape
No
Triangle
Circle^ No^
Yes Square^ Yes
Yes Square Red^
→ ∧
Yes
Triangle Red^
→
∧
Yes Blue
→ Solution: • Sort rules by observed accuracy on the training data; treat the rules as an^ ordered set. E.g:^
Decision list: If, Then, else
Learning Rules
CS346-Spring
Temperature
Humidity
Wind
PlayTennis
1
Sunny
Hot^
High
Weak
No
2
Sunny
Hot^
High
Strong
No
3
Overcast
Hot
High
Weak
Yes
4
Rain
Mild
High
Weak
Yes
5
Rain
Cool^
Normal
Weak
Yes
6
Rain
Cool^
Normal
Strong
No
7
Overcast
Cool
Normal
Strong
Yes
8
Sunny
Mild
High
Weak
No
9
Sunny
Cool^
Normal
Weak
Yes
10
Rain
Mild
Normal
Weak
Yes
11
Sunny
Mild
Normal
Strong
Yes
12
Overcast
Mild
High
Strong
Yes
13
Overcast
Hot^
Normal
Weak
Yes
14
Rain
Mild
High
Strong
No
Learning Rules
CS346-Spring
If e is false positive for H (it is negative, H says it’s positive)^ •^
H : a specialization of H that is consistent with Examples
-^ Else if e is false negative for H (it is positive, H says it’s negative)^ •
H : a generalization of H that is consistent with Examples
-^ If no consistent specialization/generalization can be found^ •
Fail;
-^ return H •^ The Algorithm needs to choose generalizations and specializations^ (there may be several). If it gets into trouble it has to backtrack to an^ earlier decision or otherwise it fails.
Learning Rules
CS346-Spring
Learn the rule structure and the set of rules simultaneously, greedily. •^ Generalization:^ •
Remove a conjunct
(sunny and normal
to
sunny)
-^ Add a disjunct
(sunny
to^
sunny or cool)
-^ Specialization:^ •
Add a conjunct • Remove a disjunct
When to add and when to remove?Credit Assignment problem
True Concept
