Concept Learning in Machine Learning, Lecture notes of Machine Learning

Download Concept Learning in Machine Learning and more Lecture notes Machine Learning in PDF only on Docsity!

Concept LearningMitchell, Chapter 2^ CptS 570 Machine Learning

School of EECS

Washington State University

Outline^ „^ Definition^ „^ General-to-specific ordering overhypotheses^ „^ Version spaces and the candidateelimination algorithm^ „^ Inductive bias

Example: Enjoy Sport^ „^ Learn a concept for predicting whether youwill enjoy a sport based on the weather^ „^ Training examples^ Example^ „^ What is the general concept?

Sky^ AirTemp

Humidity^

Wind^ Water

Forecast^

EnjoySport

1 Sunny

Warm^

Normal^ Strong

Warm^ Same

Yes

2 Sunny

Warm^

High^ Strong

Warm^ Same

Yes

3 Rainy

Cold^

High^ Strong

Warm^ Change

No

4 Sunny

Warm^

High^ Strong

Cool^ Change

Yes

Learning Task: Enjoy Sport^ „^ Task T^ „^ Accurately predict enjoyment^ „^ Performance P^ „^ Predictive accuracy^ „^ Experience E^ „^ Training examples each with attributevalues and class value (yes or no)

Concept Learning Task^ „^ Given^ „^ Instances

X : Possible days „ Each described by the attributes: Sky, AirTemp, Humidity,Wind, Water, Forecast „^ Target function

c : EnjoySport

Æ^ {0,1}

„^ Hypotheses

H : Conjunctions of literals „ E.g., „^ Training examples

D „^ Positive and negative examples of the target function „^ <x,c(x)>, …, <x^11

,c(x)>mm „^ Determine^ „^ A hypothesis

h^ in^ H^ such that

h(x) = c(x)

for all^ x^ in

D

Terminology^ „^ Instances or instance space

X

„^ Set of all possible input items „^ E.g.,^ x^ = <Sunny, Warm, Normal, Strong, Warm, Same> „^ | X | = 32222*2 = 96 „ Target concept

c^ :^ X^ Æ^

„^ Concept or function to be learned „^ E.g.,^ c(x)

=1 if EnjoySport=yes,

c(x) =0 if EnjoySport=no

„^ Training examples

D^ = { <x, c(x)>

},^ x^ ∈^ X

„^ Positive examples,

c(x)^ = 1, members of target concept „^ Negative examples,

c(x)^ = 0, non-members of target concept

Terminology^ „^ Inductive learning hypothesis^ „^ Any hypothesis approximating the targetconcept well, over a sufficiently large set oftraining examples, will also approximatethe target concept well for unobservedexamples

Concept Learning as Search^ „^ Learning viewed as a search throughhypothesis space

H^ for a hypothesis

consistent with the training examples „ General-to-specific ordering ofhypotheses^ „^ Allows more directed search of

H

General-to-Specific Orderingof Hypotheses^ „^ Hypothesis

h is more general than or equal^1 to hypothesis

h iff^ ∀^2

x^ ∈^ X ,^ h

(x)=1^ ← 1

h(x)=1^2

„^ Written

h≥h^1 g^2 „^ h strictly more general than^1

h ( h >^2

h ) g 2

when^ h

≥h and 1 g^2

h≥h^2 g^1 „^ Also implies

h≤h ,^2 g^1

h more specific than^2

h^1

„^ Defines partial order over

H

Finding Maximally-SpecificHypothesis^ „^ Find the most specific hypothesiscovering all positive examples^ „^ Hypothesis

h^ covers positive example

x

if^ h(x) = 1 „ Find-S algorithm

Find-S Example

Find-S Algorithm^ „^ Will^ h

ever cover a negative example? „ No, if^ c^ ∈^ H^

and training examples consistent „^ Problems with Find-S^ „^ Cannot tell if converged on target concept^ „^ Why prefer the most specific hypothesis?^ „^ Handling inconsistent training examples due toerrors or noise^ „^ What if more than one maximally-specificconsistent hypothesis?

Representing Version Spaces^ „^ The^ general boundary

G^ of version space

VS is H,D^

the set of its maximally general members „ The^ specific boundary

S^ of version space

VS is H,D^

the set of its maximally specific members „ Every member of the version space lies in or betweenthese members^ „^ “Between” means more specific than

G^ and more general

than^ S „ Thm. 2.1. Version space representation theorem „ So, version space can be represented by just

G^ and^ S

Version Space Example Version space resulting from previous four EnjoySport examples.