Maching learning -data mining, Apuntes de Minería de Datos

¡Descarga Maching learning -data mining y más Apuntes en PDF de Minería de Datos solo en Docsity!

Regression for Data Mining

Mgt. 2206 – Introduction to Analytics

Matthew Liberatore

Thomas Coghlan

Learning Objectives



To understand the application of regression

analysis in data mining



Linear/nonlinear



Logistic (Logit)



To understand the key statistical measures of

fit



To learn how to run and interpret regression

analyses using SAS Enterprise Miner

software

Linear Regression Analysis



Analysis of the strength of the linear

relationship between predictor

(independent) variables and outcome

(dependent/criterion) variables.



In two dimensions (one predictor, one

outcome variable) data can be plotted on a

scatter diagram.

E

E (

( y

y ) =

) = 



0

0





1

1

(x))

(x))

Expected value of

y (outcome)

Intercept

Term

coefficient

Predictor

variable

Estimation Process

Regression Model

Regression Model

y

y

= 



00





11

x

x





Regression Equation

Regression Equation

E

E (

( y

y ) =

) = 



00





11

x

x

Unknown Parameters

Unknown Parameters





0

0

,

, 



1

1

Sample Data:

Sample Data:

x y

x y

x

x

11

y

y

11

.. .. .. ..

x

x

n

n

y

y

n

n

b

b

0

0

and

and b

b

1

1

provide estimates of

provide estimates of





00

and

and 



11

Estimated

Estimated

Regression Equation

Regression Equation

Sample Statistics

Sample Statistics

b

b

0

0

,

, b

b

1

1

0 1

ˆ

y  b  b x

E

E (

( y

y ): Outcome

): Outcome

x: Predictor

x: Predictor

Slope

Slope

1

1

is negative

is negative

Regression line

Regression line

Intercept

Intercept

0

0

Simple Linear Regression Equation:

Negative Linear Relationship

Negative Linear Relationship

E

E (

( y

y ): Outcome

): Outcome

x: Predictor

x: Predictor

Simple Linear Regression Equation:

No Relationship

No Relationship

• •

E

E (

( y

y ): Outcome

): Outcome

x: Predictor

x: Predictor

Intercept

Intercept

0

0

Simple Linear Regression Equation:

Parabolic Relationship

Parabolic Relationship

•••••••••••••••••••••••

Example



List Variables we have



Determine a DV of interest



Is there a way to predict DV?



Slope for the Estimated Regression

Equation

1

2

( )( )

( )

i i

i

x x y y

b

x x

 





1

2

( )( )

( )

i i

i

x x y y

b

x x

 





Least Squares Method



y

y

Intercept for the Estimated Regression Equation

Intercept for the Estimated Regression Equation

Least Squares Method

Least Squares Method

0 1

b  y  b x

0 1

b  y  b x

where:

where:

x

x

ii

=

= value of independent variable for

value of independent variable for i

i th

th

observation

observation

n

n

= total number of observations

total number of observations

_

_

y

y = mean value for dependent variable

= mean value for dependent variable

_

_

x

x = mean value for independent variable

= mean value for independent variable

y

y

i

i

=

= value of dependent variable for

value of dependent variable for i

i th

th

observation

observation

Example: Kwatts vs. Temp

• 59.2 9, Temp Kwatts
• 61.9 9,
• 55.1 10,
• 66.2 10,
• 52.1 10,
• 69.9 11,
• 46.8 12,
• 76.8 13,
• 79.7 15,
• 79.3 15,
• 80.2 17,
• 83.3 17,

Is the Relationship Linear?

KWatts vs. Temp

0

2,

4,

6,

8,

10,

12,

14,

16,

18,

20,

40 45 50 55 60 65 70 75 80 85 90

Temp

KWatts

KWatts

Coefficient of Determination



How “strong” is relationship between predictor &

outcome? (Fraction of observed variance of

outcome variable explained by the predictor

variables).



Relationship Among SST, SSR, SSE

where:

where:

SST = total sum of squares

SST = total sum of squares

SSR = sum of squares due to regression

SSR = sum of squares due to regression

SSE = sum of squares due to error

SSE = sum of squares due to error

SST = SSR + SSE

SST = SSR + SSE

2

( )

i

y  y



2

( )

i

y  y



2

ˆ

( )

i

 y  y



2

ˆ

( )

i

 y  y



2

ˆ

( )

i i

 y  y



2

ˆ

( )

i i

 y  y



Coefficient of Determination (

Coefficient of Determination ( r

r

2

2

)

)

where:

where:

SSR = sum of squares due to regression

SSR = sum of squares due to regression

SST = total sum of squares

SST = total sum of squares

r

r

22

= SSR/SST

= SSR/SST