Least-squares regression
Cautions about correlation and
regression
Outline:
•Least-squares regression.
–Equations of regression line: slope,
intercept
–Residuals and residual plot
–Outliers and influential observations
•Cautions about correlation and regression
1
Least-Squares Regression
Regression describes the relationship between two
variables in the situation where one variable can
be used to explain or predict the other.
The regression line is a straight line that describes
how a response variable ychanges as an
explanatory variable xchanges.
2
Fitting the Regression Line to Data
Since we intend to predict yfrom x, the errors of
interest are mispredictions of yfor a fixed x.
The least-squares regression line of yon xis
the line that minimizes sum of squared errors.
This is the least squares criterion.
Given pairs of observations (x1,y1), . . . , (xn, yn),
the regression line is given by
ˆy=a+bx
where b=rsy
sxand a= ¯y−b¯x.
3
Interpreting the Regression Model
•The response in the model is denoted ˆyto
indicate that these are predictd yvalues, not
the true observed yvalues. The “hat”
denotes prediction.
•The slope of the line indicates how much ˆy
changes for a unit change in x.
•The intercept is the value of ˆyfor x= 0. It
may or not have a physical interpretation,
depending on whether or not xcan take
values near 0.
•To make a prediction for an unobserved x,
just plug it in and calculate ˆy.
•Note that the line need not pass through the
observed data points. In fact, it often will not
pass through any of them.
4
