Download Simple Linear Regression - Basic Statistics for Behavioral Sciences - Lecture Notes and more Study notes Statistics for Psychologists in PDF only on Docsity!
Ch. 5. Simple Linear Regression
I. Situation A. One group and two numerical variables (X and Y). B. Want to develop a straight line for the prediction of y using x in the future (Y regresses on X).
II. Mathematical Model A. Deterministic (ideal) Model yi = β 0 + β 1 xi
where, yi : criterion variable β 0 : y-intercept, the value of y when x= β 1 : slope of the line, Δy/Δx xi : predictor variable. e.g.,
B. Statistical (reality) Model (in the population)
yi = β 0 + β 1 xi + εi
where, εi : error associated with the prediction.
y^i = β 0 + β 1 xi
where y^i : predicted value of y.
Thus, yi = β 0 + β 1 xi + εi = y^i + εi and, εi = yi - y^i
C. Sample Regression Model
yi = b 0 + b 1 xi + ei
y^i = b 0 + b 1 xi
ei = yi - y^i
III. Assumptions A. Assumption 1: Existence
For a given value of x, y has a certain (usually normal) distribution with a finite mean (μY|X) and variance (σ²Y|X). B. Assumption 2: Independence For a given value of x, y is independently distributed. C. Assumption 3: Linearity The mean values of y for a given x make a straight- line function with the x values. D. Assumption 4: Homoscedasticity The variance of y-score distribution is the same across all given x values. E. Assumption 5: Normal Distribution For each given x value the y scores are normally distributed. F. Assumption 6: Error-free Measurement of X The x values are measured without measurement error.
IV. Developing the Best-Fitting Straight Line A. Situation
- In the area of behavioral sciences we hardly have a perfect regression line.
- Therefore, our job is to develop the best-fitting straight line through a swarm of points. B. Criterion
- (Ordinary) Least-Squares Criterion: The sum of the squared deviation scores of y about the predicted y is smaller than the sum of squared deviation scores about any other value. Σ(Y - Y^)² ≤ Σ(Y - Y^ + C)²
- Minimum-Variance Criterion: The sample slope (b 1 ) and y-intercept (b 0 ) are unbiased and should result in the minimum variance around the parameters (β 1 and β 0 ).
- Fortunately, the two criteria give us an identical solution. C. Solution for the regression (Formula for the b 1 and b 0 ) D. Accuracy of the Regression Line
- SSE (Sum of Squared Error) a) Want to know the average error (Y - Y^) of the prediction. b) As we experienced in the computation of SS with the mean, the sum of error is always
C. Testing β 0 = β 0 (0)^ (usually β 0 (0)^ = 0).
- Test if Y-intercept, β 0 = 0 or a certain value.
- Procedure Ho: β 0 = β 0 (0)^ H1: β 0 ≠ β 0 (0) (or directional hypotheses) α = .05 or .01 t(crit), df = n- TS t(obs) = Decision: If |t(obs)| ≥ t(crit), reject Ho.
- CI = b 0 ± tα/2 (SE), where
2
2 | ( 1 )
X
YX n s
X
n
SE s −
D. Testing μY|Xo = μY|Xo(0)^ (certain value).
- Testing if the conditional mean, μY|Xo = μY|Xo(0).
- Procedure Ho: μY|Xo = μY|Xo(0)^ H1: μY|Xo ≠ μY|Xo(0) (or directional hypotheses) α = .05 or .01 t(crit), df = n- TS y^Xo - μY|Xo(0) t(obs) = ──────────── SE
Decision: If |t(obs)| ≥ t(crit), reject Ho.
- CB (confidence band) = y^Xo ± tα/2 (SE), where
2
2 0 | ( 1 )
X
YX n s
X X
n
SE s −
E. PI (prediction interval)
- Building an interval for actual Y values based on y^ for given Xo.
- PI = y^Xo ± tα/2 (SE), where
2
2 0 | ( 1 )
X
YX n s
X X
n
SE s −
