Download Simple Linear Regression: Probabilistic Models and Least Squares Approach and more Study notes Data Analysis & Statistical Methods in PDF only on Docsity!
Chapter 11
Simple Linear Regression
Probabilistic Models
General form of Probabilistic Models
Y = Deterministic Component + Random
2
p
Error
where
E(y) = Deterministic Component
Probabilistic Models
First Order (Straight-Line) Probabilistic Model
y = β + β x + ε
3
Probabilistic Models
5 steps of Simple Linear Regression
- Hypothesize the deterministic component
- Use sample data to estimate unknown model
4
parameters
- Specify probability distribution of , estimate
standard deviation of the distribution
- Statistically evaluate model usefulness
- Use for prediction, estimatation, once model is
useful
Fitting the Model: The Least
Squares Approach
Reaction Time versus Drug Percentage
Subject Amount of Drug x (%) Reaction Time y (seconds) 1 1 1 2 2 1 3 3 2 4 4 2
5
4 4 2 5 5 4
Fitting the Model: The Least
Squares Approach
Least Squares Line has:
•Sum of errors (SE) = 0
•Sum of Squared errors (SSE) is smallest of all straight line
models
y x 0 1
6
models
Formulas:
Slope: y-intercept
i i xy i i i i
x y SS x x y y x y n
1
ˆ xy
xx
SS
SS
0 1
βˆ = y −βˆ x
2 2 2 (^ ) ( )
i xx i i
x SS x x x n
11/26/
2
Fitting the Model: The Least
Squares Approach
Preliminary Computations x i yi x^2 i x yi i 1 1 1 1 2 1 4 2 3 2 9 6 4 2 16 8 5 4 25 20 ∑ ∑
2 ∑ ∑
7
Totals (^) ∑ xi = (^15) ∑ yi = (^10) ∑ x 2 i^ = (^55) ∑ x yi i = 37
Comparing Observed and Predicted Values for the Least Squares Prediction Equation x y^ ˆ y^ = −.1 +.7 x ( y − ˆ y ) ( y − y ˆ)^2 1 1 .6 .4. 2 1 1.3 -.3. 3 2 2.0 0.0. 4 2 2.7 -.7. 5 4 3.4 .6. Sum of Errors = 0 SSE = 1.
Model Assumptions
- Mean of the probability distribution of ε is 0
- Variance of the probability distribution of ε is constant
for all values of x
- Probability distribution of ε is normal
8
y
- Values of ε are independent of each other
An Estimator of σ
2
Estimator of σ
for a straight-line model
2
2
SSE SSE s Degrees of freedom for error n
= = −
9
eg ees of f eedom fo e o n
( )
( )
1
2 (^2 )
2
yy xy
i yy i i
SSE SS SS
y
SS y y y
n
s s Estimated Standard Errorof the Regression Model
= − β
∑ ∑
Assessing the Utility of the Model:
Making Inferences about the Slope β 1
Sampling Distribution of (^1) βˆ
σ
10
1
ˆ
xx
SS
β
σ =
Assessing the Utility of the Model: Making
Inferences about the Slope β 1
A Test of Model Utility: Simple Linear Regression
One-Tailed Test Two-Tailed Test
H 0 : β 1 =0 H 0 : β 1 =
11
H (^) a : β 1 <0 (or H (^) a : β 1 >0) H (^) a : β 1 ≠ 0
Rejection region: t< -tα
(or t< -tα when H (^) a : β 1 >0)
Rejection region: |t|> tα/
Where tα and tα/2 are based on (n-2) degrees of freedom
1
1 1
ˆ
xx
Test statistic t
s s SS
β
Assessing the Utility of the Model: Making
Inferences about the Slope β 1
A 100(1-α)% Confidence Interval for β 1
where 1 2 ˆ
ˆ β ± t s
12
1
t s α (^) β
β ±
1
ˆ
xx
s s
SS
β
=
