Download Multiple Regression Model - Computer Processing Data - Exam 1 | STAT 479 and more Exams Statistics in PDF only on Docsity!
Multiple Regression Model
y = β 0 + β 1 x 1 + · · · + βkxk + where y: response or the dependent variable, x 1 , x 2 ,... , xk: the explanatory variables (or independent variables or predictors) β 0 , β 1 ,... , βk: unknown constants (“the coefficients”) : an unobservable random variable, the error in observing y
- Under this model, E(y) = μ(x) = β 0 + β 1 x 1 + · · · + βkxk
- Multiple regression data: n observations or cases of k + 1 values (yi, xi 1 , xi 2 ,... , xik), i = 1, 2 ,... , n
- For making statistical inference, it is usually assumed that 1 , 2 ,... , n is random sample from the N (0, σ^2 ) distribution.
- Estimation of Parameters: Minimize the sum of squares of residuals: Q = ∑n i=
{yi − (β 0 + β 1 x 1 i + · · · + βk xki)}^2
with respect to βˆ 0 , βˆ 1 ,... , βˆk
- Set the partial derivatives of Q with respect to each of the β coefficients equal to zero. The resulting set of equations are linear in the β’s and is called the normal equations.
- Least squares estimates: the solution to these equations of the β’s denoted by βˆ 0 , βˆ 1 ,... , βˆk
- The prediction equation:
yˆ = βˆ 0 + βˆ 1 x 1 + · · · + βˆk xk
y = X β + where
y =
y 1 y 2 ... yn
, X =
1 x 11 · · · xk 1 1 x 12 · · · xk 2 ... ... ... 1 x 1 n xkn
, β =
β 0 β 1 ... βk
n
- Minimize the sum of squares: Q = (y − Xβ)′(y − Xβ)
- The normal equations: X′Xβ = X′y
- The solution: βˆ = (X′X)−^1 X′y where (X′X)−^1 =inverse of the X′X matrix (assuming it is nonsingular)
- X′X, a (k + 1) × (k + 1) matrix, and its inverse are imporatnt in multiple regression computations.
- Testing the hypothesis:
H 0 : β 1 = β 2 = · · · = βk = 0vs.Ha : at least one β 6 = 0.
- Predicted or Fitted Values: ˆy = X βˆ where yˆi = βˆ 0 + βˆ 1 x 1 i + · · · + βˆk xki, i = 1,... , n
- Residuals: e = y − yˆ where e = (e 1 , e 2 ,... , en)′^ and ei = yi − yˆi, i = 1,... , n.
- Hat Matrix, H:
yˆ = X βˆ = X(X′X)−^1 X′y = Hy where H = X(X′X)−^1 X′^ is an n × n symmetric matrix. The ith diagonal element of H satisfies, 1 n
≤ hii ≤
d
- Standard Error of yˆi: Since ˆy = Hy, it can be shown that Var(ˆyi) = σ^2 hii. Thus the standard error of ˆyi = s
hii, for i = 1, 2 ,... , n.
Since e = y − yˆ = y − Hy = (I − H)y, it can be shown that Var(ei) = σ^2 (1 − hii) Thus the standard error of ei = s
√ (1 − hii) for i = 1, 2 ,... , n.
- A (1 − α)100% Confidence Interval for the Mean E(yi): yˆi ± tα/ 2 ,(n−k−1) × s
hii
- A (1 − α)100% Prediction Interval for yi:
yˆi ± tα/ 2 ,(n−k−1) × s
1 + hii
- Studentized Residuals: a standardized version of the or- dinary residuals
- An internally studentized residuals: ri divide the residuals by their standard errors
ri = ei/(s
1 − hii) for i = 1,... , n.
- The statistic maxi |ri|: used to test for the presence of a single y-outlier using Tables B.8 and B.
H 0 : No Outliers vs. Ha : A Single Outlier Present
Reject H 0 : if maxi |ri| exceeds the appropriate percentage point.
- An externally studentized residuals: ti divide residuals by s^2 (i), the MSE from a regression model fitted with the ith case deleted ti = ei/(s(i)
1 − hii) for i = 1,... , n.
- Influenial Cases: Those when deleted causes major changes in the fitted model
- Cook’s D statistic: A diagnostic tool that measures the influence of the ith case
Di =
k′
ei s
1 − hii
2 ^ hii 1 − hii
k′^
r i^2
^ hii 1 − hii
where k′^ = k + 1.
Di = const × studentized residual^2 × monotone increasing func
- A large Di may be caused by a large ri, or a large hii, or both.
- cases with large leverages may not be influential because ri is small → that these cases actually fit the model well.
- Cases with relatively large values for both hii and ri should be of more concern.
- A rule of thumb: cases with Cook’s D larger than 4/n are flagged for further investigation.
