Download Sampling Distribution of OLS Estimators for Hypothesis Testing in Linear Regression and more Study notes Economics in PDF only on Docsity!
k
1
u
x , .., x
u N , .
y
Chapter 4: Multiple regression analysis: Inference
Proposition 1
classical linear model
Wooldridge, Introductory Econometrics, 2d ed.
MLR6 (Normality) The population error is in- dependent of the explanatory variables and is normally distributed with zero mean and constant variance:
We have discussed the conditions under which OLS estimators are unbiased, and derived the variances of these estimators under the Gauss-Markov assumptions. The Gauss- Markov theorem establishes that OLS estimators have the smallest variance of any linear unbiased estimators of the population parameters. We must now more fully characterize the sampling distribution of the OLS estimators–beyond its mean and variance–so that we may test hypotheses on the population parameters. To make the sampling distribution tractable, we add an assumption on the distribution of the errors:
This is a much stronger assumption than we have previously made on the distribution of the errors. The assumption of normality, as we have stated it, subsumes both the assumption of the error process being independent of the explanatory variables, and that of homoskedasticity. For cross-sectional regression analysis, these six assumptions define the. The rationale for normally distributed errors is often phrased in terms of the many factors influencing being additive, appealing to the Central Limit Theorem to suggest that the sum of a large number of random factors will be normally distributed. Although we might have reason in a particular context to doubt
j
0 1 1 2 2
j (^) j j
j (^) j b j
j
j
k k
Testing an hypothesis on a single
b N , V ar b
b
s
N ,
b
b
k
y x x ... x u
this rationale, we usually use it as a working hypothesis. Various transformations–such as taking the logarithm of the dependent variable–are often motivated in terms of their inducing normality in the resulting errors. What is the importance of assuming normality for the error process? Under the assumptions of the classical linear model, normally distributed errors give rise to normally distributed OLS estimators: (1) which will then imply that:
(2)
This follows since each of the can be written as a linear combination of the errors in the sample. Since we assume that the errors are independent, identically distributed normal random variates, any linear combination of those errors is also normally distributed. We may also show that any linear combination of the is also normally distributed, and a subset of these estimators has a joint normal distribution. These properties will come in handy in formulating tests on the coefficient vector. We may also show that the OLS estimators will be approximately normally distributed (at least in large samples), even if the underlying errors are not normally distributed.
To test hypotheses about a single population parameter, we start with the model containing regressors:
(3)
b
j j
j
p-value
if the null hypothesis is true
the estimated coefficient to its estimated standard error. If the null hypothesis is to be rejected, the “t-stat” must be larger (in absolute value) than the critical point on the t-distribution. The “t-stat” will have the same sign as the estimated coefficient, since the standard error is always positive. Even if is actually zero in the population, a sample estimate of this parameter– will never equal exactly zero. But when should we conclude that it could be zero? When its value cannot be distinguished from zero. There will be cause to reject this null hypothesis if the value, scaled by its standard error, exceeds the threshold. For a “two-tailed test,” there will be reason to reject the null if the “t-stat” takes on a large negative value or a large positive value; thus we reject in favor of the alternative hypothesis (of in either case. This is a two-sided alternative, giving rise to a two-tailed test. If the hypothesis is to be tested at, e.g., the 95% level of confidence, we use critical values from the t-distribution which isolate 2.5% in each tail, for a total of 5% of the mass of the distribution. When using a computer program to calculate regression estimates, we usually are given the “ ” of the estimate–that is, the tail probability corresponding to the coefficient’s t-value. The p-value may usefully be considered as the probability of observing a t-statistic as extreme as that shown. If the t-value was equal to, e.g., the 95% critical value, the p-value would be exactly 0.05. If the t-value was higher, the p-value would be closer to zero, and vice versa. Thus, we are looking for small p-values as indicative of rejection. A p-value of 0.92, for instance, corresponds to an
0
0 :^0
j A (^) j
j
j
H
H >
H
b
Testing other hypotheses about
hypothesis that can be rejected at the 8% level of confidence–thus quite irrelevant, since we would expect to find a value that large 92% of the time under the null hypothesis. On the other hand, a p-value of 0.08 will reject at the 90% level, but not at the 95% level; only 8% of the time would we expect to find a t-statistic of that magnitude if was true. What if we have a one-sided alternative? For instance, we may phrase the hypothesis of interest as:
(6)
Here, we must use the appropriate critical point on the t-distribution to perform this test at the same level of confidence. If the point estimate is positive, then we do not have cause to reject the null. If it is negative, we may have cause to reject the null if it is a sufficiently large negative value. The critical point should be that which isolates 5% of the mass of the distribution in that tail (for a 95% level of confidence). This critical value will be smaller (in absolute value) than that corresponding to a two-tailed test, which isolates only 2.5% of the mass in that tail. The computer program always provides you with a p-value for a two-tailed test; if the p-value is 0.08, for instance, it corresponds to a one-tailed p-value of 0.04 (that being the mass in that tail).
Every regression output includes the information needed to test the two-tailed or one-tailed hypotheses that a population parameter equals zero. What if we want to test a different hypothesis about the value of that parameter? For instance, we
Confidence intervals
. di (_b[bdrms]-20000)/_se[bdrms] -. test
test
test regress
perform this test. In this particular case–of an hypothesis involving a single regression coefficient–we may show that this F-statistic is merely the square of the associated t-statistic. The p-value would be the same in either case. The estimated coefficient is 15198.19, with an estimated standard error of 9483.517. Plugging in these values to (8) yields a t-statistic:
which, squared, is the F-statistic shown by the command. Just as with tests against a null hypothesis of zero, the results of the command may be used for one-tailed tests as well as two-tailed tests; then, the magnitude of the coefficient matters (i.e. the fact that the estimated coefficient is about $15,000 means we would never reject a null that it is less than $20,000), and the p-value must be adjusted for one tail. Any number of commands may be given after a command in Stata, testing different hypotheses about the coefficients.
As we discussed in going over Appendix C, we may use the point estimate and its estimated standard error to calculate an hypothesis test on the underlying population parameter, or we may form a confidence interval for that parameter. Stata makes that easy in a regression context by providing the 95% confidence interval for every estimated coefficient. If you want to use some other level of significance, you may either use
1 2 3
1 2 3
1 2 3
(^0 1 2 )
1 2 3
1
bdrms
bdrms
b b b
n k
log = log + log + log + log +
CRT S
Q AL K E
Q A L K E
K, L, E
CRT S
H
b b b
s
t
level() regress regress price bdrms sqrft, level(90) set level
Testing hypotheses about a single linear combination of the parameters
the option on (e.g. ) or you may change the default level for this run with. All further regressions will report confidence intervals with that level of confidence. To connect this concept to that of the hypothesis test, consider that in the above example the 95% confidence interval for extended from -3657.581 to 34053.96; thus, an hypothesis test with the null that takes on any value in this interval (including zero) will not lead to a rejection.
Economic theory will often suggest that a particular linear combination of parameters should take on a certain value: for instance, in a Cobb-Douglas production function, that the slope coefficients should sum to one in the case of constant returns to scale ( : (9)
where are the factors capital, labor, and energy, respectively. We have added an error term to the double-log- transformed version of this model to represent it as an empirical relationship. The hypothesis of may be stated as:
(10) The test statistic for this hypothesis is quite straightforward: (11)
and its numerator may be easily calculated. The denominator,
0 0 0
L K L K L K
CRTS
CRTS
H , or
H
, or
H
Testing multiple linear restrictions
test test labor - 2*cap = 0
test labor=2*cap
test
is 2/3. This implies that the labor coefficient should be twice the capital coefficient, or:
(12)
Note that this does not allow us to test a nonlinear hypothesis on the parameters: but considering that a ratio of two parameters is a constant is not a nonlinear restriction. In the latter form, we may specify it to Stata’s command as:
In fact, Stata will figure out that form if you specify the hypothesis as:
(rewriting it in the above form), but it is not quite smart enough to handle the ratio form. It is easy to rewrite the ratio form into one of the other forms. Either form will produce an F-statistic and associated p-value related to this single linear hypothesis on the parameters which may be used to make a judgment about the hypothesis of interest.
When we use the command, an F-statistic is reported– even when the test involves only one coefficient–because in general, hypothesis tests may involve more than one restriction on the population parameters. The hypotheses discussed above– even that of , involving several coefficients–still only represent one restriction on the parameters. For instance, if
1
k n k
n b ,n
joint tests
each
0 2 2 2 2 2
( +1) 1 2 2 2
y u
y.
R
R
R
SSE
SST
F
SSE/k
SSR/ n k
F
R /k
R / n k
F
F , t
t
is imposed, the elasticities of the factors of production must sum to one, but they may individually take on any value. But in most applications of multiple linear regression, we concern ourselves with of restrictions on the parameters. The simplest joint test is that which every regression reports: the so-called “ANOVA F” test, which has the null hypothesis that of the slopes is equal to zero. Note that in a multiple regression, specifying that each slope individually equals zero is not the same thing as specifying that their sum equals zero. This “ANOVA” (ANalysis Of VAriance) F-test is of interest since it essentially tests whether the entire regression has any explanatory power. The null hypothesis, in this case, is that the “model” is that is, none of the explanatory variables assist in explaining the variation in We cannot test any hypothesis on the of a regression, but we will see that there is an intimate relationship between the and the ANOVA F:
(13)
where the ANOVA F, the ratio of mean square explained variation to mean square unexplained variation, is distributed as under the null hypothesis. For a simple regression, this statistic is which is identical to that is, the square of the statistic for the slope coefficient, with precisely the
lsalary.
. test frstbase scndbase shrtstop thrdbase catcher ( 1) frstbase = 0. ( 2) scndbase = 0. ( 3) shrtstop = 0. ( 4) thrdbase = 0. ( 5) catcher = 0. F( 5, 345) = 2. Prob > F = 0.
scndbase
. test frstbase shrtstop thrdbase catcher ( 1) frstbase = 0. ( 2) shrtstop = 0. ( 3) thrdbase = 0. ( 4) catcher = 0. F( 4, 345) = 1. Prob > F = 0.
At the 95% level of significance, these coefficients are not each zero. That result, of course, could be largely driven by the coefficient:
So perhaps it would be sensible to remove these four, which even when taken together do not explain a meaningful fraction of the variation in But this illustrates the point of the joint hypothesis test: the result of simultaneously testing several hypotheses (that, for instance, individual coefficients are equal to zero) cannot be inferred from the results of the individual tests. If each coefficient is significant, then a joint test will surely reject the joint exclusion restriction; but the converse is assuredly false.
2
2
test
r ur ur
SSRs
F
SSR SSR /q
SSR / n k
SSR R ,
SSR
q,
s.
F
F
q n k
Notice that a joint test of exclusion restrictions may be easily conduced by Stata’s command, by merely listing the variables whose coefficients are presumed to be zero under the null hypothesis. The resulting test statistic is an F with as many numerator degrees of freedom as there are coefficients (or variables) in the list. It can be written in terms of the residual sums of squares ( of the “unrestricted” and “restricted” models:
(14) Since adding variables to a model will never decrease (nor decrease the “restricted” model–in which certain coefficients are not freely estimated from the data, but constrained–must have at least as large as the “unrestricted” model, in which all coefficients are data-determined at their optimal values. Thus the difference in the numerator is non- negative. If it is a large value, then the restrictions severely diminish the explanatory power of the model. The amount by which it is diminished is scaled by the number of restrictions, and then divided by the unrestricted model’s If this ratio is a large number, then the “average cost per restriction” is large relative to the explanatory power of the unrestricted model, and we have evidence against the null hypothesis (that is, the statistic will be larger than the critical point on an table with and degrees of freedom. If the ratio is smaller than the critical value, we do not reject the null hypothesis, and conclude that the restrictions are consistent with the data. In
5 5
(y + X )).
test regress y x1 x2 x3 x4 x test x test x2+x3+x4=1, accum test x5=-1, accum test
( 1) years = 0. ( 2) frstbase + scndbase + shrtstop = 1. ( 3) sbases = -1. F( 3, 347) = 38. Prob > F = 0. accum test
instance, constraining to -1 implies that you should form a new dependent variable, Alternatively, if you are using a computer program that can test linear restrictions, you may use its features. Stata will test general linear restrictions of this sort with the command:
The final command in this sequence will print an F-statistic for the set of three linear restrictions on the regression: for instance,
The “ ” option on the command indicates that these tests are not to be performed separately, but rather jointly. The final F-test will have three numerator degrees of freedom, because you have specified three linear hypotheses to be jointly applied to the coefficient vector. This syntax of test may be used to construct any set of linear restrictions on the coefficient vector, and perform the joint test for the validity of those restrictions. The test statistic will reject the null hypothesis (that the restrictions are consistent with the data) if its value is large relative to the underlying F-distribution.
