Download Intermediate Econometrics - GLS Estimation - Notes | ECG 561 and more Study notes Econometrics and Mathematical Economics in PDF only on Docsity!
Notes For Intermediate Econometrics - 6
Paul L. Fackler - North Carolina State University
April 27, 2001
GLS Estimation
The linear regression mo del has the form
yt = xt + et :
The statistical issues that arise in this mo del dep end on the nature of the error term et. In the simplest case, it is assumed that
et N (0; 2 ) E [et es ] = 0 ; for t 6 = s E [et xt ] = 0 : In matrix notation the mo del is
Y = X + e;
with e N (0; 2 In ) and where Y is an n 1 vector and X is an n k matrix. In economics, however, it is often true that one or more of these assumptions is invalid. The Generalized Least Squares (GLS) estimator addresses the situation when E [et xt ] = 0 but when the n n covariance matrix of the n residuals, e, is given by a matrix :
e N (0; ):
Here is an n n matrix; the ij th element of is the covariance b etween ei and ej and ii (i.e., a diagonal element), is the variance of ei. If the error variance matrix is not prop ortional to the identity matrix then the usual expression for the OLS estimator is invalid ( 2 (X >^ X ) ^1 ). To see what it should b e notice that
O LS (^) = (X > (^) X ) (^1) X > (^) (X + e) = + (X > (^) X ) (^1) X > (^) e:
Hence the variance/covariance of O^ LS^ is given by
V ar ( O^ LS^ ) = E [( O^ LS^ )( O^ LS^ )>^ ] = E [(X >^ X ) ^1 X >^ ee>^ X (X >^ X ) ^1 ] = (X >^ X ) ^1 X >^ E [ee>^ ]X (X >^ X ) ^1 = (X >^ X ) ^1 X >^ X (X >^ X ) ^1 :
The GLS estimator can b e written in matrix form as GLS (^) = (X > (^) (^1) X ) (^1) X > (^) (^1) Y :
The covariance of this estimator is easy to nd by noting that
GLS (^) = (X > (^) (^1) X ) (^1) X > (^) (^1) (X + e) = + (X > (^) (^1) X ) (^1) X > (^) (^1) e:
Thus
C ov [ GLS^ ] = E [( GLS^ )( GLS^ )>^ ]
= (X >^ ^1 X ) ^1 X >^ ^1 E [ee>^ ] ^1 X (X >^ ^1 X ) ^1 = (X >^ ^1 X ) ^1 X >^ ^1 ^1 X (X >^ ^1 X ) ^1 = (X >^ ^1 X ) ^1 :
Supp ose we can nd a matrix U such that ^1 = U >^ U. Then the GLS estimator can b e written as
GLS (^) = (X > (^) U > (^) U X ) (^1) X > (^) U > (^) U Y = ( X~ > (^) X~ ) (^1) X~ > (^) Y~ ;
where X~ = U X and Y~ = U Y. Thus, the GLS estimator is like doing OLS with transformed data. The appropriate transformation is the one that makes the transformed residuals indep endent and standard normal:
e~ = U e N (0; In ):
It is imp ossible, however, to estimate in its most general form. Instead, one must make assumptions ab out the form that takes and then estimate the parameters of the assumed form. An imp ortant case arises when the residual exhibits so-called rst order auto correlation, also called rst order serial correlation. In this case
et = et 1 + vt ;
where vt N (0; 2 v ) and E [vt vs ] = 0, for t 6 = s.
To determine the covariance of et and es , consider rst that
E [et et 1 ] = E [e^2 t 1 ] + E [vt et 1 ] = E [e^2 t 1 ] = V ar [e^2 t 1 ]:
We can continue in this way to demonstrate that
E [et et s ] = s^ V ar [e^2 t s ]:
It can b e shown that, if jj < 1, the residuals all have the same variance, which we will call 2 (it
is not the same as v^2 ; but rather 2 = v^2 =(1 ^2 )). With this we can now see that the covariance
matrix can b e written
= (^) e^2
1 ^2 : : : n ^1 1 : : : n ^2 ^2 1 : : : n ^3 : : : : : : : : : : : : : : : n ^1 n ^2 n ^3 : : : 1
and
qt =
s d (^) d s s (^) d +
s d s (^) d yt^ ^
d s s (^) d zt^ +^ (^
s (^) ed t ^
d (^) es t ) = 1 q + 2 q yt + 3 q zt + v (^) tq
(to see if you understand what is going on, see if you can gure out the variance/covariance matrix for v dt and v (^) ts ). What this gives us is a set of so-called reduced-form equations that have only the exogenous variables on the right hand side. These equations can b e estimated eÆciently (if there is no het- eroskdasticity or serial correlation) using OLS. Estimates of are not really what we want, however. So we need to reverse the algebra and determine how the so-called structural parameters are related to the reduced form parameters (you really should do the math once just to see what is going on; it's just algebra). In this case there is a exact relationship b etween the structural and the reduced forms. This is known as an exactly identi ed mo del. In some cases there are more reduced form parameters than structural ones, implying that the assumed structure placed restrictions of the reduced form parameters. Known as an overidenti ed mo del, this provides an opp ortunity to test whether the structural mo del is supp orted by the data. On the other hand, if there are more structural pa- rameters than reduced form parameters, the mo del is under-identi ed. In this case there will not b e a unique way to estimate the structural parameters. From the p oint of view of the statistical mo del, there are more parameters than can b e estimated. In this case one either must b e willing to set some parameter values arbitrarily or b e content not to have estimates of all of the structural parameters. When a mo del is exactly identi ed, all of the structural parameters are unique functions of the reduced form parameters. This is the case in the demand and supply mo del ab ove. Using the reduced form parameters to estimate the structural ones is an approach known as indirect least squares. There are a variety of other metho ds available, but that is a topic for another time, as is a discussion of how to compute the variance of the structural parameter estimates from the variance of the reduced form parameter estimates.
Binary Choice Mo dels
Maximum likeliho o d estimators are commonly used to estimate parameters of so called binary choice mo dels. Supp ose you observe a binary (0/1) variable, Y , and would like to know how the realization of that variable dep ends on some set of exogenous explanatory variables, X. The standard approach to this problem is to assume that the probability that Y = 1 is given by
P r ob[Y = 1 jX ] = F (X );
for some assumed cumulative probability function F. This mo del makes two imp ortant assumptions. First, that the probability dep ends on a single index, z , which is a linear function of X , z = X.
Second, that the functional form of F is given. Typically F is either assumed to b e the standard normal (Gaussian distribution) or the logistic distribution
F (z ) =
ez 1 + ez^
The log likeliho o d function for this mo del is
l ( ) =
X^ n
i=
yi ln(F (Xi )) + (1 yi ) ln(1 F (Xi ));
i.e., we sum the log-probabilities of getting yi = 1 over all the observations that yi = 1 and add that to the sum over all the observations for which yi = 0 of the log-probabilities that yi = 0. To obtain the asymptotic distribution asso ciated with the maximum-likeliho o d estimator rst di erentiate with resp ect to zi :
@ l @ zi
yi f (zi ) F (zi )
(1 yi )f (zi )
(1 F (zi ))
where f (z ) = F 0 (z ). Using the chain rule, the fact that @ zi =@ = Xi and summing gives
X^ n
i=
yi f (zi ) F (zi )
(1 yi )f (zi )
1 F (zi )
Xi :
To obtain the information matrix we use
I ( ) = E
@ l @
(^) @ l @
@ li @
(^) @ l i @
y (^) i^2 F (zi )^2
2 yi (1 yi )
Fi (1 F (zi ))
(1 yi )^2
(1 F (zi ))^2
f (zi )^2 X (^) i> Xi :
But E [y i^2 ] = F (zi ), E [(1 yi )^2 ] = 1 F (zi ) and E [yi (1 y 1 )] = 0. Hence the information matrix is
I ( ) =
X^ n
i=
f (zi )^2
F (zi )(1 F (zi ))
X (^) i> Xi ;
and (as usual) the asymptotic covariance of the MLE estimator of is the inverse of the information matrix. A numb er of measures have b een prop osed to examine the quality of a binary choice mo del. One measure is the p ercent of correct predictions. If F (zi ) > 0 : 5 we predict that yi = 1 and we predict that yi = 0 if F (zi ) < 0 :5. A convenient expression for the numb er of correct predictions is the numb er of observations for which the sign of
(F (zi ) 0 :5)(yi 0 :5)
is p ositive. This can b e compared to the p ercent correct in a naive mo del in which no exogenous information is used. In this case the prediction is always either 0 or 1, dep ending on whether the
