Download Properties of Large and Small Samples: Consistency, Unbiasedness, and Distribution and more Study notes Engineering in PDF only on Docsity!
Large Sample and Small Sample
Just as their names imply, large sample property is about the property of a statistic when n! 1, here n is the sample size. While small sample property is about the property when n is Ćxed, and Ćnite. Basically, these properties could be summarized in the following table.
Large Sample Small Sample Consistency !p^ LLN Unbiasedness E Asymptotic Distribution !d^ CLT Exact Distribution Asymptotic EĀ¢ ciency AVar Minimal Variance Var The Ćrst question is why we need large sample theory. Basically, if we donĆt derive the large sample properties, then we almost know nothing about the statistic we are using. For most of the cases, we donĆt know much about the exact small sample properties of our statistic except some Monte Carlo simulation results(actually, Monte Carlo is using much more than we are assuming). The solution to this problem is that we use some approximation to the exact distribution. There are two most popular methods to approximate the exact distribution: large sample theory and bootstrap. Here, I will concentrate on the large sample theory. Another justiĆcation of large sample theory is that it lies behind the philosophy of frenquencist, that is, we could "potentially" get inĆnite observations, and we could and should check what will happen as n! 1. In summary, large sample theory is the benchmark for any further development. Any statistic b^ is a function of the sample(that is, iid observations), that is, b^ = f (z 1 ; ; zn) = g(n; F ), where F is the distribution of zi. Large sample theory is letting n go to 1 and check what will happen, while bootstrap is Ćxing n, but substituting F by Fn, the empirical distribution. Email: [email protected]
Example 1 (Midterm 2000, 1) The model is
yi = xi + ei, E[eijxi] = 0
where xi, and ei are scalar. We consider the estimator
e (^) = yx =
P^ n iP=1n^ yi i=1^ xi We assume that xi and ei have Ćnite 4th moments and that fyi; xigare a random sample(iid). (a)Find E[e^ jX]; (b)Find V ar^ e^ jX^ ; (c)Show that e^ !p^ as n! 1. Does this require any additional assumptions? (d)Find the asymptotic distribution of pn(e^ ) as n! 1; (e)Without imposing any additional assumption, is e^ necessarily less eĀ¢ cient than OLS? Solution: e^ =
P^ n i=1^ (Px ni^ +ei) i=1^ xi
P^ n iP=1n^ ei i=1^ xi
(a) This is conditional unbiasedness, a small sample property. E^ he^ jX^ i = E
P^ n iP=1n^ ei i=1^ xi
jX
P^ n i=1P^ E n[eijxi] i=1^ xi
= 0, so E[e^ jX] =. (b) This is about conditional variance, a small sample property. From Gauss-Markov Theorem, OLS is somewhat best in small sample(homoskedastic)environment, this estimator is not OLS estimator, so neednĆt be best in this case. V ar^ e^ jX^ = E
e ^ ^2 jX
= E
P^ n iP=1n^ ei i=1^ xi
A
2 jX
(^75) = iP=1 ^ n (^) Pn^ E[e^2 i^ jxi] i=1^ xi
P^ n iPn=1^ ^2 i i=1^ xi
(c) This is consistency, a large sample property. By LLN, we have (^1) n iP=1n ei !p^ E[ei] = 0, and (^) n^1 iP=1n xi !p^ E[xi] = . If 6 = 0, then e^ =
(^1) n iP=1n ei n^1 iP=1n^ xi^ !p^0 ^ = 0^ (why could we have this convergence?). So we requires the assumption that 6 = 0. (d) This is about asymptotic normality, a large sample property.
DeĆnition 1 If an(Tn ) !d^ Y , 0 < E[Y 2 ] < 1 , an! 1 or an! a > 0 and bn(T (^) n^0 ) !d^ Z, 0 < E[Z^2 ] < 1 , bn! 1 or bn! b > 0 , then the asymptotic relative eĀ¢ ciency of T (^) n^0 w.r.t. Tn is deĆned to be E[ aY (^2) n^2 ] = E[ bZ (^2) n^2 ] .e.g., an = bn = pn, Y N (0; ^21 ), Z N (0; ^22 ), then the asymptotic relative eĀ¢ ciency of T (^) n^0 w.r.t. Tn is ^2122 , that is, if ^21 > ^22 , then ^2122 > 1 and T (^) n^0 is more eĀ¢ cient than Tn.
Example 2 (Midterm 2005, 2) Take the model yi = xi + ei with E[xiei] = 0. Suppose you have two independent random samples of observations (yi; xi) of size n 1 and n 2. Let b^1 and b^2 denote the least-squares estimate of on each sample. Let e^ =^ b^1 + b^2 = 2 denote the average of the two estimates. Let b^ denote the least-squares estimate on the combined sample. Which is more eĀ¢ cient, e or b^? When are they asymptotically equivalent?
