Download Bootstrapping Confidence Intervals in Statistics: Coverage Error Analysis - Prof. Peter G. and more Study notes Data Analysis & Statistical Methods in PDF only on Docsity!
METHODOLOGY AND THEORY
FOR THE BOOTSTRAP
(Fifth set of two lectures)
Main topic of these lectures: Completion of work on confidence intervals, and sur- vey of miscellaneous topics
Revision of confidence intervals
Recall that ˆηα is the α-level quantile of the bootstrap distribution of T ∗^ = n^1 /^2 (ˆθ∗^ − ˆθ)/ˆσ∗:
P (T ∗^ ≤ ˆηα | X ) = α.
A one-sided percentile-t confidence interval for an unknown parameter θ, based on the bootstrap estimator ˆθ and having nominal cov- erage α, is therefore
Jˆ 1 = ˆJ 1 (α) = (−∞, θˆ − n−^1 /^2 ˆσ ˆη 1 −α).
Revision (continued)
It has coverage error O(n−^1 ):
P {θ ∈ Jˆ 1 (α)} = α + O(n−^1 ).
A conventional two-sided interval, for which the nominal coverage is also α, is obtained from two one-sided intervals:
Jˆ 2 (α) = Jˆ 1
{ 1 2 (1 +^ α)
} \ Jˆ 1
{ 1 2 (1^ −^ α)
}
=
[ θˆ − n−^1 /^2 σˆ ˆη(1+α)/ 2 , θ^ ˆ − n−^1 /^2 ˆσ ˆη(1−α)/ 2
) .
Unsurprisingly, the actual coverage of Jˆ 2 also equals α + O(n−^1 ):
P {θ ∈ Jˆ 2 (α)} = α + O(n−^1 ).
Coverage of two-sided percentile intervals
To calculate the coverage of Iˆ 22 (α), recall that
P {θ ∈ Iˆ 12 (α)} = α + n−^1 /^2 {P 1 (zα) − Q 1 (zα)} φ(zα)
Since P 1 and Q 1 are even polynomials, and z(1+α)/ 2 = −z(1−α)/ 2 , then
P 1 (z(1+α)/ 2 ) − Q 1 (z(1+α)/ 2 ) = P 1 (z(1−α)/ 2 ) − Q 1 (z(1−α)/ 2 )
Therefore,
P {θ ∈ Iˆ 22 (α)} = P
[ θ ∈ Iˆ 12
{ 1 2 (1 +^ α)
}]
−P
[ θ ∈ Iˆ 12
{ 1 2 (1^ −^ α)
}]
= 12 (1 + α) − 12 (1 − α) + O(n−^1 ) = α + O(n−^1 ).
Coverage of two-sided percentile intervals (continued)
Therefore, owing to the parity properties of polynomials in Edgeworth expansions, this two- sided percentile confidence interval has cover- age error O(n−^1 ). The same result holds true for the “other” type of percentile confidence interval, of which the one-sided form is
K̂ 1 (α) = (−∞, θˆ + n−^1 /^2 ˆσ ξˆα).
Its one- and two-sided forms have coverage
P {θ ∈ K̂ 1 (α)} = α + O(n−^1 /^2 ) , P {θ ∈ K̂ 2 (α)} = α + O(n−^1 ).
Exercise: (1) Derive the latter property.
(2) Show that, when computing percentile confidence intervals, as distinct from percen- tile-t intervals, we do not actually need the value of ˆσ. (It has been included for didactic reasons, to clarify our presentation of theory, but it cancels in numerical calculations.)
Other bootstrap confidence intervals
It is possible to correct bootstrap confidence intervals for skewness without Studentising. The best-known examples of this type are the “accelerated bias corrected” intervals pro- posed by Bradley Efron, based on explicit cor- rections for skewness.
It is also possible to construct bootstrap con- fidence intervals that are optimised for length, for a given level of coverage.
The coverage accuracy of bootstrap confi- dence intervals can be reduced by using the iterated bootstrap to estimate coverage error, and then adjust for it. Each application gen- erally reduces coverage error by a factor of n−^1 /^2 in the one-sided case, and n−^1 in the two-sided case. Usually, however, only one application is computationally feasible.
Other bootstrap confidence intervals (cont.)
Although the percentile-t approach has obvi- ous advantages, these may not be realised in practice in the case of small samples. This is because bootstrapping the Studentised ratio involves simulating the ratio of two random variables, and unless sample size is sufficiently large to ensure reasonably low variability of the denominator in this expression, poor cov- erage accuracy can result.
Note too that percentile-t confidence intervals are not transformation-invariant, whereas in- tervals based on the percentile method are.
From some viewpoints, particularly that of good coverage performance in a very wide range of settings (an analogue of “robust- ness”), the most satisfactory approach is the coverage-corrected form (using the iterated bootstrap) of first type of percentile method interval, i.e. of ˆI 12 and ˆI 22 in one- and two- sided cases, respectively.
Bootstrap for time series with structural model
We call the model structural because the pa- rameters describe only the structure of the way in which the disturbances drive the pro- cess. In particular, no assumptions are made about the disturbances, apart from standard moment conditions. In this sense the setting is nonparametric, rather than parametric.
The best known examples of structural mod- els are those related to linear time series, for example the moving average
Xj = μ +
∑^ p i=
θi j−i+1 ,
or an autoregression such as
Xj − μ =
∑^ p i=
ωi (Xj−i+1 − μ) + j ,
where μ, θ 1 ,... , θp, ω 1 ,... , ωp, and perhaps also p, are parameters that have to be es- timated.
Bootstrap for time series with structural model (continued, 1)
In this setting the usual bootstrap approach to inference is as follows:
(1) Estimate the parameters of the structural model (e.g. μ and ω 1 ,... , ωp in the autore- gression example), and compute the residuals (i.e. “estimates” of the j’s), using standard methods for time series.
(2) Generate the “estimated” time series, in which true parameter values are replaced by their estimates and the disturbances are re- sampled from among the estimated ones, ob- taining a bootstrapped time series X 1 ∗,... , X n∗, for example (in the autoregressive case)
X j∗ − μ̂ =
∑^ p i=
ω̂ i (X j∗−i+1 − μ̂ ) + ∗ j.
Bootstrap for time series with structural model (continued, 2)
All the standard properties we have already noted, founded on Edgeworth expansions, ap- ply without change provided the time series is sufficiently short-range dependent. Early work on theory in the structural time series case includes that of
Bose, A. (1988). Edgeworth correction by bootstrap in autoregressions. Ann. Statist. 16, 1709–1722.
Bootstrap for time series with structural model (continued, 3)
It is common in this setting not to be able to “estimate” n disturbances j, based on a time series of length n. For example, in the context of autoregressions we can generally estimate no more than n − p of the distur- bances. But this does not hinder application of the method; we merely resample from a set of n − p, rather than n, values of ˆj.
Usually it is assumed that the disturbances have zero mean. We reflect this property em- pirically, by centring the ˆj’s at their “sample” mean before resampling.
Block bootstrap for time series
Just as in the case of a structural time series, the block bootstrap aims to construct simu- lated versions “of” the time series, which can then be used for inference in a conventional way.
The method involves sampling blocks of con- secutive values of the time series, say XI+1,
... , XI+b, where 0 ≤ I ≤ n − b is chosen in some random way; and placing them one af- ter the other, in an attempt to reproduce the series. Here, b denotes block length.
Assume we can generated blocks XIj+1,... , XIj+b, for j ≥ 1, ad infinitum in this way. Cre- ate a new time series, X 1 ∗, X 2 ∗,.. ., identical to:
XI 1 +1,... , XI 1 +b, XI 2 +1,... , XI 2 +b,...
The resample X 1 ∗,... , X n∗ is just the first n val- ues in this sequence.
Block bootstrap for time series (contin- ued, 1)
There is a range of methods for choosing the blocks. One, the “fixed block” approach, in- volves dividing the series X 1 ,... , Xn up into m blocks of b consecutive data (assuming n = bm), and choosing the resampled blocks at random. In this case the Ij’s are indepen- dent and uniformly distributed on the values 1 , b + 1,... , (m − 1)b + 1. The blocks in the fixed-block bootstrap do not overlap.
Another, the “moving blocks” technique, al- lows block overlap to occur. Here, the Ij’s are independent and uniformly distributed on the values 0, 1 ,... , n − b.
Difficulties with the block bootstrap
The main problem with the block bootstrap is that the block length, b, which is a form of smoothing parameter, needs to be chosen. Using too small a value of b will corrupt the dependence structure, increasing the bias of the bootstrap method; and choosing b too large will give a method which has relatively high variance, and consequent inaccuracy.
Another difficulty is that the percentile-t ap- proach cannot be applied in the usual way with the block bootstrap, if it is to enjoy high levels of accuracy. This is because the corrup- tion of dependence at places where adjacent blocks joins, significantly affects the relation- ship between the numerator and the denomi- nator in the Studentised ratio, with the result that the block bootstrap does not effectively capture skewness. However, there are ways of removing this problem.
Successes of the block bootstrap
Nevertheless, the block bootstrap, and related methods, give good performance in a range of problems where no other techniques work effectively, for example inference for certain sorts of nonlinear time series.
The block bootstrap also has been shown to work effectively with spatial data. There, the blocks are sometimes referred to as “tiles,” and either of the fixed-block or moving-block methods can be used.
References for block bootstrap
Carlstein, E. (1986). The use of subseries values for estimating the variance of a gen- eral statistic from a stationary sequence. Ann. Statist. 14, 1171–1179.
Hall, P. (1985). Resampling a coverage pat- tern. Stochastic Process. Appl. 20, 231–
