No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
The Rao-Wu Rescaling Bootstrap: From theory to practice
Abstract
At Statistics Canada, variance estimation for complex surveys is mainly carried out using replication methods. The two replication methods which have been used in the last decade are the delete-one Primary Sampling Unit (PSU) jackknife and, more recently, the bootstrap. As Valliant (2007) rightly points out there are several variants of the bootstrap introduced by Efron (1979) in the i.i.d. case which are being used in survey sampling and we have to be clear which one is being referred to; at Statistics Canada we use solely the Rao-Wu rescaling bootstrap for production. Even though it was introduced in 1988 (see Rao and Wu (1988)), first implementations occurred only in the late 1990s. The Rao-Wu bootstrap is performed on both with and without replacement designs for which it yields sensible variance estimates for a variety of estimators including percentiles, a claim not matched by the delete-one PSU jackknife. Because of this, surveys which initially relied on the jackknife often make the switch to the bootstrap when the occasion arises, like during a survey re-design. The reader can get a good overview of what has been tried in survey sampling with regard to the bootstrap from Rust and Rao (1996), Shao (2003) and Lahiri (2003). The Rao-Wu variant of the original bootstrap procedure is appealing to survey users because it is simple to implement, it yields adequate variance estimates when the sample sizes are small (which is common with stratified designs) and it comes in the form of bootstrap weights. For any given methodology, making the transition from theory to practice is a challenge, and the Rao-Wu bootstrap
Multiple imputation (MI) is a well-established method to handle item-nonresponse in sample surveys. Survey data obtained from complex sampling designs often involve features that include unequal probability of selection, clustering and stratification. Because sample design features are frequently related to survey outcomes of interest, the theory of MI requires including them in the imputation model to reduce the risks of model misspecification and hence to avoid biased inference. However, in practice multiply-imputed datasets from complex sample designs are typically imputed under simple random sampling assumptions and then analyzed using methods that account for the design features. Less commonly-used alternatives such as including case weights and/or dummy variables for strata and clusters as predictors typically require interaction terms for more complex estimators such as regression coefficients, and can be vulnerable to model misspecification and difficult to implement.
We develop a simple two-step MI framework that accounts for complex sample designs using a weighted finite population Bayesian bootstrap (FPBB) method to generate draws from the posterior predictive distribution of the population. Imputations may then be performed assuming IID data. We propose different variations of the weighted FPBB for different sampling designs, and evaluate these methods using three studies. Simulation results show that the proposed methods have good frequentist properties and are robust to model misspecification compared to alternative approaches. We apply the proposed method to accommodate missing data in the Behavioral Risk Factor Surveillance System, the National Automotive Sampling System and the National Health and Nutrition Examination Survey III when estimating means, quantiles and a variety of model parameters.
Many statistical offices have been moving towards an increased use of administrative data sources for statistical purposes, both as a substitute and as a complement to survey data. Moreover, the emergence of big data constitutes a further increase in available sources. As a result, statistical output in official statistics is increasingly based on complex combinations of sources.
The quality of such statistics depends on the quality of the primary sources and on the ways they are combined.
This paper analyses the appropriateness of the current set of output quality measures for multiple source statistics, it explains the need for improvement and outlines directions for further work. The usual approach for measuring the quality of the statistical output is to assess quality through the measurement of the input
and process quality. The paper argues that in multisource production environment this approach is not sufficient. It advocates measuring quality on the basis of the output itself – without analysing the details of the inputs and the production process – and proposes directions for further development.
This article considers variance estimation for Horvitz–Thompson–type estimated totals based on survey data with imputed non-respondents and with nonnegligible sampling fractions. A method based on a variance decomposition is proposed. Our method can be applied to complicated situations where a composite of some deterministic and/or random imputation methods is used, including using imputed data in subsequent imputations. Although here linearization or Taylor expansion–type techniques are adopted, replication methods such as the jackknife, balanced repeated replication, and random groups can also be used in applying our method to derive variance estimators. Using our method, variance estimators can be derived under either the customary design-based approach or the model-assisted approach, and are asymptotically unbiased and consistent. The Transportation Annual Survey conducted at the U.S. Census Bureau, in which nonrespondents are imputed using a composite of cold deck and ratio type imputation methods, is used as an example as well as the motivation for our study.
Theoretical properties of nonresponse adjustments based on adjustment cells are studied, for estimates of means for the whole population and in subclasses that cut across adjustment cells. Three forms of adjustment are considered: weighting by the inverse response rate within cells, post-stratification on known population cell counts, and mean imputation within adjustment cells. Two dimensions of covariate information x are distinguished as particularly useful for reducing nonresponse bias: the response propensity p̂(x) and the conditional mean ŷ(x) of the outcome variable y given x. Weighting within adjustment cells based on p̂(x) controls bias, but not necessarily variance. Imputation within adjustment cells based on ŷ(x) controls bias and variance. Post-stratification yields some gains in efficiency for overall population means, and smaller gains for means in subclasses of the population. A simulation study similar to that of Holt & Smith (1979) is described which explores the mean squared error properties of the estimators. Finally, some modifications of response propensity weighting to control variance are suggested. /// L'estimation de moyennes de la population et de sous-populations pour une enquête soumise à nonréponse est discutée. Trois méthodes d'ajustement pour nonréponse sont comparés: le weighting, la post-stratification et l'imputation de moyennes en classes d'ajustement. Deux dimensions des covariates x sont distinguées pour la réduction du biais de nonréponse, la tendence de réponse p̂(x) et la moyenne conditionnelle E(yǀx) du variable y sujet à nonréponse. Les caractéristiques de méthodes qui font usage de ces dimensions pour la création de classes d'ajustement sont recherchées au moyen de théorie et de simulation.
Methods for standard errors and confidence intervals for nonlinear statistics —such as ratios, regression, and correlation coefficients—have been extensively studied for stratified multistage designs in which the clusters are sampled with replacement, in particular, the important special case of two sampled clusters per stratum. These methods include the customary linearization (or Taylor) method and resampling methods based on the jackknife and balanced repeated replication (BRR). Unlike the jackknife or the BRR, the linearization method is applicable to general sampling designs, but it involves a separate variance formula for each nonlinear statistic, thereby requiring additional programming efforts. Both the jackknife and the BRR use a single variance formula for all nonlinear statistics, but they are more computing-intensive. The resampling methods developed here retain these features of the jackknife and the BRR, yet permit extension to more complex designs involving sampling without replacement. The sampling designs studied include (a) stratified cluster sampling in which the clusters are sampled with replacement, (b) stratified simple random sampling without replacement, (c) unequal probability sampling without replacement, and (d) two-stage cluster sampling with equal probabilities and without replacement. Our proposed resampling methods may be viewed as extensions to complex survey samples of the bootstrap, and in the case of design (c), of the BRR as well. We obtain and study the properties of variance estimators of and confidence intervals for the parameter of interest θ, based on the bootstrap histogram of the t statistic. The variance estimators reduce to the standard ones in the special case of linear statistics. These confidence intervals take account of the skewness in the distribution of , unlike the intervals based on the normal approximation. For case (a), the sampled clusters are resampled in each stratum independently by simple random sampling with replacement, and this procedure is replicated many times. The estimate for each resampled cluster is properly scaled such that the resulting variance estimator of reduces to the standard unbiased variance estimator in the linear case. For case (b), the sampled units are resampled in each stratum as in case (a), but a different scaling is used. Different resampling procedures and scalings are employed for case (c). A two-stage resampling procedure is developed for case (d). Results of a simulation study under a stratified simple random sampling design show that the bootstrap intervals track the nominal error rate in each tail better than the intervals based on the normal approximation, but the bootstrap variance estimators are less stable than those based on the linearization or the jackknife.
We discuss several nonparametric methods for attaching a standard error to a point estimate: the jackknife, the bootstrap, half-sampling, subsampling, balanced repeated replications, the infinitesimal jackknife, influence function techniques and the delta method. The last three methods are shown to be identical. All the methods derive from the same basic idea, which is also the idea underlying the common parametric methods. Extended numerical comparisons are made for the special case of the correlation coefficient.
The analysis of survey data requires the application of special methods to deal appropriately with the effects of the sample design on the properties of estimators and test statistics. The class of replication techniques represents one approach to handling this problem. This paper discusses the use of these techniques for estimating sampling variances, and the use of such variance estimates in drawing inferences from survey data. The techniques of the jackknife, balanced repeated replication (balanced half-samples), and the bootstrap are described, and the properties of these methods are summarized. Several examples from the literature of the use of replication in analysing large complex surveys are outlined.
Introduction to the bootstrap
- B Efron
- R Lepage
Efron, B. and R. Lepage (1992). Introduction to the bootstrap. Part of Exploring the limits of the bootstrap, edited by
Raoul Lepage and Lynne Billard. Wiley, New-York.
A design-based perspective on missing data variance
- B E Fay
Fay, B.E. 1991. A design-based perspective on missing data variance. Proceedings of the 1991 Annual Research
Committee, US Bureau of Census, pp. 429-440.
How to avoid getting all tied up bootstrapping a survey: A walk-through featuring the National Longitudinal Survey of Children and Youth. Statistics Canada Working Paper
- C Girard
Girard, C. 2007. How to avoid getting all tied up bootstrapping a survey: A walk-through featuring the National
Longitudinal Survey of Children and Youth. Statistics Canada Working Paper HSMD-2007-001E.
Rethinking the NLSCY weighting methodology: the lessons learned and the innovations introduced
- C Girard
- M Ferland
- S Franklin
- M Tremblay
Girard, C., M. Ferland, S. Franklin and M. Tremblay (co-editors). 2009. Rethinking the NLSCY weighting
methodology: the lessons learned and the innovations introduced. Statistics Canada Working Paper HSMD-2009-002E.
Inférence en presence d'imputation simple dans les enquêtes: un survol
- D Haziza
Haziza, D. 2005. Inférence en presence d'imputation simple dans les enquêtes: un survol. Journal de la Société
Française de Statistique tome 146, no.4.
An overview of the pros and cons of linearization versus replication in establishment surveys
- R Valliant
Valliant, R. 2007. An overview of the pros and cons of linearization versus replication in establishment surveys.
Proceedings of the Third International Conference on Establishment Surveys (ICES III).