Bayes, buttressed by design-based ideas, is the best overarching paradigm for sample survey inference
Section 5. Conclusion: Ten reasons to be Bayesian for survey inference
My examples are intended to give some idea of the richness of Bayesian modeling possible for survey data, but they are far from exhaustive. Bayes is also useful in areas I have not touched on, including multistage sampling, time series, latent class and factor analysis models, measurement error, the combination of data from multiple sources, the creation of synthetic data for disclosure avoidance, and so on. I conclude by summarizing my reasons for advocating the Bayesian approach to survey inference:
- The design-based approach is asymptotic, and too limited to handle the varied problems of inference from surveys, whether probability or non-probability based.
- The Bayesian approach is both unified and flexible enough to handle the various problems encountered in surveys, and it includes superpopulation modeling as a form of large-sample inference.
- Carefully-chosen Bayesian models can yield credible intervals that have good design-based properties in repeated sampling. In particular, weighting, stratification and post-stratification can be modeled via covariates, and clustering incorporated via Bayesian hierarchical models. Flexible models that incorporate design features render hybrid approaches such as model-assisted estimation (e.g., Särndal, Swensson and Wretman, 1992) unnecessary.
- Early critiques of the modeling approach concern models that do not incorporate design-features, and hence are vulnerable to model misspecification. Such models can and should be avoided.
- The Bayesian calculus of integrating out nuisance parameters provides inferences that have good frequentist properties in small as well as large samples.
- The approach of being design-based for some problems and model-based for others leads to logical inconsistencies (see, for example, Little, 2012, Section 4.3); the Bayesian approach yields inferences that are unified and logically consistent.
- The specification of prior distributions in the Bayesian approach is a strength, not a weakness, because it provides additional modeling flexibility. For some problems, weak “objective” priors yield results that parallel standard frequentist solutions. For other problems, stronger “subjective” priors provide useful answers for models that are not identified, as in MNAR nonresponse.
- Computational challenges in the Bayesian approach have been greatly reduced by recent methodological advances and expanded computing power.
- Modeling puts survey research in the mainstream of statistical modeling for other types of data. The particular features of survey research complex sampling design, and the focus on finite population quantities, are well handled by the Bayesian paradigm.
- The Bayesian approach does not negate the utility of probability sampling for design, which is enormously valuable for achieving robust inferences that limit the need for debatable assumptions concerning representativeness of the sample.
Acknowledgements
I thank the Waksberg Award committee for providing me the opportunity to present this work, and Yajuan Si and a referee for helpful comments.
References
Andridge, R.H., and Little, R.J. (2011). Proxy pattern-mixture analysis for survey nonresponse. Journal of Official Statistics, 27, 2, 153-180.
Andridge, R.R., West, B.T., Little, R.J.A., Boonstra, P.S. and Alvarado-Leiton, F. (2019). Indices of non-ignorable selection bias for proportions estimated from non-probability samples. Journal of the Royal Statistical Society, Series C, Applied Statistics, 68, 5, 1465-1483.
Basu, D. (1971). An essay on the logical foundations of survey sampling, Part I. In Foundations of Statistical Inference, (Eds., V.P. Godambe and D.A. Sprott), Toronto: Holt, Rinehart and Winston, 203-242.
Berger, J.O., and Wolpert, R.L. (1988). The likelihood principle. Institute of Mathematical Statistics Lecture Notes-Monograph Series, 6, 1-199.
Binder, D.A. (1982). Non-parametric Bayesian models for samples from finite populations. Journal of the Royal Statistical Society, Series B, 44,3, 388-393.
Birnbaum, A. (1962). On the foundations of statistical inference (with discussion). Journal of the American Statistical Association, 57, 269-326.
Boonstra, P.S., Little, R.J., West, B.T., Andridge, R.R. and Alvaredo-Leiton, F. (2021). A simulation study of diagnostics for bias in non-probability samples. Journal of Official Statistics, 37, 3, 751-769.
Brewer, K. (2013). Three controversies in the history of survey sampling. Survey Methodology, 39, 2, 249-262. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2013002/article/11883-eng.pdf.
Brown, C.H. (1990). Protecting against nonrandomly missing data in longitudinal studies. Biometrics, 46, 143-155.
Buja, A., Berk, R., Brown, L., George, E., Pitkin, E., Zhan, L. and Zhang, K. (2019). Models as approximations 1: Consequences illustrated with linear regression. Statistical Science, 34, 4, 580-583.
Chen, Q., Elliott, M.R., Haziza, D., Yang, Y., Ghosh, M., Little, R.J., Sedransk, J. and Thompson, M. (2017). Approaches to improving survey-weighted estimates. Statistical Science, 32, 2, 227-248.
Dean, N., and Pagano, M. (2015). Evaluating confidence interval methods for binomial proportions in clustered surveys. Journal of Survey Statistics and Methodology, 3, 484-503.
Elliott, M.R. (2007). Bayesian weight trimming for generalized linear regression models. Survey Methodology, 33, 1, 23-34. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2007001/article/9849-eng.pdf.
Elliott, M.R., and Little, R.J. (2000). Model-based approaches to weight trimming. Journal of Official Statistics, 16, 191-210.
Elliott, M.R., and Valliant, R. (2017). Inference for nonprobability samples. Statistical Science, 32, 2, 249-264.
Ericson, W.A. (1969). Subjective Bayesian models in sampling finite populations (with discussion). Journal of the Royal Statistical Society, Series B, 31, 195-233.
Fienberg, S.E. (2011). Bayesian models and methods in public policy and government settings. Statistical Science, 26, 2, 212-226.
Firth, D., and Bennett, K.E. (1998). Robust models in probability sampling. Journal of the Royal Statistical Society, Series B, 60, 3-21.
Franco, C., Little, R.J., Louis, T.A. and Slud, E.V. (2019). Comparative study of confidence intervals for proportions in complex sample surveys. Journal of Survey Statistics and Methodology, 7, 3, 334-364.
Gelman, A. (2007). Struggles with survey weighting and regression modeling (with discussion). Statistical Science, 22, 2, 153-188.
Gelman, A., and Little, T.C. (1997). Poststratification into many categories using hierarchical logistic regression. Survey Methodology, 23, 2, 127-135. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/1997002/article/3616-eng.pdf.
Ghosh, M., and Meeden, G. (1997). Bayesian Methods for Finite Population Sampling. London: Chapman and Hall.
Ghosh, M., Reid, N. and Fraser, D.A.S. (2010). Ancillary statistics: A review. Statistica Sinica, 20, 1309-1332.
Hájek, J. (1971). Comment on a paper by D. Basu. In Foundations of Statistical Inference, (Eds., V.P. Godambe and D.A. Sprott), Toronto: Holt, Rinehart and Winston, 236.
Hansen, M.H., Madow, W.G. and Tepping, B.J. (1983). An evaluation of model-dependent and probability-sampling inferences in sample surveys (with discussion). Journal of the American Statistical Association, 78, 776-793.
Holt, D., and Smith, T.M.F. (1979). Post stratification. Journal of the Royal Statistical Society, Series A, 142, 1, 33-46.
Horvitz, D.G., and Thompson, D.J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663-685.
Kish, L. (1992). Weighting for unequal Pi. Journal of Official Statistics, 8, 183-200.
Kish, L. (1995). The hundred years’ wars of survey sampling. Statistics in Transition, 2, 813-830. Reproduced as Chapter 1 of Leslie Kish: Selected Papers, (2003, Eds., G. Kalton and S. Heeringa), New York: John Wiley & Sons, Inc.
Kish, L., and Frankel, M.R. (1974). Inferences from complex samples (with discussion). Journal of the Royal Statistical Society, Series B, 36, 1-37.
Lazzeroni, L.C., and Little, R.J. (1998). Random-effects models for smoothing post-stratification weights. Journal of Official Statistics, 14, 61-78.
Leon-Novelo, L.G., and Savitsky, T.D. (2019). Fully Bayesian estimation under informative sampling. Electronic Journal of Statistics, 13, 1608-1645.
Little, R.J. (1993). Post-stratification: A modeler’s perspective. Journal of the American Statistical Association, 88, 1001-1012.
Little, R.J. (1994). A class of pattern-mixture models for normal missing data. Biometrika, 81, 3, 471-483.
Little, R.J. (2003a). The Bayesian approach to sample survey inference. In Analysis of Survey Data, (Eds., R.L. Chambers and C.J. Skinner), New York: John Wiley & Sons, Inc., 49-57.
Little, R.J. (2003b). Bayesian methods for unit and item nonresponse. In Analysis of Survey Data, (Eds., R.L. Chambers and C.J. Skinner), New York: John Wiley & Sons, Inc., 289-306.
Little, R.J. (2004). To model or not to model? Competing modes of inference for finite population sampling. Journal of the American Statistical Association, 99, 546-556.
Little, R.J. (2006). Calibrated Bayes: A Bayes/frequentist roadmap. The American Statistician, 60, 3, 213-223.
Little, R.J. (2012). Calibrated Bayes: An alternative inferential paradigm for official statistics (with discussion). Journal of Official Statistics, 28, 3, 309-372.
Little, R.J. (2014). Survey sampling: Past controversies, current orthodoxies, and future paradigms. In Past, Present and Future of Statistical Science, COPSS 50th Anniversary Volume, (Eds., X. Lin, D.L. Banks, C. Genest, G. Molenberghs, D.W. Scott and J.-L. Wang), CRC Press.
Little, R.J. (2015). Calibrated Bayes, an inferential paradigm for official statistics in the era of big data. Statistical Journal of the International Association of Survey Statisticians,31, 4, 555-563.
Little, R.J. (2019). Comment on “Models as approximations 1: Consequences illustrated with linear regression” by A. Buja et al. Statistical Science, 34, 4, 580-583.
Little, R.J. (2022). A note about the definition of propensity weights. To appear in Journal of Survey Statistics and Methodology.
Little, R.J., and Rubin, D.B. (2019). Statistical Analysis with Missing Data, 3rd edition. New York: John Wiley & Sons, Inc.
Little, R.J., and Vartivarian, S. (2005). Does weighting for nonresponse increase the variance of survey means? Survey Methodology, 31, 2, 161-168. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2005002/article/9046-eng.pdf.
Little, R.J., and Wu, M.M. (1991). Models for contingency tables with known margins when target and sampled populations differ. Journal of the American Statistical Association, 86, 87-95.
Little, R.J., West, B.T., Boonstra, P.S. and Hu, J. (2020). Measures of the degree of departure from ignorable sample selection. Journal of Survey Statistics and Methodology, 8, 5, 932-964.
Royall, R.M., and Cumberland, W.G. (1981). The finite population linear regression estimator and estimator of its variance-an empirical study. Journal of the American Statistical Association, 76, 924-930.
Royall, R.M., and Cumberland, W.G. (1985). Conditional coverage properties of finite population confidence intervals. Journal of the American Statistical Association, 80, 355-359.
Rubin, D.B. (1976). Inference and missing data. Biometrika, 53, 581-592.
Rubin, D.B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 1, 34-58.
Rubin, D.B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151-1172.
Rubin, D.B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, Inc.
Rubin, D.B. (2019). Conditional calibration and the sage statistician. Survey Methodology, 45, 2, 187-198. Paper available at https://www150.statcan.gc.ca/n1/pub/12-001-x/2019002/article/00010-eng.pdf.
Ruppert, D., Wand, M.P and Carroll, R.J. (2003). Semiparametric Regression, Cambridge University Press.
Särndal, C.-E., and Lundström, S. (2010). Design for estimation: Identifying auxiliary vectors to reduce nonresponse bias. Survey Methodology, 36, 2, 131-144. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2010002/article/11376-eng.pdf.
Särndal, C.-E., Swensson, B. and Wretman, J.H. (1992). Model Assisted Survey Sampling. New York: Springer Verlag.
Schouten, B., Cobben, F. and Bethlehem, J. (2009). Indicators for the representativeness of survey response. Survey Methodology, 35, 1, 101-113. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2009001/article/10887-eng.pdf.
Sedransk, J. (2008). Assessing the value of Bayesian methods for inference about finite population quantities. Journal of Official Statistics, 24, 495-506.
Si, Y, Trangucci, R., Gabry, J.S. and Gelman, A. (2020). Bayesian hierarchical weighting adjustment and survey inference. Survey Methodology, 46, 2, 181-214. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2020002/article/00003-eng.pdf.
Smith, T.M.F. (1976). The foundations of survey sampling: A review (with discussion). Journal of the Royal Statistical Society, Series A, 139, 183-204.
Smith, T.M.F. (1994). Sample surveys 1975-1990: An age of reconciliation? (with discussion). International Statistical Review, 62, 5-34.
Szpiro, A.A., Rice, K.M. and Lumley, T. (2010). Model-robust regression and a Bayesian “sandwich” estimator. Annals of Applied Statistics, 4, 4, 2099-2113.
Valliant, R., Dorfman, A.H. and Royall, R.M. (2000). Finite Population Sampling and Inference: A Prediction Approach. New York: John Wiley & Sons, Inc.
West, B.T., Little, R.J., Andridge, R.R., Boonstra, P.S., Ware, E.B., Pandit, A. and Alvarado-Leiton, F. (2021). Measures of selection bias in regression coefficients estimated from non-probability samples. The Annals of Applied Statistics, 15(3), 1556-1581.
Yang, Y., and Little, R.J. (2015). A comparison of doubly robust estimators of the mean with missing data. Journal of Statistical Computation and Simulation, 85, 16, 3383-3403.
Zangeneh, S.Z., and Little, R.J. (2015). Bayesian inference for the finite population total in heteroscedastic probability proportional to size samples. Journal of Survey Statistics and Methodology, 3, 162-192.
Zangeneh, S.Z., and Little, R.J. (2022). Likelihood based estimation of the finite population mean with post-stratification information under nonignorable nonresponse. To appear in International Statistical Review.
Zhang, G., and Little, R.J. (2009). Extensions of the penalized spline of propensity prediction method of imputation. Biometrics, 65, 3, 911-918.
Zheng, H., and Little, R.J. (2003). Penalized spline model-based estimation of the finite population total from probability-proportional-to-size samples. Journal of Official Statistics, 19, 2, 99-117.
Zheng, H., and Little, R.J. (2005). Inference for the population total from probability-proportional-to-size samples based on predictions from a penalized spline non parametric model. Journal of Official Statistics, 21, 1-20.
- Date modified: