Survey Methodology
Design-based conformal prediction
- Release date: January 3, 2024
Abstract
Conformal prediction is an assumption-lean approach to generating distribution-free prediction intervals or sets, for nearly arbitrary predictive models, with guaranteed finite-sample coverage. Conformal methods are an active research topic in statistics and machine learning, but only recently have they been extended to non-exchangeable data. In this paper, we invite survey methodologists to begin using and contributing to conformal methods. We introduce how conformal prediction can be applied to data from several common complex sample survey designs, under a framework of design-based inference for a finite population, and we point out gaps where survey methodologists could fruitfully apply their expertise. Our simulations empirically bear out the theoretical guarantees of finite-sample coverage, and our real-data example demonstrates how conformal prediction can be applied to complex sample survey data in practice.
Key Words: Conformal prediction; Machine learning; Cross validation; Predictive modeling; Complex sample survey designs.
Table of contents
- Section 1. Introduction
- Section 2. Introduction to conformal inference
- Section 3. Methods
- Section 4. Examples
- Section 5. Extensions
- Section 6. Conclusion
- Acknowledgements
- Appendix
- References
How to cite
Wieczorek, J. (2023). Design-based conformal prediction. Survey Methodology, Statistics Canada, Catalogue No. 12‑001‑X, Vol. 49, No. 2. Paper available at http://www.statcan.gc.ca/pub/12-001-x/2023002/article/00007-eng.htm.
Note
- Date modified: