Integration of data from probability surveys and big found data for finite population inference using mass imputation
Section 1. Introduction

• Previous
• Next

In finite population inference, probability sampling is the gold standard for obtaining a representative sample from the target population. Because the selection probability is known, the subsequent inference from a probability sample is often design-based and respect the way in which the data were collected; see Särndal, Swensson and Wretman (2003), Cochran (2007), Fuller (2009) for textbook discussions. However, large-scale survey programs continually face heightened demands coupled with reduced resources. Demands include requests for estimates for domains with small sample sizes and desires for more timely estimates. Simultaneously, program budget cuts force reductions in sample sizes, and decreasing response rates make nonresponse bias an important concern. Baker, Brick, Bates, Battaglia, Couper, Dever, Gile and Tourangeau (2013) address the current challenges in using probability samples for finite population inferences.

To meet the new challenges, statistical offices face the increasing pressure to utilize convenient but often uncontrolled big data sources (also called big found data), such as satellite information (McRoberts, Tomppo and Næsset, 2010), mobile sensor data (Palmer, Espenshade, Bartumeus, Chung, Ozgencil and Li, 2013), and web survey panels (Tourangeau, Conrad and Couper, 2013). Couper (2013), Citro (2014), Tam and Clarke (2015), and Pfeffermann, Eltinge and Brown (2015) articulate the promise of harnessing big data for official and survey statistics but also raise many issues regarding big data sources. While such data sources provide timely data for a large number of variables and population elements, they are non-probability samples and often fail to represent the target population of interest because of inherent selection biases. Tam and Kim (2018) also cover some ethical challenges of big data for official statisticians and discuss some preliminary methods of correcting for selection bias in big data. See Keiding and Louis (2016), Elliott and Valliant (2017), Buelens, Burger and van den Brakel (2018), and Beaumont (2020) for recent reviews of the challenges in using non-probability samples for inferences.

To utilize modern data sources in statistically defensible ways, it is important to develop statistical tools for data integration for combining a probability sample with big non-probability data. Data integration for finite population inference is similar to the problem of combining randomized clinical trial studies and non-randomized epidemiological studies for causal inference of treatment effects (Keiding and Louis, 2016). We are particularly interested in developing data integration under the setup where the study variable is observed in the big data only, but some other variables are commonly observed in both data. In this case, survey statisticians and biostatisticians have provided different methods for combining information from multiple data sources. Lohr and Raghunathan (2017), Yang and Kim (2020), and Rao (2020) provide a review of statistical methods of data integration for finite population inference. Existing methods for data integration can be categorized into three types as follows.

The first type is the so-called propensity score adjustment (Rosenbaum and Rubin, 1983). In this approach, the probability of a unit being selected into the big sample, which is referred to as the propensity score, is modeled and estimated for all units in the big data sample. The subsequent adjustments, such as propensity score weighting or stratification, can then be used to adjust for selection biases; see, e.g., Lee and Valliant (2009), Valliant and Dever (2011), Elliott and Valliant (2017). Stuart, Bradshaw and Leaf (2015), Stuart, Cole, Bradshaw and Leaf (2011), Buchanan, Hudgens, Cole, Mollan, Sax, Daar, Adimora, Eron and Mugavero (2018) use propensity score weighting to generalize results from randomized trials to a target population. O’Muircheartaigh and Hedges (2014) propose propensity score stratification for analyzing a nonrandomized social experiment. One of the notable disadvantages of the propensity score methods is that they rely on an explicit propensity score model and are biased if the model is mis-specified (Kang and Schafer, 2007).

The second type uses calibration (Deville and Särndal, 1992; Kott, 2006; Dong, Yang, Wang, Zeng and Cai, 2020). This technique can be used to calibrate auxiliary information in the big data sample with that in the probability sample, so that after calibration the big data sample is similar to the target population (DiSogra, Cobb, Chan and Dennis, 2011). Because calibration does not require parametric modeling, it is attractive to survey practitioners. However, this approach requires the information (such as the moments) of the auxiliary variables for the population is known or at least can be estimated from a probability sample.

The third type is mass imputation, where the imputed values are created for the whole elements in the probability sample. In the usual imputation for missing data analysis, the respondents in the sample provide a training dataset for developing an imputation model. In the mass imputation, an independent big data sample is used as a training dataset, and imputation is applied to all units in the probability sample. While the mass imputation idea for incorporating information from big data is very natural, the literature on mass imputation itself is sparse. Breidt, McVey and Fuller (1996) discuss mass imputation for two-phase sampling. Rivers (2007) proposes a mass imputation approach using nearest neighbor imputation but the theory is not fully developed. Kim and Rao (2012) develop a rigorous theory for mass imputation using two independent probability samples. Chipperfield, Chessman and Lim (2012) discuss composite estimation when one of the surveys is mass imputed. Bethlehem (2016) discuss practical issues in sample matching. Recently, Kim and Wang (2019) develop a theory for mass imputation for big data using a parametric model approach. However, the parametric model assumptions do not necessarily hold in practice. In order for mass imputation to be more useful and practical, the assumptions should be as weak as possible.

We summarize our contributions in this paper below: 

1. We first develop a formal framework for mass imputation incorporating information from big data into a probability sample and present rigorous asymptotic results for the mass imputation estimators. Our framework covers the nearest neighbor imputation estimator of Rivers (2007). Unlike Kim and Wang (2019), we do not make strong parametric model assumptions for mass imputation. Thus, the proposed method is appealing to survey practitioners.
2. We also investigate two strategies for improving the nearest neighbor imputation estimator, one using $k$ nearest neighbor imputation (Mack and Rosenblatt, 1979) and the other using generalized additive models (Wood, 2006). In $k$ nearest neighbor imputation, instead of using one nearest neighbor, we identify multiple nearest neighbors in the big data sample and use the average response as the imputed value. This method is popular in the international forest inventory community for combining ground-based observations with imagines from remote sensors (McRoberts et al., 2010). In this paper, we establish asymptotic results for the $k$ nearest neighbor estimator. In the second strategy, we investigate modern techniques of prediction for mass imputation with flexible models. We use generalized additive models (Wood, 2006) to learn the relationship of the outcome and covariates from the big data and create predictions for the probability samples. We note that this strategy can apply to a wider class of semi- and non-parametric estimators such as single index models, Lasso estimators (Belloni, Chernozhukov, Chetverikov and Kato, 2015), and machine learning methods such as random forests (Breiman, 2001).
3. Using a novel calibration weighting idea, we propose an efficient mass imputation estimator and develop its asymptotic results. The efficiency gain is justified under a purely design-based framework and no model assumptions are used. We consider the case when additionally the membership to the big data can be determined throughout the probability sample. The key insight is that the subsample of units in Sample A with the big data membership constitutes a second-phase sample from the big data sample, which acts as a new population. We calibrate the information in the second-phase sample to be the same as the new acting population. The calibration process in turn improves the accuracy of the mass imputation estimator without specifying any model assumptions.

The structure of the paper is as follows. In Section 2, we introduce the basic setup. In Section 3, we present the methodology for the nearest neighbor imputation and establish its asymptotic properties. In Section 4, we investigate two strategies for improving the nearest neighbor imputation estimator, one using $k$ nearest neighbor imputation and the other using generalized additive models. In Section 5, we propose a regression calibration technique to improve the efficiency of the mass imputation estimators when additionally the big data membership is observed throughout the probability sample. In Section 6, we demonstrate that the proposed estimators are robust and efficient by simulation studies based on artificial data and real-life data from U.S. Census Bureau’s Monthly Retail Trade Survey. In Section 7, we present a case study applying the proposed method to integrate national health survey data and national health insurance records. Section 8 concludes with a discussion.

• Previous
• Article main page
• Next