Integration of data from probability surveys and big found data for finite population inference using mass imputation
Section 2. Basic setup

• Previous
• Next

2.1   Notation: Two data sources

Let $F_N\mathrm{<mo>=</mo>}\left\{(X_i\mathrm{,}{Y}_i):i\in U\right\}$ with $U\mathrm{<mo>=</mo>}\left\{\mathrm{1,}\dots \mathrm{,}N\right\}$ denote a finite population, where $X_i\mathrm{<mo>=</mo>}\left(X_i^1\mathrm{,}\dots \mathrm{,}{X}_i^p\right)$ is a $p$ -dimensional vector of covariates, and $Y_i$ is the study variable. We assume that $F_N$ is a random sample from a superpopulation model $\zeta ,$ and $N$ is known. Our objective is to estimate the general finite population parameter ${\mu }_g\mathrm{<mo>=</mo>}{N}^{-1}{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>\; 1}}^{N}g\left(Y_i\right)}$ for some known $g\left(\text{}\text{}\cdot \text{}\right).$ For example, if $g\left(\text{}Y\text{}\right)\mathrm{<mo>=</mo>}Y,$ ${\mu }_g\mathrm{<mo>=</mo>}{N}^{-1}{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>\; 1}}^N{Y}_i}$ is the population mean of $Y.$ If $g\left(\text{}Y\text{}\right)\mathrm{<mo>=</mo>}1\left(Y\mathrm{<mo><</mo>}c\right)$ for some constant $c,$ ${\mu }_g\mathrm{<mo>=</mo>}{N}^{-1}{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>\; 1}}^{N}1\left(Y_i\mathrm{<mo><</mo>}c\right)}$ is the population proportion of $Y$ less than $c.$

Suppose that there are two data sources, one from a probability sample, referred to as Sample A, and the other from a big data source, referred to as Sample B. Table 2.1 illustrates the observed data structure. Sample A contains observations $O_A\mathrm{<mo>=</mo>}\left\{(d_i\mathrm{<mo>=</mo>}{\pi }_i^{-1}\mathrm{,}{X}_i):i\in A\right\}$ with sample size $n\mathrm{<mo>=</mo>}\left|A\right|\mathrm{,}$ where ${\pi }_i\mathrm{<mo>=</mo>}P\left(i\in A\right)$ is known throughout Sample A, and Sample B contains observations $O_B\mathrm{<mo>=</mo>}\left\{(X_i\mathrm{,}{Y}_i):i\in B\right\}$ with sample size $N_B\mathrm{<mo>=</mo>}\left|B\right|.$ Often the probability sample contains many other items but we only use those items overlapping with our big data.  Although the big data source has a large sample size, the sampling mechanism is often unknown, and we cannot compute the first-order inclusion probability for Horvitz-Thompson estimation. The naive estimators without adjusting for the sampling process are subject to selection biases. On the other hand, although the probability sample with sampling weights represents the finite population, it does not observe the study variable.

2.2   Assumptions

Let $f\left(Y|X\right)$ be the conditional density function of $Y$ given $X$ in the superpopulation model $\zeta .$ Let $f\left(\text{}X\text{}\right)$ and $f\left(X|{\delta }_B\mathrm{<mo>=</mo>}1\right)$ be the density function of $X$ in the finite population and Sample B, respectively, where ${\delta }_B$ is the indicator of selection to Sample B. We first make the following assumptions.

Assumption 1 (Ignorability). Conditional on $X,$  the density of $Y$  in Sample B follows the superpopulation model; i.e., $f\left(Y|X\mathrm{<mo>;</mo>}{\delta }_B\mathrm{<mo>=</mo>}1\right)\mathrm{<mo>=</mo>}f\left(Y|X\right).$

Assumptions 1 and 2 constitute the strong ignorability condition (Rosenbaum and Rubin, 1983). This setup has previously been used by several authors; see, e.g., Rivers (2007), Vavreck and Rivers (2008). Assumption 1 states the ignorability of the selection mechanism to Sample B conditional upon the covariates. Assumption 1 also implies that $P\left({\delta }_B\mathrm{<mo>=</mo>}1|X\mathrm{,}Y\right)\mathrm{<mo>=</mo>}P\left({\delta }_B\mathrm{<mo>=</mo>}1|X\right).$ This assumption holds if the set of covariates contains all predictors for the outcome that affect the possibility of being selected in Sample B. Under this assumption, the missing outcomes in Sample A are missing at random (Rubin, 1976).

Assumption 2 (Common support). The vector of covariates $X\in R^p$ has a compact and convex support, with its density bounded and bounded away from zero. There exist constants $C_l$  and $C_u$  such that $C_l\le f\left(\text{}X\text{}\right)/f\left(X|{\delta }_B\mathrm{<mo>=</mo>}1\right)\le C_u$  almost surely.

Assumption 2 implies that the support of $X$ in Sample B is the same as that in the finite population. This assumption can also be formulated as a positivity assumption that $P\left({\delta }_B\mathrm{<mo>=</mo>}1|X\right)\mathrm{<mo>></mo>}0$ for all $X.$ Assumption 2 does not hold if certain units would never be included in the big data sample. The plausibility of this assumption can be judged by subject matter knowledge. For diagnosis purpose, we can examine the distribution of the estimated propensity scores or the distribution of the propensity score weights in Sample A. Values of propensity score close to zero or extreme large values of the propensity score weights indicate the possible positivity violation. We assume all covariates are continuous. Categorical variables can be handled by first defining imputation classes using the partition of the categories and then estimating the average of the outcome using the nearest neighbor imputation within imputation classes. In our context, Sample B is a big data sample and therefore the size of donors for each imputation class can be reasonable large.

• Previous
• Article main page
• Next