Comments on “Statistical inference with non-probability survey samples” – Miniaturizing data defect correlation: A versatile strategy for handling non-probability samples
Section 2. A finite-population deterministic identity for actual error

• Previous
• Next

To demonstrate the fruitfulness of the finite-population framework, consider the estimation of the population mean, denoted by $\overline{G},$ of $\mathrm{<mo>\{</mo>}{G}_i=G\mathrm{<mo\; stretchy="false">(</mo>}{X}_i\mathrm{<mo\; stretchy="false">)</mo><mo>:</mo>}i\in N\mathrm{<mo>\}</mo>},$ where $N=\mathrm{<mo>\{</mo>1,}\dots \mathrm{,}N\mathrm{<mo>\}</mo>}$ indexes a finite population, and the $X_i’\text{s}$ are data collected on individual $i.$ For each $i,$ let $R_i=1$ if $G_i$ (or rather $X_i)$ is recorded in our sample, and $R_i=0$ otherwise; hence the sample size is $n_R={\displaystyle {\sum }_{i\mathrm{<mo>=</mo>1}}^N}{R}_i.$ We stress that this is an all-encompassing indicator, which can (and should) be decomposed into $R_i=r_i^{\mathrm{<mo\; stretchy="false">(</mo>1<mo\; stretchy="false">)</mo>}},\dots ,r_i^{\mathrm{<mo\; stretchy="false">(</mo>}J\mathrm{<mo\; stretchy="false">)</mo>}},$ when the data collection consists of $J$ stages (e.g., $r_i^{\mathrm{<mo\; stretchy="false">(</mo>1<mo\; stretchy="false">)</mo>}}$ indicates whether or not the $i^{\text{th}}$ individual is sampled, and $r_i^{\mathrm{<mo\; stretchy="false">(</mo>2<mo\; stretchy="false">)</mo>}}$ whether the individual responded or not once sampled).

Let $\mathrm{<mo>\{</mo>}{W}_i\mathrm{,}i\in S\mathrm{<mo>\}</mo>}$ be a set of weights to be determined, where the index set $S=\mathrm{<mo>\{</mo>}i\mathrm{<mo>:</mo>}{R}_i=\mathrm{1<mo>\}</mo>},$ such that ${\displaystyle {\sum }_{i\in S}}{W}_i>0.$ Let ${\overline{G}}_W$ be the weighted sample average, expressible in three ways:

$${\overline{G}}_W=\frac{{\displaystyle {\sum }_{i\in S}{W}_i{G}_i}}{{\displaystyle {\sum }_{i\in S}{W}_i}}=\frac{{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>}1}^N{R}_i{W}_i{G}_i}}{{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>}1}^N{R}_i{W}_i}}=\frac{{\text{E}}_I\mathrm{<mo\; stretchy="false">(</mo>}{\tilde{R}}_I{G}_I\mathrm{<mo\; stretchy="false">)</mo>}}{{\text{E}}_I\mathrm{<mo\; stretchy="false">(</mo>}{\tilde{R}}_I\mathrm{<mo\; stretchy="false">)</mo>}}\mathrm{,}(2.1)$$

where ${\tilde{R}}_I=R_I{W}_I,$ and ${\text{E}}_I$ is taken with respect to the uniform distribution over the index set $N.$ The first expression in (2.1) simply defines a weighted sample average. With the help of $R_i,$ the second expression turns the sample averages into finite-population averages. This trivial re-expression is fundamental because it explicates the role of $R_i$ in influencing the behavior of ${\overline{G}}_W$ as an estimator of $\overline{G}.$ The third expression reveals a divine probability through $I,$ the finite-population index (FPI) variable, by utilizing the fact that averaging is the same as taking expectation over a uniformly distributed random index $I.$ All finite-population moments then can be expressed via ${\text{E}}_I.$

In particular, we can express the actual error of ${\overline{G}}_W$ via the following identity, where the first expression can be traced back to Hartley and Ross (1954), who used it to express biases in ratio estimators. The second expression was given in Meng (2018) with a slightly different (but equivalent) expression:  

$${\overline{G}}_W-\overline{G}=\frac{{\text{Cov}}_I\mathrm{<mo\; stretchy="false">(</mo>}{\tilde{R}}_I\mathrm{,}{G}_I\mathrm{<mo\; stretchy="false">)</mo>}}{{\text{E}}_I\mathrm{<mo\; stretchy="false">[</mo>}{\tilde{R}}_I\mathrm{<mo\; stretchy="false">]</mo>}}={\rho }_{\tilde{R},G}\times \sqrt{\frac{N-n_W}{n_W}}\times {\sigma }_G\mathrm{.}(2.2)$$

Here ${\rho }_{\tilde{R},G}={\text{Corr}}_I\mathrm{<mo\; stretchy="false">(</mo>}{\tilde{R}}_I\mathrm{,}{G}_I\mathrm{<mo\; stretchy="false">)</mo>}$ is the finite-population correlation between ${\tilde{R}}_I$ and $G_I,$ ${\sigma }_G^2$ is the finite-population variance of $G_I,$ and $n_W$ is the effective sample size due to using weights (Kish, 1965)

$$n_W=\frac{n_R}{1+{\text{CV}}_W^2}\mathrm{,}(2.3)$$

with ${\text{CV}}_W$ being the coefficient of variation (i.e., standard deviation/mean) of $\mathrm{<mo>\{</mo>}{W}_i\mathrm{,}i\in S\mathrm{<mo>\}</mo>.}$

The expression (2.2) is an algebraic identity because it holds for any instances of $\left\{\mathrm{<mo\; stretchy="false">(</mo>}{G}_i\mathrm{,}{R}_i{W}_i\mathrm{<mo\; stretchy="false">)</mo>,}i\in N\right\}.$ Hence no model assumptions are imposed, not even the assumption that $R$ (or any quantity) is random, echoing the comment by Mary Thompson, as quoted in Wu (2022), that “the sample inclusion indicator $R$ is a random variable is itself an assumption”. The only requirement is that the recorded $G_i$ is unchanged from the $G_i’\text{s}$ in the target population. (But note this requirement has two components: (1) there is no over-coverage, that is, everyone in the sample belongs to the target population, e.g., no non-eligible voters are surveyed when the target population is eligible voters, and (2) there is no measurement error; extensions to the cases with measurement errors are available, but not pursued in this article.) When we use equal weights, the three factors on the right-hand side of (2.2) reflect respectively (from left to right) data defect, data sparsity, and problem difficulty, as detailed in Meng (2018) and further illustrated in Bradley, Kuriwaki, Isakov, Sejdinovic, Meng and Flaxman (2021) in the context of COVID-19 vaccination surveys.

In particular, when all weights are equal, ${\rho }_{\tilde{R},G}$ is termed as data defect correlation (ddc) in Meng (2018) because it measures the lack of representativeness of the sample via capturing the dependence of inclusion/recording indicator on the attributes ‒ the higher the dependence, the more biased the sample average becomes for estimating population averages. With the basic strategies of probabilistic sampling or inverse probability weighting, ddc will be zero on average because $\text{E}\mathrm{<mo\; stretchy="false">(</mo>}{W}_i{R}_i\mathrm{<mo\; stretchy="false">)</mo>}=1,$ and it is of $O_p\mathrm{<mo\; stretchy="false">(</mo>}{N}^{-1/2}\mathrm{<mo\; stretchy="false">)</mo>}$ order because it is essentially an average of $N$ independent terms (Meng, 2018). Our general goal here therefore is to bring down ddc to $O_p\mathrm{<mo\; stretchy="false">(</mo>}{N}^{-1/2}\mathrm{<mo\; stretchy="false">)</mo>}$ for non-probability samples, which we shall refer to as “miniaturizing ddc” because $N^{-1/2}$ is typically a minuscule number in practice.

When we use weights, the first term ${\rho }_{\tilde{R},G}$ captures the data defect that still exists after the weighting adjustment, since no weights are perfect in practice. Identity (2.2) shows the impact of the weights on both data quality and data quantity. The impact on the nominal effective sample size $n_W$ is never positive because $n_W\le n_R$ as seen in (2.3). Incidentally, the exactness of (2.3) reveals that this well-known expression is in fact not an approximation (which is often attributed to Kish (1965)), but an exact formula for the reduction of the sample size due to weighting if the weighting had no impact on ddc. However, weighting can have a major positive impact on reducing the overall error by judiciously choosing weights to significantly decrease ddc, though apparently at the price of $n_W<n_R.$ Of course, this is exactly the aim of the quasi-randomization framework, as discussed below. Most importantly, however, (2.2) leads to a unified insight about the variety of methods reviewed in Wu (2022), including an intuitive explanation of the doubly robust property, which has been receiving increased attention for integrating data sources including both probability and non-probability samples (e.g., Yang, Kim and Song, 2020).

Indeed, Zhang (2019, Section 3.1) used the first expression in (2.2) to define a unified non-parametric asymptotic (NPA) non-informativeness assumption, which requires that the numerator ${\text{Cov}}_I\mathrm{<mo\; stretchy="false">(</mo>}{\tilde{R}}_I\mathrm{,}{G}_I\mathrm{<mo\; stretchy="false">)</mo>}$ goes to zero, while keeping the denominator ${\text{E}}_I\mathrm{<mo\; stretchy="false">[</mo>}{\tilde{R}}_I\mathrm{<mo\; stretchy="false">]</mo>}$ positive, as $N\to \infty .$ This unification permits Zhang (2019) to evaluate the quasi-randomization approach and regression modeling via a common criterion. The ddc framework echoes this unification, as discussed in Section 3 below, with Section 4 stressing the same broad message as emphasized by Zhang (2019). Section 5 harvests another low-hanging fruit of the ddc formulation, since it provides an immediate explanation of the celebrated double robustness. Section 6 then ventures into a much harder area of engineering a more representative sub-sample out of a large non-representative sample, a worthwhile trade-off because data quality is far more important than data quantity (Meng, 2018), as briefly reviewed below.

• Previous
• Article main page
• Next