An assessment of accuracy improvement by adaptive survey design
Section 3. Analyzing the deviation from unbiased estimation

• Previous
• Next

3.1  Decomposing the CAL deviation

We decompose the CAL deviation, ${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }\left(b_r-b_s\right),$ in order to better understand its behaviour when improved balance is gained by adaptive data collection. We may anticipate that ${\Delta }_{\text{CAL}}$ is reduced, if not to zero so at least to a degree, and under conditions that we seek to identify. There are no immediately apparent signs that ${\overline{x}}_s^{\prime }\left(b_r-b_s\right)$ would be reduced when ${\overline{x}}_r$ draws closer to the fixed ${\overline{x}}_s.$ A more in-depth analysis is needed to explain a reduction in ${\Delta }_{\text{CAL}},$ if any.

When the chosen $x$ -vector is of fairly high dimension, the perfect balance ${\overline{x}}_r={\overline{x}}_s$ (and therefore $\text{IMB}=0)$ is hard to achieve in a data collection; one can strive to come close. Some insight into the nature of ${\Delta }_{\text{CAL}}$ is nevertheless gained by setting ${\overline{x}}_r={\overline{x}}_s;$ using (2.6) we then have

$${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }\left(b_r-b_s\right)={\overline{x}}_r^{\prime }{b}_r-{\overline{y}}_s={\overline{y}}_r-{\overline{y}}_s=\left(1-P\right)\left({\overline{y}}_r-{\overline{y}}_{nr}\right)\ne 0,(3.1)$$

$${\widehat{Y}}_{\text{CAL}}-{\widehat{Y}}_{\text{FUL}}=\widehat{N}{\Delta }_{\text{CAL}}=\widehat{N}\left({\overline{y}}_r-{\overline{y}}_s\right)={\widehat{Y}}_{\text{EXP}}-{\widehat{Y}}_{\text{FUL}}.$$

Hence, the perfect balance, ${\overline{x}}_r={\overline{x}}_s,$ does not reduce ${\Delta }_{\text{CAL}}$ to zero; it gives ${\widehat{Y}}_{\text{CAL}}={\widehat{Y}}_{\text{EXP}}=\widehat{N}{\overline{y}}_r.$ This latter equation seems a paradox at first, because ${\widehat{Y}}_{\text{CAL}}$ is supposed to be better than the rudimentary ${\widehat{Y}}_{\text{EXP}}.$ But we find that under perfect balance they are equal. The paradox is resolved by noting that ${\overline{y}}_r-{\overline{y}}_s$ is likely to be smaller, even considerably so, when the response $r$ is made to be perfectly balanced on $x$ $–$ in that respect closer to $s$ $–$ than when it is not. And it is not true that auxiliary information is not used; such information is precisely what it takes to get the perfect balance.

Empirical evidence on the relationship between ${\Delta }_{\text{CAL}}$ and IMB was given in Särndal and Lundquist (2014). For the survey data analyzed in that article, ${\Delta }_{\text{CAL}}$ drops significantly when IMB is reduced but does not approach zero when IMB tends to zero. These empirical results suggest that ${\Delta }_{\text{CAL}}$ levels out at a certain non-zero value.

Under perfect balance, the deviation (3.1) has the same expression, ${\Delta }_{\text{CAL}}=\left(1-P\right)\left({\overline{y}}_r-{\overline{y}}_{nr}\right),$ for any $x$ -vector. But its value is not the same, because the set of units in a perfectly balanced response $r$ will be different from one $x$ -vector to the next. For given $x$ -vector, the deviation increases with the rate of nonresponse $1-P$ and with the divergence ${\overline{y}}_r-{\overline{y}}_{nr}$ between response and nonresponse $y$ -means. Perfect balance and therefore $\text{IMB}=0$ hold in particular for the trivial $x$ -vector of no interest in practice, $x_k=1$ for all $k;$ in that case the formula ${\Delta }_{\text{CAL}}=\left(1-P\right)\left({\overline{y}}_r-{\overline{y}}_{nr}\right)$ is a reminder of an elementary but often cited note on the biasing effect of nonresponse: Already Cochran (1977, page 361) gives an analogous expression for the bias of the response set mean, in the setting of a population modeled to consist of a response stratum (units responding with probability one) and nonresponse stratum (units responding with probability zero). The message is: Bias increases with the rate of nonresponse and with the separation between response and nonresponse means.

We seek to decompose ${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }\left(b_r-b_s\right)$ into two components such that one of them is reducible to zero under the perfect balance ${\overline{x}}_r={\overline{x}}_s.$ Several splits may meet this “null requirement” on one of the two terms. Särndal and Lundquist (2017) examine one such split, ${\Delta }_{\text{CAL}}={\overline{e}}_r+u_r,$ where ${\overline{e}}_r={\displaystyle {\sum }_r{d}_k{e}_k}/{\displaystyle {\sum }_r{d}_k}={\overline{x}}_r^{\prime }\left(b_r-b_s\right)$ is the mean, over the response $r,$ of the residuals from the regression of $y$ on $x$ fitted on the sample $s,$ $e_k=y_k-x_k^{\prime }{b}_s,$ and $u_r$ is the remainder: ${\Delta }_r-{\overline{e}}_r=u_r=-{\left({\overline{x}}_r-{\overline{x}}_s\right)}^{\prime }\left(b_r-b_s\right).$ These authors call ${\overline{e}}_r$ the regression inconsistency, to express that the regression model for the full sample fails $–$ or is inconsistent $–$ when applied to the response: The residuals $e_k$ have zero mean, ${\overline{e}}_s=0,$ over the sample $s,$ but non-zero mean, ${\overline{e}}_r\ne 0,$ over the response $r.$ The inconsistency is a (not surprising) result of “missing not at random”.

We have $u_r=0$ under the perfect balance ${\overline{x}}_r={\overline{x}}_s,$ so then ${\Delta }_{\text{CAL}}={\overline{e}}_r.$ Särndal and Lundquist (2017) give empirical evidence that when the imbalance IMB is reduced through adaptive data collection, both ${\Delta }_{\text{CAL}}$ and ${\overline{e}}_r$ are reduced, neither of them to zero, and that the ratio ${\overline{e}}_r/{\Delta }_{\text{CAL}}$ tends slowly toward 1 when IMB approaches zero.

3.2  Contrasting the response with the nonresponse

Some insight is gained by seeking a decomposition of ${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }\left(b_r-b_s\right)$ so that a first term remains roughly constant when the imbalance is reduced, while a second term tends to zero in that process. Such a decomposition would thus isolate a part of the deviation that is untouched by adaptive design and indicate why adaptive design is at best “partially successful” for bias removal. The decomposition in Result 3.1 is driven by a wish to contrast the response set $r$ with the nonresponse set $nr=s-r.$ The notation is given in (2.2) for the $x$ -means and in (2.5) for the regression vectors.

Result 3.1. The CAL deviation ${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }\left(b_r-b_s\right)$ has the decomposition ${\Delta }_{\text{CAL}}=D_1+D_2$ where

$$D_1=\left(1-P\right){\overline{x}}_s^{\prime }\left(b_r-b_{nr}\right);D_2=-P\left(1-P\right){\left({\overline{x}}_r-{\overline{x}}_{nr}\right)}^{\prime }\left(b_r-b_{nr}\right).$$

Equivalent expressions are

$${\Delta }_{\text{CAL}}=\left(1-P\right){\overline{x}}_{nr}^{\prime }\left(b_r-b_{nr}\right);D_2=P{\left({\overline{x}}_s-{\overline{x}}_r\right)}^{\prime }\left(b_r-b_{nr}\right).$$

The Appendix, part 1, shows how the components $D_1$ and $D_2$ are derived from the definition (2.7) of ${\Delta }_{\text{CAL}}.$ The result invites several comments:

1. Balancing the response during data collection can bring ${\overline{x}}_r$ close to ${\overline{x}}_s$ and thus a reduction of both IMB and $D_2$ toward zero. Under the perfect balance ${\overline{x}}_r={\overline{x}}_s,$ $D_2=0$ but $D_1\ne 0.$
2. To get $D_1=0$ would require $b_r=b_{nr}.$ This does not happen even under perfect balance. The sample $s$ and its mean ${\overline{x}}_s$ are fixed. To eliminate the term $D_1$ seems out of reach. But does a “better composition” of the response set $r$ $–$ closeness of ${\overline{x}}_r$ to ${\overline{x}}_s$ and lower IMB $–$ have at least some favourable influence on the difference $b_r-b_{nr}$ and therefore on $D_1?$

Simple general answers are hard to obtain. Intuitively, a transfer of some units, during data collection, from nonresponse $nr=s-r$ to response $r$ suggests that $b_r-b_{nr}$ is not much affected, leaving $D_1$ rather constant. This becomes more explicit when $x$ is a group vector, that is, of the form $x_k={\left(0,\dots ,1,\dots ,0\right)}^{\prime }\text{};$ see Section 4. We shall call $D_2$ the reducible term and $D_1$ the resisting term of the deviation ${\Delta }_{\text{CAL}}=D_1+D_2.$ It should be noted that here we consider a fixed choice of $x$ -vector. Changing that vector may of course have a certain effect both on $D_1$ and on $D_2.$

• Previous
• Article main page
• Next