An assessment of accuracy improvement by adaptive survey design
Section 2. Probability sampling followed by nonresponse

• Previous
• Next

2.1  Population, sample, response set, nonresponse set

We denote by $U=\left\{1,2,\dots ,k,\dots ,N\right\}$ a finite population of size $N$ from which a probability sample $s$ has been drawn, giving unit $k$ the inclusion probability $\text{}{\pi }_k$ and the sampling weight $d_k=1/{\pi }_k\text{}.$ Let $r$ be the response set, that is, the set of units for which the value $y_k$ of the survey variable $y\text{-}$categorical or continuous $–$ is observed. We thus have observations $y_k$ for $k\in r\subset s\subset U,$ but they are missing for the nonresponse set denoted $nr=s-r.$ Using the observed $y_k$ we wish to estimate the population $y\text{-}$total $Y={\displaystyle {\sum }_U{y}_k}.$ A summation ${\displaystyle {\sum }_{k\in A}\text{}}$ over a set of units $k\in A\subseteq U$ is written simply as ${\displaystyle {\sum }_A\text{}.}$

That “non-respondents are not like respondents” is a well-established fact, and neither group is sufficiently like the whole sample. This causes bias in estimates of the total $Y.$ This article focuses on the contrast between response set and nonresponse set, something that has been fruitful earlier, in concepts such as the fraction of missing information.

Adaptive data collection is a dynamic process. The response and nonresponse sets, and associated quantities such as means and regression coefficients, are temporarily defined. A more complete notation would distinguish response sets $r^{\left(a\right)}\text{},$ $a=1,2,\dots ,$ in hierarchical sequence,

$$r^{\left(1\right)}\subseteq r^{\left(2\right)}\subseteq \dots \subseteq r^{\left(a\right)}\subseteq \dots$$

Here $r^{\left(a\right)}$ is the set of units having delivered the value $y_k$ after $a$ call attempts (or, alternatively, after $a$ data collection days), and $n{r}^{\left(a\right)}=s-r^{\left(a\right)}$ is the corresponding nonresponse set. But to simplify, we let $r$ refer to any one of the increasingly larger response sets. Tools for data collection, such as Statistics Sweden’s WinDATI system, allow us to record all contact attempt and to intervene and redirect the data inflow, so as to get in the end a better balanced final response set.

The response rate and corresponding non-response rate (both $d$ -weighted) are

$$P={\displaystyle {\sum }_r{d}_k}/{\displaystyle {\sum }_s{d}_k};Q=1-P={\displaystyle {\sum }_{nr}{d}_k}/{\displaystyle {\sum }_s{d}_k}.$$

Further notation for response, nonresponse and full sample is given for the auxiliary vector $x$ in Section 2.2, for the survey variable $y$ and for the regression vector of $y$ on $x$ in Section 2.3.

2.2  Auxiliary vector and response imbalance

An auxiliary vector $x$ is chosen following a structured selection of auxiliary variables from a supply of such variables. In the Scandinavian countries, the supply is vast, from administrative sources, in surveys of individuals and households. Much can be said about principles, or “attempts at optimality”, that might guide this choice, but this selection procedure is not a topic in this article.

There is a designated auxiliary vector $x$ of dimension $J\ge 1.$ Its value $x_k$ is assumed known for all units $k\in s.$ (The case where the population total of $x_k$ is known is not considered.) In an important special case, $x$ is a group vector of dimension $J,$ of the form $x_k={\left(0,\dots ,1,\dots ,0\right)}^{\prime }\text{},$ that is, with $J-1$ entries “0” and a single entry “1” to identify the group that contains unit $k.$ The groups are non-overlapping and exhaustive; the group vector codes $J$ properties of the units. For example, two education categories crossed with three income categories gives $J=2\times 3=6$ groups. But categorical $x$ -variables used in the vector $x$ need not all be fully crossed; in many applications they are not.

We use vectors $x$ with a property that brings mathematical convenience in many derivations: A vector $\mu$ not depending on $k$  exists such that

$${\mu }^{\prime }{x}_k=1\text{for}\text{all}k.(2.1)$$

This is satisfied by a majority of $x$ -vectors of interest, or can be made to hold. For example, if there is a single continuous auxiliary variable $x,$ and $x_k={\left(1,x_k\right)}^{\prime }\text{},$ then $\mu ={\left(1,0\right)}^{\prime }$ satisfies the requirement. In the group vector case, the requirement is satisfied by $\mu ={\left(1,\dots ,1,\dots ,1\right)}^{\prime }.$ For an example where $x$ is categorical but not a group vector, suppose two Education categories are crossed with three Income categories and that Gender is added to the vector $x_k$ as a univariate 0/1 variable, then the dimension is $J=6+1=7,$ the number of distinct values $x_k$ is $6\times 2=12,$ and $\mu ={\left(1,1,1,1,1,1,0\right)}^{\prime }$ satisfies the requirement (2.1).

The $(d$ -weighted) $x$ -vector means $–$ all computable $–$ are

$${\overline{x}}_r={\displaystyle {\sum }_r{d}_k{x}_k}/{\displaystyle {\sum }_r{d}_k;}{\overline{x}}_{nr}={\displaystyle {\sum }_{nr}{d}_k{x}_k}/{\displaystyle {\sum }_{nr}{d}_k;}{\overline{x}}_s={\displaystyle {\sum }_s{d}_k{x}_k}/{\displaystyle {\sum }_s{d}_k.}(2.2)$$

We need the second order moments; $J\times J$ computable matrices assumed non-singular:

$${\Sigma }_r={\displaystyle {\sum }_r{d}_k{x}_k{x}_k^{\prime }}/{\displaystyle {\sum }_r{d}_k;}{\Sigma }_{nr}={\displaystyle {\sum }_{nr}{d}_k{x}_k{x}_k^{\prime }}/{\displaystyle {\sum }_{nr}{d}_k;}{\Sigma }_s={\displaystyle {\sum }_s{d}_k{x}_k{x}_k^{\prime }}/{\displaystyle {\sum }_s{d}_k.}(2.3)$$

The imbalance (IMB) of the response is computed on the auxiliary vector values $x_k$ known for $k\in s,$ see Särndal (2011a). It is a measure of the contrast between response and full sample, or alternatively between response and nonresponse. For given vector $x$ and sample $s,$ the imbalance is defined as

$$\text{IMB}=P^2{\left({\overline{x}}_r-{\overline{x}}_s\right)}^{\prime }{\Sigma }_s^{-1}\left({\overline{x}}_r-{\overline{x}}_s\right).$$

A more telling notation would be $\text{IMB}\left(r,x|s\right)$ but for simplicity we use just IMB. Because ${\overline{x}}_r-{\overline{x}}_s=\left(1-P\right)\left({\overline{x}}_r-{\overline{x}}_{nr}\right),$ we can express IMB as a contrast between response and nonresponse:

$$\text{IMB}={\left\{P\left(1-P\right)\right\}}^2{\left({\overline{x}}_r-{\overline{x}}_{nr}\right)}^{\prime }{\Sigma }_s^{-1}\left({\overline{x}}_r-{\overline{x}}_{nr}\right).(2.4)$$

An objective for an adaptive data collection design is to create a final response set $r$ with low IMB, for the given probability sample $s.$

The dimension and the composition of the $x$ -vector are important determinants of the IMB-value. A property of IMB, for fixed $r$ and $s,$ is that it increases when we extend a given $x$ -vector by adding more $x$ -variables to it. The increase reflects the intuition that it is harder to make a higher number of $x$ -variable means come close. The trivial $x$ -vector, $x_k=1$ for all $k,$ gives $\text{IMB}=0,$ but it is of virtually no interest in practice. We have $0\le \text{IMB}\le P\left(1-P\right)\le 0.25$ for any $s,$ any $r,$ and any $x$ -vector. Those are broad conditions; for most survey data sets, the computed IMB-value is much below the upper bound, often in the range 0.01 to 0.05.

2.3  The survey variable and its regression on $x$

We turn to the survey variable $y.$ Its value $y_k$ is observed for $k\in r$ so linear regression fit of $y$ on $x$ can be carried out for the set $r.$ Although not feasible in a survey with nonresponse, a linear regression fit of $y$ on $x$ also exists, conceptually, for $nr=s-r$ and for $s.$ Fitted on the response $r,$ the linear regression coefficient vector (by $d$ -weighted least squares) is $b_r={\left({\displaystyle {\sum }_r{d}_k{x}_k{x}_k^{\prime }}\right)}^{-1}{\displaystyle {\sum }_r{d}_k{x}_k{y}_k}.$ Analogously, the other two regression fits give regression vectors $b_{nr}$ and $b_s.$ Expressed with the second-order moment $x$ -matrices in (2.3), the three regression vectors are

$$b_r={\Sigma }_r^{-1}{\displaystyle {\sum }_r{d}_k{x}_k{y}_k}/{\displaystyle {\sum }_r{d}_k};b_{nr}={\Sigma }_{nr}^{-1}{\displaystyle {\sum }_{nr}{d}_k{x}_k{y}_k}/{\displaystyle {\sum }_{nr}{d}_k};b_s={\Sigma }_s^{-1}{\displaystyle {\sum }_s{d}_k{x}_k{y}_k}/{\displaystyle {\sum }_s{d}_k\text{}.}(2.5)$$

The $y$ -means are

$${\overline{y}}_r={\displaystyle {\sum }_r{d}_k{y}_k}/{\displaystyle {\sum }_r{d}_k};{\overline{y}}_{nr}={\displaystyle {\sum }_{nr}{d}_k{y}_k}/{\displaystyle {\sum }_{nr}{d}_k};{\overline{y}}_s={\displaystyle {\sum }_s{d}_k{y}_k}/{\displaystyle {\sum }_s{d}_k}\text{}.$$

The following properties hold as a result of (2.1):

$${\overline{y}}_r={\overline{x}}_r^{\prime }{b}_r;{\overline{y}}_{nr}={\overline{x}}_{nr}^{\prime }{b}_{nr};{\overline{y}}_s={\overline{x}}_s^{\prime }{b}_s.(2.6)$$

A rudimentary estimator, under nonresponse, of the population total $Y={\displaystyle {\sum }_U{y}_k}=N{\overline{y}}_U$ uses straight expansion (EXP) of the response mean:

$${\widehat{Y}}_{\text{EXP}}=\widehat{N}{\displaystyle {\sum }_r{d}_k{y}_k}/{\displaystyle {\sum }_r{d}_k}=\widehat{N}{\overline{y}}_r,$$

where $\widehat{N}={\displaystyle {\sum }_s{d}_k}.$ In ${\widehat{Y}}_{\text{EXP}},$ the weighting is uniform, without any use of auxiliary information. The bias can be high. A use of the auxiliary values $x_k$ known for $k\in s$ is usually an improvement; ${\displaystyle {\sum }_s{d}_k{x}_k}$ is a known and unbiased (Horvitz-Thompson) estimator of the population total ${\displaystyle {\sum }_U{x}_k},$ so we seek weights $d_k{w}_k$ to satisfy the calibration equation ${\displaystyle {\sum }_r{d}_k{w}_k{x}_k}={\displaystyle {\sum }_s{d}_k{x}_k}.$ A solution (not a unique one) is

$$w_k={\left({\displaystyle {\sum }_s{d}_k{x}_k}\right)}^{\prime }{\left({\displaystyle {\sum }_r{d}_k{x}_k}{x}_k^{\prime }\right)}^{-1}{x}_k=\left(1/P\right){\overline{x}}_s^{\prime }{\Sigma }_r^{-1}{x}_k.$$

The resulting linear calibration (CAL) estimator of $Y={\displaystyle {\sum }_U{y}_k}$ is ${\widehat{Y}}_{\text{CAL}}={\displaystyle {\sum }_r{d}_k{w}_k{y}_k},$ or equivalently

$${\widehat{Y}}_{\text{CAL}}=\widehat{N}{\overline{x}}_s^{\prime }{b}_r.$$

A reason why we expect ${\widehat{Y}}_{\text{CAL}}$ to be less biased than ${\widehat{Y}}_{\text{EXP}}$ $–$ especially when $y$ and $x$ are well related $–$ is that the weights $d_k{w}_k$ must respect the constraint of an unbiased computable quantity, ${\displaystyle {\sum }_s{d}_k{x}_k},$ on the right hand side of the calibration equation. But despite the adjustment weighting, ${\widehat{Y}}_{\text{CAL}}$ has non-negligible, possibly considerable, remaining bias.

As a benchmark we use the unbiased estimator of $Y$ requiring full (FUL) response $–$ therefore hypothetical under nonresponse $–$ namely, the Horwitz-Thompson estimator

$$Y_{\text{FUL}}={\displaystyle {\sum }_s{d}_k{y}_k}=\widehat{N}{\overline{y}}_s.$$

We refer to the three estimators types as EXP, CAL and FUL. In fact, CAL represents a family of estimators, corresponding to all possible choices of $x$ -vector. For a given suitable $x$ -vector, we shall closely examine the CAL deviation ${\Delta }_{\text{CAL}},$ defined as the difference between the biased CAL and the unbiased FUL, scaled by dividing by the (estimated if necessary) population size:

$$\left({\widehat{Y}}_{\text{CAL}}-{\widehat{Y}}_{\text{FUL}}\right)/\widehat{N}={\Delta }_{\text{CAL}},$$

where

$${\Delta }_{\text{CAL}}={\overline{x}}_s^{\prime }{b}_r-{\overline{y}}_s={\overline{x}}_s^{\prime }\left(b_r-b_s\right).(2.7)$$

Deviation is not bias. The deviation ${\Delta }_{\text{CAL}}$ is an outcome for a response set $r$ from a specific sample $s.$ On the other hand, the bias of the CAL estimator is the expected value of ${\Delta }_{\text{CAL}}$ over all $r$ from a given $s,$ and over all $s$ from $U.$ If a response $r$ with zero deviation ${\Delta }_{\text{CAL}}$ could be realized whatever the sample $s,$ then the CAL estimator would have zero bias, because it is then always equal to the unbiased Horvitz-Thompson estimator. But to get zero deviation, by adaptive data collection or in some other way, is unrealistic in practice. Nevertheless, it makes good sense to attempt to make the deviation small for the given sample $s,$ since it suggests coming closer to the unbiased estimate. One can say that to get unbiased estimation is not the objective, because impossible when there is nonresponse, but a realistic objective is to reduce deviation from the unbiased estimate.

• Previous
• Article main page
• Next