“Optimal” calibration weights under unit nonresponse in survey sampling
Section 2. Calibration estimation

• Previous
• Next

2.1  Calibration estimators under full response

Starting with the full response situation $\left(r\mathrm{<mo>=</mo>}s\right)$ and following the procedure as established by Deville and Särndal (1992), the calibration estimator is defined as

$${\widehat{t}}_{y\text{cal}}={\displaystyle \sum _s}{w}_{ks}{y}_k,$$

where the sample dependent weights $w_{ks}$ are chosen so that

$${\displaystyle \sum _s}{w}_{ks}{x}_k\mathrm{<mo>=</mo>}{t}_x\mathrm{,}(\text{the}\text{calibration}\text{equation})(2.1)$$

while also minimizing the quadratic distance measure

$${\left(w_s-w_{0s}\right)}^{\prime }R\left(w_s-w_{0s}\right)\mathrm{,}$$

where $w_s\mathrm{<mo>=</mo>}{\left(w_{ks}\right)}_{k\in s},$ $w_{0s}\mathrm{<mo>=</mo>}{\left(1/{\pi }_k\right)}_{k\in s}\mathrm{<mo>=</mo>}{\left(d_k\right)}_{k\in s}$ and $R$ is diagonal. (Alternative distance measures are considered in both Deville and Särndal (1992) and Haziza and Lesage (2016).)

In other words, given the constraint (2.1) the $w_{ks}$ should be “as close as possible” to the design weights $d_k,$ which is desirable since ${\displaystyle {\sum }_s}{d}_k{y}_k$ is an unbiased estimator of $t_y.$

The resulting weights are

$$w_s=w_{0s}+R^{-1}{x}^{\prime }{\left(X{R}^{-1}{X}^{\prime }\right)}^{-1}\left(t_x-{\widehat{t}}_x\right).$$

It turns out that the model assisted homoskedastic GREG estimator ${\widehat{t}}_{yr}$ (Särndal, Swensson and Wretman (1992)) is a calibration estimator for which

$$R\mathrm{<mo>=</mo>}{\left(w_{0s}{I}_{n_s}\right)}^{-1}\mathrm{,}$$

where $I_{n_s}$ is the unit diagonal matrix of size $n_s.$

Another calibration estimator is the optimal regression estimator ${\widehat{t}}_{y\text{opt}}$ (see e.g., Rao (1994) and Montanari (1998)), for which

$$R={\left(\frac{{\pi }_{kl}-{\pi }_k{\pi }_l}{{\pi }_{kl}{\pi }_k{\pi }_l}\right)}_{k\mathrm{,}l\in s}^{-1}\mathrm{,}$$

as shown by Andersson and Thorburn (2005).

Asymptotically, this estimator has (in a design-based sense) minimum variance among linear regression type estimators.

2.2  Calibration estimators under nonresponse

In the nonresponse case, a possible calibration estimator is

$${\displaystyle \sum _r}{w}_{kr}{y}_k\mathrm{,}$$

where it should hold that

$${\displaystyle \sum _r}{w}_{kr}{x}_k\mathrm{<mo>=</mo>}X\mathrm{,}(2.2)$$

where $X\mathrm{<mo>=</mo>}{\displaystyle {\sum }_U}{x}_k^{*},$ if the auxiliary information is known up to the population level. Otherwise, $X\mathrm{<mo>=</mo>}{\displaystyle {\sum }_s}{d}_k{x}_k^o,$ the unbiased estimator of $t_x.$ (We can also combine the two types of information in the constraint $X.)$

For a variety of cases weights fulfilling the requirement (2.2) are presented by e.g., Särndal and Lundström (2005). Using the direct approach, where all information is used in one single calibration, we get

$$w_{kr}=d_k\left(1+x_k^{\prime }{\left({\displaystyle \sum _r}{d}_k{x}_k{x}_k^{\prime }\right)}^{-1}\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right)\right).(2.3)$$

The resulting estimator will henceforth be denoted ${\widehat{t}}_{y\text{cal}}.$ (Other approaches, including two-step procedures, are presented and investigated by e.g., Andersson and Särndal (2016).)

An evident question to ask is: What is the underlying distance measure generating these weights? Särndal and Lundström (2005) do not comment on this particular issue, but according to Lundström and Särndal (1999), we should choose $“w_k$ ‘as close as possible’ to the $d_k”,$ which does not seem quite adequate under nonresponse. Going back to Lundström (1997) we will find that the corresponding distance measure is actually

$${\left(w_r-w_{0r}\right)}^{\prime }{\left(w_{0r}{I}_{n_r}\right)}^{-1}\left(w_r-w_{0r}\right)\mathrm{,}$$

where $w_r\mathrm{<mo>=</mo>}{\left(w_{kr}\right)}_{k\in r}$ and $w_{0r}\mathrm{<mo>=</mo>}{\left(d_k\right)}_{k\in r}.$

If we have a random mechanism generating the response set $r$ from the sample $s$ with probabilities ${\theta }_k$ of inclusion, we can view the nonresponse situation as a two-phase design and this is the assumption we will make in the following. Then we should minimize the distance between $w_{kr}$ and $d_k\cdot \left(1/{\theta }_k\right).$ Using some modelling ${\theta }_k$ can be estimated by ${\widehat{\theta }}_k,$ to be put to use for the distance minimization. But in this paper we will not go in the direction of model-based inference. In order to reduce the bias effect under nonresponse one could instead in the distance measure think of comparing $w_{kr}$ not with $d_k,$ but with $d_{k\mathrm{,}\text{alt}}\mathrm{<mo>=</mo>}{d}_k\cdot c,$ where $c$ is a constant larger than 1, aiming to compensate for the “average” nonresponse effect.

However, Lundström (1997) shows that in many important cases, namely when one can find a vector $\mu$ for which ${\mu }^{\prime }{x}_k\mathrm{<mo>=</mo>\; 1},$ for all $k,$ the multiplicative increase in $d_{k\mathrm{,}\text{alt}}$ implies the same resulting calibration weights $w_{kr}.$ This follows from the result that if ${\mu }^{\prime }{x}_k\mathrm{<mo>=</mo>\; 1},$ for all $k\in U,$ we can simplify the expression (2.3) of $w_{kr}$ as

$$w_{kr}\mathrm{<mo>=</mo>}{d}_k{x}_k^{\prime }{\left({\displaystyle \sum _r}{d}_k{x}_k{x}_k^{\prime }\right)}^{-1}X.$$

Thus, we have an invariance property for the weights. The result holds also when the population is partitioned into groups and the initial weights are inflated with a constant within each group. Note that if we include a constant, e.g., “ 1” , as a first component of the auxiliary vector $x_k,$ we can simply let ${\mu }^{\prime }\mathrm{<mo>=</mo>}\left(\mathrm{1,}\mathrm{0,}\dots \mathrm{,}0\right)$ to achieve ${\mu }^{\prime }{x}_k\mathrm{<mo>=</mo>\; 1.}$

With this as a background we propose to use alternative “optimal” weights resulting from the distance measure

$${\left(w_r-w_{0r}\right)}^{\prime }{\left(\frac{{\pi }_{kl}-{\pi }_k{\pi }_l}{{\pi }_{kl}{\pi }_k{\pi }_l}\right)}_{k\mathrm{,}l\in r}^{-1}\left(w_r-w_{0r}\right)\mathrm{,}$$

leading to ${\widehat{t}}_{y\text{opt}}.$ $({\pi }_{kl}$ denotes the inclusion probability for the pair $(k\mathrm{,}l)).$

It is to be observed that as for the full response situation, there are cases for which the “optimal” weights are identical to (2.3), as e.g., under simple random sampling.

Using quotation marks around optimal is deliberate, but under full response optimal has a very clear meaning. As mentioned earlier, the optimal regression estimator has asymptotically minimum variance among linear regression estimators. Adding nonresponse where the nonresponse mechanism is at least partially unknown, makes it difficult to define optimality criteria in a proper way.

For this “optimal” measure it might be fruitful to replace $d_k$ with $d_{k\mathrm{,}\text{alt}},$ where we include in $d_{k\mathrm{,}\text{alt}}$ the reciprocal of an estimate of the average response probability ${\overline{\theta }}_U\mathrm{<mo>=</mo>}{\displaystyle {\sum }_U}{\theta }_k/N.$ One simple candidate is

$${\widehat{\overline{\theta }}}_U\mathrm{<mo>=</mo>}{n}_r/n_s,$$

thus yielding $d_{k\mathrm{,}\text{alt}}\mathrm{<mo>=</mo>}{d}_k\cdot \left(n_s/n_r\right).$ Another natural choice is

$${\widehat{\overline{\theta }}}_U\mathrm{<mo>=</mo>}{\displaystyle \sum _r}{d}_k/{\displaystyle \sum _s}{d}_k,(2.4)$$

since $E\left({\displaystyle {\sum }_s}{d}_k\right)\mathrm{<mo>=</mo>}N$ and $E\left({\displaystyle {\sum }_r}{d}_k\right)\mathrm{<mo>=</mo>}{\displaystyle {\sum }_U}{\theta }_k\mathrm{<mo>=</mo>}N\overline{\theta },$ which lead to $E\left({\displaystyle {\sum }_r}{d}_k/{\displaystyle {\sum }_s}{d}_k\right)\approx {\overline{\theta }}_U.$ The resulting modified estimator is denoted by ${\widehat{t}}_{y\text{optm}}.$ (Also observe that $E\left(n_r/n_s\right)\approx {\displaystyle {\sum }_U}\left({\theta }_k/d_k\right)/{\displaystyle {\sum }_U}\left(1/d_k\right).$

In the following simulation study we will focus on a sampling design where generally ${\widehat{t}}_{y\text{cal}}\ne {\widehat{t}}_{y\text{opt}},$ namely Poisson sampling. The independence of drawings simplifies the “optimal” distance measure:

$${\displaystyle \sum _r}\frac{{\pi }_k^2}{1-{\pi }_k}{\left(w_{kr}-d_k\right)}^2\mathrm{<mo>=</mo>}{\displaystyle \sum _r}\frac{{\left(w_{kr}-d_k\right)}^2}{d_k\left(d_k-1\right)}$$

and minimization yields

$$w_{kr}\mathrm{<mo>=</mo>}{d}_k\left(1+\left(d_k-1\right)x_k^{\prime }{\left({\displaystyle \sum _r}{d}_k\left(1-d_k\right)x_k{x}_k^{\prime }\right)}^{-1}\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right)\right).$$

For the modified “optimal” estimator $d_k$ is replaced by $d_{k\text{alt}}\mathrm{<mo>=</mo>}{d}_k\cdot \left(1/{\widehat{\overline{\theta }}}_U\right),$ with ${\widehat{\overline{\theta }}}_U$ as in (2.4).

2.2.1  Bias for calibration estimators under nonresponse

We can write ${\widehat{t}}_{y\text{cal}}$ as

$${\widehat{t}}_{y\text{cal}}\mathrm{<mo>=</mo>}{\displaystyle \sum _r}{d}_k{y}_k+{\widehat{B}}_{U\mathrm{<mo>;</mo>}\theta }\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right)\mathrm{,}(2.5)$$

where ${\widehat{B}}_{U\mathrm{<mo>;</mo>}\theta }\mathrm{<mo>=</mo>}\left({\displaystyle {\sum }_r}{d}_k{x}_k^{\prime }{y}_k\right){\left({\displaystyle {\sum }_r}{d}_k{x}_k{x}_k^{\prime }\right)}^{-1}.$ In order to arrive at an approximate expression for the bias of ${\widehat{t}}_{y\text{cal}}$ and subsequently ${\widehat{t}}_{y\text{opt}}$ and ${\widehat{t}}_{y\text{optm}},$ we follow the derivation in Särndal and Lundström (2005) and first note that ${\widehat{t}}_{y\text{cal}}$ can be rewritten as

$${\widehat{t}}_{y\text{cal}}\mathrm{<mo>=</mo>}{\displaystyle \sum _r}{d}_k{y}_k+B_{U\mathrm{<mo>;</mo>}\theta }\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right)+\left({\widehat{B}}_{U\mathrm{<mo>;</mo>}\theta }-B_{U\mathrm{<mo>;</mo>}\theta }\right)\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right)\mathrm{,}$$

where $B_{U\mathrm{<mo>;</mo>}\theta }\mathrm{<mo>=</mo>}\left({\displaystyle {\sum }_U}{\theta }_k{x}_k^{\prime }{y}_k\right){\left({\displaystyle {\sum }_U}{\theta }_k{x}_k{x}_k^{\prime }\right)}^{-1}.$

If we let ${\widehat{t}}_{y\text{cal}}-t_y\mathrm{<mo>=</mo>}{A}_1+A_2,$ where $A_1\mathrm{<mo>=</mo>}{\displaystyle {\sum }_r}{d}_k{y}_k-t_y+B_{U\mathrm{<mo>;</mo>}\theta }\left(X-{\displaystyle {\sum }_r}{d}_k{x}_k\right)$ and $A_2\mathrm{<mo>=</mo>}\left({\widehat{B}}_{U\mathrm{<mo>;</mo>}\theta }-B_{U\mathrm{<mo>;</mo>}\theta }\right)\left(X-{\displaystyle {\sum }_r}{d}_k{x}_k\right),$ it can further be shown that

$$A_1\mathrm{<mo>=</mo>}{\displaystyle \sum _r}{d}_k{e}_{\theta k}-{\displaystyle \sum _U}{e}_{\theta k}+B_{U\mathrm{<mo>;</mo>}\theta }^o\left({\displaystyle \sum _s}{d}_k{x}_k^o-{\displaystyle \sum _U}{x}_k^o\right)\mathrm{,}$$

where $e_{\theta k}\mathrm{<mo>=</mo>}{y}_k-B_{U\mathrm{<mo>;</mo>}\theta }{x}_k$ and $B_{U\mathrm{<mo>;</mo>}\theta }^o\mathrm{<mo>=</mo>}{\left({\displaystyle {\sum }_U}{\theta }_k{x}_k^o{x}_k^o{}^{\prime }\right)}^{-1}{\displaystyle {\sum }_U}{\theta }_k{x}_k^o{y}_k.$

Then

$$E\left({\widehat{t}}_{y\text{cal}}\right)-t_y\approx E\left(A_1\right)\mathrm{<mo>=</mo>}{\displaystyle \sum _U}{\theta }_k{e}_{\theta k}-{\displaystyle \sum _U}{e}_{\theta k}\mathrm{<mo>=</mo>}-{\displaystyle \sum _U}\left(1-{\theta }_k\right)e_{\theta k}\mathrm{,}$$

since it can be argued that ${\widehat{B}}_{U\mathrm{<mo>;</mo>}\theta }$ is a consistent estimator of $B_{U\mathrm{<mo>;</mo>}\theta }$ and therefore $E\left(A_2\right)\approx 0.$

The approximation for the bias of ${\widehat{t}}_{y\text{cal}}$ is called the nearbias:

$$\text{nearbias}\left({\widehat{t}}_{y\text{cal}}\right)\mathrm{<mo>=</mo>}-{\displaystyle \sum _U}\left(1-{\theta }_k\right)e_{\theta k}.$$

The nearbias of ${\widehat{t}}_{y\text{cal}}$ is zero if ${\theta }_k\mathrm{<mo>=</mo>\; 1},$ for all $k\in U$ and/or $y_k\mathrm{<mo>=</mo>}{B}_{U\mathrm{<mo>;</mo>}\theta }{x}_k,$ for all $k\in U.$

Then, if we consider ${\widehat{t}}_{y\text{opt}},$ we have that

$${\widehat{t}}_{y\text{opt}}\mathrm{<mo>=</mo>}{\displaystyle \sum _r}{d}_k{y}_k+\left(X-{\displaystyle \sum _r}{d}_k{x}_k\right){\widehat{C}}_{U\mathrm{<mo>;</mo>}\theta }\mathrm{,}(2.6)$$

where

$${\widehat{C}}_{U\mathrm{<mo>;</mo>}\theta }\mathrm{<mo>=</mo>}\left({\displaystyle \sum _{k\in r}}{\displaystyle \sum _{l\in r}}\frac{{\pi }_{kl}-{\pi }_k{\pi }_l}{{\pi }_{kl}}\frac{x_k^{\prime }}{{\pi }_k}\frac{y_l}{{\pi }_l}\right){\left({\displaystyle \sum _{k\in r}}{\displaystyle \sum _{l\in r}}\frac{{\pi }_{kl}-{\pi }_k{\pi }_l}{{\pi }_{kl}}\frac{x_k}{{\pi }_k}\frac{x_l^{\prime }}{{\pi }_l}\right)}^{-1}.$$

Since ${\widehat{t}}_{y\text{opt}}$ can be written as (2.6), which is of the same form as for ${\widehat{t}}_{y\text{cal}}$ in (2.5), we will again arrive at the nearbias expression

$$\text{nearbias}\left({\widehat{t}}_{y\text{opt}}\right)\mathrm{<mo>=</mo>}-{\displaystyle \sum _U}\left(1-{\theta }_k\right)e_{\theta k}\mathrm{,}(2.7)$$

where $e_{\theta k}\mathrm{<mo>=</mo>}{y}_k-C_{U\mathrm{<mo>;</mo>}\theta }{x}_k$ and with ${\theta }_{kl}$ denoting the response probability for the pair $(k\mathrm{,}l):$

$$C_{U\mathrm{<mo>;</mo>}\theta }=\left({\displaystyle \sum _{k\in U}}{\displaystyle \sum _{l\in U}}{\theta }_{kl}\left({\pi }_{kl}-{\pi }_k{\pi }_l\right)\frac{x_k^{\prime }}{{\pi }_k}\frac{y_l}{{\pi }_l}\right){\left({\displaystyle \sum _{k\in U}}{\displaystyle \sum _{l\in U}}{\theta }_{kl}\left({\pi }_{kl}-{\pi }_k{\pi }_l\right)\frac{x_k}{{\pi }_k}\frac{x_l^{\prime }}{{\pi }_l}\right)}^{-1}.$$

If we use the alternative weighting $d_{k\mathrm{,}\text{alt}}\mathrm{<mo>=</mo>}{d}_k\cdot \left(1/\widehat{\overline{\theta }}\right)\mathrm{<mo>=</mo>}{d}_k\cdot \left({\displaystyle {\sum }_s}{d}_k/{\displaystyle {\sum }_r}{d}_k\right),$ we get that

             nearbias $\left({\widehat{t}}_{y\text{optm}}\right)\mathrm{<mo>=</mo>}E\left({\displaystyle \sum _r}{d}_{k\mathrm{,}\text{alt}}{e}_{\theta k}-{\displaystyle \sum _U}{e}_{\theta k}\right)\approx {\displaystyle \sum _U}\frac{{\theta }_k}{{\overline{\theta }}_U}{e}_{\theta k}-{\displaystyle \sum _U}{e}_{\theta k}\mathrm{<mo>=</mo>}-{\displaystyle \sum _U}\left(1-\frac{{\theta }_k}{{\overline{\theta }}_U}\right)e_{\theta k}\mathrm{,}$

where ${\displaystyle {\sum }_U}\left(1-\left({\theta }_k/{\overline{\theta }}_U\right)\right)\mathrm{<mo>=</mo>\; 0},$ to be compared with (2.7), where ${\displaystyle {\sum }_U}\left(1-{\theta }_k\right)\mathrm{<mo>=</mo>}N\left(1-{\overline{\theta }}_U\right).$

Unless ${\mu }^{\prime }{x}_k\mathrm{<mo>=</mo>\; 1},$ for all $k\in U,$ an equivalent expression can be obtained for ${\widehat{t}}_{y\text{cal}}.$ On the other hand, if the restriction ${\mu }^{\prime }{x}_k\mathrm{<mo>=</mo>\; 1},$ for all $k\in U$ does hold, it can be shown (Särndal and Lundström (2005)) that

$$\text{nearbias}\left({\widehat{t}}_{y\text{cal}}\right)\mathrm{<mo>=</mo>}-{\displaystyle \sum _U}{e}_{\theta k}\mathrm{,}$$

which holds independently of the sampling design and which is a result completely in line with the aforementioned invariance property of the calibration weights.

• Previous
• Article main page
• Next