Estimation of response propensities and indicators of representative response using population-level information
Section 6. Discussion

• Previous
• Next

The extension of sample-based to population-based estimators of R-indicators is comprised of two steps: 1) the estimation of response propensities, and 2) the estimation of the R-indicators based on these propensities. The population-based estimation of response propensities is straightforward when linear models are assumed for response propensities and response influences. The linear link function is reasonable when estimating response propensities under typical response rates seen for large-scale national social surveys as shown in the evaluation study in Section 4. The sample-based estimators contain sample covariance matrices and sample frequencies that can be replaced by population covariance matrices or population frequencies. We identified two types of settings: when population cross-products are available or when auxiliary information is restricted to marginal population counts only. We labelled the corresponding estimators as Type 1 and Type 2 estimators, respectively. The Type 2 setting is more restrictive than the Type 1 setting.

Following the estimation of population-based response propensities, we have constructed population-based estimators for the R-indicator and examined their properties both theoretically and empirically. The estimators are applied to samples drawn from real data from the 1995 Israel Census Data where “true” propensities were calculated according to realistic assumptions for national household social surveys. Thus, we have addressed the first two research questions at the beginning of the paper: How to extend sample-based response propensities and R-indicators to population-based response propensities and R-indicators? and What are the statistical properties of population-based R-indicators?

There are many options for the estimation of R-indicators depending on the response to the survey. We used propensity weighted response means as the propensities are available. However, any calibration method can be used such as linear weighting or adjustment classes. In fact, the set of auxiliary variables used for the estimation of the R-indicators may be a subset of the auxiliary variables used for the estimation of propensities and influences. Parsimonious models may prove to be more efficient as it is known that propensity-weighting may seriously affect the precision of the estimators. This is a topic for future research.

The two properties we examined are the bias and standard errors of the proposed population-based R-indicators. As expected the bias and standard errors are dependent on the size of the sample and the type of auxiliary information available where the smaller the sample, the larger the bias and the standard error. When samples are smaller, it becomes more difficult to distinguish sampling variation from response variation. Clearly, the confidence intervals become larger as there is less information in small samples.

The bias-adjusted Type 1 estimators (population cross-products) perform better than the bias-adjusted Type 2 estimators (population marginal counts). This is as expected given that they employ more information. However, the unadjusted Type 2 estimators have better RRMSE properties than the unadjusted Type 1 estimators. This is a surprising result and points to a suboptimal use of the population cross-products when they are used as “plug-ins” and do not account for any sampling variation. The standard errors of the population-based estimators are larger than their sample-based counterparts.

The evaluation study in scenario RR3 shows that, for very high response rates, the population-based R-indicators provide higher standard errors and larger bias, mainly due to propensities being estimated outside of the interval [0, 1]. For this reason, we proposed a composite estimator with varying smoothing parameters dependent on the response rate. Standard errors were reduced but at the cost of increased bias.

From the analyses it becomes apparent that the bias of the Type 1 and Type 2 estimators depends on the number of auxiliary variables, but this dependence was modest in our evaluations. The bias may increase when using detailed models with many variables for the estimation of response propensities. The rationale behind this is that detailed models allow for more sampling variation to be picked up as bias.

The population-based R-indicators have a number of caveats:

Firstly, the choice of auxiliary information that is available at a national level may be more limiting than sample-based auxiliary information depending on the availability of registers and administrative data. The selection of auxiliary variables should depend on whether they are correlated with the survey target variables. Also, it is strongly recommended that population statistics that are based on registers or administrative data are used rather than those based on weighted survey counts from other surveys since these statistics may not reflect the true population distribution accurately. One would draw erroneous conclusions about the representativeness of the response if the population estimates are biased.

Secondly, we make the assumption that the survey measures the same quantities as in the population information and we do not investigate the effect of possible departures from this assumption. However, we note that there is an imminent risk of measurement errors when comparing the representativeness of survey questions to population statistics. It must be ascertained that the survey questions that are employed have the same definitions and classifications as the population tables. Hence, it is best to avoid questions that are prone to measurement errors, such as questions that require a strong cognitive effort or that may lead to socially desirable answers.

Thirdly, in settings where only population information is available, options to improve representativeness during data collection are much more limited since there is no individual auxiliary information available for the nonrespondents. Nonetheless, in these settings, assessments of representativeness may still be useful in the design of advance and reminder letters, in interviewer training and in paradata collection.

Finally, we do not consider hybrid settings where the R-indicator is based on both linked data and population tables. In addition, we do not deal with the case where we could use weighted survey estimates if there is no aggregated population information available. This will impact on both the bias and variance estimates for the population based R-indicators. Such extensions are relatively straightforward but will be left to future papers.

The research into population-based R-indicators is still at the beginning stage and it is too early to provide a definitive answer to the last research question presented in the introduction regarding the feasibility and practicability of R-indicators based on aggregate population auxiliary information. As mentioned in the introduction, further usage of these R-indicators are being explored in the context of evaluating and monitoring streamed administrative data and assessing the representativeness of linked records. In addition, Schouten et al. (2011) introduced partial R-indicators under sample-based auxiliary information for evaluating the lack of representativeness due to a specific auxiliary variable or category. These were used for monitoring and evaluating data collection. Schouten and Shlomo (2017) demonstrate the use of partial R-indicators for adaptive survey designs. It is straightforward, similarly, to define population-based partial R-indicators and this will be a subject of future work.

Regarding the evaluation study presented in Section 4 on survey representativeness, it is based on real data under realistic assumptions of response probabilities typically found in social surveys conducted at national statistical institutes. Future research needs to assess whether alternative estimators can be constructed that are more precise, and, consequently, allow for stronger conclusions regarding the nature of response. A natural avenue to explore is an iterative approach through a modification of the EM-algorithm, in which the score of the nonrespondents on the auxiliary variables is estimated and used to update response propensity estimates.

We did not consider population-based estimation for other types of models such as logistic or probit regression. As shown in the numerical evaluation in Section 4, differences in sample-based estimators between the linear and logistic link function are in general small, but when the response rates get very close to 1, they become more evident. For these cases, developing other link functions for population-based estimation is a subject of future research. This would be a useful and natural extension to the theory of R-indicators as these models are often used in practice and avoid propensities outside the [0, 1] interval.

Acknowledgements

Part of the research presented here was developed within project RISQ (Representativity Indicators for Survey Quality, www.risq-project.eu), funded by the European 7^th Framework Programme. We thank the members of the RISQ project: Katja Rutar from Statisti $č$ ni Urad Republike Slovenije, Geert Loosveldt and Koen Beullens from Katholieke Universiteit, Leuven, Øyvin Kleven, Johan Fosen and Li-Chun Zhang from Statistisk Sentralbyrå, Norway, Ana Marujo from the University of Southampton, UK and Paul Knottnerus, Centraal Bureau voor de Statistiek, for their valuable input.

The first author was supported by a STSM Grant from the COST Action IS1004 and by the ex 60% University of Bergamo, Biffignandi grant.

Appendix A

Analytic approximation to the bias of Type 1 ${\tilde{R}}_{{\tilde{\rho }}_{T1}}$ estimators

First, we compute the bias of ${\tilde{S}}_{{\tilde{\rho }}_{T1}}^2$ under general sampling design. Letting ${\widehat{m}}_1=N^{-1}{\displaystyle {\sum }_r{d}_i}$ and ${\widehat{m}}_2=N^{-1}{\displaystyle {\sum }_r{d}_i{\tilde{\rho }}_{i\text{},T1}},$ then we can write

$$B\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)=E\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)-S_{\rho }^2=\frac{N}{N-1}\left\{E\left({\widehat{m}}_2\right)-V\left({\widehat{m}}_1\right)-{\left[E\left({\widehat{m}}_1\right)\right]}^2\right\}-\frac{N}{N-1}\left\{\frac{1}{N}{\displaystyle \sum _{i\in U}{\rho }_i^2}-{\overline{\rho }}_U^2\right\}.(\text{A}\text{.1})$$

Note that

$$\begin{array}{ll}E\left({\widehat{m}}_2\right)\hfill & =E\left(\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{s}_i{r}_i{\tilde{\rho }}_{i\text{},T1}}\right)=\frac{1}{N}{\displaystyle \sum _{i\in U}{x}_i^T{T}_1^{-1}{E}_s\left\{E_m\left(d_i^2{s}_i{r}_i{x}_i+{\displaystyle \sum _{\begin{array}{l}k\in U\\ k\ne i\end{array}}{d}_i{d}_k{s}_i{s}_k{r}_i{r}_k{x}_k|s}\right)\right\}}\hfill \\ \hfill & =\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_1^{-1}{x}_i}+\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_1^{-1}{\displaystyle \sum _{\begin{array}{l}k\in U\\ k\ne i\end{array}}{d}_k{\pi }_{ik}{\rho }_k{x}_k}},\hfill \\ E\left({\widehat{m}}_1\right)\hfill & =E\left(\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{s}_i{r}_i}\right)=E_s\left(\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{s}_i{\rho }_i}\right)={\overline{\rho }}_U,\hfill \end{array}$$

and

$$\begin{array}{ll}V\left({\widehat{m}}_1\right)\hfill & =V_s\left\{E_m\left({\widehat{m}}_1|s\right)\right\}+E_s\left\{V_m\left({\widehat{m}}_1|s\right)\right\}\hfill \\ \hfill & =V_s\left\{\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{s}_i{\rho }_i}\right\}+E_s\left\{\frac{1}{N^2}{\displaystyle \sum _{i\in U}{d}_i^2{s}_i{\rho }_i\left(1-{\rho }_i\right)}\right\}\hfill \\ \hfill & =\frac{1}{N^2}{\displaystyle \sum _{i\in U}{\displaystyle \sum _{k\in U}{d}_i{d}_k{\Delta }_{ik}{\rho }_i{\rho }_k}}+\frac{1}{N^2}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i\left(1-{\rho }_i\right)},\hfill \end{array}$$

where ${\Delta }_{ik}={\pi }_{ik}-{\pi }_i{\pi }_k$ and ${\pi }_{ik}$ are the second-order sample inclusion probabilities. Hence, the bias of ${\tilde{S}}_{{\tilde{\rho }}_{T1}}^2$ with respect to the joint distribution of sampling design and the response mechanism is given by

$$\begin{array}{ll}B\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)\hfill & =\frac{N}{N-1}[\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_1^{-1}{x}_i}+\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_1^{-1}{\displaystyle \sum _{\begin{array}{l}k\in U\\ k\ne i\end{array}}{d}_k{\pi }_{ik}{\rho }_k{x}_k}}\hfill \\ \hfill & {}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{}}}}}}}}}}}}}}}}}-\frac{1}{N^2}{\displaystyle \sum _{i\in U}{\displaystyle \sum _{k\in U}{d}_i{d}_k{\Delta }_{ik}{\rho }_i{\rho }_k}}-\frac{1}{N^2}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i\left(1-{\rho }_i\right)}-\frac{1}{N}{\displaystyle \sum _{i\in U}{\rho }_i^2}].(\text{A}\text{.2})\hfill \end{array}$$

Under simple random sampling without replacement, (A.2) can be simplified to

$$B^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)=\frac{N}{N-1}\left[\frac{1}{n}{\displaystyle \sum _{i\in U}{\rho }_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}{x}_i^T{T}_1^{-1}{x}_i}+\frac{n-1}{n\left(N-1\right)N}{\displaystyle \sum _{i\in U}{\rho }_i^2}-\frac{{\overline{\rho }}_U}{n}-\left(1-\frac{n}{N}\right)\frac{S_{\rho }^2}{n}\right].$$

A response-set based estimator of $B^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)$ is

$${\tilde{B}}_{{\tilde{\rho }}_{T1}}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)=\frac{N}{N-1}\left[\frac{N}{n^2}{\displaystyle \sum _{i\in r}\left\{1-\frac{n-1}{N-1}{\tilde{\rho }}_{i\text{},T1}\right\}{x}_i^T{T}_1^{-1}{x}_i}+\frac{n-1}{n^2\left(N-1\right)}{\displaystyle \sum _{i\in r}{\tilde{\rho }}_{i\text{},T1}}-\left(1-\frac{n}{N}\right)\frac{{\tilde{S}}_{{\tilde{\rho }}_{T1}}^2}{n}-\frac{n_r}{n^2}\right].$$

More generally, the Horvitz-Thompson response-set estimator for (A.2) under complex sampling is given by

$$\begin{array}{ll}{\tilde{B}}_{{\tilde{\rho }}_{T1}}\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)\hfill & =\frac{N}{N-1}\{\frac{1}{N}{\displaystyle \sum _{i\in r}{d}_i\left(d_i-{\tilde{\rho }}_{i\text{},T1}\right)x_i^T{T}_1^{-1}{x}_i}-\frac{1}{N^2}{\displaystyle \sum _{i\in r}{d}_i^3{\Delta }_{ii}{\tilde{\rho }}_{i\text{},T1}{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{}}}}}}}}}}}}}}}}}}\hfill \\ \hfill & -\frac{1}{N^2}{\displaystyle \sum _{i\in r}{\displaystyle \sum _{\begin{array}{l}k\in r\\ k\ne i\end{array}}{d}_i{d}_k\frac{{\Delta }_{ik}}{{\pi }_{ik}}}}-\frac{1}{N^2}{\displaystyle \sum _{i\in r}{d}_i^2\left(1-{\tilde{\rho }}_{i\text{},T1}\right)}\hfill \\ \hfill & +\frac{1}{N}{\displaystyle \sum _{i\in r}{x}_i^T{T}_1^{-1}{\displaystyle \sum _{\begin{array}{l}k\in r\\ k\ne i\end{array}}{x}_k\left(d_i{d}_k-\frac{1}{{\pi }_{ik}}\right)}}\}.\hfill \end{array}$$

Appendix B

Analytic approximation to the bias of Type 2 ${\tilde{R}}_{{\tilde{\rho }}_{T2}}$ estimators

The strategy to compute an analytical bias adjustment for ${\tilde{S}}_{{\tilde{\rho }}_{T2}}^2$ is to first approximate ${\tilde{\rho }}_{i\text{},T2}$ by a linear estimator using Taylor linearization techniques. Next, compute an approximate bias adjustment for ${\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\text{},$ by inserting the linear approximation for ${\tilde{\rho }}_{i\text{},T2}$ into ${\widehat{m}}_2.$

In the following, define, for $j=1,\dots ,p$ and $j^{\prime }=1,\dots ,p,$ the estimated totals

$${\widehat{t}}_0={\displaystyle {\sum }_s{d}_k{r}_k},{\widehat{t}}_{j{j}^{\prime }}={\displaystyle {\sum }_s{d}_k{r}_k{z}_{jk}{z}_{j^{\prime }k}},\text{and}{\widehat{t}}_j={\displaystyle {\sum }_s{d}_k{r}_k{x}_{jk}},$$

where $z_k=\left(x_k-{\overline{x}}_U\right)$ and $z_{jk}=\left(x_{jk}-{\overline{x}}_{jU}\right).$ Let $\widehat{t}$ be a $p$ -vector with components ${\widehat{t}}_j,$ and $\widehat{F}$ be the symmetric $\left(p\times p\right)$ -matrix with elements ${\widehat{t}}_{j{j}^{\prime }}.$ We may write

$${\tilde{\rho }}_{i\text{},T2}=x_i^T{\left[N{\widehat{t}}_0^{-1}\widehat{F}+N{\overline{x}}_U{\overline{x}}_U^T\right]}^{-1}\widehat{t}=x_i^T{\widehat{T}}_2^{-1}\widehat{t}.$$

Define now the population totals

$$t_0={\displaystyle {\sum }_U{\rho }_k},F={\displaystyle {\sum }_U{\rho }_k}{z}_k{z}_k^T,\text{and}t={\displaystyle {\sum }_U{\rho }_k{x}_k}.$$

Notice that ${\widehat{t}}_0$ is unbiased for $t_0,$ $\widehat{F}$ is unbiased for $F,$ and $\widehat{t}$ is unbiased for $t.$ Let $T_2=N{t}_0^{-1}F+N{\overline{x}}_U{\overline{x}}_U^T.$

Proposition 1. The estimator ${\tilde{\rho }}_{i\text{},T2}$ defined in (2.7) may be approximated by

$${\tilde{\rho }}_{i\text{},T2}\cong x_i^T{T}_2^{-1}\left(N{t}_0^{-2}F\right)T_2^{-1}t\left({\widehat{t}}_0-t_0\right)-x_i^T{T}_2^{-1}N{t}_0^{-1}\left(\widehat{F}-F\right)T_2^{-1}t+x_i^T{T}_2^{-1}\widehat{t}.$$

Proof. Following standard Taylor linearization (see Särndal, Swensson and Wretman, 1992 and Bethlehem, 1988), the estimator ${\tilde{\rho }}_{i\text{},T2}$ may be approximated by

$${\tilde{\rho }}_{i\text{},T2}\cong {\rho }_{i\text{},T2}^{*}+a_0\left({\widehat{t}}_0-t_0\right)+{\displaystyle \sum _{j=1}^p{\displaystyle \sum _{j^{\prime }\le j}{a}_{j{j}^{\prime }}\left({\widehat{t}}_{j{j}^{\prime }}-t_{j{j}^{\prime }}\right)}}+{\displaystyle \sum _{j=1}^p{a}_j\left({\widehat{t}}_j-t_j\right)},(\text{B}\text{.1})$$

where ${\rho }_{i\text{},T2}^{*}=x_i^T{T}_2^{-1}t,$ and

$$\begin{array}{lll}{a}_0\hfill & ={\frac{\partial {\tilde{\rho }}_{i\text{},T2}}{\partial {\widehat{t}}_0}}_{|\begin{array}{c}{\widehat{t}}_0=t_0\\ \widehat{F}=F\\ \widehat{t}=t\end{array}}\hfill & =x_i^T\left[-{\widehat{T}}_2^{-1}\left(-N{\widehat{t}}_0^{-2}\widehat{F}\right){\widehat{T}}_2^{-1}\right]{\widehat{t}}_{|\begin{array}{c}{\widehat{t}}_0=t_0\\ \widehat{F}=F\\ \widehat{t}=t\end{array}}=x_i^T{T}_2^{-1}\left(N{t}_0^{-2}F\right)T_2^{-1}t,\hfill \\ a_{j{j}^{\prime }}\hfill & ={\frac{\partial {\tilde{\rho }}_{i\text{},T2}}{\partial {\widehat{t}}_{j{j}^{\prime }}}}_{|\begin{array}{c}{\widehat{t}}_0=t_0\\ \widehat{F}=F\\ \widehat{t}=t\end{array}}\hfill & =-x_i^T{T}_2^{-1}\left(N{t}_0^{-1}{\Lambda }_{j{j}^{\prime }}\right)T_2^{-1}t,\hfill \\ a_j\hfill & ={\frac{\partial {\tilde{\rho }}_{i\text{},T2}}{\partial {\widehat{t}}_j}}_{|\begin{array}{c}{\widehat{t}}_0=t_0\\ \widehat{F}=F\\ \widehat{t}=t\end{array}}\hfill & =x_i^T{T}_2^{-1}{\lambda }_j,\hfill \end{array}$$

where ${\Lambda }_{j{j}^{\prime }}$ is a $\left(p\times p\right)-$ matrix with ones in positions $\left(j,j^{\prime }\right)$ and $\left(j^{\prime },j\right)$ and zeros elsewhere and ${\lambda }_j$ is a $p$ -vector with the $j^{\text{th}}$ component equal to one and zeros elsewhere. Inserting the partial derivatives into (B.1) gives the result.

Proposition 2. Under simple random sampling, an approximate bias for ${\tilde{S}}_{{\tilde{\rho }}_{T2}}^2$ with respect to the joint distribution of sampling design and the response mechanism is given by

$$\begin{array}{ll}{B}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)\hfill & =\frac{N}{N-1}\{t_0^{-2}\frac{N}{n}{\displaystyle \sum _U{c}_i{\rho }_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}}\hfill \\ \hfill & -t_0^{-1}\frac{N}{n}{\displaystyle \sum _U{b}_i{\rho }_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}{z}_i{z}_i^T{T}_2^{-1}t}+\frac{1}{n}{\displaystyle \sum _U{\rho }_i{x}_i^T{T}_2^{-1}{x}_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}}\hfill \\ \hfill & +\frac{n-1}{n\left(N-1\right)}{\displaystyle \sum _U{\rho }_i{\rho }_{i\text{},T2}^{*}}-\left(1-\frac{n}{N}\right)\frac{S_{\rho }^2}{n}-\frac{{\overline{\rho }}_U}{n}+\frac{1}{nN}{\displaystyle \sum _U{\rho }_i^2}-\frac{1}{N}{\displaystyle \sum _U{\rho }_i^2}\},\hfill \end{array}$$

where $c_i=x_i^T{T}_2^{-1}F{T}_2^{-1}t,$ $b_i=x_i^T{T}_2^{-1}$ and ${\rho }_{i\text{},T2}^{*}=x_i^T{T}_2^{-1}t.$

A response-set based estimator of $B^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)$ is

$$\begin{array}{ll}{\tilde{B}}_{{\tilde{\rho }}_{T2}}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)\hfill & =\frac{N}{N-1}\{\frac{1}{n_r^2}{\displaystyle \sum _r\left\{1-\frac{n-1}{N-1}{\tilde{\rho }}_{i\text{},T2}\right\}}{x}_i^T{\widehat{T}}_2^{-1}\widehat{F}{\widehat{T}}_2^{-1}\widehat{t}\hfill \\ \hfill & -\frac{N}{n{n}_r}{\displaystyle \sum _r\left\{1-\frac{n-1}{N-1}{\tilde{\rho }}_{i\text{},T2}\right\}{x}_i^T{\widehat{T}}_2^{-1}{z}_i{z}_i^T{\widehat{T}}_2^{-1}\widehat{t}}\hfill \\ \hfill & +\frac{N}{n^2}{\displaystyle \sum _r\left\{1-\frac{n-1}{N-1}{\tilde{\rho }}_{i\text{},T2}\right\}}{x}_i^T{\widehat{T}}_2^{-1}{x}_i\hfill \\ \hfill & +\frac{n-1}{n^2\left(N-1\right)}{\displaystyle \sum _r{\tilde{\rho }}_{i\text{},T2}}-\left(1-\frac{n}{N}\right)\frac{{\tilde{S}}_{{\tilde{\rho }}_{T2}}^2}{n}-\frac{n_r}{n^2}\}.\hfill \end{array}$$

Proof. Thanks to Proposition 1, ${\widehat{m}}_2$ defined in Appendix A may be approximated as follows

$$\begin{array}{ll}{\widehat{m}}_2\hfill & =\frac{1}{N}{\displaystyle \sum _U{d}_i{s}_i{r}_i{\tilde{\rho }}_{i\text{},T2}}\hfill \\ \hfill & \cong \frac{1}{N}{\displaystyle \sum _U{d}_i{s}_i{r}_i{x}_i^T{T}_2^{-1}\left(N{t}_0^{-2}F\right)T_2^{-1}t\left({\widehat{t}}_0-t_0\right)}\hfill \\ \hfill & -\frac{1}{N}{\displaystyle \sum _U{d}_i{s}_i{r}_i{x}_i^T{T}_2^{-1}N{t}_0^{-1}\left(\widehat{F}-F\right)T_2^{-1}t}+\frac{1}{N}{\displaystyle \sum _U{d}_i{s}_i{r}_i{x}_i^T{T}_2^{-1}\widehat{t}}\hfill \\ \hfill & =:A+B+C.\hfill \end{array}$$

The expected values of the terms $A,B,$ and $C$ are

$$\begin{array}{ll}E\left(A\right)\hfill & =t_0^{-2}{\displaystyle \sum _{i\in U}{c}_i{d}_i{\rho }_i}+t_0^{-2}{\displaystyle \sum _{i\in U}{c}_i{d}_i{\displaystyle \sum _{k\ne i}{d}_k{\rho }_i{\rho }_k{\pi }_{ik}}}-t_0^{-1}{\displaystyle \sum _{i\in U}{c}_i{\rho }_i},\hfill \\ E\left(B\right)\hfill & =-t_0^{-1}{\displaystyle \sum _{i\in U}{d}_i{b}_i{\rho }_i{z}_i{z}_i^T{T}_2^{-1}t}-t_0^{-1}{\displaystyle \sum _{i\in U}{d}_i{b}_i{\displaystyle \sum _{k\ne i}{d}_k{\rho }_i{\rho }_k{\pi }_{ik}{z}_k{z}_k^T{T}_2^{-1}t}}+t_0^{-1}{\displaystyle \sum _{i\in U}{\rho }_i{b}_{i}F{T}_2^{-1}t},\hfill \end{array}$$

and

$$E\left(C\right)=\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_2^{-1}{x}_i}+\frac{1}{N}{\displaystyle \sum _{i\in U}{d}_i{\rho }_i{x}_i^T{T}_2^{-1}{\displaystyle \sum _{k\ne i}{d}_k{\rho }_k{\pi }_{ik}{x}_k}}.$$

It follows that, under simple random sampling, $E\left({\widehat{m}}_2\right)$ becomes

$$\begin{array}{ll}{E}^{\text{SRS}}\left({\widehat{m}}_2\right)\hfill & =t_0^{-2}\frac{N}{n}{\displaystyle \sum _U{c}_i{\rho }_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}}-t_0^{-1}\frac{N}{n}{\displaystyle \sum _U{b}_i{\rho }_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}}{z}_i{z}_i^T{T}_2^{-1}t\hfill \\ \hfill & +\frac{1}{n}{\displaystyle \sum _U{\rho }_i{x}_i^T{T}_2^{-1}{x}_i\left\{1-\frac{n-1}{N-1}{\rho }_i\right\}+\frac{n-1}{n\left(N-1\right)}}{\displaystyle \sum _U{\rho }_i{\rho }_{i\text{},T2}^{*}}.\hfill \end{array}$$

So the total bias under simple random sampling is obtained by inserting $E^{\text{SRS}}\left({\widehat{m}}_2\right)$ computed above into (A.1) and following the proof in Appendix A for the other terms.

The response-set based estimator ${\tilde{B}}_{{\tilde{\rho }}_{T2}}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)$ of $B^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)$ is obtained by substituting $t_0$ with ${\widehat{t}}_0=N{n}_r/n,$ $F$ with $\widehat{F}=N{n}^{-1}{\displaystyle {\sum }_r{z}_k{z}_k^T},$ $T_2$ with ${\widehat{T}}_2=N{\widehat{t}}_0^{-1}\widehat{F}+N{\overline{x}}_U{\overline{x}}_U^T,$ and $t$ with $\widehat{t}=N{n}^{-1}{\displaystyle {\sum }_r{x}_k}.$

Note that the bias adjustment ${\tilde{B}}_{{\tilde{\rho }}_{T2}}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)$ corresponds to “plugging-in” Type 2 quantities $({\tilde{\rho }}_{i\text{},T2}$ instead of ${\tilde{\rho }}_{i\text{},T1},$ matrix ${\widehat{T}}_2$ instead of $T_1,$ and ${\tilde{S}}_{{\tilde{\rho }}_{T2}}^2$ instead of ${\tilde{S}}_{{\tilde{\rho }}_{T1}}^2)$ into the analytical bias adjustment ${\tilde{B}}_{{\tilde{\rho }}_{T1}}^{\text{SRS}}\left({\tilde{S}}_{{\tilde{\rho }}_{T1}}^2\right)$ developed for ${\tilde{S}}_{{\tilde{\rho }}_{T1}}^2$ with two additional terms due to the linearization of ${\widehat{T}}_2$ .

More generally, the Horvitz-Thompson response-set estimator under complex sampling for the bias adjustment of Type 2 population-based R-indicator is given by

$$\begin{array}{ll}{\tilde{B}}_{{\tilde{\rho }}_{T2}}\left({\tilde{S}}_{{\tilde{\rho }}_{T2}}^2\right)\hfill & =\frac{N}{N-1}\{\frac{1}{N}{\displaystyle \sum _{i\in r}{d}_i\left(d_i-{\tilde{\rho }}_{i\text{},T2}\right)x_i^T{\widehat{T}}_2^{-1}{x}_i}-\frac{1}{N^2}{\displaystyle \sum _{i\in r}{d}_i^3{\Delta }_{ii}{\tilde{\rho }}_{i,T2}}-\frac{1}{N^2}{\displaystyle \sum _{i\in r}{\displaystyle \sum _{\begin{array}{l}k\in r\\ k\ne i\end{array}}{d}_i{d}_k\frac{{\Delta }_{ik}}{{\pi }_{ik}}}}\hfill \\ \hfill & -\frac{1}{N^2}{\displaystyle \sum _{i\in r}{d}_i^2\left(1-{\tilde{\rho }}_{i\text{},T2}\right)}\text{+}\frac{1}{N}{\displaystyle \sum _{i\in r}{x}_i^T{\widehat{T}}_2^{-1}{\displaystyle \sum _{\begin{array}{l}k\in r\\ k\ne i\end{array}}{x}_k\left(d_i{d}_k-\frac{1}{{\pi }_{ik}}\right)}}\hfill \\ \hfill & +{\left({\displaystyle \sum _{k\in r}{d}_k}\right)}^{-2}{\displaystyle \sum _{i\in r}{d}_i^2{x}_i^T{\widehat{T}}_2^{-1}\widehat{F}{\widehat{T}}_2^{-1}\widehat{t}\text{+}{\left({\displaystyle \sum _{k\in r}{d}_k}\right)}^{-2}{\displaystyle \sum _{i\in r}{d}_i{x}_i^T{\widehat{T}}_2^{-1}\widehat{F}{\widehat{T}}_2^{-1}\widehat{t}{\displaystyle \sum _{k\ne i}{d}_k}}}\hfill \\ \hfill & {}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{{}^{}}}}}}}}}}}}}}}}}-{\left({\displaystyle \sum _{k\in r}{d}_k}\right)}^{-1}{\displaystyle \sum _{i\in r}{d}_i^2{x}_i^T{\widehat{T}}_2^{-1}{z}_i{z}_i^T{\widehat{T}}_2^{-1}\widehat{t}-{\left({\displaystyle \sum _{k\in r}{d}_k}\right)}^{-1}{\displaystyle \sum _{i\in r}{d}_i{x}_i^T{\widehat{T}}_2^{-1}{\displaystyle \sum _{k\ne i}{d}_k{z}_k{z}_k^T{\widehat{T}}_2^{-1}\widehat{t}}}}\}.\hfill \end{array}$$

References

Beaumont, J.-F., Bocci, C. and Haziza, D. (2014). An adaptive data collection procedure for call prioritization. Journal of Official Statistics, 30, 607-621.

Bethlehem, J. (1988). Reduction of nonresponse bias through regression estimation. Journal of Official Statistics, 4, 251-260.

Booth, J.G., Butler, R.W. and Hall, P. (1994). Bootstrap methods for finite populations. Journal of the American Statistical Association, 89 (428), 1282-1289.

Brick, J.M., and Jones, M.E. (2008). Propensity to respond and nonresponse bias. METRON – International Journal of Statistics, LXVI (1), 51-73.

Copas, J.B. (1983). Regression, prediction and shrinkage. Journal of the Royal Statistical Society, Series B, 45, 311-354.

Copas, J.B. (1993). The shrinkage of point scoring methods. Journal of the Royal Statistical Society, Series C, 42, 315-331.

De Heij, V., Schouten, B. and Shlomo, N. (2015). RISQ manual 2.1. Tools in SAS and R for the computation of R-indicators and partial R-indicators, available at www.risq-project.eu.

Deville, J.-C., and Särndal, C.-E. (1992). Calibration estimators in survey sampling. Journal of the American Statistical Association, 87, 376-382.

Efron, B., and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York: Chapman and Hall.

Kreuter, F. (2013). Improving Surveys with Process and Paradata, Edited monograph, New Jersey: John Wiley & Sons, Inc.

Little, R.J.A. (1986). Survey nonresponse adjustments for estimates of means. International Statistical Review, 54, 139-157.

Little, R.J.A. (1988). Missing-data adjustments in large surveys. Journal of Business and Economic Statistics, 6, 287-301.

Little, R.J.A., and Rubin, D.B. (2002). Statistical Analysis with Missing Data, Hoboken, New Jersey: John Wiley & Sons, Inc.

Lundquist, P., and Särndal, C.-E. (2013). Aspects of responsive design with applications to the Swedish Living Conditions Survey. Journal of Official Statistics, 29 (4), 557-582.

MOA (2015). User Instruction Gold Standard, Dutch Market Research Association, available at www.moaweb.nl/sevrices/services/gouden-standaard.html.

Rosenbaum, P.R., and Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.

Särndal, C.-E. (2011). The 2010 Morris Hansen Lecture: Dealing with survey nonresponse in data collection, in estimation. Journal of Official Statistics, 27 (1), 1-21.

Särndal, C.-E., and Lundquist, P. (2014). Accuracy in estimation with nonresponse: A function of degree of imbalance and degree of explanation. Journal of Survey Statistics and Methodology, 2 (4), 361-387.

Särndal, C.-E., and Lundström, S. (2005). Estimation in Surveys with Nonresponse, New York: John Wiley & Sons, Inc.

Särndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling, New York: Springer.

Schouten, B., and Shlomo, N. (2017). Selecting adaptive survey design strata with partial R-indicators. International Statistical Review, 85 (1), 143-163.

Schouten, B., Calinescu, M. and Luiten, A. (2013). Optimizing quality of response through adaptive survey designs. Survey Methodology, 39, 1, 29-58. Paper available at https://www150.statcan.gc.ca/n1/pub/12-001-x/2013001/article/11824-eng.pdf.

Schouten, B., Cobben, F. and Bethlehem, J. (2009). Indicators for the representativeness of survey response. Survey Methodology, 35, 1, 101-113. Paper available at https://www150.statcan.gc.ca/n1/pub/12-001-x/2009001/article/10887-eng.pdf.

Schouten, B., Shlomo, N. and Skinner, C. (2011). Indicators for monitoring and improving representativeness of response. Journal of Official Statistics, 27, 231-253.

Schouten, B., Cobben, F., Lundquist, P. and Wagner, J. (2016). Does more balanced survey response imply less non-response bias? Journal of the Royal Statistical Society, Series A, 179 (3), 727-748.

Schouten, B., Bethlehem, J., Beulens, K., Kleven, Ø., Loosveldt, G., Rutar, K., Shlomo, N. and Skinner, C. (2012). Evaluating, comparing, monitoring and improving representativeness of survey response through R-indicators and partial R-indicators. International Statistical Review, 80 (3), 382-399.

Shlomo, N., Skinner, C. and Schouten, B. (2012). Estimation of an indicator of the representativeness of survey response. Journal of Statistical Planning and Inference, 142, 201-211.

Van der Laan, D., and Bakker, B. (2015). Indicators for the representativeness of linked sources, NTTS 2015 Proceedings, available at https://ec.europa.eu/eurostat/cros/system/files/NTTS2015%20proceedings.pdf.

Wagner, J. (2012). A comparison of alternative indicators for the risk of nonresponse bias. Public Opinion Quarterly, 76 (3), 555-575.

Wagner, J. (2013). Adaptive contact strategies in telephone and face-to-face surveys. Survey Research Methods, 7 (1), 45-55.

Wagner, J., and Hubbard, F. (2014). Producing unbiased estimates of propensity models during data collection. Journal of Survey Statistics and Methodology, 2, 323-342.

Wolter, K.M. (2007). Introduction to Variance Estimation, 2^nd Ed. New York: Springer.

• Previous
• Article main page
• Next