Variance estimation under monotone non-response for a panel survey
Section 5. A simulation study

• Previous
• Next

In this section, several artificial populations are generated according to the model described in Section 5.1. In Section 5.2, we consider several estimators for a change between totals, which illustrates the heuristic reasoning in Section 4. A Monte Carlo experiment is presented in Section 5.3, and several variance estimators for estimating a total, a ratio or a parameter change are compared. The results from Tables 5.1 and 5.2 are readily reproducible using the R code provided in the Supplementary Material.

5.1  Simulation set-up

We consider seven populations of size 10,000, each containing three variables of interest $y_{i1},$ $y_{i2}$ and $y_{i3}$ observed at times $t\mathrm{<mo>=</mo>\; 1,}2$ and 3, respectively. The variables of interest are generated according to the superpopulation model

$$y_{i1}\mathrm{<mo>=</mo>}{\alpha }^0+{\alpha }^a{x}_{ai}+{\alpha }^b{x}_{bi}+\sigma u_{i1}\mathrm{,}(5.1)$$

$$y_{i2}\mathrm{<mo>=</mo>}\rho y_{i1}+\sigma u_{i2}\mathrm{,}(5.2)$$

$$y_{i3}\mathrm{<mo>=</mo>}\rho y_{i2}+\sigma u_{i3}\mathrm{.}(5.3)$$

The auxiliary variables $x_{ai}$ and $x_{bi}$ are independently generated from a Gamma distribution with shape and scale parameters 2 and 1. Two auxiliary variables $x_{ci}$ and $x_{di},$ not related to the variables of interest, are generated similarly. The variables $u_{i1},$ $u_{i2}$ and $u_{i3}$ are independently generated according to a standard normal distribution. We use ${\alpha }^0\mathrm{<mo>=</mo>\; 10},$ ${\alpha }^a\mathrm{<mo>=</mo>}{\alpha }^b\mathrm{<mo>=</mo>\; 5}$ and $\sigma \mathrm{<mo>=</mo>\; 10},$ which leads to a coefficient of determination $\left(R^2\right)$ in model (5.1) approximately equal to 0.50. The parameter $\rho$ is set to 0, 0.2, 0.4, 0.6, 0.8, 1.0 and 1.2 for populations 1 to 7, respectively.

For each population, a simple random sample $s_0$ of size $n=$ 1,000 is selected. Three non-response phases are then successively simulated. At each phase $\delta \mathrm{<mo>=</mo>\; 1,}\mathrm{2,}3,$ the sub-sample of respondents $s_{\delta }$ is obtained by Poisson sampling with a response probability $p_i^{\delta }$ for unit $i,$ defined as

$$\text{logit}\left(p_i^{\delta }\right)\mathrm{<mo>=</mo>}{\beta }^{\delta 0}+{\beta }^{\delta a}{x}_{ai}+{\beta }^{\delta b}{x}_{bi}\mathrm{.}(5.4)$$

We use ${\beta }^{\delta 0}\mathrm{<mo>=</mo>}-1$ at each phase $\delta \mathrm{<mo>=</mo>\; 1,}\mathrm{2,}3.$ For $\delta \mathrm{<mo>=</mo>\; 1},$ we use ${\beta }^{1a}\mathrm{<mo>=</mo>}{\beta }^{1b}=$ 0.60, which corresponds to an average response rate of 0.75. For $\delta \mathrm{<mo>=</mo>\; 2,}3,$ we use ${\beta }^{\delta a}\mathrm{<mo>=</mo>}{\beta }^{\delta b}\mathrm{<mo>=</mo>}$ 0.75, which corresponds to an average response rate of 0.81. Inside each sub-sample $s_{\delta },$ the estimated response probabilities ${\widehat{p}}_i^{\delta }$ are obtained by means of an unweighted logistic regression.

5.2  Comparison of estimators for a difference of totals

In this section, we are interested in comparing the accuracy of two estimators for a difference of totals $\Delta \left(u\to t\right)$ for $u\mathrm{<mo>=</mo>\; 1}$ and $t\mathrm{<mo>=</mo>\; 2},$ for $u\mathrm{<mo>=</mo>\; 1}$ and $t\mathrm{<mo>=</mo>\; 3},$ and for $u\mathrm{<mo>=</mo>\; 2}$ and $t\mathrm{<mo>=</mo>\; 3.}$ We consider the estimator ${\widehat{\Delta }}_{ut}\left(u\to t\right),$ which makes use of the whole appropriate sub-samples for variables $y_{iu}$ and $y_{it},$ and the estimator ${\widehat{\Delta }}_{tt}\left(u\to t\right),$ which makes use of the common sub-sample only. These two estimators are compared through the relative difference (RD) of their variances, which are defined as follows:

$$\text{RD}\left(u\to t\right)\mathrm{<mo>=</mo>\; 100}\times \frac{V\left\{{\widehat{\Delta }}_{ut}\left(u\to t\right)\right\}-V\left\{{\widehat{\Delta }}_{tt}\left(u\to t\right)\right\}}{V\left\{{\widehat{\Delta }}_{tt}\left(u\to t\right)\right\}}\mathrm{.}(5.5)$$

The true variances are replaced by their Monte Carlo approximation, obtained by repeating $B=$ 100,000 times the sample selection and the non-response phases.

The results are presented in Table 5.1. A positive RD indicates that the use of the common sample only leads to a more accurate estimator. As could be expected, the RD increases in all cases with $\rho ,$ that is, when the correlation between $y_{it}$ and $y_{iu}$ increases. For $u\mathrm{<mo>=</mo>\; 1}$ and $t\mathrm{<mo>=</mo>\; 2},$ and for $u\mathrm{<mo>=</mo>\; 2}$ and $t\mathrm{<mo>=</mo>\; 3},$ the estimator ${\widehat{\Delta }}_{tt}\left(u\to t\right)$ is more accurate for $\rho$ greater than 0.6. For $u\mathrm{<mo>=</mo>\; 1}$ and $t\mathrm{<mo>=</mo>\; 3},$ ${\widehat{\Delta }}_{tt}\left(u\to t\right)$ is more accurate for $\rho$ greater than 0.8.

5.3  Performances of the variance estimators

In this section, we consider the artificial population 5 $\left(\rho \mathrm{<mo>=</mo>}\text{0}\text{.8}\right)$ generated as described in Section 5.1. The sample selection by means of simple random sampling of size $n=$ 1,000 and the three non-response phases are applied $B=$ 5,000 times. We are interested in evaluating the variance estimators and the simplified variance estimators, in case of estimating a total, a ratio or a change in totals.

As for the total $Y\left(t\right),$ we consider at each time $t\mathrm{<mo>=</mo>\; 1,}\mathrm{2,}\mathrm{3,}$ three estimators. The estimator ${\widehat{Y}}_t$ makes use of the weights $d_{ti}={\pi }_i^{-1}{\left({\widehat{p}}_i^{1\to t}\right)}^{-1}\text{}.$ The estimator ${\widehat{Y}}_{wt}$ makes use of the weights $w_i,$ obtained by calibrating the weights $d_{ti}$ on the population size and on the totals of the auxiliary variables $x_{ai}$ and $x_{bi}.$ The estimator ${\widehat{Y}}_{\tilde{w}t}$ makes use of the weights ${\tilde{w}}_i,$ obtained by calibrating the weights $d_{ti}$ on the population size and on the totals of the auxiliary variables $x_{ci}$ and $x_{di}.$ The working model is therefore well-specified for ${\widehat{Y}}_{wt},$ but not for ${\widehat{Y}}_{\tilde{w}t}.$ The proposed variance estimator for ${\widehat{Y}}_t$ is obtained from equation (2.16), and the simplified variance estimator is obtained by plugging in (2.16) the simplified variance estimator for non-response given in (2.17). The proposed variance estimators for ${\widehat{Y}}_{wt}$ and ${\widehat{Y}}_{\tilde{w}t}$ are obtained from equation (3.8), and the simplified variance estimators are obtained by plugging in (3.8) the simplified variance estimator for non-response given in (3.9).

We are also interested in estimating the ratio $R\left(t\right)\mathrm{<mo>=</mo>}Y\left(t\right)/Y\left(1\right)$ for $t\mathrm{<mo>=</mo>\; 2,}3.$ At each time $t,$ we consider three estimators. The estimator ${\widehat{R}}_t$ makes use of the weights $d_i.$ The proposed variance estimator is obtained from equation (3.14), by using the estimated linearized variable $u_{it}\mathrm{<mo>=</mo>}{\left({\widehat{Y}}_1\right)}^{-1}\left(y_{ti}-{\widehat{R}}_t{y}_{1i}\right).$ The simplified variance estimator is obtained by plugging in (3.14) the simplified variance estimator for non-response given in (3.15). The estimators ${\widehat{R}}_{wt}$ and ${\widehat{R}}_{\tilde{w}t}$ make use of the calibrated weights $w_i$ and ${\tilde{w}}_i.$ The proposed variance estimators are obtained from equation (3.21). The simplified variance estimators are obtained by plugging in (3.21) the simplified variance estimator for non-response given in (3.22).

Finally, we are interested in estimating the change in totals $\Delta \left(1\to t\right)$ for $t\mathrm{<mo>=</mo>\; 2,}3.$ At each time $t,$ we consider three estimators. The estimator ${\widehat{\Delta }}_{tt}\left(1\to t\right)$ makes use of the weights $d_i.$ The proposed variance estimator is obtained from equation (4.8), and the simplified variance estimator is obtained by plugging in (4.8) the simplified variance estimator for non-response given in (4.9). The estimators ${\widehat{\Delta }}_{tt\mathrm{,}w}\left(1\to t\right)$ and ${\widehat{\Delta }}_{tt\mathrm{,}\tilde{w}}\left(1\to t\right)$ make use of the calibrated weights $w_i$ and ${\tilde{w}}_i.$ The proposed variance estimators are obtained from equation (4.8), by replacing $y_{it}-y_{iu}$ by the estimated residual for the weighted regression of $y_{it}-y_{iu}$ on the calibration variables. The simplified variance estimators are obtained by plugging in (4.8) the simplified variance estimator for non-response given in (4.9).

For a proposed variance estimator $\widehat{V},$ we computed the Monte Carlo Percent Relative Bias

$${\text{RB}}_{\text{mc}}\left(\widehat{V}\right)\mathrm{<mo>=</mo>\; 100}\times \frac{B^{-1}{\displaystyle {\sum }_{b\mathrm{<mo>=</mo>\; 1}}^B{\widehat{V}}^{\left(b\right)}}-V}{V}$$

where the global variance $V$ was approximated through an independent set of 100,000 simulations. To evaluate the contribution of some component ${\widehat{V}}_a$ into the variance estimator $\widehat{V},$ we computed the contribution (in percent)

$${\text{CONTR}}_{\text{mc}}\left({\widehat{V}}_a\right)\mathrm{<mo>=</mo>\; 100}\times \frac{\frac{1}{B}{\displaystyle {\sum }_{b\mathrm{<mo>=</mo>\; 1}}^B{\widehat{V}}_a^{\left(b\right)}}}{\frac{1}{B}{\displaystyle {\sum }_{b\mathrm{<mo>=</mo>\; 1}}^B{\widehat{V}}^{\left(b\right)}}}\mathrm{.}$$

To evaluate the simplified variance estimator for the non-response ${\widehat{V}}_{\text{simp}}^{\text{nr}},$ we computed the Monte Carlo Percent Relative Bias

$${\text{RB}}_{\text{mc}}\left({\widehat{V}}_{\text{simp}}^{\text{nr}}\right)\mathrm{<mo>=</mo>\; 100}\times \frac{B^{-1}{\displaystyle {\sum }_{b\mathrm{<mo>=</mo>\; 1}}^B{\widehat{V}}_{\text{simp}}^{\left(b\right)}}-V^{\text{nr}}}{V^{\text{nr}}}\mathrm{,}$$

where the variance $V^{\text{nr}}$ due to non-response was approximated through an independent set of 100,000 simulations.

The simulation results are presented in Table 5.2. The proposed variance estimator is almost unbiased in all cases. As could be expected, the contribution of the variance due to the sampling design decreases with time, as the number of respondents decreases and as the variance due to non-response becomes larger. The simplified variance estimator is highly biased for the variance due to non-response in case of ${\widehat{Y}}_t.$ The bias decreases quickly with time, but remains large at time $t\mathrm{<mo>=</mo>\; 3.}$ The simplified variance estimator is almost unbiased for a calibrated estimator when the working model is adequately specified, but is severely biased otherwise. This is consistent with our reasoning in Section 3.1. The simplified variance estimator is almost unbiased for the three estimators of the ratio, and for the calibrated estimators of the change in totals. In case of the non-calibrated estimator for the change in totals, the bias can be as high as 30%.

• Previous
• Article main page
• Next