Empirical likelihood inference for missing survey data under unequal probability sampling
Section 5. Simulation study

• Previous
• Next

We carried out simulation studies to compare the performance of the proposed BELP and BELR intervals to that of the usual bootstrap intervals based on $\sqrt{n}\left({\widehat{\mu }}_b-\widehat{\mu }\right)$ and $R_b\left(\widehat{\mu }\right).$ We will refer to the proposed proper intervals as propBELP and propBELR, and refer to the usual naive ones as naiveBELP and naivBELR. We also report the results for EL ratio intervals (SELR) with estimated scaling constant $c_N$ based on the limiting distribution of the EL ratio established in Theorem 2.

To generate population data, we followed the simulation settings of Wu and Rao (2006) and used the model

$$y_i\mathrm{<mo>=</mo>}{\beta }_0+{\beta }_1{x}_i+\tau {ϵ}_i(5.1)$$

for $i\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}N,$ where ${\beta }_0\mathrm{<mo>=</mo>}{\beta }_1\mathrm{<mo>=</mo>\; 1},$ $x_i$ were generated from an exponential distribution with rate 1, and ${ϵ}_i$ were generated from ${\chi }_1^2-1$ distribution to have a zero mean. The $x_i$ were used as the size measure for selecting PPS samples. The value of $\tau$ was chosen so that the correlation $\rho :=\rho \left(y\mathrm{,}x\right)$ between the variable of interest $y$ and the size measure $x$ reaches a certain level. The finite populations so generated were held fixed under repeated independent simulation runs. In each particular case of the simulation study, the number of simulation runs was set to be 10,000, and the number of bootstrap replications in each simulation run was set to $B\mathrm{<mo>=</mo>}$3,000. In Sections 5.1 and 5.2, we focus on constructing 95% CIs for the population mean of $y;$ in Section 5.3, we consider 95% CIs for the population distribution function of $y$ evaluated at given values.

5.1  Case 1: Single imputation class

We first considered the simple case where there is only one imputation class for the entire population. We set the population size to be $N\mathrm{<mo>=</mo>}$5,000, and drew PPS samples with replacement from the population generated from model (5.1). The $\tau$ value in model (5.1) was chosen such that $\rho \mathrm{<mo>=</mo>}$0.3. The sample size was set to $n\mathrm{<mo>=</mo>}$80 and 250, and for each sample size, we examined two settings of response probability, $P\mathrm{<mo>=</mo>}$0.4 and 0.8. For each combination of $n$ and $P,$ we used two settings for the number of draws in imputation, $J\mathrm{<mo>=</mo>\; 1}$ and 5. Note that, unlike the original setting used by Wu and Rao (2006), where a constant was added to all the size measures to avoid extremely small values, we intentionally avoided adding any constant to the size measures to test the case where inclusion probabilities contain very small values and differ largely in size.

The coverage probabilities and average lengths of the proposed and naive bootstrap-EL intervals for the population mean are shown in Table 5.1. When $J\mathrm{<mo>=</mo>\; 1},$ it is clear that in all the cases the proposed propBELR intervals have the most accurate coverage probabilities and shorter average lengths than the naiveBELR intervals. The naivBELR intervals show 1%-2% of over-coverage relative to the 95% nominal coverage. Both propBELP and naivBELP intervals perform much worse than the BELR intervals in terms of coverage probability, and show serious under-coverage. The propBELP intervals, however, have much better coverage probabilities than the naivBELP intervals. The naivBELP and propBELP intervals can be shown to have exactly the same lengths, so their lengths are shown in a single column titled “BELP” under “Average Length” in Table 5.1 and also in the other tables. Given that both the propBELP and naiveBELP intervals have notable under-coverage, their lengths are not comparable to those of the BELR intervals. Moreover, when sample size increases, all the BELP intervals show improved coverage probabilities, while the coverages of the BELR intervals are stable in terms of change in the sample size. The SELR intervals also show significant under-coverage, although they have slightly better coverage than the propBELP intervals. The average lengths of all the intervals decrease as the sample size increases.

When $J$ is increased to 5, the differences between the naivBELR and propBELR intervals, and those between the naivBELP and propBELP intervals, become nearly negligible. This observation agrees with our theoretical finding given in Remark 1, that is, as $J$ increases, the differences between the proposed intervals and naive intervals will diminish as $n\to \infty .$ The average lengths of the propBELR intervals remain slightly shorter than those of the naivBELR intervals. The coverage probabilities of both the propBELR intervals and the SELR intervals do not change substantially as $J$ increases from 1 to 5.

A striking observation is that the BELP intervals perform much worse than the BELR intervals. Unreported simulation studies suggest that this is likely due to the use of unequal probability sampling. When simple random sampling is used, the performance of the propBELP interval is found to be close to that of the propBELR interval, which is also observed by Cai et al. (2019). In addition, if a constant is added to the size measures to avoid extremely small values, we observed that the performance of the proposed propBELP interval increases greatly while that of the naivBELP increases slightly. This is further illustrated in the simulation study presented in Section 5.2. A clear advantage of the proposed BELR interval over the proposed BELP interval is that the performance of the BELR interval is not significantly affected by the variation in inclusion probabilities.

The above simulation results show that the naivBELR intervals have similar performance to the propBELR intervals with a slight over-coverage. Does this imply that the naivBELR interval is also asymptotically correct? To answer this question, we conducted a large-sample simulation study. In this study, we set the population size to be $N\mathrm{<mo>=</mo>}$25,000, and considered sample sizes $n\mathrm{<mo>=</mo>}$500, 1,000, 1,500, 2,000 and 3,000. The population data were generated from model (5.1) and PPS samples were drawn with replacement from the population repeatedly and independently. The response probability $P$ was fixed at 0.8 and the number of random draws $J$ in the imputation was set to 1.

The simulation results based on large samples are reported in Table 5.2. In all the cases, the coverage probabilities of the propBELR intervals are precisely 95%. However, the naivBELR intervals always exihbit 1%-1.5% of over-coverage regardless of how large the sample size is. This shows that the naivBELR intervals are asymptotically biased. The coverage probabilities of the propBELP intervals improve as the sample size increases, and when $n$ is beyond 2,000, they are satisfactorily close to the nominal coverage of 95%. The naivBELP intervals, however, have lower than 90% coverage probabilities in all the cases, implying that they are asymptotically incorrect. The SELR intervals also improve as the sample size $n$ increases. However, they improve slower than the propBELP intervals: when $n\mathrm{<mo>=</mo>}$250, the SELR intervals have slightly better coverage (Table 5.1), but when $n$ increases to 3,000, the coverage probability of the propBELP intervals becomes 94.5% while that of the SELR intervals is only 93.0%.

5.2  Case 2: Multiple imputation classes

We now turn to the case of multiple imputation classes, i.e., $K\mathrm{<mo>></mo>\; 1.}$ We still focus on constructing 95% CIs for the population mean of $y.$ We drew Rao-Sampford (Rao, 1965; Sampford, 1967) PPS samples without replacement from a finite population generated from model (5.1). In this study, we added the constant 1 to all the size measures generated from the standard exponential distribution to avoid extremely small values. We considered two settings of the sample size and population size combinations: (a) $n\mathrm{<mo>=</mo>}$150 and $N\mathrm{<mo>=</mo>}$5,000, corresponding to a sampling fraction of 3%, and (b) $n\mathrm{<mo>=</mo>}$500 and $N\mathrm{<mo>=</mo>}$50,000, corresponding to a sampling fraction of 1%. The reason that we reduced the sampling fraction in setting (b) is that for the large sample size $n\mathrm{<mo>=</mo>}$500, the rejective Rao-Sampford PPS samples are difficult to generate when the sampling fraction is greater than 1%. For each sample size setting, we considered two levels of correlation between $y$ and the size measure, $\rho \mathrm{<mo>=</mo>}$0.3 and 0.8. Under each of the above sample size and correlation setting, we tested three cases for the number of random draws in the imputation: $J\mathrm{<mo>=</mo>}$1, 3 and 5.

We set the number of imputation classes to $K\mathrm{<mo>=</mo>}$3, and use the models considered by Fang, Hong and Shao (2009) to generate the class variable $z$ and response probabilities for different imputation classes. The class variable $z$ was generated with a proportional-odds model for all population units $i\mathrm{<mo>=</mo>\; 1,}\dots ,N,$

$$\log \frac{P\left(z_i\le k|y_i\right)}{P\left(z_i\le k|y_i\right)}\mathrm{<mo>=</mo>}k+b{y}_i\text{for}k\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}K-1$$

with $b\mathrm{<mo>=</mo>}$-0.2. For each sampled unit $i\in s,$ the response probability for $y_i$ was generated according to the model

$$P\left({\delta }_i|z_i\mathrm{<mo>=</mo>}k\right)\mathrm{<mo>=</mo>}\frac{\exp \left(\text{-}0.1+\gamma k\right)}{1+\exp \left(\text{-}0.1+\gamma k\right)}$$

with $\gamma \mathrm{<mo>=</mo>}$0.7, where $k\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}K.$ This model yields response probabilities $P_1\mathrm{<mo>=</mo>}$0.646, $P_2\mathrm{<mo>=</mo>}$0.786, and $P_3\mathrm{<mo>=</mo>}$0.881.

The simulation results for the sample size $n\mathrm{<mo>=</mo>}$150 are reported in Table 5.3. In all the cases, the propBELR intervals have the most accurate coverage probabilities and shorter average lengths than the naivBELR intervals. The naivBELR intervals show over-coverage when $J\mathrm{<mo>=</mo>\; 1},$ and their coverages improve as $J$ increases. The coverage probabilities of the propBELP intervals are lower than the nominal level, but are much improved compared to the serious under-coverage that we have observed in the simulation study of Section 5.1 where no constant was added to the size measures to avoid extremely small values. The SELR intervals exhibit slight under-coverage when $\rho \mathrm{<mo>=</mo>}$0.3; they perform equally well as the propBELR intervals when $\rho \mathrm{<mo>=</mo>}$0.8. The naivBELP intervals perform the worst with significant under-coverage in all the cases.

The simulation results for the larger sample size $n\mathrm{<mo>=</mo>}$500 are shown in Table 5.4. As in the case of the smaller sample size $n\mathrm{<mo>=</mo>}$150, the propBELR and propBELP intervals outperform their naive counterparts, and the propBELR intervals perform better than the propBELP intervals in terms of coverage probability. Under both sample sizes, 150 and 500, the coverage probabilities of the propBELR intervals are nearly identical to the nominal level, and those of the propBELP intervals improve as the sample size increases, suggesting that the proposed BELR and BELP intervals are asymptotically correct. However, the coverage probabilities of the naivBELR and naivBELP intervals do not improve as the sample size increases, indicating that they are asymptotically incorrect. The SELR intervals have approximately the same performance as the propBELR intervals in terms of both coverage probabilities and average lengths under the large sample size setting.

It is worth noting that the average lengths of all the intervals are shorter when the correlation between $y$ and the size measure is higher. This agrees with the classical estimation theory of survey sampling that using a size measure that is highly correlated with the variable of interest leads to small variance of the estimator.

5.3  Case 3: CIs for population distribution function

We now present the simulation results on 95% CIs of the finite-population distribution function of $y$ at a given value $t,$ $F_N\left(t\right)\mathrm{<mo>=</mo>}{N}^{-1}{\displaystyle {\sum }_{i\in U}1\left(y_i\le t\right)}.$ As noted in the Introduction, $F_n\left(t\right)$ can be represented as the solution to the estimating equation ${\displaystyle {\sum }_{i\in U}}\left(h\left(y_i\right)-\mu \right)\mathrm{<mo>=</mo>\; 0}$ in $\mu$ by taking $h\left(y\right)\mathrm{<mo>=</mo>}1\left(y\le t\right).$ We took the same settings for data generation as used in simulation Case 2 in Section 5.2. For each $\rho$ value, we considered three $t$ values fixed at the ${25}^{\text{th}},$ ${50}^{\text{th}}$ and ${75}^{\text{th}}$ percentiles of the data-generating distribution of $y$ implied by model (5.1). For $\rho \mathrm{<mo>=</mo>}$0.3, these $t$ values are 0.81, 1.95 and 3.99, and for $\rho \mathrm{<mo>=</mo>}$0.8, they are 2.09, 2.68 and 3.56. The sample size was set to $n\mathrm{<mo>=</mo>}$80.

The simulation results for the cases $\rho \mathrm{<mo>=</mo>}$0.3 and $\rho \mathrm{<mo>=</mo>}$0.8 are shown in Table 5.5 and 5.6, respectively. Consistent to what we have observed in simulation Case 2, the propBELR intervals still perform the best among the competitors. The SELR intervals show slight under-coverage compared to the propBELR intervals when $\rho \mathrm{<mo>=</mo>}$0.3 at the ${75}^{\text{th}}$ percentile $(t\mathrm{<mo>=</mo>}$3.99); otherwise they perform similarly. The naivBELR intervals again exhibit approximately 1% of over-coverage when $J\mathrm{<mo>=</mo>\; 1}$ and improve as $J$ increases. The propBELP intervals perform better than the naivBELP intervals, but both show significant under-coverage.

• Previous
• Article main page
• Next