Assessing the coverage of confidence intervals under nonresponse. A case study on income mean and quantiles in some municipalities from the 2015 Mexican Intercensal Survey
Section 3. Empirical study based on data from the populations MIC2015_Oax and MIC2015_Oax_trunc

• Previous
• Next

The population census considered in this work consisted of 208,101 inhabitants with complete responses in a vector $x$ of six auxiliary variables: age, educational level, employment status, gender, indigenous language, and marital status. The variable of interest $y$ corresponds to the monthly income.

A logistic regression with some two-way interactions was fitted to the 208,101 observations, with response variable $I_k\mathrm{<mo>=</mo>}1$ if an income value was given by individual $k,$ and $I_k\mathrm{<mo>=</mo>}0$ if it was not, and the vector $x$ of six explanatory variables. This model was then applied to the population of 161,296 individuals, which corresponds to those with $I_k\mathrm{<mo>=</mo>}1.$ This led to a set of response propensity values $\varphi \mathrm{<mo>=</mo>}\mathrm{<mo\; stretchy="false">(</mo>}{\varphi }_1\mathrm{,}\dots ,{\varphi }_{\text{161,296}}\mathrm{<mo\; stretchy="false">)</mo>\; .}$

The set of 161,296 respondents, with response propensities ${\varphi }_1\mathrm{,}\dots \mathrm{,}{\varphi }_{\text{161,296}},$ is referred to as MIC2015_Oax population. The distribution of income in this population is highly asymmetric partly due to the presence of some very large values. When removing the 80 observations with income larger than or equal to 50,000, we obtain a truncated population referred to as MIC2015_Oax_trunc, which is also used in our experiments.

The step distribution function of income has only 913 and 887 jumps in each population respectively, with some large jumps at income values that are near the quantiles of interest. In particular, $y\mathrm{<mo>=</mo>}$ 2,571 accounts for 7.3% of the distribution and is very close to the quantile $Y_{\text{0}\text{.5}};$ $y\mathrm{<mo>=</mo>}643$ (1.1%) and $y\mathrm{<mo>=</mo>}857$ (4.2%) are close to $Y_{\text{0}\text{.1}};$ whereas $y\mathrm{<mo>=</mo>}$ 6,429 (2.8%) and $y\mathrm{<mo>=</mo>}$ 7,000 (1.1%) are close to $Y_{\text{0}\text{.9}}.$

3.1 Numerical results

For each population, the coverage rate of the CI for each method was estimated as follows:

1. A simple random sample $s$ of size $n\in \mathrm{<mo>\{</mo>}\text{1,000;}\text{5,000}\mathrm{<mo>\}</mo>}$ was selected.
2. For each unit $k\in s$ with response propensity ${\varphi }_k,$ we generated $I_k$ from a $\text{Bernoulli}\mathrm{<mo\; stretchy="false">(</mo>}{\varphi }_k\mathrm{<mo\; stretchy="false">)</mo>\; .}$
3. Two cases were considered: a) full response and b) average nonresponse rate of 22.5%. For the latter, a logistic regression with two-way interactions, with $I$ as the response and the six explanatory variables, was selected by forward selection using the BIC criterion. The estimated response probabilities ${\widehat{\varphi }}_k$ were obtained with the selected model.
4. 90% CIs were computed using linearization (Lin), empirical likelihood (EL) and the Woodruff (W) method for quantiles.
5. Steps 1 to 4 were repeated $M\mathrm{<mo>=</mo>}$ 5,000 times and the coverage rate for each method and for each parameter was calculated as the proportion of CIs that covered the corresponding parameter value.

Table 3.1 shows the results for $n\mathrm{<mo>=</mo>}$ 5,000. Table 3.2 shows the absolute value of the percent relative bias, $\text{RB}\mathrm{<mo>=</mo>}100\left(\overline{\widehat{\theta }}-{\theta }_0\right)/{\theta }_0$ with ${\overline{\widehat{\theta }}}_0\mathrm{<mo>=</mo>}{\displaystyle \sum {\widehat{\theta }}_{0i}/M},$ and of the percent relative root mean square error, $\text{RRMSE}\mathrm{<mo>=</mo>}100{\left\{{\displaystyle \sum {({\widehat{\theta }}_{0i}-{\theta }_0)}^2}/M\right\}}^{\mathrm{1\; <mo>/</mo>\; 2}}/{\theta }_0.$ Figure 3.1 presents the distribution of the $M\mathrm{<mo>=</mo>}$ 5,000 estimates for the nonresponse scenario for each parameter; the corresponding distributions for the full response scenario are qualitatively similar. The results for $n\mathrm{<mo>=</mo>}$ 1,000 are omitted since they were similar to those obtained with $n\mathrm{<mo>=</mo>}$ 5,000. From Tables 3.1 and 3.2 and Figure 3.1, we make the following remarks:

1. For $\overline{Y},$ Lin and EL methods perform similarly: they have a poor performance (coverage as low as 72.9%) for MIC2015_Oax, and a good one for MIC2015_Oax_trunc, reaching the nominal level with similar tail error rates and CI average length. Figure 3.1 a) shows that the distribution of $\widehat{\overline{Y}}$ is symmetric for MIC2015_Oax_trunc and highly asymmetric for MIC2015_Oax; this asymmetry seems to be related to the 80 extreme income values not present in MIC2015_Oax_trunc.
2. For quantiles, Lin method has a poor performance with the shortest CI average length in both scenarios, in spite of the expected overestimation of the variance in the nonresponse scenario. This method relies on the normality of ${\widehat{Y}}_q,$ but Figures 3.1 b), c) and d) show that the distribution of ${\widehat{Y}}_q$ is far from being symmetric and unimodal, with modes around income values with high frequency. Especially for $Y_{\text{0}\text{.1}},$ where the coverage rate is as low as 31.4%, the distribution of ${\widehat{Y}}_{\text{0}\text{.1}}$ is multimodal with a high proportion of values that are farther from $Y_{\text{0}\text{.1}}$ than half the CI average length. EL and W methods generally perform well, except for $Y_{\text{0}\text{.5}}$ in the full response scenario and for $Y_{\text{0}\text{.9}}$ in MIC2015_Oax. The low coverages seem to be related to the observed high frequency of the two income values 2,571 (7.3%) and 6,429 (2.8%). The first one is very close to $Y_{\text{0}\text{.5}}\mathrm{<mo>=</mo>}$ 2,570 and some of the CIs for $Y_{\text{0}\text{.5}}$ are too narrow when ${\widehat{Y}}_{\text{0}\text{.5}}\mathrm{<mo><</mo>}$ 2,571. The second one is farther from $Y_{\text{0}\text{.9}}$ in MIC2015_Oax than in MIC2015_Oax_trunc, reducing the proportion of CIs that cover $Y_{\text{0}\text{.9}}$ when ${\widehat{Y}}_{\text{0}\text{.9}}\approx$ 6,429, see Figure 3.1 d), where $Y_{\text{0}\text{.9}}\mathrm{<mo>=</mo>}$ 6,921 in MIC2015_Oax and $Y_{\text{0}\text{.9}}\mathrm{<mo>=</mo>}$ 6,856 in MIC2015_Oax_trunc.
3. Table 3.2 shows that the RB is small, less than 3.3%, for all parameters. When only a simple adjustment with the percentage of nonresponse is applied (not shown in this note), the RB is larger and all the methods have a very poor performance. These results suggest that the use of a propensity model helps to obtain a RB comparable with that of the full response case. For $Y_{\text{0}\text{.1}},$ the empirical double expansion estimator is even less biased than the one associated with the full response scenario; however their RRMSE are comparable and the largest among those for the parameters of interest, since the distribution of the estimators is multimodal in both scenarios, see Figure 3.1 b).

• Previous
• Article main page
• Next