Multiple imputation of missing values in household data with structural zeros
Section 5. Empirical study

• Previous
• Next

To evaluate the performance of the NDPMPM as an imputation method, as well as the speed up strategies, we use data from the public use microdata files from the 2012 ACS, available for download from the United States Census Bureau (http://www2.census.gov/acs2012_1yr/pums/). We construct a population of 764,580 households of sizes $\mathcal{H}\mathrm{<mo>=</mo>}\left\{\mathrm{2,}\mathrm{3,}4\right\},$ from which we sample $n\mathrm{<mo>=</mo>}\text{5,000}$ households comprising $N\mathrm{<mo>=</mo>}\text{13,181}$ individuals. We work with the variables described in Table 5.1, which mimic those in the U.S. decennial census. The structural zeros involve ages and relationships of individuals in the same house; see the Appendix for a full list of rules that we used. We move the household head to the household level as in Section 4.1 to take advantage of the computational gains.

We introduce missing values using the following scenario. We let household size and age of household heads be fully observed. We randomly and independently blank 30% of each variable for the remaining household-level variables. For individuals other than the household head, we randomly and independently blank 30% of the values for gender, race and Hispanic origin. We make age missing with rates 50%, 20%, 40% and 30% for values of the relationship variable in the sets {2}, {3, 4, 5, 10}, {7, 9} and {6, 8, 11, 12, 13}, respectively. We make the relationship variable missing with rates 40%, 25%, 10%, and 55% for values of age in the sets $\left\{x\mathrm{<mo>:</mo>}x\le 20\right\},$ $\left\{x\mathrm{<mo>:</mo>}\mathrm{20\; <mo><</mo>}x\le 50\right\},$ $\left\{x\mathrm{<mo>:</mo>}\mathrm{50\; <mo><</mo>}x\le 70\right\},$ and $\left\{x\mathrm{<mo>:</mo>}x\mathrm{<mo>></mo>\; 70}\right\},$ respectively. This results in approximately 30% missing values for both variables. About 8% of the individuals in the sample are missing both the age and relationship variable, and 2% are missing gender, age, and relationship jointly. This mechanism results in data that technically are not missing at random, but we use the NDPMPM approach regardless to examine its potential in a complicated missingness mechanism. Actual rates of item nonresponse in census data tend to be smaller than what we use here, but we use high rates to put the NDPMPM through a challenging stress test. We also introduce missing values using a missing completely at random scenario with rates in the 10% range across all the variables. In short, the results are similar to those here, though more accurate due to the lower rates of missingness. See the Appendix for the results.

We estimate the NDPMPM using two approaches, both using the rejection step S9' in Section 3. The first approach considers ${\psi }_2\mathrm{<mo>=</mo>}{\psi }_3\mathrm{<mo>=</mo>}{\psi }_4\mathrm{<mo>=</mo>\; 1},$ i.e., without using the cap-and-weight approach, while the second approach considers ${\psi }_2\mathrm{<mo>=</mo>}{\psi }_3\mathrm{<mo>=</mo>}1/2$ and ${\psi }_4\mathrm{<mo>=</mo>}1/3.$ For each approach, we run the MCMC sampler for 10,000 iterations, discarding the first 5,000 as burn-in and thinning the remaining samples every five iterations, resulting in 1,000 MCMC post burn-in iterates. We set $F\mathrm{<mo>=</mo>\; 30}$ and $S\mathrm{<mo>=</mo>\; 15}$ for each approach based on initial tuning runs. Across the approaches, the effective number of occupied household-level clusters usually ranges from 13 to 16 with a maximum of 25, while the effective number of occupied individual-level clusters across all household-level clusters ranges from 3 to 5 with a maximum of 10. For convergence, we examined trace plots of $\alpha ,$ $\beta ,$ and weighted averages of a random sample of the multinomial probabilities in (2.3) and (2.4) (since the multinomial probabilities themselves are prone to label switching).

For both methods, we generate $L\mathrm{<mo>=</mo>\; 50}$ completed datasets, $Z=\left(Z^{\mathrm{<mo\; stretchy="false">(</mo>\; 1\; <mo\; stretchy="false">)</mo>}}\mathrm{,}\dots \mathrm{,}{Z}^{\left(50\right)}\right)$ , using the posterior predictive distribution of the NDPMPM, from which we estimate all marginal distributions, bivariate distributions of all possible pairs of variables, and trivariate distributions of all possible triplets of variables. We also estimate several probabilities that depend on within household relationships and the household head to investigate the performance of the NDPMPM in estimating complex relationships. We obtain confidence intervals using multiple imputation inferences (Rubin, 1987). As a brief review, let $q$ be the completed-data point estimator of some estimand $Q,$ and let $u$ be the estimator of variance associated with $q.$ For $l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L,$ let $q^{\left(l\right)}$ and $u^{\left(l\right)}$ be the values of $q$ and $u$ in completed dataset $Z^{\left(l\right)}\text{}.$ We use ${\overline{q}}_L\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{l\mathrm{<mo>=</mo>\; 1}}^L}{q}^{\left(l\right)}/L$ as the point estimate of $Q.$ We use $T_L\mathrm{<mo>=</mo>}\left(1+1/L\right)b_L+{\overline{u}}_L$ as the estimated variance of $\overline{q},$ where $b_L={\displaystyle {\sum }_{l\mathrm{<mo>=</mo>\; 1}}^L}{\left(q^{\left(l\right)}-{\overline{q}}_L\right)}^2/\left(L-1\right)$ and ${\overline{u}}_L\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{l\mathrm{<mo>=</mo>\; 1}}^L}{u}^{\left(l\right)}/L.$ We make inference about $Q$ using $\left({\overline{q}}_L-Q\right)\sim t_v\left(\mathrm{0,}{T}_L\right),$ where $t_v$ is a $t-$ distribution with $v\mathrm{<mo>=</mo>}\left(L-1\right){\left(1+{\overline{u}}_L/\left[\left(1+1/L\right)b_L\right]\right)}^2$ degrees of freedom.

Figures 5.1 and 5.2 display the value of ${\overline{q}}_{50}$ for each estimated marginal, bivariate and trivariate probability plotted against its corresponding estimate from the original data, without missing values. Figure 5.1 shows the results for the NDPMPM with the rejection sampler, and Figure 5.2 shows the results for the NDPMPM using the cap-and-weight approach. For both approaches, the point estimates are close to those from the data before introducing missing values, suggesting that the NDPMPM does a good job of capturing important features of the joint distribution of the variables. Figure 5.2 in particular also shows that the cap-and-weight approach did not degrade the estimates.

Table 5.2 displays 95% confidence intervals for several probabilities involving within-household relationships, as well as the value in the full population of 764,580 households. The intervals include the two based on the NDPMPM imputation engines and the interval from the data before introducing missingness. For the latter, we use the usual Wald interval, $\widehat{p}\pm \text{1}\text{.96}\sqrt{\widehat{p}\left(1-\widehat{p}\right)/n},$ where $\widehat{p}$ is the corresponding sample percentage. For the most part, the intervals from the NDPMPM with the full rejection sampling are close to those based on the data without any missingness. They tend to include the true population quantity. The NDPMPM imputation engine results in noticeable downward bias for the percentages of households where everyone is the same race, with bias increasing as the household size gets bigger. This is a challenging estimand to estimate accurately via imputation, particularly for larger households. Hu et al. (2018) identified biases in the same direction when using the NDPMPM (with household head data treated as individual-level variables) to generate fully synthetic data, noting that the bias gets smaller as the sample size increases. The NDPMPM fits the joint distribution of the data better and better as the sample size grows. Hence, we expect the NDPMPM imputation engine to be more accurate with larger sample sizes, as well as with smaller fractions of missing values.

The interval estimates from the cap-and-weight method are generally similar to those for the full rejection sampler, with some degradation particularly for the percentages of same race households by household size. This degradation comes with a benefit, however. Based on MCMC runs on a standard laptop, the NDPMPM using the cap-and-weight approach and moving household heads’ data values to the household level is about 42% faster than the NDPMPM with household heads’ data values moved to the household level.

• Previous
• Article main page
• Next