A new double hot-deck imputation method for missing values under boundary conditions
Section 5. Conclusion

• Previous
• Next

We proposed a method for multiple imputation of missing variables when the missing values are logically bounded, which is often encountered in censuses or sample surveys. The existing imputation methods with an additional truncation or acceptance/rejection step produced biased estimates, depending on the extent of asymmetry of the boundary information. Their imputation values shrank toward the boundary with lower correlation with the missing variable. However, by employing a proportioned residual draw, boundary information matching, and a double hot-deck procedure, our DBM-PRD method produced more accurate and efficient estimates for the mean and percentiles, regardless of missingness rates, missing data mechanism, and distributions of the missing variable.

Moreover, our DBM-PRD imputation method is resistant to asymmetric boundary information in the sense that its imputed values do not depend on the extent of asymmetry of the boundary information. Especially, when there are two or more variables for the boundary information, or when reliability of the lower boundary information is suspected, DBM-PRD imputation is a powerful tool for estimating the parameters of interest accurately.

The DPM-PRD method also will work for a single imputation. There may be cases when (especially in official statistics) a single definitive output dataset is needed, and when users do not have the sophistication to deal with multiple imputation.

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government Ministry of Science and ICT (MSIT) (NRF-2018R1C1B5043739). This work was supported by a Korea University Grant (K1910711) and Hankuk University of Foreign Studies Research Fund.

Appendix

Proof of Theorem 1

It suffices to show that $Y_{jU}^{*}\le C_{jU}$ and $Y_{jL}^{*}\ge C_{jL}$ because of the constraints in the imputation step.

1. If ${\widehat{Y}}_j^{\text{miss}}\in S^0,$ then ${\tilde{r}}_{jU}^{*}$ is sampled from $R_U^0$ whose element ${\tilde{r}}_{iU}\le 1$ for all $i$ because $Y_i\le C_{iU}$ and $C_{iU}-{\widehat{Y}}_i\mathrm{<mo>></mo>}0$ for ${\tilde{r}}_{iU}\in R_U^0.$ Since ${\tilde{r}}_{jU}^{*}$ is one of such ${\tilde{r}}_{iU}’s,$ we have ${\tilde{r}}_{jU}^{*}\le 1.$ Furthermore $C_{jU}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo>></mo>}0$ gives

$$Y_{j\mathrm{,}U}^{*}={\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{j\mathrm{,}U}^{*}\left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\le {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jU}.(\text{A}\text{.1})$$

• Similarly, if ${\widehat{Y}}_j^{\text{miss}}\in S^0,$ then ${\tilde{r}}_{jL}^{*}$ is randomly selected from $R_L^0$ whose element ${\tilde{r}}_{iL}\le 1$ for any $i$ because $Y_i\ge C_{iL}$ and $C_{iL}-{\widehat{Y}}_i\mathrm{<mo><</mo>}0$ for $i\in R_L^0.$ Since ${\tilde{r}}_{jL}^{*}$ is one of such ${\tilde{r}}_{iL},$ ${\tilde{r}}_{jL}^{*}\le 1.$ Using $C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo><</mo>}0$ because ${\widehat{Y}}_j^{\text{miss}}\in S^0,$ we have

$$Y_{j\mathrm{,}L}^{*}\mathrm{<mo>=</mo>}{\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{jL}^{*}\left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\ge {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jL}.(\text{A}\text{.2})$$

2. If ${\widehat{Y}}_j^{\text{miss}}\in S^{+},$ then ${\tilde{r}}_{jU}^{*}$ is sampled from $R_U^{+}$ whose element ${\tilde{r}}_{iU}\ge 1$ for all $i$ because $Y_i\le C_{i\mathrm{,}U}$ and $C_{iU}-{\widehat{Y}}_i\mathrm{<mo><</mo>}0.$ Since ${\tilde{r}}_{jU}^{*}$ is one of such ${\tilde{r}}_{iU}’s,$ we have ${\tilde{r}}_{jU}^{*}\ge 1.$ Furthermore $C_{jU}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo><</mo>}0$ gives

$$Y_{j\mathrm{,}U}^{*}\mathrm{<mo>=</mo>}{\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{j\mathrm{,}U}^{*}\left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\le {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jU}.(\text{A}\text{.3})$$

• Similarly, if ${\widehat{Y}}_j^{\text{miss}}\in S^{+},$ then ${\tilde{r}}_{jL}^{*}$ is randomly selected from $R_L^{+}$ whose element ${\tilde{r}}_{iL}\le 1$ for any $i$ because $Y_i\ge C_{iL}$ and $C_{iL}-{\widehat{Y}}_i\mathrm{<mo><</mo>}0$ for $i\in R_L^{+}.$ Since ${\tilde{r}}_{jL}^{*}$ is one of such ${\tilde{r}}_{iL},$ ${\tilde{r}}_{jL}^{*}\le 1.$ Using $C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo><</mo>}0$ because of ${\widehat{Y}}_j^{\text{miss}}\in S^{+},$ we have

$$Y_{j\mathrm{,}L}^{*}\mathrm{<mo>=</mo>}{\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{jL}^{*}\left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\ge {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jL}.(\text{A}\text{.4})$$

3. If ${\widehat{Y}}_j^{\text{miss}}\in S^{-},$ then ${\tilde{r}}_{jU}^{*}$ is sampled from $R_U^{-}$ whose element ${\tilde{r}}_{iU}\le 1$ for all $i$ because $Y_i\le C_{iU}$ and $C_{iU}-{\widehat{Y}}_i\mathrm{<mo>></mo>}0$ for ${\tilde{r}}_{iU}\in R_U^{-}.$ Since ${\tilde{r}}_{jU}^{*}$ is one of such ${\tilde{r}}_{iU}’s,$ we have ${\tilde{r}}_{jU}^{*}\le 1.$ Furthermore $C_{jU}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo>></mo>}0$ gives

$$Y_{j\mathrm{,}U}^{*}\mathrm{<mo>=</mo>}{\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{j\mathrm{,}U}^{*}\left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\le {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}U}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jU}.(\text{A}\text{.5})$$

• Similarly, if ${\widehat{Y}}_j^{\text{miss}}\in S^{-},$ then ${\tilde{r}}_{jL}^{*}$ is randomly selected from $R_L^{-}$ whose element ${\tilde{r}}_{iL}\ge 1$ for any $i$ because $Y_i\ge C_{iL}$ and $C_{iL}-{\widehat{Y}}_i\mathrm{<mo>></mo>}0$ for $i\in R_L^{-}.$ Since ${\tilde{r}}_{jL}^{*}$ is one of such ${\tilde{r}}_{iL},$ ${\tilde{r}}_{jL}^{*}\ge 1.$ Using $C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\mathrm{<mo>></mo>}0$ because ${\widehat{Y}}_j^{\text{miss}}\in S^{-},$ we have

$$Y_{j\mathrm{,}L}^{*}\mathrm{<mo>=</mo>}{\widehat{Y}}_j^{\text{miss}}+{\tilde{r}}_{jL}^{*}\left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\ge {\widehat{Y}}_j^{\text{miss}}+1\times \left(C_{j\mathrm{,}L}-{\widehat{Y}}_j^{\text{miss}}\right)\mathrm{<mo>=</mo>}{C}_{jL}\mathrm{.}(\text{A}\text{.6})$$

References

Andridge, R.R., and Little, R.J.A. (2010). A review of hot-deck imputation for survey non-response. International Statistical Review, 78(1), 40-64.

Cambois, E., Robine, J.-M. and Mormiche, P. (2007). Did the prevalence of disability in France really fall in the 1990s? A discussion of questions asked in the French Health Survey. Population-E, 62(2), 313-337.

Crossley,T.F., and Kennedy, S. (2002). The reliability of self-assessed health status. Journal of Health Economics, 21, 4, 643-658.

Gelman, A., Van Mechelen, I., Verbeke, G., Heitjan, D.F. and Meulders, M. (2005). Multiple imputation for model checking: Completed-data plots with missing and latent data. Biometrics, 61(1), 74-85.

Geraci, M., and McLain, A. (2018). Multiple imputation for bounded variables. Psychometrika, 83(4), 919-940.

Honaker, J., King, G. and Blackwell, M. (2012). Amelia II: A program for missing data. R package, version 1.6.4.

Kim, N.H., Kwon, T.Y., Yu, S., Kim, N.H., Choi, K.M., Baik, S.H., Park, Y. and Kim, S.G. (2017). Increased vascular disease mortality risk in prediabetic Korean adults is mainly attributable to ischemic stroke. Stroke, 48, 4, 840-845.

Kim, N.H., Lee, J., Kim, T.J., Kim, N.H., Choi, K.M., Baik, S.H., Choi, D.S., Pop-Busui, R., Park, Y. and Kim, S.G. (2015). Body mass index and mortality in the general population and in subjects with chronic disease in Korea: A nationwide cohort study (2002-2010). PLoS ONE, 10(10).

Ko, M.J., Jo, A.J., Park, C.M., Kim, H.J., Kim, Y.J. and Park, D.W. (2016). Level of blood pressure control and cardiovascular events: SPRINT criteria versus the 2014 hypertension recommendations. Journal of the American College of Cardiology, 67(24), 2821-2831.

Ko, S., Yoon, S.J., Kim, D., Kim, A.R., Kim, E.J. and Seo, H.Y. (2016). Metabolic risk profile and cancer in Korean men and women. Journal of Preventive Medicine and Public Health, 49(3), 143-152.

Kwon, T.Y., and Park, Y. (2015). A new multiple imputation method for bounded missing values. Statistics and Probability Letters, 107, 204-209.

Kwon, T.Y., and Park, Y. (2016). Reliability of self-reported data for prevalence and health life expectancy studies: Comparison with sample cohort DB of National Health Insurance Services. The Korean Journal of Applied Statistics, 39(7), 1329-1346.

Kwon, T.Y., Lim, J. and Park, Y. (2017). Health life expectancy in Korea based on sample cohort database of National Health Insurance Services. The Korean Journal of Applied Statistics, 30(3), 475-486.

Lee, J., Lee, J.S., Park, S.H., Shin, S.A. and Kim, K. (2016). Cohort profile: The National Health Insurance Service-National Sample Cohort (NHIS-NSC), South Korea. International Journal of Epidemiology, doi: 10.1093/ije/dyv319.

Little, R. (1988). Missing-data adjustments in large surveys. Journal of Business and Economic Statistics, 6, 287-296.

Raghunathan, T.E., Lepkowski, J.M., Van Hoewyk, J. and Solenberger, P. (2001). A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology, 27, 1, 85-95. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2001001/article/5857-eng.pdf.

Raghunathan, T.E., Solenberger, P.W. and Van Hoewyk, J. (2002). Iveware: Imputation and variance estimation software. Ann Arbor, MI: Survey Methodology Program, Survey Research Center, Institute for Social Research, University of Michigan.

Rim, T.H., Kim, D.W., Han, J.S. and Chung, E.J. (2015). Retinal vein occlusion and the risk of stroke development: A 9-year nationwide population-based study. Ophthalmology, 122(6), 1187-1194.

Rubin, D.B. (1978). Multiple imputation in sample surveys - A phenomenological Bayesian approach to nonresponse. Proceedings of the Survey Research Methods Section, American Statistical Association, 20-34.

Rubin, D.B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, Inc.

Rubin, D.B. (2004). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, Inc.

Rubin, D.B., and Schenker, N. (1986). Multiple imputations for interval estimation from simple random sampling with ignorable nonresponse. Journal of American Statistical Association, 81(394), 366-374.

Schafer, J.L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8(1), 3-15.

Schafer, J.L., Ezzati-Rice, T.M., Johnson, W., Khare, M., Little, R.J.A. and Rubin, D.B. (1996). The NHANES III multiple imputation project. Race/Ethnicity, 60(21.2).

Scheffer, J. (2002). Dealing with missing data. Research Letters in the Information and Mathematical Sciences, 3, 153-160.

Schenker, N., Raghunathan, T.E., Chiu, P.L., Makuc, D.M., Zhang, G. and Cohen, A.J. (2006). Multiple imputation of missing income data in the National Health Interview Survey. Journal of the American Statistical Association, 101(475), 924-933.

Schenker, N., and Taylor, J.M.G. (1996). Partially parametric techniques for multiple imputation. Computational Statistics & Data Analysis, 22(4), 425-446.

Su, Y.-S., Gelman, A., Hill, J. and Yajima, M.(2011). Multiple imputation with diagnostics (mi) in R: Opening windows into the black box. Journal of Statistical Software, 45(2), 1-31.

van Buuren, S., and Groothuis-Oudshoorn, K. (2010). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1-67.

Yucel, R.M., and Demirtas, H. (2010). Impact of non-normal random effects on inference by multiple imputation: A simulation assessment. Computational Statistics & Data Analysis, 54(3), 790-801.

Yucel, R.M., He, Y. and Zaslavsky, A.M. (2008). Using calibration to improve rounding in imputation. The American Statistician, 62(2), 125-129.

• Previous
• Article main page
• Next