A new double hot-deck imputation method for missing values under boundary conditions
Section 1. Introduction
Survey nonresponse (or item nonresponse) arises in many censuses or sample surveys, and several methods for filling in such missed items have been proposed. Some missing variables in a survey are logically bounded. For example, in the U.S. National Health Interview Survey, some families did not report exact income but did report income categories, which provides bounds of the exact family income values. When personal earnings within a family are reported for some family members but not for the others, the sum of the reported personal earnings gives a lower bound of the family income (Schenker, Raghunathan, Chiu, Makuc, Zhang and Cohen, 2006). Geraci and McLain (2018) addressed several examples of bounded missing variables in surveys which include psychometric scales, clinical scores, and school grades.
Waves in panel surveys and panel data sets often provide logical constraints for missing variables. In the periodic health screening data released by the national health service of Korea, the missing smoking period of a smoker at the current wave is bounded below by his/her smoking period reported at the previous wave and bounded above by his/her age. We observed from 2011 and 2013 health screening data of Korea that the 2013 data contain up to 73.5% missing values for smoking periods when we treat the smoking periods that violate such logical constraints as missing. In particular, the mean age of respondents to the smoking periods question is 9 years younger than that of non-respondents, implying that the missing mechanism of the smoking period is not missing completely at random (MCAR) and hence imputation is required.
Geraci and McLain (2018) proposed a quantile-based imputation method for one-sided or two-sided missing variables in which upper and lower values are fixed constants, and showed that their method had advantages, especially when the sample size is moderately large and the true model is strictly non-linear. The most common way to accomodate the logical boundaries to a multiple imputation method is by adopting truncation or an acceptance/rejection step. However, our simulation studies shows that this additional step to existing multiple imputation methods introduces bias as long as the two-sided boundaries are asymmetric.
We propose a new regression-based multiple imputation method without an acceptance/rejection or truncation procedure. This new method utilizes the two-sided boundaries for imputing missing values, automatically meets the boundary constraints, and includes the imputation method given in Kwon and Park (2015) as a special case. We call this new method double hot-deck boundary information matching proportioned residual draw method (DBM-PRD), because two hot-deck steps are used to reduce the number of donor candidates and to choose an appropriate proportional residual that is defined by the usual residual divided by the distance between an observation and its lower or upper bound. This proportioned residual was used in Kwon and Park (2015).
Hot-deck imputation that replaces a missing value with a “similar” observation can improve the imputation performance relative to imputation methods that are derived only from model-assisted schemes. Andridge and Little (2010) showed that, in particular, when a model is used to define matches, hot-deck is less vulnerable to model misspecification than model-assisted methods.
Multiple imputation incorporates imputation uncertainty into statistical inference by substituting missed values several times. The basic method given in Rubin (1978) is to impute the missing value with a sampled value from the normal posterior distribution. It has been extended for imputation of missing values with a logical boundary (Raghunathan, Lepkowski, Van Hoewyk and Solenberger, 2001) by using a truncated normal posterior distribution (T-NORM). Rubin and Schenker (1986) and Rubin (2004) adopted the empirical distribution of the observed standardized residuals based on a fitted regression model. They proposed an imputation method adjusted for uncertainty of the mean and variance (MV), which imputes the missing value with its predictive mean plus residuals that are randomly chosen from their empirical distribution.
Extended from these basic ideas of hot-deck and multiple imputation, most existing methods assume the distribution of the data (usually normal) and employ a truncated distribution (usually truncated normal) to meet a logical boundary of the missing value (van Buuren and Groothuis-Oudshoorn, 2010; Honaker, King and Blackwell, 2012; Su, Gelman, Hill, Yajima, 2011; Raghunathan et al., 2001; Raghunathan, Solenberger and Van Hoewyk, 2002). The predictive mean matching method (PMM) imputes a missing value with a randomly selected observation having a similar predictive mean to that of missing value (Little, 1988). Schenker and Taylor (1996) proposed the local residual draw method (LRD), which replaces each missing value with its predictive mean plus a randomly drawn residual whose predictive mean is close to that of missing value. Instead of the residual in LRD, Kwon and Park (2015) used the proportioned residual whose distance from the predicted mean to its boundary value is close to that of the missing value in order to meet an one-sided boundary imposed on the variables of interest.
The DBM-PRD method is essentially along the same lines as Kwon and Park (2015). However, DBM-PRD employs one more matching procedure to take into account two-sided boundaries and to resolve asymmetric boundary information. This additional matching is based on which boundary is closer to the predictive mean of each missing value. Meanwhile, DBM-PRD imputes the missing value with its predictive mean plus a proportioned residual multiplied by the distance between the predicted mean and the corresponding upper or lower bound. Although DBM-PRD belongs to a mixed method as it uses a regression model in the first step and a double hot-deck imputation of missing values in the second step, the DBM-PRD method is new in that it directly adjusts for the boundary information instead of truncating the designated distribution and uses the boundary information to determine the similarity between observations and missings.
This paper consists of five sections. Our new imputation method is described and its properties discussed in Section 2. By simulation studies in Section 3, we compare our method with T-NORM and MV, PMM, and LRD with additional truncation procedure to meet boundary constraints and PRD-series methods to examine the effect of the double hot-deck step of our method, DBM-PRD. In Section 4, we apply DBM-PRD and the existing imputation methods to the 2013 health screening data of Korea for missing values of smoking periods. Finally, a brief conclusion is found in Section 5.
- Date modified: