A new double hot-deck imputation method for missing values under boundary conditions
Section 3. Simulation

• Previous
• Next

We use the following abbreviations for imputation methods discussed in Section 1 and 2; OBS (available cases), T-NORM (truncated normal imputation in Rubin (1978); Raghunathan et al. (2001)), MV (method adjusted for uncertainty of the mean and variance in Rubin and Schenker (1986)), PMM (predictive mean matching in Little (1988)), LRD (local residual draw method in Schenker et al. (2006)) and three variations of DBM-PRD denoted by PRD, SPRD and ${\text{SPRD}}_2.$ We compare these eight imputation methods with our DBM-PRD where a truncation procedure is added in MV, PMM, and LRD to accommodate the boundary constraints, denoted by T-MV, T-PMM, and T-LRD, respectively.

We consider a sample size of 1,000 with a 20% or 50% missing rates from the following linear model:

$Y_i\mathrm{<mo>=</mo>}{X}_i+{\epsilon }_i\mathrm{,}$  where  $C_{iL}\le Y_i\le C_{iU}$  for  $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,n\mathrm{,}(3.1)$

and $X_i’s$ are independently generated from $N\left(\mathrm{2,}2\right)$ and i.i.d. ${\epsilon }_i$ are simulated from $N\left(\mathrm{0,}{\sigma }_Y\right)$ or the t-distribution with degree of freedom $t_{df}$ . The boundary values $C_{i\mathrm{,}U}$ and $C_{i\mathrm{,}L}$ are generated with $Y_i+\left|Z_{i\mathrm{,}U}\right|$ and $Y_i-\left|Z_{i\mathrm{,}L}\right|$ where $Z_{i\mathrm{,}U}~N\left(\mathrm{0,}{\sigma }_U\right)$ and $Z_{i\mathrm{,}L}~N\left(\mathrm{0,}{\sigma }_L\right),$ respectively. We set $\text{Cor}\left(X\mathrm{,}Y\right)$ to be 0.7 or 0.9 by adjusting ${\sigma }_Y$ (or $t_{df}),$ and $\text{Cor}\left(Y\mathrm{,}{C}_U\right)$ and $\text{Cor}\left(Y\mathrm{,}{C}_L\right)$ to be between 0 and 0.9 by adjusting ${\sigma }_U$ and ${\sigma }_L.$ The correlation $\text{Cor}\left(Y\mathrm{,}{C}_U\right)$ $\left(\text{Cor}\left(Y\mathrm{,}{C}_L\right)\right)$ denoted by ${\rho }_{y\mathrm{,}{c}_u}$ $\left({\rho }_{y\mathrm{,}{c}_l}\right)$ indicates that the upper bound $C_U$ has stronger information for $Y$ than the lower bound $C_L$ when ${\rho }_{y\mathrm{,}{c}_u}$ is greater than ${\rho }_{y\mathrm{,}{c}_L}$ in absolute value.

Two types of missing mechanisms are considered. First, 20% of $Y$ values are randomly chosen and treated as missing to reflect the “missing completely at random (MCAR)” missing mechanism. Second, we set 80% of $Y_i’s$ to missing when the corresponding $X_i$ is greater than its mean and 20% of $Y_i’s$ to missing when the corresponding $X_i$ is less than its mean. This results in approximately 50% of $Y_i’s$ with missing values overall and reflects “missing at random (MAR)”. Note that no imputation is needed for missing values under MCAR, while imputation for missing values under MAR is required (Scheffer, 2002).

We repeat each simulation scenario 1,000 times with the number of imputations $M$ equal to 5 and the number of possible donors in the selection pool for imputation $m_d$ equal to 6. A possible donor size $m_d$ is allowed to be smaller than 6 when there is not enough sample to compose a donor, but there is no such case when the sample size is 1,000. We choose the commonly used fixed numbers $M\mathrm{<mo>=</mo>}5$ and $m_d\mathrm{<mo>=</mo>}6$ (Geraci and McLain, 2018; Schafer, Ezzati-Rice, Johnson, Khare, Little and Rubin, 1996; Schenker and Taylor, 1996), because it is known that such a setup does not affect the performance of imputation methods significantly as shown in Schafer (1999) and Schenker and Taylor (1996).

The imputation methods are compared in terms of estimation accuracy and efficiency for population quantities: mean $\left(\mu \right)$ and the 5^th, 25^th, 50^th, 75^th, and 95^th percentiles. Statistical inference after multiple imputation proceeds as in Rubin (2004) and Schafer et al. (1996). We use the mean absolute error (MAE), root mean squared error (RMSE), a coverage rate of 95% confidence interval (CR) and an average width of 95% confidence interval (AWCI) as evaluation criteria for measuring the estimation accuracy and efficiency (Yucel and Demirtas, 2010; Yucel, He and Zaslavsky, 2008; Gelman, Van Mechelen, Verbeke, Heitjan and Meulders, 2005).

3.1  Simulation results under MCAR

Figure 3.1 shows the distribution of $\widehat{\mu }-\mu$ (bias) in 1,000 simulated data sets with 20% of MCAR missing values under $\left({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.8,0)$ and ${\rho }_{x\mathrm{,}y}\mathrm{<mo>=</mo>}\mathrm{0.7.}$ Since no imputation is necessary for missing values under MCAR in the estimation of mean and variance of $Y,$ OBS is unbiased for the mean of $Y,$ as expected. However, Figure 3.1 shows that all imputation methods, except for DBM-PRD, reveal an over-estimation problem. Observe that the lower boundary has strong information for $Y({\rho }_{y\mathrm{,}{c}_l}\mathrm{<mo>=</mo>}0.8)$ but the upper boundary has no information $({\rho }_{y\mathrm{,}{c}_u}\mathrm{<mo>=</mo>}0).$ Except for OBS and DBM-PRD, this asymmetric boundary information pushes up imputed values in the other imputation methods. To see the effect of asymmetric boundary information on imputation accuracy, different values of $\left({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)$ are considered in Table 3.1.

When upper and lower boundaries provide boundary information for $Y$ in a symmetric way (i.e., $\left({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.9,0.9))$ all imputation methods are comparable and are competitive with OBS. However, in the presence of asymmetric boundary information $({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u})\mathrm{<mo>=</mo>}(0.8,0)$ or $\mathrm{<mo\; stretchy="false">(</mo>}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\mathrm{<mo\; stretchy="false">)</mo>}\mathrm{<mo>=</mo>}(0.5,0.8),$ the estimation accuracy of the existing T-NORM, T-MV, T-PMM, and T-LRD is much worse than OBS and DBM-PRD. In particular, the coverage rate of 95% CIs (CR) is dramatically decreased as the degree of asymmetry increases. On the other hand, those of the PRD series (i.e., PRD, SPRD, DBM-PRD) are resistant to such asymmetry, indicating that the proportioned residual draw is resistant to asymmetric boundary information. Among the PRD series, DBM-PRD outperforms PRD and SPRD and is even better than OBS in terms of MAE and RMSE.

Notice that, except OBS and DBM-PRD, the imputed values by all other imputation methods make the distribution of $Y$ lean toward the boundary with weaker boundary information. More precisely, all the imputation methods except OBS and DBM-PRD tend to over-estimate the true mean of $Y\left(E\left(Y\right)\mathrm{<mo>=</mo>}2\right)$ for $\left({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.8,0)$ because ${\rho }_{y\mathrm{,}{c}_u}\mathrm{<mo><</mo>}{\rho }_{y\mathrm{,}{c}_l},$ whereas they tend to under-estimate the true mean for $\left({\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.5,0.8)$ because ${\rho }_{y\mathrm{,}{c}_u}\mathrm{<mo>></mo>}{\rho }_{y\mathrm{,}{c}_l}.$ This dependency is also observed with the MAR missing mechanism as discussed in the following section.

3.2  Simulation results under MAR

Table 3.2 summarizes the results with 50% MAR missing values under normal and $t\left(t_{df}\mathrm{<mo>=</mo>}3\right)$ distribution errors. As expected, OBS which uses only observed values in estimation is much worse than any imputation method for estimating the true mean $\mu \mathrm{<mo>=</mo>\; 2.}$ The results related to asymmetric boundary information are along the same lines as the MCAR simulation results. The accuracy and efficiency of T-NORM, T-MV, T-PMM, and T-LRD are much worse than those of the PRD series when the boundary information is asymmetric. This shows the effect of the two hot-deck steps and the proportioned residual draw on the accuracy and efficiency of mean estimation. Except for the symmetric boundary information under normal and $t$ distributions, the CRs of T-NORM, T-MV, T-PMM and T-LRD are less than 50%, much smaller than the target 95%. All imputation methods except DBM-PRD produce the empirical distribution of $Y$ biased to the boundary with weaker boundary information under both normal and $t$ error distributions. This implies that only DBM-PRD is resistant to asymmetric boundary information and error distributions regardless of the missing mechanism. As a result, DBM-PRD outperforms the other imputation methods in all simulation scenarios.

In order to see the effect of the double hot-deck procedures employed in DBM-PRD, we compared DBM-PRD with SPRD. The effect of the first hot-deck step can be examined by comparing DBM-PRD and SPRD as the first hot-deck step of DBM-PRD is removed in SPRD. DBM-PRD is consistently better than SPRD regardless of the evaluation measure as long as the boundary information is asymmetric. The CR of SPRD relative to that of DBM-PRD, becomes worse as the boundary information becomes more skewed to one side. Thus, the first hot-deck step has an important role in resisting asymmetry of the boundary information.

The role of the second hot-deck step can be checked by comparing SPRD and PRD where PRD does not adopt both hot-deck steps. SPRD is better than PRD when the boundary information is moderately asymmetric in normal error or $t$ distributed error, implying that the second hot-deck step works for a heavier tail distribution than normal. This comparison is further discussed in the following evaluation for percentile estimation.

As we described before, the ${\text{SPRD}}_2$ is the same as SPRD except for the method of matching to construct possible donors. The possible donor in ${\text{SPRD}}_2$ consists of ${\tilde{r}}_{i\mathrm{,}U}$ and ${\tilde{r}}_{i\mathrm{,}L}$ whose predicted mean ${\widehat{Y}}_i^{\text{obs}}$ is close to ${\widehat{Y}}_j^{\text{miss}}.$ Thus, by comparing SPRD and ${\text{SPRD}}_2,$ we examine the effect of the boundary information matching used in DBM-PRD. Table 3.2 shows that SPRD outperforms ${\text{SPRD}}_2$ regardless of boundary information and error distributions, implying that the boundary information matching works better than the usual mean matching for imputation of bounded missing data.

We also investigate the percentile estimation by evaluating how well each imputation method estimates the probability that $Y$ is greater than 5%, 25%, 50%, 75%, 95% quantiles. Denote the $p^{\text{th}}$ quantile by $y^{-1}\left(p\right)$ satisfying $P\left(Y\mathrm{<mo>></mo>}{y}^{-1}\left(p\right)\right)\mathrm{<mo>=</mo>}1-p.$ The five percentiles are chosen to test how different the true distribution of $Y$ and the estimated distribution of $Y$ are for different imputation methods. Table 3.3 shows the results of percentile estimation when 50% missing values under MAR with normal error when $\left({\rho }_{x\mathrm{,}y}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.7,0.5,0.8)$ and (0.7, 0.8, 0). From the first line of each table, the existing methods clearly produce distributions skewed to the right when $\left({\rho }_{x\mathrm{,}y}\mathrm{,}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.7,0.5,0.8)$ because ${\rho }_{y\mathrm{,}{c}_u}\mathrm{<mo>></mo>}{\rho }_{y\mathrm{,}{c}_l},$ while they produce distributions skewed to the left when $\mathrm{<mo\; stretchy="false">(</mo>}{\rho }_{x\mathrm{,}y}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\text{}\mathrm{<mo\; stretchy="false">)</mo>}\mathrm{<mo>=</mo>}(0.7,0.8,0)$ because ${\rho }_{y\mathrm{,}{c}_u}\text{}\mathrm{<mo><</mo>}{\rho }_{y\mathrm{,}{c}_l}.$

The degree of skewness is considerably weakened in the PRD series where DBM-PRD shows the best performance in percentile estimation. When the boundary information is moderately asymmetric (i.e., $\left({\rho }_{x\mathrm{,}y}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.7,0.5,0.8)),$ the second hot-deck step is important whereas the first hot-deck step is less important because SPRD is better than PRD but is comparable to DBM-PRD. On the other hand, when $\left({\rho }_{x\mathrm{,}y}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_l}\text{}\mathrm{,}{\rho }_{y\mathrm{,}{c}_u}\right)\mathrm{<mo>=</mo>}(0.7,0.8,0)$ the first hot-deck step is more important to choose a correct boundary due to the extremely asymmetric boundary information, and DBM-PRD is best among PRD series because only it contains the first hot-deck step. In addition, SPRD is better than ${\text{SPRD}}_2$ in percentile estimation. In summary, double hot-deck procedures including boundary information matching and proportioned residual draw are essential not only for boundary restrictions but also for asymmetric boundary information and even for symmetric boundary information and heavy tail distributions.

• Previous
• Article main page
• Next