Semiparametric quantile regression imputation for a complex survey with application to the Conservation Effects Assessment Project
Section 1. Introduction
Missing data have important implications for analyses of survey data. Missing data can arise because sampled units refuse to participate in the survey, are difficult to locate, do not respond to sensitive questions, or drop out of longitudinal studies. If the missing values are related to the variable of interest, an analysis of the complete data with no modification for missing values, is biased. Weighting and imputation are two broad classes of missing data adjustments.
Two types of weighting adjustments are calibration (D’arrigo and Skinner, 2010 and Kott, 2006) and propensity score estimation (Kim and Riddles, 2012). In calibration, the weights for the respondents are adjusted so that the weighted sum of an auxiliary variable for the respondents is equal to the corresponding mean for the full sample or a population mean. In propensity score estimation, the sampling weight is multiplied by the inverse of an estimated response probability.
Imputation completes the data set, replacing missing response variables with imputed values. Imputation can simplify analyses in the presence of item nonresponse and improve consistency in results across users. We consider imputation of a response which may be missing, using an auxiliary variable that is observed for the full sample. To allow flexibility in the model assumptions, we use a semiparametric quantile regression model to describe the relationship between and
A diverse range of imputation procedures exists (Kim and Shao, 2013). Parametric fractional imputation (Kim, 2011) and parametric multiple imputation (Rubin, 2004) generate imputed values from an estimate of a fully parametric model for the conditional distribution of the response given covariates. Hot deck imputation (i.e., Andridge and Little, 2010), in contrast, includes, a class of nonparametric procedures in which imputed values are selected from respondents. In some hot deck procedures, weights are assigned according to a proximity measure, defined by imputation classes (Brick and Kalton, 1996) or a metric (Rubin, 2004; Little, 1988) such as a kernel distance (Wang and Chen, 2009). Nonparametric imputation is more robust to model misspecification than fully parametric methods, but estimators based on nonparametric procedures can have poor efficiency in small samples. Semiparametric quantile regression imputation (QRI) is a compromise between nonparametric and fully parametric imputation procedures. In QRI, the imputed values for a single missing value are the estimated quantiles of the distribution of the missing observation conditional on a function of auxiliary variables. Because a semiparametric model for the quantile function is used, QRI is robust to model misspecification, and because values are imputed from estimated quantiles, QRI is resistant to extreme values. Chen and Yu (2016) develop QRI for simple random sampling from an infinite population. We extend Chen and Yu (2016) to allow unequal selection probabilities.
Many imputation procedures rely on a missing at random (MAR) assumption (Rubin, 1976). A common assumption is that the response variable which may be missing) is conditionally independent of the missing indicator (1 if a response is provided and 0 otherwise) given the observed data. A direct application of this MAR definition to a complex survey specifies independence of the response variable and missing indicator variable conditional on the auxiliary variable and the sample inclusion indicators (Little, 1982; Pfeffermann, 2011). Berg, Kim and Skinner (2016) call the missing at random assumption that is defined conditional on the sample inclusion indicators sample missing at random. An alternative assumption, called population missing at random (Berg et al., 2016), is that the response variable is conditionally independent of the missing indicator given the auxiliary variable in the superpopulation, unconditional on the sample inclusion indicators. Berg et al. (2016) show that these two assumptions are not equivalent. We discuss these MAR concepts precisely in Section 2 and develop our procedure to be sufficiently flexible to accommodate either condition.
Our interest in semiparametric quantile regression for a complex survey is motivated in part by the Conservation Effects Assessment Project (CEAP), a complex survey intended to quantify soil and nutrient loss from crop fields. Because distributions of the response variables are highly skewed and contain extreme values, specification of an adequate fully parametric imputation model is difficult, and hot deck imputation procedures may have large variances. We investigate the use of QRI to address these issues in imputation for CEAP.
We demonstrate the theoretical validity and applicability of semiparametric quantile regression imputation in the context of a complex survey. Section 2 and Section 3, respectively, present the imputation algorithm and asymptotic properties. Section 4 and Section 5 demonstrate the properties of QRI through the CEAP application and simulations, respectively. Section 6 concludes with a summary and a discussion of areas for future research.
- Date modified: