Model-assisted calibration of non-probability sample survey data using adaptive LASSO
Section 5. Application to National Health Interview Survey (NHIS)

• Previous
• Next

5.1  NHIS and ACS data

We next apply LASSO calibration to National Health Interview Survey (NHIS) 2013 to estimate the total number of adults (age 18 or older) diagnosed with cancer in the population. The National Health Interview Survey is a nationally representative sample of non-institutionalized civilian households collected by a multi-stage area-probability sampling (Centers for Disease Control and Prevention, 2005). Each month, health-related data on a cross-sectional sample of people in selected households are obtained by face-to-face interviews. The data provides pseudo-primary-sampling-unit (PSU), pseudo-strata, and sampling weights to allow for weighted estimates with complex survey design. In addition to health-related measures, NHIS also collects demographic data. Our goal is to assess our LASSO estimator by treating the unweighted NHIS sample as reflective of a non-probability sample, and explore how GREG and LASSO calibration compare with the design-weighted estimator.

To calibrate NHIS on a set of demographic and income-related variables, we use the American Community Survey (ACS) 2013 micro-data as the benchmark data. ACS samples are households selected through multi-stage area-probability sampling from 3,143 counties of the U.S. The design of ACS is to improve estimates of small areas between the decennial census long-form samples. Around three million households are selected each year, with measures collected on household types and individual demographics within the households. ACS also collects data from group-quarters, which are excluded from this analysis. For ACS 2013, the sample size for adults is 2,317,301. The NHIS 2013 sample size is 34,201 after removing 242 cases with missing values on demographic variables. For the purposes of this analysis, we treat the weighted estimates from the ACS as known population totals, a reasonable assumption given the differences in sample size.

5.2  Estimators

The outcome variable of interest is whether a person has been diagnosed with cancer. Define the binary indicator for the outcome variable:

$$y_i=\{\begin{array}{ll}\mathrm{1:}\hfill & \text{if}\text{person}i\text{has}\text{been}\text{diagnosed}\text{with}\text{cancer}\hfill \\ \mathrm{0:}\hfill & \text{otherwise}\text{.}\hfill \end{array}$$

We first use the NHIS 2013 sampling weights, $w^{\text{NHIS}},$ and design variables to obtain an unbiased estimate of the population total, $T_y={\displaystyle {\sum }_{i\mathrm{<mo>=</mo>\; 1}}^N}{y}_i.$ Then we assume that the NHIS 2013 sample is collected from a simple-random-sampling, with initial design weights $d^A\mathrm{<mo>=</mo>}N/n,$ where $N$ is the population total obtained from ACS, and $n$ is the sample size of NHIS. We calibrate $d^A$ by a set of demographic and income variables with traditional GREG calibration and LASSO calibration. Finally, as a compromise between GREG and LASSO, we consider model-assisted calibration to a linear model for $y_i$ instead of the LASSO using (2.7); note that, when ${\widehat{\mu }}_i$ is computed using the same linear model as in GREG, the point estimates of the total will correspond, even though the calibration weights will differ. Thus, we generate seven estimates:

1.  ${\widehat{T}}_y^{\text{NHIS}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{NHIS}}{y}_i$: Estimate obtained with NHIS weights.
2.  ${\widehat{T}}_y^{\text{HTSRS}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}\left(N/n\right)y_i$: Estimate obtained with weights $d^A\mathrm{<mo>=</mo>}N/n.$
3.  ${\widehat{T}}_y^{\text{GREG1}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{GREG1}}{y}_i$: Estimate obtained by calibrating $d^A$ with GREG using all calibration variables.
4.  ${\widehat{t}}_y^{\text{GREG1MC}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{GREG1MC}}{y}_i$: Estimate obtained by model-assisted calibration to linear model using predictors in GREG1.
5.  ${\widehat{T}}_y^{\text{GREG2}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{GREG2}}{y}_i$: Estimate obtained by calibrating $d^A$ with GREG using only calibration variables chosen using backward stepwise variable selection.
6.  ${\widehat{t}}_y^{\text{GREG2MC}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{GREG2MC}}{y}_i$: Estimate obtained by model-assisted calibration to linear model using predictors in GREG2.
7.  ${\widehat{T}}_y^{\text{LASSO}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\in s_A}}{w}_i^{\text{LASSO}}{y}_i$: Estimate obtained by model-assisted calibration with LASSO.

The variance of ${\widehat{T}}_y^{\text{NHIS}}$ is the linearization variance estimate of total, accounting for sampling-stratum, primary-sampling-units, and survey weights in the NHIS 2013 sample. Variances of HTSRS, GREG1, and GREG2 are linearization variance estimates with weights $d^A\text{},$ $w^{\text{GREG1}}\text{},$ and $w^{\text{GREG2}}$ respectively. We obtain the variance of LASSO estimator through naive bootstrap.

5.3  Working models

Table 5.1 lists calibration variable names, labels, values, and distributions in this analysis. The first column is the unweighted distribution of variables in the NHIS sample. The second column contains variable distributions in the NHIS sample, weighted by $w^{\text{NHIS}}$ person-level weights. The third column is the distribution of variables in the population obtained from the ACS benchmark data. Missing income category is included as a separate category to capture the difference in missing patterns between NHIS and ACS. Including a missing category also allows us to maintain the analytic sample size. Relative to ACS, the unweighted NHIS sample has higher proportions of females, widowed/divorced/separated, and fewer proportion of non-Hispanic whites. After weighting, the NHIS distributions of gender and race are close to the benchmark’s, and only marital status categories show some differences.

We use an unweighted linear model with backward-stepwise variable selection to determine the working model for GREG2. The final variables included in the model for GREG2 are age, education, race, employment status (yes/no), and family income. For standard GREG and LASSO calibration, we use all available variables.

5.4  Results

Table 5.2 lists the estimates, standard errors (SE), root mean square error treating the correctly weighted NHIS as the true value (RMSE), percent-deviate from the NHIS estimate: $\mathrm{<mi>\%</mi>}\text{deviate}\mathrm{<mo>=</mo>}100\left(\widehat{T}-{\widehat{T}}_y^{\text{NHIS}}\right)/{\widehat{T}}_y^{\text{NHIS}}\text{},$ and the standard deviation and minimum and maximum of the weights associated with a given estimator. We treat NHIS estimate as the unbiased estimate because it is calculated with probability-based sampling weights provided by NHIS. Without any weighting adjustment, HTSRS shows a positive bias of 5.9%. The GREG2 estimator reduces this bias from 5.9% to 2.0%, the GREG1 estimator reduces bias to 1.8%, while LASSO estimator reduces the bias to 0.9%. By definition, use of the model-assisted estimator using linear predictors will yield the same estimator as the GREG model; however the variability is substantially reduced. In this analysis, if NHIS were a non-probability sample, without weighting adjustment, we would have over-counted the number of adults with cancer by 1.18 million. With traditional calibration, the error is reduced to an over-count of 365 thousand (without variable selection) or 392 thousand (with variable selection). LASSO calibration further reduces the over-count to 175 thousand.

As expected, the standard error of the NHIS estimate is the largest, as it properly incorporates complex survey design. If the calibration working model correctly captures the relationship between the outcome variable and the calibration variables, we anticipate that the calibration estimator standard errors to be smaller than HTSRS estimator’s. This is not the case for either of the GREG estimator, where the standard error is larger than HTSRS’s, although the RMSE is smaller due to the reduction in bias. In addition, the standard GREG estimator has a standard error about 2.0% greater than the backward selection GREG estimator, a feature offset by its estimated 6.6% reduction in bias (although this is insufficient to reduce RMSE); use of the model-assisted GREG estimator does reduce the standard error, and the root mean square error, by 5-7% and 2-3% respectively, over the standard GREG estimates. For LASSO calibration, we do observe a smaller standard error than HTSRS’s, even with the bootstrap variance estimate that tends to overestimate. Without using the correct design weights, LASSO calibration produced the most accurate estimate of a population total while providing the smallest standard error among the estimators in this application. This is in spite of the fact that the standard deviation of the LASSO calibration weights were only about one-seventh as variable as the GREG weights, reflected in the smaller standard error of the estimator itself and greatly reduced RMSE.

• Previous
• Article main page
• Next