Estimation of response propensities and indicators of representative response using population-level information
Section 5. Application to the Dutch Health Survey

• Previous
• Next

In this section, we apply the population-based Type 1 and Type 2 estimators to the Dutch Health Survey conducted by Statistics Netherlands. We employ three auxiliary variables that are part of the gold standard for Dutch market research companies and compare population-based performance to sample-based performance.

The Dutch Health Survey (HS) was commissioned in 1998 as a repeated cross-sectional survey among the full population registered in the Dutch Population Register, but excluding the institutionalized population. It uses a two-stage, self-weighting sampling design in which the first stage is formed by municipalities and the second stage by persons living in the selected municipalities. Until 2012, the HS was a face-to-face survey. In 2012, it changed to a mixed-mode design involving online and face-to-face interviews. Over the years, the sample size was reduced considerably from around 35,000 to around 18,000. We use the 2002 HS data, one of the last years with the original sample size. The net sample size is 33,584 persons.

To calibrate national and regional samples, Dutch market research companies use the so-called Gold Standard population statistics produced by Statistics Netherlands (MOA, 2015). The Gold Standard is an explicitly defined set of auxiliary variables that affiliated companies include in their survey questionnaires. Three of these variables are age, gender and marital status. We focus on these three in the application.

Table 5.1 contains the HS sample and response distributions, and the Statistics Netherlands’ population distributions for the three variables. Joint population distributions, needed to estimate the Type 1 population-based covariance matrices, are also available, but not given here. In practice, the sample distribution is, of course, unknown. The three variables show a different picture: for age and marital status, the response distribution is closer to the sample distribution than to the population distribution, and population-based response propensities give more variation. For gender, the population distribution is closer to the response distribution and less variation is found.

We estimate Type 1 and Type 2 population-based R-indicators in Table 5.3. For the composite estimator, we used the estimated smoothing parameter ${\tilde{\lambda }}_{\text{opt}}$ based on the population-based response propensities. We also include an estimate for ${\lambda }_{\text{opt}}$ calculated using sample-based response propensities. The latter cannot normally be computed and is included for comparison only. Table 5.2 contains the estimated smoothing parameter ${\tilde{\lambda }}_{\text{opt}}$ based on both the population-based response propensities and the sample-based response propensities. The sample-based ${\tilde{\lambda }}_{\text{opt}}$ are larger and tend to have a stronger smoothing effect. However, all ${\tilde{\lambda }}_{\text{opt}}$ are relatively small.

Table 5.3 contains the various population-based R-indicators. For comparison, the sample-based R-indicator is also provided where we used the logistic link function. The linear link function produced the same result. We can conclude that the population-based R-indicators, using only response and population distributions, are different from the sample-based R-indicators, using response and sample distributions. This difference increases, as expected, when Type 2 indicators are used. The composite estimators perform slightly better than the non-composite estimators, but there is still a considerable difference. This is not due to a biased smoothing parameter, as the difference is only modestly smaller when sample-based propensities are used to estimate the smoothing parameter. Furthermore, after bias adjustment, the differences between the composite estimators for sample-based and population-based propensities vanish.

A conclusion from the application is that the lower population-based R-indicators result from the large differences between sample and population distributions of the auxiliary variables. For a sample size of 33,584 persons, a test of the differences between sample and population distributions is significant for all three variables at the 5% level. The available Dutch Health Survey net sample does not contain sampling units with frame and/or other administrative errors as well as out-of-scope populations such as institutionalized persons. This modification plus some additional, small tailoring to interviewer workloads, most likely caused sample distributions to differ from the original population counts. This points at the “Achilles heel” of population-based R-indicators: it is imperative that there is no disparity between definitions and populations.

• Previous
• Article main page
• Next