Reducing the response imbalance: Is the accuracy of the survey estimates improved? Section 1. Introduction
The problem of accurate estimation despite considerable nonresponse needs to be examined from two time dependent angles: First, ways to handle the data collection, then ways to handle the estimation with the data that were finally collected. The first activity may require substantial resources. In a telephone survey, the daily scheduling of contact attempts, the interaction with the interviewers, and consideration for their workloads, can be expensive efforts. The estimation stage is administratively simpler; there is a search for the best auxiliary variables for a calibrated nonresponse adjustment weighting, whereupon the computation of estimates is usually carried out with existing software.
The data collection is in focus in the literature on Responsive Design; Groves (2006), Groves and Heeringa (2006) are early references. Adaptive survey designs are discussed in Wagner (2008). One idea in this research tradition is that a data collection that extends over a period of time might be inspected at suitable decision points, where action may be taken to realize in the end a well-balanced set of respondents. Schouten, Calinescu and Luiten (2013) explain how adaptive survey designs may be tailored to optimize response rates and reduce nonresponse selectivity, with cost aspects taken into account. Much exploratory work has been carried out on responsive (or adaptive) design. Seeking well balanced or representative response can be pursued as a goal in itself. Different avenues have been explored: Case prioritization, (Peytchev, Riley, Rosen, Murphy and Lindblad 2010); stopping rules to halt data collection attempts for specific sample units, (Rao, Glickman and Glynn 2008; Wagner and Raghunathan 2010); uses of paradata more generally to manage the survey response, (Couper and Wagner 2011).
Measuring and controlling the imbalance belongs in the data collection phase. The imbalance statistic (see Section 3) has a central role in this article; it was used for example in Särndal (2011), Lundquist and Särndal (2013), Särndal and Lundquist (2014a, 2014b). It is related to the indicator (R for representativity); see Schouten, Cobben and Bethlehem (2009) and Bethlehem, Cobben and Schouten (2011).
The second time slice relies on estimation theory to resolve the challenge of nonresponse, primarily how to achieve low bias in the estimates. Viewed strictly as an estimation problem, it is an activity in itself, after a completed data collection. The set of responding units is fixed; the data on those units is a “frozen” supply. The choice of auxiliary variables plays a crucial role. The “best ones” should be selected. This aspect has been dealt with extensively, as in Särndal and Lundström (2005). Two factors are traditionally cited as important for the accuracy of the estimates: The degree to which the chosen auxiliary variables can explain the study variable and the degree to which these variables can explain the 0/1 response indicator showing presence or not in the set of respondents. Each of the two degrees of explanation is partial at best, not perfect. The two roles of the auxiliary variables interact, as recognized for example in Little and Vartivarian (2005). An extensive review of weighting adjustment procedures for nonresponse is given in Brick (2013).
The supply of auxiliary variables depends on the survey environment. In Scandinavia, surveys on individuals and households can draw on extensive sources administrative registers of auxiliary variables. This is increasingly so in other countries also.
One view holds that the estimation is the all-important step: Whatever may be accomplished at the data collection stage balancing, improved representativeness is perhaps superfluous; achieving best possible accuracy in the estimates can be dealt with effectively at the estimation stage, by clever use of the auxiliary variables in a nonresponse adjustment weighting or in other ways. This point of view is supported for example in Beaumont, Bocci and Haziza (2014).
Nevertheless, it is clear that measurable aspects of the data collection will influence the accuracy of the estimates that are ultimately produced. One such measure is the imbalance statistic defined in Section 3. In this article, the two time dependent activities are taken into account: Balancing the response should be combined with efficient estimation methods, to get in the end the best possible (most accurate) estimates. Such a view underlies, for example, Schouten, Cobben, Lundquist and Wagner (2014).
The motivation for this paper is as follows: Methods exist for different courses of action stopping rules, case prioritization, and others during data collection, so as to get in the end a favourable response set Särndal and Lundquist (2014a, 2014b) used the imbalance statistic given in Section 3 as a tool to achieve low imbalance in the final response set. Considering that auxiliary variables will also be used in the estimation, to what extent, if any, will better accuracy in the estimates follow from low imbalance in the preceding data collection? There are encouraging signs, as in Särndal and Lundquist (2014a), that lower imbalance creates some accuracy improvement, although modest. That work was empirical; in this article we give mathematical/analytical support for a similar conclusion.
The contents are arranged as follows: The survey background (Section 2) and the imbalance statistic (Section 3) are presented. The regression relationship that of the study variable on the auxiliary vector is important (Section 4), notably for the estimator (called CAL) obtained by calibrated nonresponse weight adjustment (Section 5). The deviation of the calibration (CAL) estimator from the (unbiased) estimator requiring full response is analyzed (Section 6, Section 7, Section 8), showing how deviation depends on imbalance. Two results are presented on statistical properties (mean and variance) of the CAL deviation. In particular, the variance of that deviation is shown to be, approximately, a linear function of the imbalance statistic. Hence the deviation is likely to be smaller, and estimates more accurate, if the imbalance can be reduced during data collection. The theoretical results are empirically validated (Section 9) using data from an Estonian household survey. The statistical software R is used; R Core Team (2014). A discussion (Section 10) concludes the article. Three appendices provide the necessary proofs and derivations.
- Date modified: