An assessment of accuracy improvement by adaptive survey design
Section 5. Empirical evidence

• Previous
• Next

The empirical work reported in this section illustrates some of the theory in earlier sections. We used survey data from Statistics Sweden’s Labour Force Survey (LFS) in its 2012 edition. Results are given in Tables 5.1 to 5.4.

We created the LFS2012 data set by combining the 12 wave-one samples in 2012. The monthly first-wave size counts approximately 2,650 units (persons). This LFS2012 data set is treated as a simple random sample $s$ of size $n=$  32,265. Response or nonresponse in the actual data collection is recorded and available for all those units. The response rate in the actual data collection was 70.6%. The data and the construction of the experimental response sets in the tables are described in further detail in Särndal and Lundquist (2017).

In analyzing the LFS2012 data set, we used different $x$ -vectors obtained by crossing binary $x$ -variables: Educ, equal to 1 for a person with high education, 0 otherwise; Owner, equal to 1 for an owner of his/her place of residence, 0 otherwise; Origin, equal to 1 for person born in Sweden, 0 otherwise; Civil, equal to 1 for married or widower, 0 otherwise; Gender, equal to 1 for male, 0 otherwise. Results are presented here for two $x$ -vectors.

The $x$ -vector in Tables 5.1 and 5.3 represents the crossing of the first three binary variables: $x=x_1=$ (Educ  $\times$  Owner  $\times$  Origin); its dimension, equal to the number of groups, is $J=2^3=8.$ The $x$ -vector in Tables 5.2 and 5.4 was obtained crossing also by the binary Civil: $x=x_2=$ (Educ  $\times$  Owner  $\times$  Origin  $\times$  Civil), of dimension $J=2^4=16.$

In this experimental study, we used two $y$ -variables, Employed and Income. Both are register variables; with values $y_k$ available for all units $k\in s.$ They are thus “pseudo $y$-variables” rather than real survey $y$ -variables. Knowing $y_k$ for $k\in s$ allows us to compute regression coefficients and $y$ -means both for the response and for the nonresponse. The $y$ -variable is Employed in Table 5.1 (with $x=x_1)$ and in Table 5.2 (with $x=x_2).$ Employed is binary, with value $y_k=1$ if $k$ is an employed person, zero otherwise. The $y$ -variable is Income in Table 5.3 (with $x=x_1)$ and in Table 5.4 (with $x=x_2).$ Income is a continuous register variable, available from the Swedish tax register. We standardized Income to have zero mean and unit variance over $s.$ Considerable variability and skewness of Income creates some volatility in the results. One individual $y_k$ -value can have considerable impact within a smallish group, as compared with the more stable performance of the $0/1$ $y$ -variable Employed.

The four rows of Tables 5.1 to 5.4 refer to a series of four different response sets. The important feature is that they are, by construction, sets with progressively lower IMB. The first, Act, is the response set recorded in the actual 2012 Labour Force Survey data collection for the 32,265 units (persons). The last three response sets, A65, A63, A60, taken from Särndal and Lundquist (2017), are constructed from Act by the threshold method to have successively lower imbalance IMB.

To illustrate, the response set A65 was created from Act by dropping, at each of a sequence of intervention points, those responding units in Act whose computed response propensity exceeds the threshold 0.65. This tends to even out differences in response propensity, so this construction reduces IMB, and the overall response rate $P$ drops somewhat. The response sets denoted A63 and A60 were obtained by setting the threshold at 0.63 and 0.60, respectively; IMB and $P$ are again reduced.

The table columns show: Response rate $P,$ imbalance IMB (multiplied by 10²), the components $D_1$ and $D_2$ of the deviation ${\Delta }_{\text{CAL}}=D_1+D_2$ (all three multiplied by 10²), the $D_2$ proportion of ${\Delta }_{\text{CAL}},$ $\text{Prop}{D}_2=100\times D_2/{\Delta }_{\text{CAL}},$ and finally the $D_1$ size relative, $\text{Rel}{D}_1=D_1/\text{mean}\left(D_1\right),$ where mean $\left(D_1\right)$ is the arithmetic mean of the four table values of $D_1.$ We use $\text{Rel}{D}_1$ to see if it is near one for all rows, in line with the contention that $D_1$ is little affected by a reduced imbalance IMB.

The results in Tables 5.1 to 5.4 prompt the following observations, of which the second and third are particularly interesting, in that they confirm what theory in earlier sections suggests, namely that when IMB is reduced, $\text{Prop}{D}_2$ drops quite distinctly, whereas $\text{Rel}{D}_1$ stays very close to one.

1. In all four tables, $D_1$ and $D_2$ have the same sign. Both are positive, and the reduced IMB (from first to fourth row) brings a reduction in ${\Delta }_{\text{CAL}}=D_1+D_2,$ due almost entirely to the drop in $D_2.$
2. In each table, the relatives $\text{Rel}{D}_1$ are not far from 1. Thus $D_1$ is remarkably constant over the four rows (response sets), thus insensitive to the reduced IMB.
3. In each table, $D_2$ and $\text{Prop}{D}_2$ are decreasing over the four rows, as theory makes us expect. In fact, $\text{Prop}{D}_2$ tends to zero with IMB. The effect of the $y$-variable is important; $\text{Prop}{D}_2$ is considerably greater for Income than for Employed.
4. The change of $x$ -vector is an important influence on $D_1,$ for both $y$ -variables. Going from the smaller $x=x_1$ (Tables 5.1 and 5.3) to the more extensive $x=x_2$ (Tables 5.2 and 5.4) brings considerable reduction in $D_1,$ whereas $D_2$ changes very little.

We also examined the distribution of the $J$ group divergences ${\delta }_j,$ $j=1,\dots ,J,$ for the vectors $x_1$ (with $J=8),$ $x_2$ (with $J=16)$ and $x_3=$ (Educ  $\times$  Owner  $\times$  Origin  $\times$  Civil  $\times$  Gender) (with $J=32).$ For both Employed and for Income, and for all four response sets, there are, not unexpectedly, a few large positive ${\delta }_j$ and a certain skewness in the distribution. For both variables, $\overline{\delta }={\displaystyle {\sum }_{j=1}^J{W}_j{\delta }_j}$ is clearly positive. That is, on average over the groups, $y$-means are, for these data, higher for respondents than for non-respondents. It is a feature of those particular $y$ -variables.

For $x_3,$ a plot of the 32 divergences ${\delta }_j$ against the nonresponse differential $Q_j-Q$ shows a majority of points in the vicinity of zero on both axes, and scattered values in the four quadrants of the plot. The plot does suggest a positive, although not very pronounced, correlation between ${\delta }_j$ and $Q_j-Q,$ which is what makes the covariance term $D_2$ positive.

• Previous
• Article main page
• Next