How to decompose the non-response variance: A total survey error approach
Section 3. Unit-level error decomposition of variance components

• Previous
• Next

This section describes the approach used to evaluate the contribution of a given nonresponding unit, $\lambda \in s_m,$ to the estimated total variance for the estimation of a total for a given variable.

The unit-level error decomposition, ${\delta }_{\lambda },$ of the total variance for a given unit, $\lambda ,$ is defined as the difference between the estimated total variance, and the projected total variance, i.e., ${\delta }_{\lambda }\left({\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)\right)\equiv {\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)-{\widehat{V}}_{\text{TOT}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right).$ The superscript $\left(\lambda \right)$ is used to indicate projected quantities when unit $\lambda$ is converted to a respondent. So, ${\delta }_{\lambda }\left({\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)\right)$ can be seen as the expected gain, in terms of total variance, of converting a nonrespondent unit $\lambda$ to a respondent.

In order to get ${\delta }_{\lambda }\left({\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)\right),$ $\lambda$ is moved from $s_m$ to $s_r,$ generating the new partition $P_s^{\left(\lambda \right)}$ of the sample from $P_s$ where $P_s^{\left(\lambda \right)}=\left\{s_r^{\left(\lambda \right)}\text{},s_m^{\left(\lambda \right)}\right\},$ $s_r^{\left(\lambda \right)}=s_r\cup \left\{\lambda \right\}$ and $s_m^{\left(\lambda \right)}=s_m\backslash \left\{\lambda \right\},$ as illustrated in Figure 3.1.

Some assumptions are necessary to decompose the variance components. It is recognized that these assumptions may not perfectly hold in reality. However, they can be used to generate accurate results, as shown in the simulation in Section 4. The required assumptions are:

1. Projected reported value: let $\lambda \in s_m$ be converted to a response and let $y_{\lambda }^{\left(\lambda \right)}=y_{\lambda }^{*}.$
2. Projected imputation parameters: $\forall k\in s_m,$ ${\widehat{\mu }}_k^{\left(\lambda \right)}={\widehat{\mu }}_k$ and ${\widehat{\sigma }}_k^{\left(\lambda \right)}={\widehat{\sigma }}_k.$
3. Projected imputation relationship matrix: $\forall k\in s_m$ and $\forall l\in s_r,$ ${\phi }_{lk}^{(\lambda )}=0$ if $l=\lambda$ or if $k=\lambda$ or ${\phi }_{lk}^{\left(\lambda \right)}={\phi }_{lk}$ otherwise. Similarly, ${\phi }_{0k}^{\left(\lambda \right)}=0$ if $k=\lambda$ or ${\phi }_{0k}^{\left(\lambda \right)}={\phi }_{0k}$ otherwise.

Assumption 1 implies that if a nonresponding unit, $\lambda ,$ would have been converted to a respondent, its reported value is equal to its imputed value. This is not true generally, but the imputed value is our best estimate. The expectation is that this imputed value is close enough to the reported value to estimate the error on the variance components. This assumption will have an impact when the sampling variance is decomposed.

Assumption 2 states that the estimated parameters of the imputation model would remain unchanged if $\lambda$ were a respondent. In the case of a consistent imputation model parameter estimator, this assumption becomes more realistic when $s_r$ is larger.

Finally, assumption 3 means that the imputation relationship between nonrespondents and respondents remains unchanged, except when unit $\lambda$ is involved. In other words, the converted unit, $\lambda ,$ is no longer imputed from respondents, but will not be used to impute other nonresponding units. Figure 3.2 shows how assumption 3 is reflected in terms of the phi matrix.

Therefore, the compensation weight, $W_{dl}^{\left(\lambda \right)}\text{},$ of a responding unit, $\forall l\in s_r,$ is projected as

$$\begin{array}{ll}{W}_{dl}^{\left(\lambda \right)}\hfill & ={\displaystyle \sum _{k\in s_m^{\left(\lambda \right)}}{w}_k{d}_k{\phi }_{lk}^{\left(\lambda \right)}}\hfill \\ \hfill & ={\displaystyle \sum _{k\in s_m}{w}_k{d}_k{\phi }_{lk}}-w_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }\hfill \\ \hfill & =W_{dl}-w_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }.(3.1)\hfill \end{array}$$

The marginal weight from the converted unit $\lambda$ is withdrawn from the original compensation weight, $W_{dl},$ to obtain the new $W_{dl}^{\left(\lambda \right)}\text{}.$ Note that $W_{d\lambda }^{\left(\lambda \right)}={\displaystyle {\sum }_{k\in s_m^{\left(\lambda \right)}}{w}_k{d}_k{\phi }_{\lambda k}^{\left(\lambda \right)}}=0$ because ${\phi }_{\lambda k}^{\left(\lambda \right)}=0$ under assumption 3. As mentioned above, it means that $\lambda$ isn’t used to impute nonrespondents.

In the next subsections, the unit-level error decomposition for unit $\lambda$ is computed for the four variance components, as described in Section 2.3.

3.1  Unit-level error decomposition of the naive sampling variance

The quantity ${\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)$ depends on the $y-$ values, the final weights and the first-order and second-order selection probabilities. The unit-level error decomposition of the naive sampling variance component ${\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)$ is trivial since the assumption that unit $\lambda$ goes from $s_m$ to $s_r$ does not change weights and selection probabilities. Under assumption 1, the projected reported value $y_{\lambda }^{\left(\lambda \right)}$ is set to $y_{\lambda }^{*}$ so that ${\widehat{V}}_{\text{ORD}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right)={\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)$ when $\lambda$ is converted to a responding unit. Consequently, the decomposition of ${\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)$ is given by

$${\delta }_{\lambda }\left({\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)\right)\equiv {\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)-{\widehat{V}}_{\text{ORD}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right)=0.(3.2)$$

This result is consistent with the idea that the naive sampling variance point estimate will likely change, but it is not expected to decrease with an extra responding unit.

3.2  Unit-level decomposition of the correction to the sampling variance component

The unit-level error decomposition for unit $\lambda$ of the correction to the sampling variance component, ${\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right),$ is given by

$$\begin{array}{ll}{\delta }_{\lambda }\left({\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)\right)\hfill & \equiv {\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)-{\widehat{V}}_{\text{DIF}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right)\hfill \\ \hfill & ={\displaystyle \sum _{k\in s_m}\left(1-{\pi }_k\right)d_k{w}_k^2{\widehat{\sigma }}_k^2}-{\displaystyle \sum _{\lambda \in s_m^{\left(\lambda \right)}}\left(1-{\pi }_k\right)d_k{w}_k^2{\left({\widehat{\sigma }}_k^{\left(\lambda \right)}\right)}^2}.\hfill \end{array}$$

Under assumption 2, ${\widehat{\sigma }}_k^{\left(\lambda \right)}={\widehat{\sigma }}_k,$ so that

$${\delta }_{\lambda }\left({\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)\right)=\left(1-{\pi }_{\lambda }\right)d_{\lambda }{w}_{\lambda }^2{\widehat{\sigma }}_{\lambda }^2.(3.3)$$

The astute reader will notice that the actual sampling variance (not its estimation) should not be impacted by whether or not a unit is a respondent. However, we decided to include the impact of a unit on the sampling variance estimation in order to be coherent in the way we treat the three components $V_{\text{SAM}}\left({\widehat{t}}_d\right),$ $V_{\text{NR}}\left({\widehat{t}}_d\right)$ and $V_{\text{MIX}}\left({\widehat{t}}_d\right).$

3.3  Unit-level decomposition of the non-response variance component

The unit-level error decomposition for unit $\lambda$ of the non-response variance component ${\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)$ is given by

$$\begin{array}{ll}{\delta }_{\lambda }\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right)\hfill & \equiv {\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)-{\widehat{V}}_{\text{NR}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right)\hfill \\ \hfill & =\left({\displaystyle \sum _{l\in s_r}{W}_{dl}^2{\widehat{\sigma }}_l^2}+{\displaystyle \sum _{k\in s_m}{w}_k^2{d}_k{\widehat{\sigma }}_k^2}\right)-\left({\displaystyle \sum _{l\in s_r^{\left(\lambda \right)}}{\left(W_{dl}^{\left(\lambda \right)}\right)}^2{\left({\widehat{\sigma }}_l^{\left(\lambda \right)}\right)}^2}+{\displaystyle \sum _{k\in s_m^{\left(\lambda \right)}}{w}_k^2{d}_k{\left({\widehat{\sigma }}_k^{\left(\lambda \right)}\right)}^2}\right).\hfill \end{array}$$

Under assumptions 2 and 3, ${\widehat{\sigma }}_k^{\left(\lambda \right)}={\widehat{\sigma }}_k$ and $W_{d\lambda }^{\left(\lambda \right)}=0.$ This can be rewritten as

$${\delta }_{\lambda }\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right)=\left({\displaystyle \sum _{l\in s_r}{W}_{dl}^2{\widehat{\sigma }}_l^2}-{\displaystyle \sum _{l\in s_r}{\left(W_{dl}^{\left(\lambda \right)}\right)}^2{\widehat{\sigma }}_l^2}\right)+w_{\lambda }^2{d}_{\lambda }{\widehat{\sigma }}_{\lambda }^2.$$

Using formula (3.1), this becomes

$$\begin{array}{ll}{\delta }_{\lambda }\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right)\hfill & =\left({\displaystyle \sum _{l\in s_r}{W}_{dl}^2{\widehat{\sigma }}_l^2}-{\displaystyle \sum _{l\in s_r}{\left(W_{dl}-w_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }\right)}^2{\widehat{\sigma }}_l^2}\right)+w_{\lambda }^2{d}_{\lambda }{\widehat{\sigma }}_{\lambda }^2\hfill \\ \hfill & =\left({\displaystyle \sum _{l\in s_r}{W}_{dl}^2{\widehat{\sigma }}_l^2-\left(W_{dl}^2-2W_{dl}{w}_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }+w_{\lambda }^2{d}_{\lambda }{\phi }_{l\lambda }^2\right){\widehat{\sigma }}_l^2}\right)+w_{\lambda }^2{d}_{\lambda }{\widehat{\sigma }}_{\lambda }^2\hfill \\ \hfill & ={\displaystyle \sum _{l\in s_r}\left(2W_{dl}{w}_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }-w_{\lambda }^2{d}_{\lambda }{\phi }_{l\lambda }^2\right){\widehat{\sigma }}_l^2}+w_{\lambda }^2{d}_{\lambda }{\widehat{\sigma }}_{\lambda }^2.(3.4)\hfill \end{array}$$

3.4  Unit-level decomposition of the mixed variance component

Finally, the impact of unit $\lambda$ on the variance component term, ${\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right),$ is given by

$$\begin{array}{ll}{\delta }_{\lambda }\left({\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)\right)\hfill & \equiv {\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)-{\widehat{V}}_{\text{MIX}}^{\left(\lambda \right)}\left({\widehat{t}}_d\right)\hfill \\ \hfill & =\left(2{\displaystyle \sum _{l\in s_r}{W}_{dl}\left(w_l-1\right)d_l{\widehat{\sigma }}_l^2}-2{\displaystyle \sum _{k\in s_m}{w}_k\left(w_k-1\right)d_k{\widehat{\sigma }}_k^2}\right)\hfill \\ \hfill & -\left(2{\displaystyle \sum _{l\in s_r^{\left(\lambda \right)}}{W}_{dl}^{\left(\lambda \right)}\left(w_l-1\right)d_l{\left({\widehat{\sigma }}_l^{\left(\lambda \right)}\right)}^2}-2{\displaystyle \sum _{k\in s_m^{\left(\lambda \right)}}{w}_k\left(w_k-1\right)d_k{\left({\widehat{\sigma }}_k^{\left(\lambda \right)}\right)}^2}\right).\hfill \end{array}$$

This equation can be rewritten as follows, under assumptions 2 and 3 and equation (3.1)

$$\begin{array}{ll}{\delta }_{\lambda }\left({\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)\right)\hfill & =\left(2{\displaystyle \sum _{l\in s_r}{W}_{dl}\left(w_l-1\right)d_l{\widehat{\sigma }}_l^2}-2{\displaystyle \sum _{k\in s_m}{w}_k\left(w_k-1\right)d_k{\widehat{\sigma }}_k^2}\right)\hfill \\ \hfill & -\left(2{\displaystyle \sum _{l\in s_r}\left(W_{dl}-w_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }\right)\left(w_l-1\right)d_l{\widehat{\sigma }}_l^2}-2{\displaystyle \sum _{k\in s_m^{\left(\lambda \right)}}{w}_k\left(w_k-1\right)d_k{\widehat{\sigma }}_k^2}\right)\hfill \\ \hfill & =2{\displaystyle \sum _{l\in s_r}{w}_{\lambda }{d}_{\lambda }{\phi }_{l\lambda }\left(w_l-1\right)d_l{\widehat{\sigma }}_l^2}-2w_{\lambda }\left(w_{\lambda }-1\right)d_{\lambda }{\widehat{\sigma }}_{\lambda }^2.(3.5)\hfill \end{array}$$

In Section 2.3, the estimation of the total variance, ${\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right),$ has been defined as ${\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)={\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)+{\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)+{\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)+{\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right).$ Similarly, the impact of unit $\lambda$ on ${\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)$ is defined as

$${\delta }_{\lambda }\left({\widehat{V}}_{\text{TOT}}\left({\widehat{t}}_d\right)\right)={\delta }_{\lambda }\left({\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)\right)+{\delta }_{\lambda }\left({\widehat{V}}_{\text{DIFF}}\left({\widehat{t}}_d\right)\right)+{\delta }_{\lambda }\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right)+{\delta }_{\lambda }\left({\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)\right),$$

where ${\delta }_{\lambda }\left({\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right)\right),$ ${\delta }_{\lambda }\left({\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)\right),$ ${\delta }_{\lambda }\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right),$ and ${\delta }_{\lambda }\left({\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)\right)$ are respectively given by equations (3.2), (3.3), (3.4) and (3.5).

It can be observed (proofs are given in the appendix) that ${\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)={\displaystyle {\sum }_{k\in s_m}{\delta }_k\left({\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)\right)}$ and ${\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)={\displaystyle {\sum }_{k\in s_m}{\delta }_k\left({\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)\right)}.$ However, this linear relation doesn’t hold for ${\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right).$ This property is important to consider because, for ${\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right)$ and ${\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right),$ the sum of the unit-level errors on all nonresponding units, $k\in s_m,$ is equal to the corresponding estimated variance component. In the case of non-response variance component, the sum of the errors is different than ${\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right).$ The difference is given by

$${\displaystyle \sum _{k\in s_m}{\delta }_k\left({\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)\right)}-{\widehat{V}}_{\text{NR}}\left({\widehat{t}}_d\right)={\displaystyle \sum _{l\in s_r}\left({\left({\displaystyle \sum _{k\in s_m}{w}_k{d}_k{\phi }_{lk}}\right)}^2-{\displaystyle \sum _{k\in s_m}{w}_k^2{d}_k{\phi }_{lk}^2}\right){\widehat{\sigma }}_l^2}.(3.6)$$

This difference can be relatively small, especially in business surveys characterized with asymmetric data. This is the case when ${\max }_{k\in s_m}\left(w_k{d}_k{\phi }_{lk}\right)\cong {\displaystyle {\sum }_{k\in s_m}{w}_k{d}_k{\phi }_{lk}}.$ This is in line with the results shown by Mills et al. (2013).

Overall, the total variance can be considered as approximately linear in terms of the unit-level errors, especially in the case of sample surveys where ${\widehat{V}}_{\text{ORD}}\left({\widehat{t}}_d\right),$ ${\widehat{V}}_{\text{DIF}}\left({\widehat{t}}_d\right),$ and ${\widehat{V}}_{\text{MIX}}\left({\widehat{t}}_d\right)$ are significant contributors to the total variance.

• Previous
• Article main page
• Next