Tests for evaluating nonresponse bias in surveys Section 5. DiscussionTests for evaluating nonresponse bias in surveys Section 5. Discussion

In this paper, we considered tests for nonresponse bias after poststratification or inverse propensity weighting has been used. The arguments in the theorems could be extended to similar methods that are used to adjust for nonresponse bias such as raking, which iteratively poststratifies to marginal population totals, or calibration, which adjusts the weights so that estimated population totals agree with control totals for a set of auxiliary variables. Haziza and Lesage (2016) argued that using a two-step procedure of propensity weighting followed by calibration provides more protection against nonresponse bias than using calibration alone in a single step, because single-step calibration implies a model relating the response propensities and the calibration variables and that model may be misspecified. The tests proposed in this paper could be extended to situations in which both propensity weighting and poststratification are used, or could be used separately to assess the bias removed in each step of a two-step process.

We employed the jackknife for the replication variance estimation. However, all of the estimators are smooth functions of population totals, so other replication variance estimators such as balanced repeated replication or bootstrap could be used as well.

A challenge for evaluating nonresponse bias is the limited amount of information available for the selected sample. For some surveys all available auxiliary information is used or considered for forming poststrata, raking classes, or inverse propensity weights. The poststratified estimator for characteristics used in the poststratification has no variance or bias, so testing these or closely related characteristics will not uncover nonresponse bias in other survey variables. Auxiliary variables that are not used for nonresponse adjustments are often omitted only because they were not selected in model selection method used to form the poststrata or select variables for the logistic regression, and that typically occurs because they have low explanatory power for predicting the response indicator after the other variables are included in the model. For surveys with less frame information, it may be possible to obtain auxiliary information from other sources, such as administrative records associated with the respondents’ addresses or paradata. It is important to make sure that the variables used to test nonresponse bias are recorded consistently for respondents and nonrespondents. If, for example, $y$ is the interviewer’s curbside assessment about whether children are present in the household, that initial assessment should be used for both respondents and nonrespondents: the assessment used in the nonresponse bias analysis should not be updated after the interviewer ascertains the actual number of children in a responding household.

After testing available variables for nonresponse bias, we still do not know whether the adjustments have removed the bias for outcome variables that are available only for the respondents. Abraham, Helms and Presser (2009) and Kohut, Keeter, Doherty, Dimock and Christian (2012) found that estimates of volunteering and civic participation are higher from surveys with low response rates than from the Current Population Survey, indicating that weighting adjustments do not remove bias for civic engagement variables although they appear to remove bias for demographic variables and home ownership. But testing a wide range of auxiliary variables for residual bias may give more confidence in the results of a survey on the untested variables, or may indicate concerns about inferences from the survey for variables of interest. We recommend that survey designers plan the survey with nonresponse bias assessment in mind, and collect additional information for the selected sample whenever possible. In general, the more information that can be collected about the selected sample, the better.

The comparison of estimates using different sets of weights may be of special interest when studying responsive or adaptive design strategies such as those described in Groves and Heeringa (2006) and summarized in Tourangeau, Brick, Lohr and Li (2016). In these, later phases of the design are modified using information gleaned in the early returns. One responsive design strategy may be to estimate response rates after the first phase of the survey, and then to allocate resources in the second phase to equalize rates across subgroups of interest. In an experimental comparison of different responsive design strategies, it may be of interest to evaluate the estimated nonresponse bias from the strategies. Riddles, Marker, Rizzo, Wiley and Zukerberg (2015) compared nonresponse-weighted estimates from different data cutoff points in the U.S. Schools and Staffing Survey, to see if estimates changed with earlier truncation of data collection.

The results in Theorems 1 through 5 are expressed for probability samples. There is increased interest in using nonprobability samples to study populations (Baker, Brick, Bates, Battaglia, Couper, Dever, Gile and Tourangeau 2013). Proponents of nonprobability samples argue that with response rates sometimes below 10%, an inexpensive large nonprobability sample can have smaller mean squared error than a small probability sample. The same methods of poststratification and inverse propensity weighting are typically used with nonprobability samples. The tests proposed in this paper can be adapted for use with nonprobability samples, provided that auxiliary information is known for a collection of individuals that can serve as a stand-in for a sampling frame. For a web survey, it might be possible to compare characteristics of persons visiting the web page with those of persons completing the survey. Further research is needed in this area.

Acknowledgements

The authors thank the reviewers for their helpful suggestions that led to improvements in the article.

Appendix

The following lemma shows that the additional variability due to the stochastic response mechanism is $O (M^{2} / n) .$

Lemma 1. Suppose assumptions (A3) and (A5) are met, and that $| q_{h i k} | \leq Q$ for all $(h i k) \in U .$ Then

$E [V (\sum_{h i k \in U} Z_{h i k} w_{h i k} q_{h i k} r_{h i k} | Z)] = O (M^{2} / n) .$

Proof. By assumption (A5),

$\begin{array}{l} | E [V (\sum_{h i k \in U} Z_{h i k} w_{h i k} q_{h i k} r_{h i k} | Z)] | & = | E [\sum_{h =1}^{H} \sum_{i =1}^{N_{h}} \sum_{k =1}^{M_{h i}} \sum_{p =1}^{M_{h i}} Z_{h i k} Z_{h i p} w_{h i k} w_{h i p} Cov (r_{h i k}, r_{h i p}) q_{h i k} q_{h i p}] | \\ \leq Q^{2} E [\sum_{h =1}^{H} \sum_{i =1}^{N_{h}} \sum_{k =1}^{M_{h i}} \sum_{p =1}^{M_{h i}} Z_{h i k} Z_{h i p} w_{h i k} w_{h i p}] \\ = Q^{2} \sum_{h =1}^{H} \sum_{i =1}^{N_{h}} \sum_{k =1}^{M_{h i}} \sum_{p =1}^{M_{h i}} P [(h i) \in S] P [k \in S_{h i}, p \in S_{h i}] w_{h i k} w_{h i p} \\ = O (M^{2} / n) . \end{array}$

The last line is implied by (A3).

Proof of Theorem 1. From (2.4),

$V_{1} (\hat{θ}) = V [\sum_{c =1}^{C} \frac{1}{p_{c}} ({\hat{Y}}_{c}^{R} - {\bar{Y}}_{c}^{R} ({\hat{M}}_{c}^{R} - M_{c}^{R})) - {\hat{Y}}_{S S}]$

and

$V_{2} (\hat{θ}) = V [\sum_{c =1}^{C} \frac{{\hat{T}}_{c}}{p_{c}}] + 2 Cov [\sum_{c =1}^{C} \frac{{\hat{T}}_{c}}{p_{c}}, \sum_{c =1}^{C} \frac{({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) {\hat{M}}_{c}^{R}}{p_{c}} - {\hat{Y}}_{S S}] .$

The leading term simplifies to

$\begin{array}{l} V_{1} (\hat{θ}) & = V [\sum_{h i k \in U} Z_{h i k} w_{h i k} \sum_{c =1}^{C} δ_{c h i k} {\frac{r_{h i k}}{p_{c}} (y_{h i k} - {\bar{Y}}_{c}^{R}) - y_{h i k}}] \\ = V [E [\sum_{h i k \in U} Z_{h i k} w_{h i k} \sum_{c =1}^{C} δ_{c h i k} {\frac{r_{h i k}}{p_{c}} (y_{h i k} - {\bar{Y}}_{c}^{R}) - y_{h i k}} | Z]] \\ + E [V [\sum_{h i k \in U} Z_{h i k} w_{h i k} \sum_{c =1}^{C} δ_{c h i k} {\frac{r_{h i k}}{p_{c}} (y_{h i k} - {\bar{Y}}_{c}^{R}) - y_{h i k}} | Z]] \\ = V (\sum_{h i k \in U} Z_{h i k} w_{h i k} e_{R h i k}) + E [V [\sum_{h i k \in U} Z_{h i k} w_{h i k} \sum_{c =1}^{C} δ_{c h i k} \frac{r_{h i k}}{p_{c}} (y_{h i k} - {\bar{Y}}_{c}^{R}) | Z]] . \end{array}$

Lemma 1 and Assumption (A4), which guarantees that $1 / p_{c}$ is bounded, imply that the second term is $O (M^{2} / n) .$

To show that $V_{2} (\hat{θ}) = o (M^{2} / n),$ note that by (A4) and the Cauchy-Schwarz inequality,

$V [\frac{{\hat{T}}_{c}}{p_{c}}] \leq \frac{1}{ε^{2}} \sum_{c =1}^{C} \sum_{d =1}^{C} \sqrt{V [({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\hat{M}}_{c}^{R} - M_{c}^{R})] V [({\bar{y}}_{d}^{R} - {\bar{Y}}_{d}^{R}) ({\hat{M}}_{d}^{R} - M_{d}^{R})]} .$

Assumption (A2) implies (Fuller 2009, Theorem 1.3.2) that

$\sqrt{n} [\begin{matrix} {\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R} \\ {\hat{M}}_{c}^{R} / M_{c}^{R} - 1 \end{matrix}] \to N (0, Σ_{c})$

as $n \to \infty,$ where $Σ_{c}$ is a non-negative definite matrix. Consequently,

${(\frac{n}{M_{c}^{R}})}^{2} V [({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\hat{M}}_{c}^{R} - M_{c}^{R})] \to Σ_{c} [1,1] Σ_{c} [2,2] + 2 {(Σ_{c} [1,2])}^{2};$

applying the Cauchy-Schwarz inequality to the covariance term implies that $V_{2} (\hat{θ}) = o (M^{2} / n) .$

Proof of Theorem 2. We show that

$\tilde{V} (θ) = \sum_{h =1}^{H} \frac{n_{h}}{n_{h} - 1} \sum_{i \in S_{h}} {({\tilde{b}}_{h i} - {\tilde{b}}_{h})}^{2}$

is consistent, where

${\tilde{b}}_{h i} = \sum_{k \in S_{h i}} w_{h i k} {\sum_{c =1}^{C} \frac{1}{p_{c}} r_{h i k} δ_{c h i k} (y_{h i k} - {\bar{Y}}_{c}^{R}) - y_{h i k}} = \sum_{k \in S_{h i}} w_{h i k} {\tilde{e}}_{r h i k}$

and ${\tilde{b}}_{h} = \sum_{i \in S_{h}} {\tilde{b}}_{h i} / n_{h} .$ Arguments in Yung and Rao (2000) then imply that $(n / M^{2}) [\tilde{V} (θ) - \hat{V} (θ)]$ converges to zero in probability.

Note that

$E [{\tilde{b}}_{h i} | Z] = \sum_{k \in S_{h i}} w_{h i k} e_{R h i k},$

$\begin{array}{l} E [{\tilde{b}}_{h i}^{2} | Z] & = E [{(\sum_{k \in S_{h i}} w_{h i k} {\sum_{c =1}^{C} \frac{1}{p_{c}} [R_{h i k} + r_{h i k} - R_{h i k}] δ_{c h i k} (y_{h i k} - {\bar{Y}}_{c}^{R}) - y_{h i k}})}^{2} | Z] \\ = {(\sum_{k \in S_{h i}} w_{h i k} e_{R h i k})}^{2} + V (\sum_{k \in S_{h i}} w_{h i k} {\tilde{e}}_{r h i k} | Z), \end{array}$

and

$\begin{array}{l} E [{\tilde{b}}_{h}^{2} | Z] & = \frac{1}{n_{h}^{2}} E [\sum_{i \in S_{h}} b_{h i}^{2} + \sum_{i \in S_{h}} \sum_{j \neq i} b_{h i} b_{h j} | Z] \\ = \frac{1}{n_{h}^{2}} \sum_{i \in S_{h}} V (\sum_{k \in S_{h i}} w_{h i k} {\tilde{e}}_{r h i k} | Z) + {(\frac{1}{n_{h}} \sum_{i \in S_{h}} \sum_{k \in S_{h i}} w_{h i k} e_{R h i k})}^{2} . \end{array}$

This implies that

$\begin{array}{l} E [\sum_{i \in S_{h}} {[{\tilde{b}}_{h i} - {\tilde{b}}_{h}]}^{2}] & = E [\sum_{i \in S_{h}} {(\sum_{k \in S_{h i}} w_{h i k} e_{R h i k})}^{2} - \frac{1}{n_{h}} {(\sum_{i \in S_{h}} \sum_{k \in S_{h i}} w_{h i k} e_{R h i k})}^{2}] \\ + (1 - \frac{1}{n_{h}}) E [\sum_{i \in S_{h}} V (\sum_{k \in S_{h i}} w_{h i k} {\tilde{e}}_{r h i k} - y_{h i k} | Z)], \end{array}$

so that ${\hat{V}}_{L} (\hat{θ})$ is an approximately unbiased estimator of $V_{1} (\hat{θ}) .$ The consistency follows by (A2), which implies asymptotic normality, and the law of large numbers.

Proof of Theorem 3. For $c \neq d,$

$Cov [({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\hat{M}}_{c}^{R} - M_{c}^{R}), ({\bar{y}}_{d}^{R} - {\bar{Y}}_{d}^{R}) ({\hat{M}}_{d}^{R} - M_{d}^{R})] = o (M^{2} / n^{2})$

because $E [({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\bar{y}}_{d}^{R} - {\bar{Y}}_{d}^{R})] = o (n^{- 1})$ for simple random sampling (equation (4.26) of Lohr 2010). Consequently,

$\begin{array}{l} V (\sum_{c =1}^{C} \frac{{\hat{T}}_{c}}{p_{c}}) & = \sum_{c =1}^{C} \sum_{d =1}^{C} \frac{1}{p_{c}} \frac{1}{p_{d}} Cov [({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\hat{M}}_{c}^{R} - M_{c}^{R}), ({\bar{y}}_{d}^{R} - {\bar{Y}}_{d}^{R}) ({\hat{M}}_{d}^{R} - M_{d}^{R})] \\ = \sum_{c =1}^{C} \frac{1}{p_{c}^{2}} V [{\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}] V [{\hat{M}}_{c}^{R} - M_{c}^{R}] + o (M^{2} / n^{2}) . \end{array}$

The second term of $V_{2} (\hat{θ})$ is:

$\begin{array}{l} 2 Cov [\sum_{c =1}^{C} \frac{{\hat{T}}_{c}}{p_{c}}, \sum_{c =1}^{C} \frac{({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) {\hat{M}}_{c}^{R}}{p_{c}} - {\hat{Y}}_{S S}] \\ =2 \sum_{c =1}^{C} \sum_{d =1}^{C} \frac{1}{p_{c} p_{d}} Cov [{\hat{T}}_{c}, ({\bar{y}}_{d}^{R} - {\bar{Y}}_{d}^{R}) {\hat{M}}_{d}^{R} - p_{d} {\hat{M}}_{d}^{R} {\bar{y}}_{d}^{R} - p_{d} {\hat{Y}}_{d}^{N R}] \\ =2 \sum_{c =1}^{C} \frac{1}{p_{c}^{2}} Cov [- ({\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}) ({\hat{M}}_{c}^{R} - M_{c}^{R}), (1 - p_{c}) {\bar{y}}_{c}^{R} {\hat{M}}_{c}^{R} - {\bar{Y}}_{c}^{R} {\hat{M}}_{c}^{R} - p_{c} {\hat{Y}}_{c}^{N R}] + o (\frac{M^{2}}{n^{2}}) \\ =2 \sum_{c =1}^{C} \frac{p_{c} - 1}{p_{c}^{2}} V [{\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}] V [{\hat{M}}_{c}^{R} - M_{c}^{R}] + o (\frac{M^{2}}{n^{2}}) . \end{array}$

Combining the terms,

$V_{2} (\hat{θ}) = \sum_{c} \frac{2 p_{c} - 1}{p_{c}^{2}} V [{\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}] V [{\hat{M}}_{c}^{R} - M_{c}^{R}] + o (\frac{M^{2}}{n^{2}}) .$

We can estimate $p_{c}$ by the empirical response rate in poststratum $c,$ $V [{\bar{y}}_{c}^{R} - {\bar{Y}}_{c}^{R}]$ by $s_{c}^{2} / n_{c}^{R},$ and, under simple random sampling, $V [{\hat{M}}_{c}^{R} - M_{c}^{R}] = M_{c} p_{c} (M - M_{c} p_{c}) / n .$ The term $V_{2} (\hat{θ})$ can be negative when $p_{c} < 1 / 2$ for some poststrata; however, when $p_{c} < 1 / 2$ and $V [{\bar{y}}_{c}^{R}] >0,$ then the first-order term of the variance, $V_{1} (\hat{θ}),$ is positive and the second-order term has lower order.

Proof of Theorem 4. Condition (A4) guarantees that, asymptotically, complete separation will not occur and $R_{h i k}^{M}$ is bounded away from 0.

The derivative of $\hat{A}$ with respect to the parameters is

$\begin{array}{l} \hat{D} (r, β, θ) & = \frac{\partial \hat{A}}{\partial {(β, θ)}^{'}} \\ = [\begin{matrix} - \sum_{h i k \in S} w_{h i k} {[1 + \exp (- x_{h i k}^{^{'}} β)]}^{- 2} \exp (- x_{h i k}^{^{'}} β) x_{h i k} x_{h i k}^{^{'}} & 0 \\ - \sum_{h i k \in S} w_{h i k} r_{h i k} y_{h i k} \exp (- x_{h i k}^{^{'}} β) x_{h i k}^{^{'}} & - 1 \end{matrix}] . \end{array}$

Using successive conditioning and the independence of $r$ and $Z,$ the expected value of $\hat{D} (r, β, θ)$ is

$\begin{array}{l} D (R, β, θ) & = [\begin{matrix} - \sum_{h i k \in U} {[1 + \exp (- x_{h i k}^{^{'}} β)]}^{- 2} \exp (- x_{h i k}^{^{'}} β) x_{h i k} x_{h i k}^{^{'}} & 0 \\ - \sum_{h i k \in U} R_{h i k} y_{h i k} \exp (- x_{h i k}^{^{'}} β) x_{h i k}^{^{'}} & - 1 \end{matrix}] \\ = [\begin{matrix} - X^{'} {[I + Q]}^{- 2} Q X & 0 \\ - T^{'} Q X & - 1 \end{matrix}] . \end{array}$

Also, $Cov [vec \hat{D} (r, β, θ)] = O (M^{2} / n)$ because

$\begin{array}{l} V [\sum_{h i k \in S} w_{h i k} r_{h i k} y_{h i k} \exp (- x_{h i k}^{^{'}} β) x_{h i k}^{^{'}}] & = V [\sum_{h i k \in S} w_{h i k} R_{h i k} y_{h i k} \exp (- x_{h i k}^{^{'}} β) x_{h i k}^{^{'}}] \\ + E {V [\sum_{h i k \in S} w_{h i k} r_{h i k} y_{h i k} \exp (- x_{h i k}^{^{'}} β) x_{h i k}^{^{'}} | Z]} . \end{array}$

The first term is $O (M^{2} / n)$ by standard arguments and the second term is $O (M^{2} / n)$ by Lemma 1, noting that the boundedness of $R_{h i k}$ and $x_{h i k}$ also bound $\exp (- x_{h i k}^{^{'}} β) .$ Consequently,

$V [\begin{matrix} \hat{β} - β \\ \hat{θ} - θ \end{matrix}] = D {(R, β, θ)}^{- 1} V [\sum_{h i k \in S} w_{h i k} u (y_{h i k}, x_{h i k}, r_{h i k}, β)] D {(R, β, θ)}^{- T} + o (M^{2} / n) .$

The result in (3.3) follows because

${[D (R, β, θ)]}^{- 1} = [\begin{matrix} - C & 0 \\ T^{'} Q X C & - 1 \end{matrix}] .$

References

Abraham, K.G., Helms, S. and Presser, S. (2009). How social processes distort measurement: The impact of survey nonresponse on estimates of volunteer work in the United States. American Journal of Sociology, 114(4), 1129-1165.

Baker, R., Brick, J.M., Bates, N.A., Battaglia, M., Couper, M.P., Dever, J.A., Gile, K.J. and Tourangeau, R. (2013). Summary report of the AAPOR task force on nonprobability sampling. Journal of Survey Statistics and Methodology, 1, 90-143.

Binder, D.A. (1983). On the variances of asymptotically normal estimators from complex surveys. International Statistical Review/Revue Internationale de Statistique, 51, 279-292.

Brick, J.M. (2013). Unit nonresponse and weighting adjustments: A critical review. Journal of Official Statistics, 29(3), 329-353.

Eltinge, J.L. (2002). Diagnostics for the practical effects of nonresponse adjustment methods. In Survey Nonresponse, (Eds., R.M. Groves, D.A. Dillman, J.L. Eltinge and R.J.A. Little), New York: John Wiley & Sons, Inc., 431-443.

Fay, R.E. (1991). A design-based perspective on missing data variance. In Proceedings of the 1991 Annual Research Conference, U.S. Bureau of the Census, Washington, DC, 429-440.

Fuller, W.A. (2009). Sampling Statistics. Hoboken, NJ: Wiley.

Groves, R.M., and Heeringa, S.G. (2006). Responsive design for household surveys: Tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society, Series A (Statistics in Society), 169(3), 439-457.

Groves, R.M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.

Hamrick, K.S. (2012). Nonresponse Bias Analysis of Body Mass Index Data in the Eating and Health Module. Technical Report 1934, United States Department of Agriculture Economic Research Service, Washington, DC.

Harris-Kojetin, B.A. (2012). Nonresponse Bias Analysis for Establishment Surveys: Guidance from the U.S. Office of Management and Budget. Paper presented at DC-AAPOR, http://www.dc-aapor.org/documents/Harris-Kojetin2012.pdf.

Haziza, D., and Lesage, É. (2016). A discussion of weighting procedures for unit nonresponse. Journal of Official Statistics, 32(1), 129-145.

Haziza, D., Thompson, K.J. and Yung, W. (2010). The effect of nonresponse adjustments on variance estimation. Survey Methodology, 36, 1, 35-43. Paper available at http://www.statcan.gc.ca/pub/12-001-x/2010001/article/11246-eng.pdf.

Kim, J.K., and Kim, J.J. (2007). Nonresponse weighting adjustment using estimated response probability. Canadian Journal of Statistics, 35(4), 501-514.

Kohut, A., Keeter, S., Doherty, C., Dimock, M. and Christian, L. (2012). Assessing the Representativeness of Public Opinion Surveys. Pew Research Center, Washington, DC.

Lohr, S.L., Hsu, V. and Montaquila, J. (2015). Using classification and regression trees to model survey nonresponse. Proceedings of the Survey Research Methods Section, American Statistical Association, 2071-2085.

Oh, H.L., and Scheuren, F.J. (1987). Weighting adjustments for unit nonresponse. In Incomplete Data in Sample Surveys, (Eds., W.G. Madow, I. Olkin and D.B. Rubin), New York: Academic Press, 2, 143-184.

Riddles, M., Marker, D.A., Rizzo, L., Wiley, E. and Zukerberg, A. (2015). Adaptive Design for the National Teacher Principal Survey. Paper presented at the AAPOR 70^th Annual Conference, Hollywood, FL.

R Core Team (2015). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

Särndal, C.-E., and Lundström, S. (2005). Estimation in Surveys with Nonresponse. Hoboken, NJ: Wiley.

SAS Institute, Inc. (2011). SAS/STAT 9.3 User’s Guide. Cary, NC: SAS Institute, Inc.

Shao, J., and Steel, P. (1999). Variance estimation for survey data with composite imputation and nonnegligible sampling fractions. Journal of the American Statistical Association, 93, 254-265.

Shao, J., and Tu, D. (1995). The Jackknife and Bootstrap. New York: Springer.

Tourangeau, R., Brick, J.M., Lohr, S. and Li, J. (2016). Adaptive and responsive survey designs: A review and assessment. Journal of the Royal Statistical Society, Series A (Statistics in Society), published online, http://onlinelibrary.wiley.com/doi/10.1111/rssa.12186/pdf.

United States Office of Management and Budget (2006). Standards and guidelines for statistical surveys. https://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf.

Yung, W., and Rao, J.N.K. (2000). Jackknife variance estimation under imputation for estimators using poststratification information. Journal of the American Statistical Association, 95(451), 903-915.

ISSN : 1492-0921

Editorial policy

Survey Methodology publishes articles dealing with various aspects of statistical development relevant to a statistical agency, such as design issues in the context of practical constraints, use of different data sources and collection techniques, total survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves. All papers will be refereed. However, the authors retain full responsibility for the contents of their papers and opinions expressed are not necessarily those of the Editorial Board or of Statistics Canada.

Submission of Manuscripts

Survey Methodology is published twice a year in electronic format. Authors are invited to submit their articles in English or French in electronic form, preferably in Word to the Editor, (statcan.smj-rte.statcan@canada.ca, Statistics Canada, 150 Tunney’s Pasture Driveway, Ottawa, Ontario, Canada, K1A 0T6). For formatting instructions, please see the guidelines provided in the journal and on the web site (www.statcan.gc.ca/SurveyMethodology).

Note of appreciation

Canada owes the success of its statistical system to a long-standing partnership between Statistics Canada, the citizens of Canada, its businesses, governments and other institutions. Accurate and timely statistical information could not be produced without their continued co-operation and goodwill.

Standards of service to the public

Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner. To this end, the Agency has developed standards of service which its employees observe in serving its clients.

Copyright

Published by authority of the Minister responsible for Statistics Canada.

Use of this publication is governed by the Statistics Canada Open Licence Agreement.

Catalogue No. 12-001-X

Frequency: semi-annual

Ottawa

Date modified:: 2016-12-20

Language selection

Search and menus

Search

Tests for evaluating nonresponse bias in surveys Section 5. DiscussionTests for evaluating nonresponse bias in surveys Section 5. Discussion

Acknowledgements

Appendix

References