Survey Methodology
Tests for evaluating nonresponse bias in surveys

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

by Sharon L. Lohr, Minsun K. Riddles and David MorgansteinNote 1

  • Release date: December 20, 2016


How do we tell whether weighting adjustments reduce nonresponse bias? If a variable is measured for everyone in the selected sample, then the design weights can be used to calculate an approximately unbiased estimate of the population mean or total for that variable. A second estimate of the population mean or total can be calculated using the survey respondents only, with weights that have been adjusted for nonresponse. If the two estimates disagree, then there is evidence that the weight adjustments may not have removed the nonresponse bias for that variable. In this paper we develop the theoretical properties of linearization and jackknife variance estimators for evaluating the bias of an estimated population mean or total by comparing estimates calculated from overlapping subsets of the same data with different sets of weights, when poststratification or inverse propensity weighting is used for the nonresponse adjustments to the weights. We provide sufficient conditions on the population, sample, and response mechanism for the variance estimators to be consistent, and demonstrate their small-sample properties through a simulation study.

Key Words: Inverse propensity weighting; Poststratification; Replication variance estimation; Responsive design.

Table of content

How to cite

Lohr, S.L., Riddles, M.K. and Morganstein, D. (2016). Tests for evaluating nonresponse bias in surveys. Survey Methodology, Statistics Canada, Catalogue No. 12-001-X, Vol. 42, No. 2. Paper available at


Date modified: