Non-response follow-up for business surveys
Section 5. Conclusions

• Previous

In Section 3, we derived an explicit expression for $n_2\left(C,\infty \right),$ the minimum sample size to expend the budget $C,$ on average, while resolving all follow-up sample units. We showed that this minimum sample size maximizes the expected response rate; thereby minimizing the bias of the non-response-adjusted Hansen and Hurwitz (1946) estimator. Our empirical investigations showed that this minimum sample size also appears to minimize the mean square error of this estimator. This can be explained by noting that the expected number of respondents remain roughly constant as the sample size increases, yielding an approximately constant variance. For the uniform follow-up response mechanism, it was possible to show theoretically that the expected number of respondents does not vary as the sample size increases (or does not vary with $K),$ confirming the empirical results.

At first glance, the idea of maximizing the expected response rate to minimize non-response bias may appear to contradict existing non-response literature. It is well known that a data collection procedure that intends to maximize the response rate for a given sample will most likely increase the non-response bias when easy-to-reach respondents differ from the other sample units. That is, increasing the response rate does not necessarily reduce non-response bias for a given sample and may actually do the opposite. Our results do not contradict this statement as we studied a different feature of the data collection design: the effect of the follow-up sample size on the expected response rate and non-response bias. It appears that this question has not been investigated in the literature. Our main conclusion is that a smaller follow-up sample size contributes to increasing the expected response rate and decreasing non-response bias.

Our conclusions may have important implications in practice. In business surveys conducted by Statistics Canada, all the mail-out non-respondents are currently followed up, and an adaptive collection procedure is used to prioritize cases (see Bosa et al., 2018). We believe that the non-response bias could be further reduced by following up only a sample of mail-out non-respondents in situations where the follow-up budget is insufficient to properly handle the volume of mail-out non-respondents. The adaptive collection procedure currently in place could continue to be used to manage data collection of the follow-up sample.

Another conclusion of our empirical investigations is that the PPS designs appeared to perform slightly better than the SRS and stratified SRS designs. However, no attempt was made to optimize the stratification or allocation of the stratified SRS design. The performance of the stratified design would likely be improved through a more efficient use of the auxiliary variable “Revenue” for stratification.

Finally, we observed that, unlike the follow-up response mechanism, the mail-out response mechanism had no impact on the bias of the non-response-adjusted Hansen and Hurwitz (1946) estimator. As a result, the mail-out non-response bias could be eliminated, even if the mail-out response probability was correlated to the variable of interest, provided that the follow-up response probability was uniform. This result is not surprising since the estimator of Hansen and Hurwitz (1946) is unbiased for any mail-out response mechanism.

Acknowledgements

The authors would like to thank three anonymous referees and the Associate Editor for their constructive comments, which led to significant improvements to the clarity of the manuscript.

References

Beaumont, J.-F., Bocci, C. and Haziza, D. (2014). An adaptive data collection procedure for call prioritization. Journal of Official Statistics, 30, 607-621.

Beaumont, J.-F., Bocci, C. and Hidiroglou, M. (2014). On weighting late respondents when a follow-up subsample of nonrespondents is taken. Paper presented at the Advisory Committee on Statistical Methods, Statistics Canada, May 2014, Ottawa.

Bosa, K., Godbout, S., Mills, F. and Picard, F. (2018). How to decompose the non-response variance: A total survey error approach. Survey Methodology, 44, 2, 291-308. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2018002/article/54957-eng.pdf.

Groves, R.M., and Heeringa, S.G. (2006). Responsive design for household surveys: Tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society, Series A, 169, 439-457.

Hansen, M.H., and Hurwitz, W.N. (1946). The problem of non-response in sample surveys. Journal of the American Statistical Association, 41, 517-529.

Peytchev, A., Riley, S., Rosen, J., Murphy, J. and Lindblad, M. (2010). Reduction of non-response bias in surveys through case prioritization. Survey Research Methods, 4, 21-29.

Schouten, B., Cobben, F. and Bethlehem, J. (2009). Indicators for the representativeness of survey response. Survey Methodology, 35, 1, 101-113. Paper available at https://www150.statcan.gc.ca/n1/pub/12-001-x/2009001/article/10887-eng.pdf.

Statistics Canada (2017). Monthly Survey of Food Services and Drinking Places (MSFSDP). Statistics Canada, http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=413027.

Thompson, K.J., Kaputa, S. and Bechtel, L. (2018). Strategies for subsampling nonrespondents for economic programs. Survey Methodology, 44, 1, 75-99. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2018001/article/54929-eng.pdf.

Tourangeau, R., Brick, J.M., Lohr, S. and Li, J. (2017). Adaptive and responsive survey designs: A review and assessment. Journal of the Royal Statistical Society, Series A, 180, 203-223.

Xie, H., Godbout, S., Youn, S. and Lavallée, P. (2011). Collection Follow-Up Operation Using Priority Scores For Business Surveys. Conference of European Statisticians, Work Session on Statistical Data Editing, Ljubljana (Slovenia).

• Previous
• Article main page