Response and nonresponse

Results

All (154)

All (154) (0 to 10 of 154 results)

1. Quality indicators for representation in administrative data
Articles and reports: 12-001-X202500200006
Description: National Statistical Institutes (NSIs) are directing resources into advancing the use of administrative data in official statistics. Administrative data, however, are not developed for the purpose of producing statistics rather as a result of an event or transaction relating to administrative procedures of organizations, public administrations and government agencies. Therefore, it is essential to check the quality of the administrative data with respect to sources of error, particularly representativeness to the target population. In this paper, we utilize the strength of probability-based reference samples or censuses that can be used to detect the lack of representativeness in administrative data and introduce quality indicators based on distance metrics and representativity indicators (R-indicators). We demonstrate their application with a simulation study and discuss a real application applied on a UK Office for National Statistics (ONS) administrative dataset.
Release date: 2025-12-23
2. Adapting to change: Online first collection initiatives to improve the Labour Force Survey response rate
Articles and reports: 75-005-M2025001
Description: Since 2010, engaging Canadians to participate in the LFS has become more challenging due to a variety of social and technological changes. The decline in the LFS response rate accelerated in 2020, exacerbated by public health measures during the COVID-19 pandemic. This technical paper presents preliminary results of two collection initiatives implemented using an online first strategy to improve the LFS response rates by confirming respondent contact information and expanding the availability of online response. Through these and other planned initiatives, Statistics Canada is working to ensure that the LFS estimates continue to provide an accurate and representative portrait of the Canadian labour market.
Release date: 2025-10-21
3. Low Response Rate from Merchants? Sample and Ask Consumers! An Application of Indirect Sampling Under a Consumer-Merchant Bipartite Network Archived
Articles and reports: 11-522-X202500100002
Description: Under the consumer-merchant bipartite network, we apply the indirect sampling approach to estimate merchant payment acceptance through a consumer payment diary. The records of in-person transactions in the consumer diary provide both the merchant sample via consumer-merchant linkages, and the merchant acceptance via consumers' responses on methods of payments used and accepted. Among merchants receiving multiple transactions during the period of the diary, we show that the derived payment acceptance from the consumer reporting is high quality in terms of very few conflicts between usage and perception, and within perceptions. Therefore, consumers are leveraged to be both sampling and reporting units in our indirect sampling application to eliminate merchant response burden. Furthermore, the necessity to proceed to weight adjustment to account for the non-recorded-merchant bias due to the relatively shorter duration of the diary (i.e., 3 days) is shown. Finally, these indirect sampling estimates are compared to the ones from a direct sampling survey, and it is found that the results are aligning well.
Release date: 2025-09-08
4. A Bias Evaluation for Probabilistic Web Panels at Statistics Canada Archived
Articles and reports: 11-522-X202500100009
Description: Three series of web panels were implemented at Statistics Canada from 2020 to 2024. Participants for these web panel series were recruited from respondents of large probabilistic social surveys (recruitment surveys), and subsequently were invited to complete a series of short online surveys. Estimates of recruitment survey variables were calculated using both recruitment survey weights and web panel weights, and these were compared; differences signal the possibility of residual bias that was not corrected by the web panel weighting process. This investigation found more significant differences than would be expected if the web panel estimator fully corrected for the bias resulting from the web panel response process. Questions related to certain topics such as politics and voting, sense of belonging, and media consumption were found to have the most significant differences between web panel estimates and recruitment survey estimates.
Release date: 2025-09-08
5. Survey-admin Hybrid Measure of Persistent Child Poverty in New Zealand Archived
Articles and reports: 11-522-X202500100018
Description: The Child Poverty Reduction Act (2018) outlines a need for the New Zealand Government to set three- and ten-yearly persistent child poverty reduction targets come end of 2024. In the absence of longitudinal survey data, a survey-administrative data hybrid method that will facilitate the production of these reduction targets and official estimates of persistent child poverty once reporting is required for the 2025/2026 financial year onwards is outlined. This hybrid approach leverages off the cross-sectional Household Economic Survey (HES), administrative-based beneficiary's family data, and recent advances developed for the construction of households within the Administrative Population Census (APC) at Statistics New Zealand. With increasing data collection challenges due to rising non-response and costs, this survey-admin hybrid method represents an alternative to longitudinal survey data collection, ensuring ongoing sustainable and quality statistics to produce persistent child poverty estimates.
Release date: 2025-09-08
6. Factors Affecting Response Propensity, with an Interest in Units Sampled Multiple Times Archived
Articles and reports: 11-522-X202500100036
Description: As the need for data has grown over the past number of years, the effect and burden of repeatedly sampling the same units for multiple surveys have become an increasing concern. Response burden is generally assumed to contribute to decreasing response rates; however, there are few empirical studies looking into this question. As part of this study, data on response to social surveys conducted at Statistics Canada between 2021 and 2023 was aggregated in order to investigate factors contributing to the observed response patterns, including the effect of having been selected multiple times. It was found that, relative to some other demographic and geographic characteristics, a unit being sampled multiple times is not an influential factor in predicting response propensity.
Release date: 2025-09-08
7. On the use of machine learning methods for the treatment of unit nonresponse in surveys
Articles and reports: 12-001-X202500100003
Description: In recent years, there has been a significant interest in machine learning in national statistical offices. Thanks to their flexibility, these methods may prove useful at the nonresponse treatment stage. In this article, we conduct an empirical investigation in order to compare several machine learning procedures in terms of bias and efficiency. In addition to the classical machine learning procedures, we assess the performance of ensemble approaches that make use of different machine learning procedures to produce a set of weights adjusted for nonresponse.
Release date: 2025-06-30
8. Inference from sampling with response probabilities estimated via calibration
Articles and reports: 12-001-X202400200013
Description: A solution to control for nonresponse bias consists of multiplying the design weights of respondents by the inverse of estimated response probabilities to compensate for the nonrespondents. Maximum likelihood and calibration are two approaches that can be applied to obtain estimated response probabilities. We consider a common framework in which these approaches can be compared. We develop an asymptotic study of the behavior of the resulting estimator when calibration is applied. A logistic regression model for the response probabilities is postulated. Missing at random and unclustered data are supposed. Three main contributions of this work are: 1) we show that the estimators with the response probabilities estimated via calibration are asymptotically equivalent to unbiased estimators and that a gain in efficiency is obtained when estimating the response probabilities via calibration as compared to the estimator with the true response probabilities, 2) we show that the estimators with the response probabilities estimated via calibration are doubly robust to model misspecification and explain why double robustness is not guaranteed when maximum likelihood is applied, and 3) we highlight problems related to response probabilities estimation, namely existence of a solution to the estimating equations, problems of convergence, and extreme weights. We present the results of a simulation study in order to illustrate these elements.
Release date: 2024-12-20
9. Authors’ response to comments on “Exploring the assumption that commercial online nonprobability survey respondents are answering in good faith”
Articles and reports: 12-001-X202400100009
Description: Our comments respond to discussion from Sen, Brick, and Elliott. We weigh the potential upside and downside of Sen’s suggestion of using machine learning to identify bogus respondents through interactions and improbable combinations of variables. We join Brick in reflecting on bogus respondents’ impact on the state of commercial nonprobability surveys. Finally, we consider Elliott’s discussion of solutions to the challenge raised in our study.
Release date: 2024-06-25
10. Comments by Aditi Sen on “Exploring the assumption that commercial online nonprobability survey respondents are answering in good faith”
Articles and reports: 12-001-X202400100010
Description: This discussion summarizes the interesting new findings around measurement errors in opt-in surveys by Kennedy, Mercer and Lau (KML). While KML enlighten readers about “bogus responding” and possible patterns in them, this discussion suggests combining these new-found results with other avenues of research in nonprobability sampling, such as improvement of representativeness.
Release date: 2024-06-25

Data (0)

Data (0) (0 results)

No content available at this time.

Analysis (153)

Analysis (153) (0 to 10 of 153 results)

1. Quality indicators for representation in administrative data
Articles and reports: 12-001-X202500200006
Description: National Statistical Institutes (NSIs) are directing resources into advancing the use of administrative data in official statistics. Administrative data, however, are not developed for the purpose of producing statistics rather as a result of an event or transaction relating to administrative procedures of organizations, public administrations and government agencies. Therefore, it is essential to check the quality of the administrative data with respect to sources of error, particularly representativeness to the target population. In this paper, we utilize the strength of probability-based reference samples or censuses that can be used to detect the lack of representativeness in administrative data and introduce quality indicators based on distance metrics and representativity indicators (R-indicators). We demonstrate their application with a simulation study and discuss a real application applied on a UK Office for National Statistics (ONS) administrative dataset.
Release date: 2025-12-23
2. Adapting to change: Online first collection initiatives to improve the Labour Force Survey response rate
Articles and reports: 75-005-M2025001
Description: Since 2010, engaging Canadians to participate in the LFS has become more challenging due to a variety of social and technological changes. The decline in the LFS response rate accelerated in 2020, exacerbated by public health measures during the COVID-19 pandemic. This technical paper presents preliminary results of two collection initiatives implemented using an online first strategy to improve the LFS response rates by confirming respondent contact information and expanding the availability of online response. Through these and other planned initiatives, Statistics Canada is working to ensure that the LFS estimates continue to provide an accurate and representative portrait of the Canadian labour market.
Release date: 2025-10-21
3. Low Response Rate from Merchants? Sample and Ask Consumers! An Application of Indirect Sampling Under a Consumer-Merchant Bipartite Network Archived
Articles and reports: 11-522-X202500100002
Description: Under the consumer-merchant bipartite network, we apply the indirect sampling approach to estimate merchant payment acceptance through a consumer payment diary. The records of in-person transactions in the consumer diary provide both the merchant sample via consumer-merchant linkages, and the merchant acceptance via consumers' responses on methods of payments used and accepted. Among merchants receiving multiple transactions during the period of the diary, we show that the derived payment acceptance from the consumer reporting is high quality in terms of very few conflicts between usage and perception, and within perceptions. Therefore, consumers are leveraged to be both sampling and reporting units in our indirect sampling application to eliminate merchant response burden. Furthermore, the necessity to proceed to weight adjustment to account for the non-recorded-merchant bias due to the relatively shorter duration of the diary (i.e., 3 days) is shown. Finally, these indirect sampling estimates are compared to the ones from a direct sampling survey, and it is found that the results are aligning well.
Release date: 2025-09-08
4. A Bias Evaluation for Probabilistic Web Panels at Statistics Canada Archived
Articles and reports: 11-522-X202500100009
Description: Three series of web panels were implemented at Statistics Canada from 2020 to 2024. Participants for these web panel series were recruited from respondents of large probabilistic social surveys (recruitment surveys), and subsequently were invited to complete a series of short online surveys. Estimates of recruitment survey variables were calculated using both recruitment survey weights and web panel weights, and these were compared; differences signal the possibility of residual bias that was not corrected by the web panel weighting process. This investigation found more significant differences than would be expected if the web panel estimator fully corrected for the bias resulting from the web panel response process. Questions related to certain topics such as politics and voting, sense of belonging, and media consumption were found to have the most significant differences between web panel estimates and recruitment survey estimates.
Release date: 2025-09-08
5. Survey-admin Hybrid Measure of Persistent Child Poverty in New Zealand Archived
Articles and reports: 11-522-X202500100018
Description: The Child Poverty Reduction Act (2018) outlines a need for the New Zealand Government to set three- and ten-yearly persistent child poverty reduction targets come end of 2024. In the absence of longitudinal survey data, a survey-administrative data hybrid method that will facilitate the production of these reduction targets and official estimates of persistent child poverty once reporting is required for the 2025/2026 financial year onwards is outlined. This hybrid approach leverages off the cross-sectional Household Economic Survey (HES), administrative-based beneficiary's family data, and recent advances developed for the construction of households within the Administrative Population Census (APC) at Statistics New Zealand. With increasing data collection challenges due to rising non-response and costs, this survey-admin hybrid method represents an alternative to longitudinal survey data collection, ensuring ongoing sustainable and quality statistics to produce persistent child poverty estimates.
Release date: 2025-09-08
6. Factors Affecting Response Propensity, with an Interest in Units Sampled Multiple Times Archived
Articles and reports: 11-522-X202500100036
Description: As the need for data has grown over the past number of years, the effect and burden of repeatedly sampling the same units for multiple surveys have become an increasing concern. Response burden is generally assumed to contribute to decreasing response rates; however, there are few empirical studies looking into this question. As part of this study, data on response to social surveys conducted at Statistics Canada between 2021 and 2023 was aggregated in order to investigate factors contributing to the observed response patterns, including the effect of having been selected multiple times. It was found that, relative to some other demographic and geographic characteristics, a unit being sampled multiple times is not an influential factor in predicting response propensity.
Release date: 2025-09-08
7. On the use of machine learning methods for the treatment of unit nonresponse in surveys
Articles and reports: 12-001-X202500100003
Description: In recent years, there has been a significant interest in machine learning in national statistical offices. Thanks to their flexibility, these methods may prove useful at the nonresponse treatment stage. In this article, we conduct an empirical investigation in order to compare several machine learning procedures in terms of bias and efficiency. In addition to the classical machine learning procedures, we assess the performance of ensemble approaches that make use of different machine learning procedures to produce a set of weights adjusted for nonresponse.
Release date: 2025-06-30
8. Inference from sampling with response probabilities estimated via calibration
Articles and reports: 12-001-X202400200013
Description: A solution to control for nonresponse bias consists of multiplying the design weights of respondents by the inverse of estimated response probabilities to compensate for the nonrespondents. Maximum likelihood and calibration are two approaches that can be applied to obtain estimated response probabilities. We consider a common framework in which these approaches can be compared. We develop an asymptotic study of the behavior of the resulting estimator when calibration is applied. A logistic regression model for the response probabilities is postulated. Missing at random and unclustered data are supposed. Three main contributions of this work are: 1) we show that the estimators with the response probabilities estimated via calibration are asymptotically equivalent to unbiased estimators and that a gain in efficiency is obtained when estimating the response probabilities via calibration as compared to the estimator with the true response probabilities, 2) we show that the estimators with the response probabilities estimated via calibration are doubly robust to model misspecification and explain why double robustness is not guaranteed when maximum likelihood is applied, and 3) we highlight problems related to response probabilities estimation, namely existence of a solution to the estimating equations, problems of convergence, and extreme weights. We present the results of a simulation study in order to illustrate these elements.
Release date: 2024-12-20
9. Authors’ response to comments on “Exploring the assumption that commercial online nonprobability survey respondents are answering in good faith”
Articles and reports: 12-001-X202400100009
Description: Our comments respond to discussion from Sen, Brick, and Elliott. We weigh the potential upside and downside of Sen’s suggestion of using machine learning to identify bogus respondents through interactions and improbable combinations of variables. We join Brick in reflecting on bogus respondents’ impact on the state of commercial nonprobability surveys. Finally, we consider Elliott’s discussion of solutions to the challenge raised in our study.
Release date: 2024-06-25
10. Comments by Aditi Sen on “Exploring the assumption that commercial online nonprobability survey respondents are answering in good faith”
Articles and reports: 12-001-X202400100010
Description: This discussion summarizes the interesting new findings around measurement errors in opt-in surveys by Kennedy, Mercer and Lau (KML). While KML enlighten readers about “bogus responding” and possible patterns in them, this discussion suggests combining these new-found results with other avenues of research in nonprobability sampling, such as improvement of representativeness.
Release date: 2024-06-25

Reference (1)

Reference (1) ((1 result))

1. Labour Force Survey Response Rates, September 2023 Archived
Surveys and statistical programs – Documentation: 75-005-M2023001
Description: This document provides information on the evolution of response rates for the Labour Force Survey (LFS) and a discussion of the evaluation of two aspects of data quality that ensure the LFS estimates continue providing an accurate portrait of the Canadian labour market.
Release date: 2023-10-30

Date modified:: 2026-01-21