Canadian Survey on Disability, 2022: Concepts and Methods Guide
8. Differences between the 2022 and 2017 cycles of the Canadian Survey on Disability

Skip to text

Text begins

The 2022 Canadian Survey on Disability was designed to be as comparable as possible to the 2017 cycle and is the first time in decades that two consecutive cycles of the CSD or its predecessors have produced a significant amount of comparable data. However, some differences do exist, as explained in this section, and should be considered carefully when making any comparisons. It is also recommended that data users and researchers refer to the 2017-2022 Concordance in Appendix E, which provides a mapping of the variables between the two cycles.

8.1 Changes to the CSD collection methods

Data collection for the 2022 CSD and the 2017 CSD were both conducted using an electronic questionnaire that could either be self-administered (rEQ mode) online or completed over the phone with the assistance of an interviewer. Approximately two in five respondents (40%) opted for the self-administered online collection in 2017, and approximately 64% did so in 2022. This change in the frequency of collection modes may have affected responses. For example, the fact that those who answered the questions in rEQ mode could see all the possible response choices, as opposed to those who have the choices read to them over the phone by an interviewer, can have an impact on the responses. This is called the “mode effect.”  Additionally, a new approach called CAPI Lite Plus was used in some of Canada’s Northern communities in Nunavut. This involved interviewers visiting dwellings to encourage participation in the CSD and handing out introduction letters in English and French and brochures in Inuktitut and Inuinnaqtun.

8.2 Lag time between 2021 Census and 2022 CSD

As mentioned above, the CSD survey frame was constructed from the responses to questions on activities of daily living in the 2021 Census long-form questionnaire and the CSD was conducted 13 to 18 months later. This time lag is greater than the time between the 2016 Census and 2017 CSD (10 to 15 months). The possibility that a respondent who reported a difficulty in the census no longer experienced it at the time of the CSD or, conversely, was institutionalized, deceased or no longer living in the country is significant and may be higher in 2022 given the increase in the time lag in comparison to 2017. Nevertheless, as was done in 2017, to prevent the underestimating of disability due to these losses in the YES population,^Note the weights of units in the 2022 NO population (for whom no collection was done) were adjusted to account for people who may have left the country or died between the census and CSD data collection. Given that this adjustment is related to the length of the time lag, it could have an effect on data comparisons. For more information on the weighting of the CSD, please see Section 6.1.

8.3 Changes to CSD content

While most questions remained the same between the 2017 and 2022 CSD, the content of the 2022 CSD has been expanded to include new indicators. Some subject areas have been revised and updated while others have been cut back. A number of new survey modules were added for 2022 in order to address existing data gaps and emerging data needs, as described more fully in Chapter 2. Thus, for some 2022 content areas, there are not comparative indicators from 2017.

Many survey questions were also updated in 2022 to better reflect current realities and to correct known areas for improvement in the 2017 CSD. For example, new options were added to several questions in the section on aids and assistive devices. Also, questions regarding the causes for main medical conditions were divided into work-related causes and non-work-related causes.

It should also be noted that some of the content from the 2017 CSD was removed in 2022 in order to balance the respondent burden created by new content additions. Several variables were added to better adhere to the Disaggregated Data Action Plan, and the CSD 2022 began collecting information on gender and sexual orientation. Further discussion of this addition can be found in Chapter 2.

8.4 COVID-19 pandemic

While it is not possible to quantify any possible impact of the COVID-19 pandemic on results from the 2022 CSD, there is no doubt that the pandemic, and social distancing measures which were implemented across Canada, have affected many aspects of Canadian life, including for persons with disabilities. It is not inconceivable that changes between 2017 and 2022 could have been influenced, to some degree, by the pandemic and this should be considered when making comparisons.

8.5 Changes to sex question and inclusion of gender

To improve inclusivity in the 2022 CSD, the sex question from 2017 was modified and a new question on gender was added. In 2017, respondents were asked their sex and in 2022, respondents were asked about sex at birth.  It is possible that there could be a conceptual difference in the way respondents interpreted sex in 2017 vs. sex at birth in 2022, especially with the absence of the gender question in 2017, which was added in 2022.

The 2022 CSD uses the concept of gender to disaggregate and disseminate 2022 CSD data, while the 2017 CSD uses sex of person. The change to disseminate data on gender rather than sex is consistent with the Treasury Board of Canada Secretariat’s Policy Direction to Modernize the Government of Canada’s Sex and Gender Information Practices (2018) and is consistent with the changes to the 2021 Census (see Filling the gaps: Information on gender in the 2021 Census (statcan.gc.ca)). Although sex and gender refer to two different concepts, the introduction of gender is not expected to have a significant impact on data analysis and historical comparability, given the small size of the transgender and non-binary populations.

More detail on the approach and definitions is described in Section 2.5.

8.6 Change in choice of quality indicator

Prior to 2022, the coefficient of variation (CV) was used to report the quality of estimates in terms of their sampling error. For the 2022 CSD, the 95% confidence interval (CI) is used instead for this purpose.

A confidence interval is associated with a confidence level, which is generally set at 95%. A 95% CI is an interval constructed around the estimate so that, if the process that generated the sample were repeated many times, the value of the interest parameter in the population would be contained in 95% of these intervals. The CIs presented with CSD estimates are produced using appropriate methods so that their coverage is close to the nominal rate, i.e., 95%. Further details on the methods used to construct confidence intervals for the 2022 CSD are available upon request by email through Statistics Canada Client Services or by phone at 1-800-263-1136.

There are many advantages in using CIs to report quality. Firstly, they are appropriate for all types of estimates. This contrasts with CVs which are not suitable for estimated proportions and estimates of differences, among others. As well, CIs provide an objective measure of the sampling error in a form that is easy to interpret: when given a 95% CI for a parameter of interest, a user can say that they are 95% confident that the CI constructed using the survey data contains the true population parameter. Lastly, CIs convey that there is uncertainty around the estimate, and that there is a range of other possible values that could have been obtained if different samples had been selected.

Readers may refer to the Canadian Survey on Disability, 2022: A User Guide to the Analytical Data Files for related information on the use of CIs to report the precision of survey estimates and the release rules related to quality.

• Previous
• Next