Canadian Survey on Disability, 2017: Concepts and Methods Guide
4. Data collection

Skip to text

Text begins

4.1 Time Frame

The Canadian Survey on Disability (CSD) was conducted from March 1 to August 31, 2017. As a post-censal survey, it followed the 2016 Census of Population which was conducted on May 10, 2016. A time lag of approximately 10 to 15 months existed between the two surveys.

4.2 Communications

In the months leading up to data collection for the 2017 CSD, preparations were made to ensure that respondents had all the information they would need about the survey. A CSD survey webpage appeared on the Statistics Canada website as part of the agency’s official survey repository and registry system, called the Integrated Metadatabase (IMDB). The CSD webpage included a survey description, background information on the survey and its methodology, and a link to the questionnaire. In addition, a special webpage of Information for Survey Participants (ISP) was developed with step-by-step information on how to participate, “Questions and Answers” about the survey, and an infographic highlighting Canadian statistics on disability from the 2012 CSD.

In order to create on-line visibility for CSD collection activities, a social media campaign for the survey was launched on Twitter and Facebook. Social media postings were made throughout the survey collection period with weekly to biweekly announcements about the survey, a variety of promotional infographics, customized images and other relevant statistical results from the 2012 CSD. 

4.3 Mode of collection

For the first time, collection for the 2017 CSD was done using an Internet-based electronic questionnaire (EQ). Respondents could answer the EQ directly online without interviewer assistance (i.e., self-response) using a secure access code they received in the mail. We refer to this type of collection using the acronym rEQ. In addition, telephone interviews were used at the start of collection with people who were less likely to respond online, as well as in the middle and at the end of collection for non-response follow-up. For this type of collection mode, the interviewers asked the questions to the respondent and entered the answers directly in the respondent’s EQ during the interview. The acronym iEQ is used to refer to this type of collection.

In order to optimize online responses while considering the availability of interviewers in the regional offices, the sample was divided into two right from the beginning: respondents most likely to respond online and others. The group of respondents most likely to respond online comprised people in households who responded to the 2016 Census online; people who reported often or always having difficulty hearing;^Note and people whose household provided an email address in the 2016 Census. Based on these criteria, two-thirds of respondents were assigned to the online collection mode (rEQ) at the start of collection. The remaining third was sent to Statistics Canada’s regional offices for telephone collection (iEQ). Due to issues with high-speed Internet access in Nunavut, the above criteria were waived and all respondents from this territory were assigned to telephone collection right from the beginning.

A few days before the start of collection, a letter of introduction was sent to all respondents to inform them of the upcoming survey and mentioning the importance of participating. Each letter included a link to the CSD web page, as well a toll-free number to call if they had questions (and a TTY number for the hearing impaired). Each respondent also received a colour infographic in both official languages showing results of the 2012 CSD. The letters were always in English and French, while Inuktitut and Inuinnaqtun versions were also sent to respondents in Nunavut.

In addition, a Braille insert was included with each letter, providing information for the visually impaired on how to contact Statistics Canada by phone or how to visit the CSD web page for more information about the survey for which they had been selected.

The introduction letters sent to respondents assigned to the rEQ method included a link to the electronic questionnaire as well as a secure access code. To encourage timely participation, they were asked to respond by March 24^Note at the latest. Respondents assigned to the iEQ method were informed that an interviewer would try to reach them by phone over the following weeks. Over the six-month collection period, up to five reminder letters were sent to rEQ respondents who had not yet submitted their questionnaire. A new deadline was hence given on the reminder letters. During the first two months of collection, interviewers only tried to make telephone contact with respondents selected for the iEQ collection mode. From the third month, interviewers also tried to call rEQ respondents who had not yet completed the questionnaire to give them the option of doing so by telephone interview. In some cases, iEQ respondents asked to complete the survey themselves using the electronic questionnaire. A procedure was established to promptly send them an email with a link to the EQ and a secure access code. Approximately two in five respondents (40%) opted to use the electronic questionnaire.

For respondents designated for telephone contact, interviewers were instructed to make all reasonable attempts to obtain a completed interview with the selected member of the household. For cases in which the timing of the interviewer’s call was inconvenient, an appointment was arranged to call back at a more convenient time. For cases in which there was no one home, call backs were made at different moments of the day and different days of the week to maximize chances of reaching the respondent. Those who refused to participate were sent a letter to explain the importance of the survey and encourage their participation, and were re-contacted by telephone.

Across Canada, respondents were interviewed in the official language of their choice — English or French.

The time required to complete the survey varied from person to person but, on average, the survey took approximately 35 minutes to complete.

4.4 Security of online survey questionnaires

The electronic data collection system for the 2017 CSD involved a secure web server for the Internet-based EQ questionnaire, which captured both the iEQ and rEQ survey data. In addition, the iEQ was framed by a Blaise software system for the purpose of interviewer case management.

Statistics Canada takes the protection of confidential information provided online very seriously. A secure login process and robust encryption are key elements in helping to prevent anyone from viewing or tampering with a respondent’s survey information when it is completed and submitted online.

To protect the security of respondents’ personal information when using the Internet, Statistics Canada has incorporated the following safeguards:

• It uses strong bi-directional encryption technologies to ensure end-to-end security of data passing between the respondent’s computer device and Statistics Canada’s web server.
• Survey data are processed and stored on a highly restricted internal network and cannot be accessed by anyone who has not taken the oath of secrecy.
• Data submitted to Statistics Canada’s web servers are encrypted before being stored and remain encrypted until they are transferred to the high security internal network.
• Powerful firewalls, intrusion detection and stringent access control procedures are used to limit access to back-end systems and databases. Survey employees who have proper authorization and who have affirmed an oath of secrecy can access survey data but only from secure Statistics Canada locations.

The Blaise software system is also used in the context of stringent safeguards for protecting respondent information, including industry standard encryption technology, firewalls and restricted internal networks.

4.5 Supervision and quality control

Data collection was closely monitored throughout the six months of field activities through coordinated efforts between three specialized teams: Statistics Canada’s regional data collection offices, the Collection Planning and Research Division and the methodologists, analysts and managers of the CSD survey team.  

Quality control began with in-depth training of regional data collection managers and senior interviewers. Training addressed both the survey content and the interviewer-led electronic questionnaire (iEQ) application. A detailed interviewers’ manual was developed and presented through a combination of classroom learning and a period of self-study. Presentations were delivered by analysts, methodologists and collection experts. Interviewers completed a series of mock interviews to become familiar with the survey’s concepts and definitions as well as the EQ screens. In addition, a special session of disability awareness training was provided by the Champion for Persons with Disabilities at Statistics Canada along with an employment equity and diversity specialist of the department. 

Regional data collection managers and senior interviewers ensured that training for the CSD was provided to all front-line interviewers across all regional offices. They also ensured that all interviewers were familiar with the concepts and procedures of the survey. In addition, they were responsible for the ongoing supervision of interviewers, including the monitoring of interviews throughout the survey collection period to ensure that standard procedures were being followed.

CSD head office managers, methodologists and analysts also provided oversight and quality control throughout data collection. CSD managers made in-person visits to the regions once the survey was underway in order to observe live interviews and provide feedback to field staff. Statistical reports on collection progress were generated and closely scrutinized by analysts and methodologists on a daily basis. Ongoing communication and feedback between the regions and head office was provided through a quick-response ticket system, along with regular meetings for coordinating field work efforts and fine-tuning collection strategies. 

4.6 Proxy interviews

Since disability is difficult to measure and very subjective, interviewers were asked to make every effort to conduct the interview with the selected person. However, in the following circumstances, a proxy interview was acceptable:

• The selected person was away for the duration of the survey.
• The selected person spoke neither English nor French.
• The selected person was unable to participate because of mental or physical health problems.
• A parent insisted on responding for his or her child.

In order to be accepted as a proxy respondent, the person responding must be:

• an adult who speaks either English or French;
• reachable during the survey’s data collection period;
• the person most knowledgeable, or among the most knowledgeable, about the selected person’s difficulties and challenges related to activity limitations and participation restrictions.

A total of 4,399 proxy interviews were conducted, 4,122 of which are considered complete, 158 out of scope,^Note and 119 incomplete. The 4,122 complete proxy interviews represent 12.2% of complete responses

The table below shows the distribution of the 4,122 proxy interviews deemed to be completed, by respondent’s age and reason for the proxy interview.

4.7 Special issues

There were logistical challenges involved in managing the CSD collection, in that the collection system used was a temporary “in-house”^Note version that didn’t allow communication or automatic transfers between the rEQ and iEQ collection modes. Employees at headquarters had to perform a series of manual operations to ensure that cases could eventually be “transferred” from one collection mode to the other. We wanted this type of transfer to be possible in order to allow respondents to use the collection mode that was most convenient for them.

The impact of this in-house collection system was all the more significant mid-way through collection when all the cases became available in the interviewers’ tasks, while also being available in rEQ mode. We wanted to prevent respondents who had just completed their questionnaire in rEQ mode being contacted by an interviewer who had no way of knowing that the case was already complete. To do this, special daily procedures were put in place. As rEQ questionnaires were received by headquarters, a series of manual operations were performed to update the interviewers’ system and remove these cases from their lists. This helped optimize the interviewers’ work by making sure they did not contact respondents who had already completed the questionnaire.

On occasion, the in-house collection system also caused the same respondent to receive more than one questionnaire. Many verifications had to be done during processing to ensure that only one of the two questionnaires was kept.

4.8 Response rate

Collection for the CSD ended with a response rate of 69.5%. This response rate is the number of complete respondents (with or without a disability) divided by the number of cases sent to collection minus the out-of-scope cases. Out of scope cases include people who died, emigrated, were institutionalized, moved to a First Nations reserve, are full-time members of the Canadian Forces living on a military base, are visitors to Canada (misclassified during the census) or who reported being less than 15 years of age at the time of the interview. Hence, the response rate reflects the percentage of cases that completed the interview relative to the number of cases that should have completed it (which is why the out-of-scope cases are excluded from the denominator).

Response rate = Completed cases / (Cases sent to collection — Out-of-scope cases)

Once the data are examined, cases which appeared to be “respondent” can sometimes be considered incomplete or out of scope, affecting the number of cases that can be used for analytical purposes. After cleaning the data and reclassifying certain cases as non-respondent, the number of respondent and out-of-scope cases were recalculated, and thus a “clean” response rate was recalculated. The tables below provide the clean response rates by province/territory and age group.

• Previous
• Next