6. The short DSQ A New Survey Measure of Disability: the Disability Screening Questions (DSQ)
6. The short DSQ

This section explains the need for a shorter version of the DSQ and the stages involved in its development.

6.1. Why a short DSQ?

The DSQ have several advantages: 1) they are entirely based on the social model of disability and take into account activity limitations to identify disability (except developmental, which is by design); 2) they cover 10 disability types; 3) they provide  better coverage of non-physical disability types with filter questions at the beginning of the questionnaire that are more inclusive than the NHS filter questions; 4) they contain all the information needed to compute a severity score based on the intensity of difficulty and frequency of activity limitation for each disability type.

As well, qualitative tests and quantitative tests showed that the DSQ yield a valid and reliable measure of disability. People respond to the DSQ as expected. Quantitative tests with the LFS and CCHS revealed that the DSQ provide very comparable measures of disability, even when administered in surveys with different contexts.

The main drawback of the DSQ is length. Based on the two major quantitative tests, it takes 3 and 4 minutes on average to administer. Most general population surveys cannot afford to add an extra 3 or 4 minutes of content, especially if the main topic is not directly related to disability.

Several ideas that were proposed for shortening the DSQ would have required dropping questions and combining others, which would confuse respondents. Extensive qualitative testing would have been needed to evaluate respondents’ ability to understand these new questions.

6.2. Minimum requirements of the short DSQ

One of the motivations for developing the DSQ was Statistics Canada’s decision to remove the Census filter questions from general population surveys because the questions, on one hand, included too many people who did not have a disability (false positives), and on the other hand, did not adequately capture the full range of PWD (false negatives). General population surveys that used these questions were reporting very different rates of disability. Part of the research strategy at ESDC was to make better use of all survey data on PWD. This meant that general population surveys that focussed on specific themes would need a short module to identify PWD to replace the old Census filter questions.

If the only goal is to replace the old Census filters, the minimum requirement for the short DSQ is to identify only PWD and not type of disability or severity.

6.3. A preliminary version of the short DSQ

The aim was to create a short version of the DSQ similar to and compatible with the long version. The same questions could be used, but the way they were administered had to change to reduce the time.

6.3.1. Identifying disability only (IDO version of the short DSQ)

If the only goal of the short DSQ was to identify PWD without details on disability type, the DSQ could be administered as is, and when the first disability type was identified, the module could stop because the respondent was identified as having a disability. This would reduce average interview time because many people have more than one disability type, and it would not be necessary to go through all the screeners to find them all. To save even more time, the same could be done with the filter questions: as soon as a first filter was “Yes,” the other filters could be skipped because the screeners would be asked. The main advantage is that this yields the same disability rate as the long version of the DSQ. The only difference is that this short version would not generate information on all disability types, and consequently, would not provide the information needed to compute a severity score.  

This was not the final version of the short DSQ, but it triggered ideas that culminated in the creation of the short DSQ. This first prototype was called the “Identify Disability Only” version (IDO).

To determine if this change would be sufficient to reduce the average interview time, the “Audit Trail” files from the LFS quantitative test of the long DSQ were used. A simulation was performed to discover which questions would have been skipped if the IDO version had been used, and how much time would have been saved. The time to administer the IDO version would average 1 minute 54 seconds.

Chart 5 illustrates how the IDO version affects the proportion of people who are asked each question, compared with the long version. The numerous questions that are skipped in the IDO version make the proportion of people asked each subsequent question smaller and smaller, compared with the original DSQ.

6.3.2. Synchronizing the IDO version of the short DSQ

Given that the IDO version still required nearly 2 minutes to administer and yielded so little information (only the presence of a disability), additional options were sought. To reduce the interview time of the IDO version, it was proposed that the answers to the filter questions be used to decide which screeners would be asked. In the original DSQ, if one filter is positive, all screeners are asked. The proposal was to “synchronize” the screeners with the filters based on their conceptual relationship. Each disability type is related to a category of disability, and each category of disability is associated with one filter question (Table 8).

Synchronizing the screeners with the filters means that if the answer to a given filter is “No,” all screeners for the disability types associated with that filter are skipped, because no disability is assumed for these types. This saves time, but yields a different disability rate than the long version. Based on results of the LFS or CCHS quantitative tests, respondents can reply “No” to a given filter, be filtered into the screener section on the basis of another filter question, and go on to report a disability for a type associated with the filter to which they had originally answered “No.” With synchronization, these people would be considered not to have the disability type since the associated filter is negative.

If the order of the questions remains the same, synchronizing requires that everyone be asked the first four filters; it is no longer possible to skip to the screener if a filter is positive. Therefore, although synchronizing saves time, time is added by asking all filters to everyone. However, a feature of this synchronized option is that instead of asking all filters at the beginning of the DSQ, followed by all the screeners, filters and screeners can be alternated. This improves the flow of the questionnaire by keeping questions on a related topic together. Hence, the first filter on Seeing and Hearing is asked. If it is negative, all screeners on Seeing and Hearing are skipped, and the second filter is asked. But if the first filter is positive, the Seeing screeners are asked, followed by, if no Seeing disability is found, the Hearing screeners. If no Hearing disability is found, the second filter question on physical disabilities is asked, and so on. With the IDO version, the process ceases as soon as a first disability is found. 

To further reduce interview time, “Rarely” answers were excluded from the definition of disability (the second definition of disability described in Section 4.4.1.2 is used). Excluding “Rarely” means skipping the questions on intensity of difficulty for the Pain, Learning, Mental health-related and Memory disability types, since these questions come after those on frequency of activity limitations, which serve to identify disability. This change has a small impact on the disability rate derived from the short synchronized IDO version of the DSQ, compared with the long DSQ.

Using data from the LFS and CCHS quantitative tests, the impact on the disability rate of synchronizing the DSQ was assessed.^{Note 1} The relationship between each disability type indicator and its corresponding filter question was examined to determine how often a disability was present when the filter question was negative. These are the cases that are lost in the synchronized version. The impact of excluding all “Rarely” answers from the definition of disability was also evaluated.

For Pain, several options were tested. Although pain is considered a physical disability, it does not necessarily trigger the physical filter question (F2). Pain can affect concentration (F3), or even vision (F1). Therefore, it was decided that the Pain screeners should be asked if any of Filters 1 through 4 was positive.

With this option for the short DSQ (synchronized and excluding “Rarely” answers), the overall disability rate would be underestimated by 1.4 (LFS) to 3.4 (CCHS) percentage points, compared with the original long DSQ and its RACXR definition.^{Note 2}

The majority (70% to 98%, depending on the disability type) of people excluded from the synchronized short DSQ reported either “No difficulty” or “Some difficulty,” and most (76% to 86%) were “Sometimes” limited. Therefore, although the disability rate is underestimated, the short DSQ exclude people with milder disabilities. Ideally, the disability rate would not be underestimated, but the aim is to create a version of the DSQ short enough for inclusion on surveys.  

A simulation of this short synchronized IDO version using the LFS "Audit Trail" files showed that it averages 1 minute 33 seconds to administer.  

6.4. Can the short DSQ do more?

The utility of the short DSQ would be increased if groups of disability types, instead of just the presence of a disability, could be identified. Operationally, this means that instead of skipping all screeners when a first disability type is identified, skipping would be done within broad categories or groups of disability types. Hence, when a first disability is found in a given category, the other screeners in this category are skipped, and the interview proceeds to the next category (given that the associated filter is positive).

Different groupings of disability types and their impact on average interview time were evaluated. The goal was to keep the average interview time of the short DSQ below 2 minutes, but obtain information on as many disability categories as possible.

6.5. Final version of the short DSQ

The final^{Note 3} version of the short DSQ can identify disabilities in five areas: Seeing, Hearing, Physical (including pain), Cognitive, and Mental health-related, plus Other. This final version of the short DSQ is estimated to have an average interview time just below 2 minutes, and has greater analytical value than the IDO version.

Figure 1 shows a flow chart of this version of the short DSQ.

Qualitative testing of this short version ensured that the new question order and flow did not create difficulties for respondents. One concern was switching between filters and screeners, particularly the slight difference in the answer categories (for the filter questions, "No,” “Sometimes,” “Often,” “Always”; for the screeners, “Never,” “Rarely,” “Sometimes,” “Often,” “Always”). However, the flow of questions was good; questions were clear; and interviews went smoothly.

An electronic version (EQ) of this short DSQ was also qualitatively tested in preparation for its use on the General Social Survey, which collects data using both Computer-Assisted Telephone Interviews (CATI) and EQ. The EQ version tested well. The main change (to both the EQ and CATI versions) was splitting the first filter question (F1) about seeing and hearing into two (F1S and F1H), which improved the flow.

6.6. Official definition of disability using short DSQ

For two reasons, the definition of what constitutes a disability in the short version of the DSQ is not exactly the same as in the long version. The first is synchronizing of the filters and screeners, which results in some screeners not being asked in the short DSQ (implying a negative response) if the corresponding filter is negative. The other reason is that to save interview time with the short version, people who are “Rarely” limited are excluded from the definition of disability (Definition 2 in Section 4.4.1.2). Table 9 summarizes the definition of disability in the short DSQ for all types including Other. For the Developmental type, the definition is based solely on having received a diagnosis.

6.7. Differences between long and short DSQ

The long and short versions of the DSQ are essentially composed of the same questions, although a few questions were not included in the short DSQ. The main differences between the two versions are the question flow and skip patterns.

Chart 6 shows the distribution of the number of questions asked of respondents to the long and short versions,^{Note 4} based on the LFS 2011 test data. Because this test used the long DSQ, it was possible to determine the number of questions asked of each respondent. To estimate the number for the short version, the LFS data were modified according to the flow patterns of the short DSQ.

For both the long and short versions, respondents answer a minimum of 5 questions (first five filters). In fact, the majority of respondents answer only the first 5 or 6 filters. The long DSQ ask all filter questions first, and if one is positive, all screeners are asked. For this reason, the number of long DSQ respondents answering 7 to 20 questions is 0. By contrast, the short DSQ start with the first filter question, and if positive, goes on to the corresponding screeners. But if negative, the second filter is asked, and so on.

On average, respondents to the long DSQ answer 13.2 questions versus 7.9 questions for respondents to the short DSQ (Table 10).  

Thus, 95% of the respondents to the short version answer no more than 17 questions; for the long version, 95% of respondents answer no more than 29 questions. For both versions, more than half of respondents answer only 5 questions. 

In the long DSQ, all screeners for all 10 disability types are asked if at least one filter question is positive. In the short DSQ, only the screeners associated with the positive filters are asked. Within categories of disability types, when a first disability type is identified, the screeners for the other types within this category are skipped. Also in the short DSQ, questions on the intensity of difficulty for Pain, Learning, Developmental, Mental health-related, and Memory were removed from the module, because they are not needed to identify disability. For Developmental, the question on activity limitation was also removed from the module in the short DSQ, because it is not needed to identify disability. In the long DSQ, it is not used to identify disability, but is needed to compute the severity score.

The differences between the two versions affect the results. With the long DSQ, 10 disability types can be identified, and for each one, a severity score can be computed, plus a global severity score that takes all disability types into account. For the short DSQ, only five disability categories (Seeing, Hearing, Physical, Cognitive, Mental health-related) can be identified. Since some of these categories include more than one disability type, all the information on each type required to evaluate the severity of an individual’s disability is not available. For example, a respondent might have a mild disability related to Dexterity due to arthritis, and also a severe disability related to Pain due to migraines. All this information is collected with the long DSQ, but with the short DSQ, only Dexterity is asked because Dexterity and Pain are in the same category (Physical disability), and Dexterity comes first. It would be misleading if only Dexterity was considered in evaluating severity for that person. Thus, the short DSQ do not allow computation of a severity score.

Based on the long DSQ, it is possible to estimate the disability rate that the short version would yield, but the inverse is not true.

Because of the strong relationship between the two versions of the DSQ, future enhancements of one can potentially be implemented in the other. For example, the next version of the long DSQ will split the first filter (F1) into two (F1S and F1H), as in the short DSQ.

• Previous
• Next