Appendix A: PALS 2006 severity scale for adults

View the most recent version.

1. Types of disability
2. Scores
3. Filter questions
4. Imputation of the 'undetermined'
5. Special cases
6. Construction of index
7. Creation of classes

An index for measuring the severity of disability was constructed on the basis of responses to the screening questions for the 2001 PALS. This index was calculated also for the 2006 PALS in order to compare results of both surveys. This document presents the methodology used to construct the adult disability index.

1. Types of disability

To construct the index, ten types of disabilities were considered: hearing, seeing, communication, mobility, agility, pain and discomfort, learning difficulties, memory problems, developmental disability and psychological conditions.

Points are assigned to each question on the basis of severity. For some types of disability, more than one question is asked. Each of these questions seeks to measure a functional limitation related to the disability. Thus, for example, to measure hearing-related disability, three questions are asked:

(1) How much difficulty do you have hearing what is said in a conversation with one other person?
(2) How much difficulty do you have hearing what is said in a conversation with at least three other persons?
(3) How much difficulty do you have hearing what is said in a telephone conversation?

A score is thus derived for each of these questions. Further on, we will see how these scores are then combined to obtain a single score per type of disability. Table 4 below shows the number of functional limitations measured by the PALS questionnaire by type of disability as well as the contexts for which these questions are asked.

2. Scores

Some questions measure the intensity of the disability, while others instead measure the frequency of its presence. Both types of questions are used in calculating scores. Points are assigned to each question on the basis of severity. Thus, when there is no disability, no points are assigned. Conversely, the maximum score is given for total disability. For example, for an intensity question, scores are assigned as follows:

• "Some difficulty": 1 point
• "A lot of difficulty": 2 points
• "Completely unable": 3 points
• Other answer (no, refusal, don't know): 0 point

For a frequency question, points are assigned as follows:

• "Yes, sometimes": 1 point
• "Yes, often or always": 2 points
• Other answer (no, refusal, don't know): 0 point

When both intensity and frequency are available for a given type of disability, the product of the scores for the two questions is used.

For each type of disability, a single value is required. Take, for example, the case of hearing, for which three different questions are asked: "How much difficulty do you have hearing what is said in a conversation with one other person?" (HEAR1), "How much difficulty do you have hearing what is said in a conversation with at least three other persons?"(HEAR2) and "How much difficulty do you have hearing what is said in a telephone conversation?" (HEAR3). Here the three scores must be combined in order to have only one score . The same is true for questions asked in different contexts: these sub-questions must be combined to have only one score for each type of disability. For example, for learning difficulties, the same question is asked for four contexts: home, work, school and other.

Since the number of questions varies depending on the disability, we standardized the indices by type of disability, so as not to over-represent types for which there are numerous questions. We do not want to assign more weight to one type of disability than to another. Where there is more than one question for a given type, the scores for these questions are summed and the sum is then standardized to obtain a score that lies between 0 and 1:

(1a)

Whereis the score for disability type T, N is the number of different questions (functional limitations) for type T,is the maximum score for the i th question for disability type T andis the score obtained for the i th question for disability type T. In some cases,may be made up of more than one question. When the same question is asked in different contexts, we take the mean of the scores for each of the contexts. For example, for learning difficulties, since the same question is asked four times, we take the mean of the four scores:

(2a)

Whereis the number of different contexts in which the i th question of disability type T is asked andis the score for question i and context j of type T.

The following is an example for communication-related disability (T=COMM). This type of disability is made up of two different questions (), COMM1 (difficulty speaking) and COMM2 (difficulty making yourself understood). COMM1 is asked in a general context (), whereas COMM2 is asked in four different contexts (family, friends, professional services, and other) (). The maximum value of COMM1 is 3, while the maximum value of COMM2 is 2 (, ):

(1b)

Whereis asked in a general context and where:

(2b)

3. Filter questions

For respondents who have a non-nil index value based on the screening questions, no additional points are assigned for answers to the filter questions. But for respondents who have no points based on the screening questions (that is, basically the "yes-no's", YES at the filter questions and NO to the screening questions), points are assigned on the basis of the four filter questions. The overall score for the filter questions is calculated in the same way as for the types of disability, based on the expressions (1a) and (2a). In this case, we have , , and.

4. Imputation of the 'undetermined'

For some respondents, we have enough information to know that they have a certain type of disability but the information for them is incomplete because either intensity, frequency or both are missing. They were initially assigned an "undetermined" flag and a score of 0, with the intention of imputing them after a score was calculated for all those for which the information was complete.

For imputation, we decided to confine ourselves to a relatively simple technique. It consists in looking for a group of respondents having the same responses to certain questions as the respondent to be imputed and imputing the mean of their scores. Here is an example:

This type of action is justified by the fact that there is a correlation between the frequency question and the intensity question. A person who answers "Yes, often or always" to the frequency question is more likely to answer "Completely unable" or "A lot of difficulty" to the intensity question than persons who answered "Yes, sometimes" to the frequency question.

Imputation specific to PALS 2006

Because of a change in the flow of the questions in PALS 2006, a component of the severity indicators associated with mobility was imputed for some respondents to ensure that the severity indicators for 2001 and 2006 remained comparable. In 2001, respondents who stated that they were unable to walk still had to answer the question AMOF_Q09 pertaining to the difficulties of moving from one room to another. In 2006, this question's placement was modified so that those who stated they were incapable of walking did not respond to this question. All the same, since this question was part of the calculation of the severity indicators for these people in 2001, it was therefore decided that a score would be imputed for this question in 2006. To do this, the average score obtained for 2001 was used. 688 adults were affected in the total by this rule.

5. Special cases

There are some types of disability for which we ask,

A) Whether a given condition reduces the quantity or number of activities that a respondent can engage in (frequency question).

If the answer is yes, we then ask,

B) How many activities does this condition prevent (at home, at work, at school, elsewhere).

A respondent is considered limited if he/she answers "Yes" to A). However, the respondent may answer "None" to each of the four contexts in question B). This situation is not corrected by the rules during processing. Since the points on the scale are assigned on the basis of the combined response to A) and B) (generally the product of the two), no points are assigned to persons in this situation (since B=0), even though they are considered as being limited for the type of disability concerned. Thus, overall, a respondent may be limited for two types of disability but have points for only one type or even, in some cases, for neither type.

We decided to assign a minimum number of points to these respondents for the types of disability for which this problem arises. Accordingly, we assign one point to everyone who answered "Yes" to A), and then we calculate the score for B) as presented above. For example, if the maximum score for a given type of disability is 6 (frequency (2) X intensity (3)), then with this change, the maximum score becomes 7 and respondents who have a "Yes" for A) and "None" for B) have a score of 1. To summarize,

"Yes" to A) and "None" to each question in B): 1 point

"Yes" to A) and at least one answer to B): 1 point + points assigned to B)

In this way, respondents who answered "None" throughout B) will necessarily have the lowest score, since they have points only because of A).

For some types of disability, a respondent is considered limited (and is assigned points accordingly) if a disability is reported even though there is no limitation. In such cases (learning difficulty and developmental disability), a point is still assigned even if the answer to the frequency question is negative. These special cases, along with the questions for which an additional point is assigned for a "Yes," are shown in Table 5 , below.

6. Construction of index

We observed an important relationship between learning difficulties and developmental disability. For a majority of persons with developmental disability, a learning difficulty was also reported. We therefore decided that when points are assigned to a respondent for a developmental disability, points cannot also be assigned for learning difficulties.

The overall score is calculated taking the average of all standardized scores. Unlike what was done in the case of children, where the presence of two age groups not having the same questions requires that two scales be calculated, the score for adults is calculated in the same way for respondents of all ages:

(3)

Where

This scale is derived for people who have an affirmative answer to the screening questions (the "yes-yes" group and the "no-yes" group) only. For the "yes-no" group, only the filter questions are used to calculate the score, and these questions are considered to represent an additional disability:

(4)

The reason why we did not consider the filter questions in (3) is that it is not desirable to have redundant information. For example, a person who has a disability related to mobility has probably answered "Yes" to the filter questions, thinking of his/her mobility-related disability (the filter questions being general in nature) and also answered "Yes" to the mobility questions.

For the "yes-no" group, the reason why they did not answer "Yes" to the screening questions is probably that we are unable to measure their type of disability with our questionnaire or that they had too mild a disability to be reported in the screening questions. For this reason, we dealt with them separately and assigned a relatively low score.

A few results concerning the overall index are shown in Tables 6 and 7. First, Table 6 presents descriptive statistics according to the number of disabilities reported. Thus, for a given number of disabilities, it shows the number of respondents having that number of disabilities, the mean and the standard deviation from the overall index for these respondents, as well as the minimum and maximum values. As may be seen, the mean increases with the number of disabilities, which is entirely desirable. The same is true for the standard deviation. When the number of disabilities is large, there may be people who have several mild disabilities and other that are quite severe, and who have a high score for a number of disabilities.

Table 7 shows, by number of disabilities, the proportion of respondents having each of the disabilities identified. Thus, row 1 of the table shows that among persons with one disability, 22.5% have a disability related to hearing, 5.9% a disability related to seeing, 1.2% to communication, etc. As may be seen, disabilities such as PAIN, HEAR and MOBI are often unaccompanied by other disabilities (30.8%, 22.5% and 14.5% respectively). Also, MOBI, AGIL and PAIN are often present together, since the rates are similar in each row starting with row 2.

Other approaches were considered in order to limit the redundancy of the information contained in the severity scale. In addition to the strong relationship between developmental disability and learning disability, there are other significant correlations between some disability types in the scale. For example, there is a strong correlation between mobility difficulties, agility difficulties and pain and discomfort. Thus, in many cases, a person who has mobility problems also has some pain and discomfort. On the other hand, seeing or hearing difficulties are more often encountered on their own.

In order to remove redundancy of information, an unequally weighted scale was considered. Instead of having a weight of 1, disability types that are strongly correlated would have a smaller weight in the global score. Since it is difficult to justify the use of unequal weights in the scale, this option was rejected.

7. Creation of classes

In order to create severity classes, the distribution of the global severity score was examined. The distribution has been separated into deciles. The first decile corresponds to the 10% of people with activity limitation with the lowest scores. The second decile corresponds to the next 10% of people with activity limitation with the lowest scores, etc. The average score was calculated for each decile and a plot of this average score as a function of the decile was produced in Chart 3.

As can be seen in Chart 3, no obvious cut-off points in the global severity score distribution exist. Several techniques were considered in order to create the severity classes. However, given the continuous nature of the severity curve and because it was desirable to employ a strategy that users would readily understand, we were unable to enter into exhaustive analyses, and we had to confine ourselves to a relatively intuitive approach. Thus, the severity classes were essentially determined by means of a graphic analysis of the data.

After discussion with some data users, it was decided that the severity scale should be cut into four severity classes , Class #1 being the less severe and Class #4 the most severe. The creation of the classes has been done in two steps.

In a first step, an attempt was made to identify a "natural cut-off point" in the scale. Although this is not obvious, one can note that the beginning of the distribution is fairly linear up to 70 th percentile and then, the slope starts to increase more and more rapidly. This cut-off point in the trend of the distribution seems to correspond to a score around 1/9. This particular score corresponds to the score of someone with the maximum score for one type of disability and no points for the other types. Many such cases were found in the sample. Of course, there is a number of ways to obtain a score of 1/9. Because of the particular interpretation of this point, the cut-off was chosen to be exactly 1/9. This cut-off creates two groups:

"Least severe"			SI	<	1/9
"Most severe"	1/9		SI		1

For example, a person with a total disability related to seeing (), but with no other disability, would fall into the most severe group. Table 8 shows the number of persons in this situation as well as the types of disability concerned. It also shows the number of persons who have more than one disability with a maximum score and no points for other disabilities. Thus, there are only 37 cases where there is a score of 1 for a one disability and where the nine other types are nil. Problems of seeing (18 cases) and pain and discomfort (13 cases) are the severe disabilities most often unaccompanied by others.

We then separated these two groups into two parts. These two boundaries correspond to respectively half and double the maximum score obtained for a given disability. Thus, respondents who have a score lower than half the maximum score for a disability are included in Class 1, while those who have a score that lies between half the maximum score for a disability and the maximum score for that disability are in Class 2. Those whose score lies between the maximum score for a disability and double that score are in Class 3, while those with a score greater than double the maximum score for a disability are in Class 4:

Class 1			SI	<	1/18
Class 2	1/18		SI	<	1/9
Class 3	1/9		SI	<	2/9
Class 4	2/9		SI		1

The advantage of this classification system is that it is easy for all users to understand and interpret. In light of the subjective nature of such a system, we preferred not to use specific terms to characterize the classes, so as to avoid misinterpretations. The only possible interpretation of these classes is that according to our measurement tool, persons in Class 4 have a more severe disability than persons in Class 3, who in turn have a more severe disability than persons in Class 2, and so forth. However, for practical purposes, these classes were assigned names. We use the terms "mild," "moderate," "severe" and "very severe" to designate classes 1 to 4 in that order. It should be noted that there is no judgment associated with the use of this terminology; the classes of severity depend on the way in which the scale is constructed.

---

You need to use the free Adobe Reader to view PDF documents. To view (open) these files, simply click on the link. To download (save) them, right-click on the link. Note that if you are using Internet Explorer or AOL, PDF documents sometimes do not open properly. See Troubleshooting PDFs. PDF documents may not be accessible by some devices. For more information, visit the Adobe website or contact us for assistance.