Health Reports
Mapping the Washington Group on Disability Statistics disability measure to the Health Utilities Index Mark 3: Development and validation of a predictive multivariable model in a general population sample

Abstract

Background

Statistics Canada routinely collects information on functional health and related concepts. Recently, the Washington Group on Disability Statistics (WG) measure of disability has been introduced to the Canadian Community Health Survey (CCHS). The WG measure is used as a tool for developing internationally comparable data on disability. In alternate cycles of the CCHS, it replaces the Health Utilities Index Mark 3 (HUI3), a generic preference-based measure of health-related quality of life. The HUI3 is used to derive evaluative health measures common in population health and economic evaluations. Since the WG measure is not preference-based, it is unable to derive these measures. To address resulting data gaps, this study empirically maps the health state utility values of the HUI3 score from the WG measure.

Data and methods

Empirical mapping used a “head-to-head” subsample of the 2017 CCHS where WG and HUI3 measures were collected from the same respondents aged 40 and over. Empirical mapping used regression models to estimate the statistical relationship between WG and HUI3 measures in addition to health and demographic variables. Out-of-sample predictive performance was assessed through descriptive statistics, mean absolute error, and other measures of predictive accuracy.

Results

The preferred estimation strategy resulted in reasonably precise estimates of the HUI3 score corresponding to trends across health and demographic characteristics and reflecting distributional properties of the HUI3 score. Inclusion of different components of the WG measure influenced predictive accuracy.     

Interpretation

Empirical mapping offers a potential method to estimate health state utility scores from the WG measure and addresses data gaps in health-related quality of life measures in the CCHS when HUI3 is not collected.

Keywords

Adult; quality of life; data interpretation, statistical; models, theoretical; health surveys/statistics and numerical data; Canada

Authors

Thomas Charters and Dafna Kohen are with the Health Analysis Division at Statistics Canada. Julie Bernier was formerly with the Health Analysis Division.

 

 

What is already known on this subject?

• The Health Utilities Index Mark 3 (HUI3) has been included in various health and social surveys from Statistics Canada for several decades. Health status information derived using the HUI3 has been used extensively in economic and population health analyses.
• The Washington Group on Disability Statistics (WG) measure of disability cannot be used to generate utility scores necessary for common evaluative health measures since it does not have preference-based scoring functions, yet it is useful as an internationally comparable measure.
• Empirical mapping provides a common method to predict health state utility values from measures that lack preference-based scoring functions but show sufficient conceptual overlap.
• Previous work has conceptually mapped the WG to HUI3 measure and found sufficient overlap in the attributes and measurement approach between the measures to justify empirical mapping.  

What does this study add?

• Empirical mapping provides a feasible means to estimate health state utility scores of the HUI3 from the WG measure. This approach is verified through tests of predictive accuracy and validation across sample characteristics.
• A predictive formula to estimate health state utility scores from the WG measure in addition to health and demographic variables in the core content of the CCHS.
• Results from this study provide a means to derive and validate health-related quality of life measures in the general Canadian population aged 40 years and over such as Quality Adjusted Life Years (QALYs) and Health-Adjusted Life Expectancy (HALE).

Introduction

Population health surveys commonly collect information on health status as represented by functional abilities. Questions assess ability levels of respondents carrying out various tasks or activities in addition to health states that may impede this functioning. Disability is a related concept, involving interactions between these elements of functional status and environmental factors that limit or restrict participation in society.^Note 1, ^Note 2 The Washington Group on Disability Statistics (WG) measure of disability was developed by an international consortium and sponsored by the United Nations Statistical Commission. The purpose of the WG was to develop an internationally comparable population-based measure of disability to be used in censuses or national surveys, through measurement of functional limitations across domains closely associated with social participation.^Note 2 To facilitate comparability across different countries and cultural contexts, the WG measure assesses functional health through difficulties with universal basic activities. While the WG measures were developed within the International Classification of Functioning, Disability and Health framework,^Note 3, ^Note 4 they do not include social or environmental factors implicit to this framework for reasons of brevity and comparability. The WG measures are intended to be used in conjunction with other information sources to highlight inequalities between limitations in health and functioning and social inclusion, and thereby targets for intervention as per the United Nations 2030 Sustainable Development Goals.^Note 5, ^Note 6, ^Note 7 The validity and reliability of the WG measure have been demonstrated in international contexts,^Note 3, ^Note 8 and the WG measure has been adopted in censuses or surveys in over 80 countries.^Note 6

Several surveys and measures have historically been used by Statistics Canada to estimate levels of disability through measures of impairment, functional health, or activity limitations. Among these, the Health Utilities Index® Mark 3^Note 9 has been incorporated into several health and social surveys for several decades.^Note 10 The HUI system was developed to provide a standardized measure to assess and compare health and health-related quality of life (HRQoL) in patient groups and the general population, and in evaluation of health interventions.^Note 11 Further, the HUI3 has been used to derive common evaluative health measures, such as health-adjusted life expectancy (HALE)^Note 12 and quality-adjusted life years (QALYs), commonly used measures in population health and economic evaluations.^Note 13 The validity, reliability, and responsiveness of the HUI3 system are well established in clinical and population health settings.^Note 9, ^Note 14, ^Note 15, ^Note 16

Since 2000, the Canadian Community Health Survey (CCHS) has been administered by Statistics Canada to provide comprehensive health information on the Canadian population.^Note 17 The HUI3 instrument has been included in the CCHS since its inception. In 2015, the CCHS underwent a major redesign which saw updates to its content, sampling methods, and administration.^Note 18 Following the redesign, the HUI3 and the questions from the Washington Group Short Set on Functioning (WG-SS) were included as part of a two-year theme content, collected in alternate cycles to optimize data collection. Inclusion of the WG-SS measure meets commitments for collection of internationally integrated data on disability.^Note 6 Both measures describe functional capacities (what can you do) intrinsic to the person (within or near the skin) rather than performance (what you do) to avoid influence by context-dependent environmental factors.^Note 19, ^Note 20 Unlike the HUI3, the WG measure is unable to generate health state utility values, since these are derived from a preference-based scoring function.^Note 21, ^Note 22 As such, the WG does not permit calculation of HALEs or QALYs.

Collection of the HUI3 in alternating years will lead to data gaps. While the WG and HUI3 measures play complementary roles in the measurement of functional health, the WG measure is not suitable for use in the calculation of important health measures used in population health and program evaluations. Mapping provides a potential solution to estimate health state utility values from the WG measure. Mapping involves the estimation of a relationship between a target measure (HUI3) and a source measure (WG).^Note 21 The relationship may be estimated through use of a statistical model or algorithm^Note 23 or through equating or linking equivalent values between instruments.^Note 24 Importantly, the validity and feasibility of mapping rely on sufficient conceptual overlap between the measures.^Note 21, ^Note 25, ^Note 26 Mapping studies are common^Note 21, ^Note 27 and include several examples of successfully mapping the HUI from other measures.^Note 28, ^Note 29, ^Note 30, ^Note 31, ^Note 32, ^Note 33 Estimation of the HUI3 health state utility score would alleviate data gaps in years when not collected and may optimize resources used in data collection. The purpose of this study is to empirically map the health state utility values of the HUI3 score from the WG measure and to validate the results. This report builds off of previous research, which has established necessary levels of conceptual overlap between these two measures (available upon request).^Note 34

Materials and methods

Data

The 2017 CCHS annual component was used in this study. The CCHS is a cross-sectional representative survey covering a range of topics relevant to the health status, health behaviours, and demographic profiles of the Canadian population aged 12 and over living in private dwellings. Persons living on Indian reserves, on Crown lands, in institutions, in remote regions, or serving in the Canadian Forces are excluded from the sample. Additionally, individuals residing in the territories are excluded from one-year survey cycles. Approximately 98% of Canadians aged 12 and over are represented in the CCHS.^Note 35 Health, demographic, and socioeconomic variables collected from the 2017 CCHS included the respondents’ sex and age, highest educational attainment, marital status, self-rated general and mental health, chronic conditions, and the WG-SS.

Mapping used a unique “head-to-head” subsample of the 2017 CCHS containing both WG and HUI3 measures from the same respondents aged 40 years and over. The subsample contains three additional variables from the Washington Group Extended Set on Functioning (WG ES-F) in addition to the multi-attribute health status classification system questionnaire and derived attribute-specific and overall scores for the HUI3. The CCHS Rapid Response file included 2,837 respondents who had been provided modules for both the HUI3 and WG ES-F, with 2,597 having non-missing responses to the HUI3 target measure.

Measures of functional health

All domains from the Washington Group Short-Set (WG-SS) were included in the core content of the annual 2017 CCHS. The WG-SS consists of six attributes, including vision, hearing, mobility, cognition, self-care, and communication. Each attribute was assessed by a single question containing four response options: “No difficulty,” “Yes-some difficulty,” “Yes- a lot of difficulty,” and “Cannot do at all.”^Note 4 The Rapid Response subsample of the 2017 CCHS contained three additional attributes from the WG ES-F (pain, anxiety, depression). Response categories for each attribute were derived from one question measuring the frequency of the attribute and a second on its intensity.^Note 3 All WG measures included an additional response category of “Not stated,” combining response categories of “Don’t know” and “Refusal.” Appendix I shows derivation of categories for the WG ES-F.

The HUI3 multi-attribute classification system consists of eight attributes: vision, hearing, speech, ambulation, dexterity, emotion, cognition, and pain, with five or six response options per attribute.^Note 9, ^Note 11 Preference-based scoring functions convert this descriptive information in utilities, cardinal scores describing preferences over various health states. The multi-attribute scoring function generates overall HUI3 utility scores through multiplicative models and describes up to 972,000 different health states. These scores range from -0.36 (implying a state worse than death) to 1.00 (perfect functional health) with 0.00 representing death.

Analytical techniques

Descriptive statistics of the CCHS Rapid Response subsample and overall CCHS population aged 40 years and over were estimated for key health, sociodemographic, and socioeconomic variables. Sampling weights were used to estimate sample means and standard errors were obtained through bootstrap repeated replications.

Regression techniques were used to estimate the statistical relationship between WG and HUI3 measures enabling prediction of the overall HUI3 score in the head-to-head subsample. The continuous HUI3 overall score (target measure) was regressed on categorical WG variables in addition to demographic, socioeconomic, and health variables of interest (source measures). Respondent health and demographic characteristics were included since these were anticipated to have associations to the HUI3 independent from that of the WG measures, and that their inclusion had potential to improve overall predictive accuracy. Characteristics were selected given their inclusion in the annual CCHS core content, having independent associations to the HUI3 score, and use in other mapping studies.^Note 28, ^Note 29, ^Note 30, ^Note 31, ^Note 32, ^Note 33 Missing data in categorical independent variables were permitted,^Note 36 being treated as distinct item scores.

Responses to the WG questions were entered into the model as discrete dummy variables with the response “no difficulty” as the reference category. Use of item scores in regression models was chosen given the potential to improve model flexibility.^Note 21 Three different sets of WG items were tested together: the WG-SS only (25 potential coefficients), and the two sets with both the WG-SS and WG ES-F, including measures of pain in both and, alternately, anxiety or depression for affect (33 potential coefficients each). Age, sex, marital status (married or common law; single, windowed, separated, or divorced; missing), and self-rated general health and mental health (for each: poor, fair, good, very good, excellent, not stated) were included in succession in 12 sets of prediction models. Self-rated health has been demonstrated to be a reliable measure,^Note 37 positively correlated to physicians’ health ratings,^Note 38 chronic disease incidence,^Note 39 and mortality.^Note 40 Inclusion of self-rated health was selected given common use of additional clinical or health state measures in mapping studies.^Note 21, ^Note 27 Age was centred at 62, the unweighted mean age, to improve interpretability of coefficients. Consistent with previous studies,^Note 28, ^Note 41, ^Note 42 age was entered in linear, quadratic, and cubic forms, accounting for declines in functional health at advanced ages, assuming a nonlinear functional relationship. All other variables were entered into the model as categorical dummies.

Models fit on one set of data may predict characteristics randomly unique to that dataset and not necessarily show comparable predictive validity if replicated in other data sources. The head-to-head dataset was randomly partitioned into an “analytical” dataset making up two-thirds of the subsample (N=1,731) used to statistical predict HUI3 scores and a “hold-out” dataset with one-third of the data (N=866) used to assess the predictive accuracy. Models that accurately predict target measures in out-samples were expected to perform well in routine use and avoid problems intrinsic to overfitting.^Note 36 In the “hold-out” dataset predictive accuracy was assessed through descriptive statistics of the predicted health state utility scores (mean, median, interquartile range, minimum, maximum) in addition to several forecast statistics, including mean absolute error (MAE), root mean squared error (RMSE), Kendall’s rank correlation coefficient, and the model adjusted R2 (from the analytical dataset). Proportions of predicted scores that differed from observed values by +/-0.03 units or more (the smallest change in HUI3 considered clinically important)^Note 11 were calculated.

Characteristics of the HUI3 score presented challenges to empirical mapping. The population distribution of the overall HUI3 score is known to be highly skewed, with most respondents having perfect or near-perfect functional health.^Note 15, ^Note 42, ^Note 43 Because of the skewed nature of the HUI3 distribution, the normality of residuals assumption may be violated through use of regression techniques during empirical mapping. Further, since the HUI3 score is calibrated between -0.36 and 1.00,^Note 9, ^Note 11 modelling must ensure predicted results fall within these theoretical bounds to be interpretable. Several different regression modelling methods and outcome transformations were explored to improve predictive accuracy, attain a more normal distribution of residuals, and better replicate the distributional properties of the HUI3 score (see^Note 44 for the complete record of mapping procedures tested).

The validity of selected models was assessed by comparing the distributions of mean observed HUI3 scores and predicted scores across key respondent characteristics in the full CCHS Rapid Response subsample. Comparisons were made across age groups (40 to 49 years, 50 to 59 years, 60 to 69 years, 70 to 79 years, 80 years and over), sex, and presence of chronic conditions (no chronic conditions, one chronic condition, two chronic conditions, three or more chronic conditions, not stated). Accurate empirical mapping would imply little difference between observed and predicted variables in respect to key demographic and health characteristics. 

Results

Sample characteristics from the head-to-head subsample (N=2,597) are compared with those from the main survey (N=37,609) for respondents aged 40 and over (Table 1). The subsample had a mean age of 59, was 52% female, was mostly educated at a postsecondary level (63%), and mostly married or in a common-law relationship (74%). Respondents were most likely to report “very good” self-perceived health (37%), “very good” self-perceived mental health (38%), and to have one chronic condition (30%). Overall, demographic, socioeconomic, and health characteristics of the CCHS Rapid Response subsample corresponded closely to those in the full sample.



In total, nine modelling strategies differing by use of regression model, transformation, and estimation of highly prevalent discrete scores were tested on 12 sets of covariates. No prespecified criteria determined model success. Two broad observations were drawn from model testing, which led to the selection of the final set of candidate models. First, regression methods led to mapped estimates of reduced variability, which inadequately corresponded to the properties of the HUI3 distribution. Second, regression of the untransformed HUI3 score led to mapped scores, which were somewhat less accurate and tended to exceed the theoretical upper bound of the HUI3 score (for more details on the modelling procedures and results from model testing, see^Note 44).

The candidate models addressed these limitations by applying a two-step procedure to empirically map the HUI3 score. First, ordinal or multinomial logistic regression was used to predict highly prevalent discrete scores of the HUI3 variable of 1.00, 0.973, and all scores less than 0.973 on source measures through regressing on a three-category variable representing these scores. After deriving predicted probabilities for each category of the three-level variable, mapped categories were assigned based on each respondent’s highest predicted probability. Table 2 shows that 16.1% of the sample had a HUI3 score of 1.00, 24.5% had a score of 0.973, and 59.5% had a score less than 0.973. Greater accuracy in predicting these categories was attained, with multinomial logistic regression in models controlling for the WG-SS, the WG ES-F (pain and anxiety), age, age², age³, sex, and marital status (Model 8) having the highest agreement between categories (66%, kappa=0.374). The highest agreement in models not using the WG ES-F was found in Model 10 (64%, kappa=0.328), which additionally included self-rated general and mental health. 



The next step regressed an arcsine transformed HUI3 score on source measures on the 63% of the “hold-out” sample projected to have HUI3 scores less than 0.973. The arcsine transformation takes the form: $arcsine\left[2*\left(\frac{HUI3+0.36}{1+0.36}\right)-1\right]$  which first binds the HUI3 score to the [-1, 1] interval necessary to facilitate transformation by the arcsine function. Mapped scores were derived from reverse-transforming predicted scores and otherwise imputing discrete scores of 1.00 or 0.973 based on projected categories from the first estimation step. Table 3 shows descriptive and forecast statistics for this two-step approach, whereby discrete scores of 1.00 and 0.973 were derived from Model 8 (Table 2) of the first estimation step. The HUI3 score in the hold-out sample had a mean of 0.848 (95% confidence interval=0.828, 0.869), a median and interquartile range of 0.919, 0.744 to 0.973, respectively, and a range of -0.16 to 1.00. Models including the WG ES-F in the second estimation step routinely performed better than those without, but indicated little difference based on selection of anxiety or depression for affect or from inclusion of other non-WG predictors. The greatest predictive accuracy based on MAE estimates was found to be 0.086, with slight improvements in forecast statistics favouring inclusion of depression. Mean predicted scores were generally higher than observed scores, although by less than the clinically important difference of 0.03. Predicted scores were constrained to the bounds of the HUI3 and aligned with the median and 75th percentiles while routinely overestimating the 25th percentile. To investigate the predictive accuracy of models not using WG ES-F measures, derivation of HUI3 categories used Model 10 covariates (Table 2). The lowest MAE was found to be 0.094, with little variation observed in predictive performance across models (Table 4).

 

To validate empirical mapping, scores of the predicted health state utility values in candidate models were compared to observed values of the HUI3 across key demographic and health factors. Selected models included Model 6 (Table 3), which included covariates for WG-SS, WG ES-F (pain and depression), age, age², age³, and sex (Candidate Model 1); and Model 4 (Table 4), which included covariates for WG-SS, age, age², age³, and sex (Candidate Model 2). Candidates were selected based on high predictive performance and exclusion of variables that did not improve predictive accuracy. Figure 1 shows the distribution of health state utility values from the HUI3 in comparison to those from Candidate Model 1. Table 5 shows the overall HUI3 score generally declined in older ages, a pattern replicated in predicted health state utility scores from both candidate models. Candidate models consistently overestimated the mean HUI3 score by age by amounts proximate to or less than the threshold for clinical significance (0.03). The distributions of predicted utility scores also followed those of sex, showing higher mean scores in men than women and overestimating scores to similar degrees. Both observed and mapped scores were highest among those with no chronic conditions and declined in those with three or more conditions or those missing this covariate.

Description of Figure 1



The title of Figure 1 is “Distribution of observed and predicted health state utility values.”

The figure illustrates a graph with two overlaid histograms, one showing the distribution of health state utility values as measured by the HUI3 and the second the distribution of values as predicted from Candidate Model 1 in the analytical head-to-head subsample of the 2017 CCHS.

The X axis reads Health state utility value and has a range of -0.5 to 1.0.

The Y axis reads Fraction and has a range of 0 to 0.4.

The figure shows the plot of health state utility values from the HUI3 score in a solid pale orange colour and the overlaid plot of predicted health state utility values in transparent bars with solid dark blue outlines. A graph of this type can show how well the empirical mapping procedure was able to replicate the distribution of the measured health state utility values from the HUI3 instrument. Both distributions are skewed, with most respondents at or close to a health state utility score of 1.0 (the theorical maximum of the HUI3 distribution and indicative of perfect functional health according to this instrument) and a long tail of respondents to the left with low health state utility scores indicative of poor functional health, including scores below the value of 0.0 indicating states worse than death. The values on the Y axis represent the proportion or fraction of cases where the height of all of the histogram bars are scaled so the sum of the bars per each plot equals one.

There are differences between the observed and predicted scores. The distribution of the HUI3 runs from ‑0.16 (in health state utility values) on the left to 1.0 on the right. From the left, the height of the bars is consistently low indicating health state utility scores at these levels were uncommon in the analytical dataset. The distribution of cases follows a raising curve to approximately 0.9 (roughly the median value) indicating a fraction of 0.14, or that about 14% of respondents had a value of the HUI3 at this level. The distribution then drops in the next bar to a fraction a bit over 0.05, before increasing to a fraction a bit above 0.4 at health state utility values at or close to 1.0. This corresponds to nearly 41% of the sample having a HUI3 score of 1.0 or 0.973.

The distribution of the mapped health state utility values from Candidate Model 1 runs from -0.3 on the left up to 1.0 on the right margin. Like the distribution of observed HUI3 scores, the distribution shows evidence of a left skew with most observations at or close to 1.0. Further, the fraction of values at or close to 1.0 is very similar at over 40% of the sample. Unlike the HUI3 distribution, the mapped values exhibit only a single peak at the health state utility value of 1.0 with the distribution increasing nearly monotonically to this point.  

The source appears below the graph as follows: Statistics Canada, 2017, Canadian Community Health Survey Rapid Response subsample.

A note appears below the source and reads: HUI3 = Health Utilities Index Mark 3.



Figure 2 shows a calibration plot between observed and mapped health state utility values from Candidate Model 1 in the hold-out dataset. Results show a calibration coefficient (slope=0.938), showing reasonably strong agreement between observed and predicted utility scores.^Note 45, ^Note 46 The statistic for calibration-in-the-large (CITL) of -0.039 reflects minor overestimation. The 95% confidence interval around this slope reflects less accurate model calibration at lower utility scores.     

Description of Figure 2



The title of Figure 2 is “Calibration of observed and predicted health state utility values”

The figure illustrates a scatter plot of health state utility values from the observed Health Utilities Index Mark 3 (HUI3) scores and the mapped values. The term expected values refers to the latter in keeping with naming conventions for this plot type. This illustrates the level of agreement between the observed and predicted values and, importantly, how predictive accuracy varies throughout the distribution of observed values. The case of a perfect model prediction is illustrated by the dashed green line at the 45-degree mark with a slope of 1.0, which passes through the intercept of (0,0). Deviations from this show that predictions may be too high or low compared with observed values, and that calibration was poor. This may occur in instances where the intercept if offset, even when the slopes are equivalent.

This graph also shows the distributions of the observed and mapped health state utility values in histogram plots aligned with measure with histograms reflecting the fraction of health state utility scores at given levels. Histogram bars are in solid deep maroon colour.   

The X axis of the calibration plot reads “Expected health state utility value (Candidate Model 1)” and has a range of -0.5 to 1.0. This is also the X axis of the histogram of expected health state utility values.

The Y axis of the calibration plot reads “Observed health state utility value (HUI3)” and has a range of -0.5 to 1.0. This is also the X axis of the histogram of observed health state utility values.

The Y axis for each histogram has a range of 0 to 0.41. This reflects the fraction of observations per histogram bar, with the sum of the bars equalling 1. This also shows that for both the observed and mapped values, about 4 in 10 respondents had scores of 1.0 or approximate to 1.0.

On the calibration plot, data points are plotted as x’s in dark green for observed (Y axis) and mapped (X axis) health state utility values. The plot follows a positive linear pattern as indicated by the diagonal distribution of data points, with the majority of data points falling in the upper right quadrant, indicative of perfect or near perfect states of functional health. A solid blue line indicates the calibration slope, with a shaded light blue area around this line representing the 95% confidence interval around the calibration slope. In the upper left corner of the calibration plot there are two reported statistics: calibration-in-the-large (CITL) = -0.039 and Slope = 0.938. Slope refers to the calibration slope coefficient.

The calibration slope or coefficient is 0.938, indicating minor deviation from the slope of 1.0 representing perfect model calibration. This can be assessed visually by comparing how the calibration slope curve compares with the dashed line where, although approximating the dashed line, it is not as steep. The 95% confidence interval around this slope is wider at lower utility values, reflecting less accurate model calibration at lower utility scores. The statistic for CITL of -0.039 relates to the placement of the intercept on the calibration plot and, as this is negative but close to 0, illustrates minor overestimation in the mapping. This can be assessed visually on the plot by assessing how the calibration slope at the intercept value of (0,0) is slightly higher than the dashed line indicating perfect calibration.

The source appears below the graph as follows: Statistics Canada, 2017, Canadian Community Health Survey Rapid Response subsample.

Two notes appear below the source. The first note reads: CITL = calibration in the large; HUI3 = Health Utilities Index Mark 3. The second note reads: Distribution of health state utility values by fraction (histogram).

Discussion

Information on health status and quality of life obtained from the HUI3 system plays important roles in economic, clinical, and population health analysis in Canada. While the adoption of the WG measure permits collection on a validated and internationally comparable measure of functional capacity, the lack of preference-based scoring functions makes it unsuited for estimation of HRQoL, HALE, QALY, and other common measures. This data gap was addressed through empirical mapping of the overall HUI3 on the WG measure and select demographic and health characteristics in a CCHS subsample of the Canadian general population aged 40 years and over.

The head-to-head subsample provided a comparatively large and detailed dataset representative of the non-institutionalized Canadian population. The preferred model included two estimation steps: first, multinomial logistic regression to predict the HUI3 score falling in categories defined as 1, 0.973, or less than 0.973, and a second step predicting the HUI3 score through linear regression of the arcsine transformed score on respondents projected to have scores less than 0.973 or imputing discrete values of 1.00 or 0.973 based on the first estimation step. Arcsine transformation has been shown to improve the distribution of residuals in regression modelling and to maintain prediction within the theoretical bounds of the HUI3 score.^Note 42 This estimation strategy was able to predict reasonably precise health state utility scores corresponded to trends across demographic and health characteristics, reflected the distributional properties of the skewed HUI3 score, and retained prediction to its theoretical bounds. Mapping using the WG-SS and WG ES-F resulted in a MAE in the candidate model of 0.086, about 6.3% of the total range of the HUI3 health state utility score (1.36), and exceeds the predictive accuracy of many mapping studies.^Note 21 While demonstrating group-level predictive accuracy, about 60% of the sample had mapped scores exceeding the minimum clinically important difference of 0.03, implying difficulties in mapping at the individual level.

Empirical mapping using only the short-set attributes of the WG measure was also able to generate reasonable measures in this population group, although with less predictive accuracy (MAE=0.094, or 6.9% of the overall HUI3 range). This finding may highlight the importance of conceptual overlap between the measures,^Note 21, ^Note 25, ^Note 26 since the WG-SS alone may not adequately match the attributes of pain and emotion contained in the HUI3 system^Note 34 and may not be adequately represented by routinely collected health data, such as self-rated general and mental health. This highlights potential benefits to extending the existing WG-SS portion, included biennially in the CCHS two-year theme content, to include WG ES-F measures for pain and affect. Interestingly, inclusion of demographic and health covariates appeared more informative in modelling discrete categories of the HUI3 representative of perfect or near-perfect functional health than modelling lower functional health. Appendix II outlines regression coefficients and methods to map the HUI3 score for both candidate models.

Limitations of this study should be noted. Empirical mapping was tested and validated on a head-to-head sample of the Canadian population aged 40 years and over and is not generalizable to younger ages. The household population under 40 years generally has higher levels of functional health as measured by the HUI3,^Note 12 and additional methods may be required to map the score to these groups. Further, some applicable categories of the WG group were absent in the head-to-head sample and may reduce reliability and replicability. Third, mapped health state utility scores were overestimated in the “hold-out” sample and across demographic categories. Generally, levels of error were greater at lower HUI3 health state utility scores, a similar finding to other mapping studies.^Note 47 Next, conceptual limitations may arise if mapped scores are used in research comparisons across demographic characteristics included in the prediction equation. Assessment and prediction coefficients excluding sociodemographic factors for these uses will be presented elsewhere.^Note 44 Finally, mapping functions were generated on a non-institutionalized general population sample and may not be appropriate for use in other population groups, such as patient data or respondents from institutional settings. Further work may incorporate methods^Note 12 to adjust population-level mapped health state utility scores for institutionalized populations.

This study offers a potential method through empirical mapping to estimate health state utility scores from the WG measure and, as such, addresses data gaps in HRQoL measurement in the CCHS. Mapping was validated through comparisons of the distribution of the overall HUI3 score and selected mapped scores over key demographic and health characteristics. Mapped health state utility values may be used in future population studies of health-adjusted life expectancy (HALE) and quality-adjusted life years (QALYs), although further validation specific to these uses is required. Future work may further expand on mapping to the population aged less than 40 years.

  

Before running the first mapping step, the user may derive a three-category variable to represent prominent categories of the HUI3 variable. The new variable, which will be named H3, should take the form: H3=1 if HUI3<0.973, H3=2 if HUI3=0.973, H3=3 if HUI3=1.00. The predicted probability of each level of HUI3 may be ascertained for each individual respondent based on their observed characteristics through the following equations:

(1) $$\text{Pr}\left(\text{H}3=3\text{|X}=\text{x}\right)=\frac{\text{exp}\left({\gamma }_0+ {\gamma }_1{X}_1+ {\gamma }_2{X}_2+\dots {\gamma }_n{X}_{n }\right)}{1 + \exp ({\beta }_0+ {\beta }_1{X}_1+ {\beta }_2{X}_2+\dots {\beta }_n{X}_{n }) + \text{exp}\left({\gamma }_0+ {\gamma }_1{X}_1+ {\gamma }_2{X}_2+\dots {\gamma }_n{X}_{n }\right)}$$

(2) $$\text{Pr}\left(\text{H}3=2\text{|X}=\text{x}\right)=\frac{\text{exp}\left({\beta }_0+ {\beta }_1{X}_1+ {\beta }_2{X}_2+\dots {\beta }_n{X}_n\right)}{1 + \exp ({\beta }_0+ {\beta }_1{X}_1+ {\beta }_2{X}_2+\dots {\beta }_n{X}_{n }) + \text{exp}\left({\gamma }_0+ {\gamma }_1{X}_1+ {\gamma }_2{X}_2+\dots {\gamma }_n{X}_{n }\right)}$$

(3) $$\text{Pr}\left(\text{H}3=1\text{|X}=\text{x}\right)=1-\left[\text{Pr}\left(\text{H}3=3\text{|X}=\text{x}\right)+ \text{Pr}\left(\text{H}3=2\text{|X}=\text{x}\right)\right]$$

Work-through using Candidate Model 1: There is a vector 36 of coefficients ${\beta }_{n=1\dots 36}$  plus the constant ${\beta }_0$  are used to predict for each individual respondent the probability that H3=2 (HUI3=0.973) and a vector of 36 coefficients ${\text{γ}}_{n=1\dots 36}$  plus the constant ${\gamma }_0$  are used to predict the probability that H3=3 (HUI3=1). The probability that the HUI3 score is less than 1 (H3=1) can be derived from equation (3). The 36 coefficients relate to item scores for the Washington Group Short Set on Functioning (WG-SS) and Washington Group Extended Set on Functioning (WG ES-F) (with pain and anxiety), age (centred at 62 and entered as linear, quadratic, and cubic forms), sex, and marital status. In some instances, applicable categories of the Washington Group on Disability Statistics (WG) measure are missing coefficients given these were not represented in the analytical dataset.

Each respondent in the dataset will now have a predicted probability of each value of H3. Based on the highest predicted probability for each value of H3, the values corresponding to H3=3 (HUI3=1) and H3=2 (HUI3=0.973) may be imputed directly onto a new mapped health state utility score that will be named HUI3map. Individual records where the value of H3=1 shows the highest predicted probability undergo an additional step. Since the predictive coefficients used to estimate this step are derived from arcsine transformation of a linearly transformed HUI3 score of the form $arcsine\left[2*\left(HUI3+0.36/1+0.36\right)\right]-1$  (Materials and methods: Empirical mapping) the equation must reverse-transform the predicted values.

(4) $$HUI3map= \left[\frac{(\sin \left({\delta }_0+ {\delta }_1{X}_1+ {\delta }_2{X}_2+\dots {\delta }_n{X}_n\right)+1)*\left(1+0.36\right)}{2}\right]-0.36$$

Where a vector of 36 coefficients coefficients ${\delta }_{n=1\dots 36}$  plus the constant ${\delta }_0$  are used to predict, for each respondent, the arcsine and linear transformed HUI3 score. Since this value is not interpretable, additional steps of reverse transformation as outlined in equation (4) are necessary.

Scenario 1:

A respondent has the following response characteristics: male sex, age 71, widowed, separated, divorced, or single, never married. Responses to the WG-SS and WG ES-F show no difficulties in any functional domain. Estimation Step 1 first takes the predicted sum of β and γ coefficients including the intercept term. For brevity, response categories that fall within the reference category are not shown or those for which the responses are 0. The age term is centred at 62, so it is entered in the prediction equation as 9 (linear), 81 (quadratic), and 729 (cubic). $\sum {\text{β}}_{1-\text{n}}{\text{X}}_{1-\text{n}}$ : 0.6253845 (intercept) + -0.01504*9 (age) + -0.001247*81 (Age²) + 0.0000585*729 (age³) + -0.237704*1 (marital status)= 0.19396. $\sum {\text{γ}}_{1-\text{n}}{\text{X}}_{1-\text{n}}$ : -0.061049 (intercept) + -0.022366* 9(age) + 0.0017927*81 (age²) + -0.0000372*729 (age³) + 0.0303471*1 (marital status)= -0.11391. From this, probabilities of falling into prominent discrete categories of 0.973, 1.0, or less than 0.973 (H3 indicator) can be derived. Pr(H3=2): exp(0.19396)/(1 + exp(0.19396) + exp(-0.11391))=0.391 while Pr(H3=3): exp(-0.11391)/(1 + exp(0.19396) + exp(-0.11391))=0.287 and Pr(H3=1) = 0.322. The probability of H3=2 is estimated by the model to be the most likely, and the value of 0.973 is imputed for this respondent and the second estimation step is not needed.

Scenario 2:

A respondent has the following response characteristics: female sex, aged 44, and married or common law. Responses to the WG-SS indicate “No difficulty” in vision, hearing, cognition, self-care, and communication and responses of “Some difficulty” to mobility. Responses to the WG ES-F show pain was experienced “every day (in between a little and a lot) OR most days (a lot),” that the person, “felt worried, nervous or anxious monthly (a little or in between a little and a lot), OR feel worried, nervous or anxious daily (a little),” and that the individual “never feels depressed OR feels depressed a few times a year.”

Estimation Step 1 first takes the predicted sum of β and γ coefficients, including the intercept term. Note that the age term is centred at 62, so is included as 44-62= -18 (linear), 324 (quadratic) and -5832 (cubic) forms. $\sum {\text{β}}_{1-\text{n}}{\text{X}}_{1-\text{n}}$ : 0.6253845 (intercept) + -0.5592043*1(mobility) + -2.278011*1 (pain) + -0.531578*1(anxiety) + -0.01504*-18 (age) + -0.001247*324 (Age2) + 0.0000585*-5832 (age3) + 0.436339*1 (sex) = -2.78162.  $\sum {\text{γ}}_{1-\text{n}}{\text{X}}_{1-\text{n}}$ :  -0.061049 (intercept) + -0.73422*1 (mobility) + -2.093236*1 (pain) + -0.629575*1 (anxiety) + -0.022366*-18 (age) + 0.0017927*324 (age²) + -0.0000372*-5832 (age³) + -0.008179*1 (sex) = -2.32589. From this, probabilities of falling into prominent discrete categories of 0.973, 1.0, or less than 0.973 (H3 indicator) can be derived. Pr(H3=2): exp(-2.7816)/(1 + exp(-2.7816) + exp(-2.3259)) = 0.0534 while Pr(H3=3): exp(-2.3259)/(1 + exp(-2.7816) + exp(-2.3259))=0.0842, leaving the difference as Pr(H3=1)=0.862. In this case, the probability of health state utility scores of 1.0 (0.0842) or 0.973 (0.0534) are lower than the probability of scores being less than 0.973, so direct imputation of one of these scores is not possible (in which case the estimation would be completed) and the second estimation step follows.

Estimation Step 2 uses the same variables except marital status and anxiety, which are not included in the predictive model in this step. The sum of the second-step δ coefficients can be expressed as $\sum {\text{δ}}_{1-\text{n}}{\text{X}}_{1-\text{n}}$ : 1.2501338 (intercept) + -0.20157005*1 (mobility) + -0.35916977*1 (pain) + -0.0015583*-18 (age) + 0.000004428*324 (age²) + -0.000002948*-5832 (age³) + 0.01516857*1 (sex) =0.75124. To reverse transform the estimation coefficients to express the health state utility score, ((SIN(0.75124)+1)*(1 + 0.36) / 2) - 0.36 = 0.78413 is used, the mapped HUI3 health state utility score for this respondent.