Development and implementation of microsimulation models of neurological conditions
by Philippe Finès, Rochelle Garner, Christina Bancej, Julie Bernier and Douglas G. Manuel
Prompted by growing attention to population aging and the potential health burden of neurological conditions (diseases, disorders and injuries to the brain and nervous systemNote 1Note 2), in June 2009, the federal Minister of Health announced the government’s commitment to a four-year study of neurological conditions, the National Population Health Study of Neurological Conditions (NPHSNC).Note 3 The goal of the NPHSNC, co-led by Neurological Health Charities Canada and the Public Health Agency of Canada (PHAC), was to build an understanding of neurological conditions and their impact on Canadians.Note 4Note 5 One of its components is a set of microsimulation models, POHEM-Neurological, that project the health and economic impacts of neurological conditions over a 20-year horizon.
The evolution of neurological conditions is a global health concern.Note 1 For example, when comparing dementia projection models, Norton et al.Note 6 found that all models predicted a significant increase in prevalence over the next 50 years. These authors suggested that microsimulation would be a useful tool in predicting future prevalence. More generally, the use of microsimulation is promoted in healthNote 7 and longitudinal analyses.Note 8
At Statistics Canada, POHEM (Population Health Model) and CRMM (Cancer Risk Management Model) comprise the family of health-specific microsimulation models. These are dynamic models with continuous-time and discrete events, dealing with specific conditions such as osteoarthritis,Note 9 cancer,Note 10 and cardiovascular disease.Note 11 For the models that employ a fully synthetic population, each person in the population is generated from birth, and their life course to death is simulated based on a set of rules or transition probabilities. The model outputs, when aggregated across the synthetic life courses, provide a realistic portrait of the Canadian population, at a point in time, on variables of interest.
The POHEM-Neurological models were designed for seven neurological conditions (NCs): Alzheimer’s disease and other dementias, cerebral palsy, epilepsy, multiple sclerosis, Parkinson’s disease, traumatic brain injury, and traumatic spinal cord injury. (Because comprehensive data were lacking, eight NCs among the 15 identified by the NPHSNC were not modelled.) By including current microdata, these models are grounded in empirical reality, thereby filling gaps on the epidemiology and health and economic impacts of these conditions in Canada. The series of seven models (one for each NC) can be considered one “meta-model” since they are built on the same framework.
This paper describes the methods and data used in the development and implementation of the POHEM-Neurological meta-model.
Development of conceptual models
The first step was development of conceptual models depicting the key drivers of the health and economic outcomes of NCs, the typical and significant health states through which persons with the NCs may progress, and their interrelationships, in order to project the health and economic impact of NCs over the next 20 years (2011 to 2031) under status quo assumptions. That is, people are aged in an unchanging world with respect to factors such as economic growth and policies; costs do not vary and are expressed as constant Canadian 2010$; and no cohort or period effects are allowed. The literature was search for existing conceptual models, and expert clinical validation was sought. Although the microsimulation component was informed by the NPHSNC Working Group on Health and Economic Modelling of Neurological Conditions—with representation from Statistics Canada as well as clinical, neuroscience and health economics experts—conceptual models were developed and validated iteratively through several focus groups that included individuals with NCs and their family caregivers. The goal was to build a unique framework that could be run for each of the seven NCs, while balancing two objectives: 1) creation of a general framework to simulate all NCs in a similar way, and 2) allowing for the inclusion of specificities of each NC.
Each (synthetic) person in the POHEM-Neurological models has the following characteristics:
- They enter the microsimulation at birth (in the “no NC” state) with a given sex; they leave at death.
- All transitions from one state to the next are generated by a random process that converts a random number into a time-to-event. One such transition moves a person from a “no NC” state to being diagnosed with the NC under study. If the calculated time-to-event does not fall within the calendar year in which the calculation is performed, the transition does not occur, and transition probability is recalculated the next year. If the calculated time-to-event falls within the year, the transition to being diagnosed with the NC (that is, an incident NC) occurs at the date generated.
- People diagnosed with cerebral palsy or epilepsy have the possibility of the NC being no longer present (“cured”).
- Each person is assigned a set of characteristics that vary during their life course (in addition to age): functional health, as measured by the Health Utilities Index Mark 3 (HUI3); presence or absence of an informal caregiver; place of residence, at home or in a long-term care (LTC) institution; and health care costs. Some of these characteristics are modified at any time throughout the year (age, HUI 3, costs, as well as incidence of an NC—as mentioned above), while others are modified only at the end of the year (institutionalization, assignment of caregiver).
- The risk of death varies by age, sex, presence of an NC, or place of residence.
When a model is run, each person is simulated one at a time, until a number sufficient to yield stable results has been generated (small Monte-Carlo errors). For POHEM-Neurological, projections have been run with 32 million persons, the oldest being born in 1872 and the final death occurring in 2100.
Parameters are used to define characteristics of the synthetic population. Their values depend not only on the diagnosis of an NC, but also on other characteristics of the person, such as sex and age (Text table 1). For example, incidence of a specific NC varies by sex and age group; health care costs vary by sex, age group, and the presence or absence of a specific NC. Results are produced in tables that contain cross-sections of the synthetic population, according to pre-determined dimensions. Those tables can be exported (for example, in Microsoft Excel worksheets) for manipulation by users.
People with a given NC may have any number of other neurological and/or non-neurological comorbidities. Attempts to capture this reality are made by simulating persons with a specific NC regardless of the presence or absence of those comorbidities, relative to a population similar to that under study, but with no NC. The analysis compares the estimates of results (for example, costs, functional health, mortality, long-term care entry) of the population with the NC to those of the population without the NC. Summing the disease-specific costs to provide an overall cost of the NCs under study is precluded, as the conditions and their costs will overlap.
Specific aspects of the model
POHEM-Neurological was based on microsimulation health models already developed at Statistics Canada,Note 9Note 10Note 11 but it was necessary to design and model additional modules in order to account for the nature of NCs.
No single, comprehensive data source from which to produce the estimates is available. A variety of sources was used to ensure broad coverage of health and economic burdens. If primary data were not available, equations (for risk, assignment of a caregiver, functional health, etc.) were obtained from the published literature or determined ad hoc. Most of these equations are not included in this article but are available from the authors on request.
NC-specific incidence rates and mortality hazards were entered as input and contributed to prevalence. Sex-age-NC-specific incidence and mortality rates were derived based on administrative data for British Columbia. Case definitions and algorithms were developed. Four conditions (Alzheimer’s disease and other dementias, epilepsy,Note 12 multiple sclerosis, Parkinson’s diseaseNote 13) were validated against electronic medical records in Ontario; the other three conditions were determined to be consistent with expert advice to the Canadian Chronic Disease Surveillance System Neurological Conditions Working Group and/or systematic review of the validation literature for NCs.Note 14 All-cause mortality rates were obtained by examining deaths among the cohorts of individuals previously determined to have prevalent NCs. As such, all-cause mortality rates reflect deaths among individuals with NCs, and not simply deaths attributed to NCs. To determine mortality risk ratios (RRs), sex-age-NC-specific mortality rates (Text table 1) were compared with mortality rates in the general population for the same sex and age. Mortality rates for the general population vary by calendar year, but not RRs.
When unmodified incidence rates were entered, the projected prevalence rates for cerebral palsy and epilepsy became implausibly high relative to those estimated directly in the source data. For both conditions, evidence shows that diagnosis at one point may not be an accurate reflection of diagnosis later in life, whether due to misclassification (cerebral palsy)Note 15 or long-term remission (epilepsy).Note 16Note 17 Therefore, an evidence-based approach was used to reconcile projected prevalence with observed prevalence rates: for these two NCs, “cure” was allowed. The meta-model thus includes a group of ancillary parameters for “cure” (Text table 1).
Assignment of functional health
HUI3 is the measure of functional healthNote 18 used in POHEM-Neurological; it is also a predictor of other life events, such as assignment of a caregiver and institutionalization (Text table 1). Its value depends on the person’s age, sex and NC status. For persons without NCs living in private households, values were obtained from population health surveys conducted by Statistics Canada; for a data source, users can choose between the Canadian Community Health Survey (CCHS)Note 19 and the National Population Health Survey (NPHS).Note 20 Means and standard errors of HUI3 were used to randomly assign (according to sex and age group) a HUI3 score to individuals without an NC who were living in a private household.
For the household population aged 15 or older who had an NC, means and standard errors of HUI3 were calculated by sex and age group based on data from the 2011 Survey on Living with Neurological Conditions in Canada (SLNCC).Note 21 For children younger than 15 with an NC (epilepsy or cerebral palsy), a population not covered by the SLNCC, values were obtained from the 2006 Participation and Activity Limitations Survey (PALS).Note 22 For household residents younger than 65 who had a NC, these HUI3 values were applied in the models. For household residents aged 65 or older, to ensure that HUI3 decreased with advancing age, the value of functional health assigned in the models was calculated as:
HUI3NC,age5 = (1-weight)*(HUI3NC,65+)+ weight*(RelativeHUI3)*(HUI3NoNC,age5)
- HUI3NC,65+ is the mean HUI3 value obtained from the SLNCC for individuals aged 65 or older with a particular NC;
- HUI3NoNC,age5 is the mean HUI3 value obtained from the general population (CCHSNote 19 or NPHSNote 20) without NCs for individuals in a specific 5-year age group;
- RelativeHUI3 is the ratio of HUI3 from SLNCC/PALS for individuals younger than 65 with NCs relative to HUI3 values for individuals younger than 65 without NCs; and
- weight is a proportion chosen by the user to determine how much of the assigned HUI3 value will be determined by the underlying data sources.
Assignment of informal caregiver
Caregiver assignment reflects the receipt of unpaid support and does not account for the actual need for or availability of caregivers. Informal care data, including receipt of care, number of hours of care, and characteristics of the caregiver, came from the SLNCCNote 21 and the 2012 General Survey (GSS)—Caregiving and Care Receiving.Note 23 Persons with and without NCs are eligible to be assigned a caregiver. Assignment occurs at the end of each year for each synthetic person, depending on their characteristics (Text table 1):
- If the person resides in a private household, caregiver assignment depends either on the potential recipient’s sex, age group, and HUI3 (if the person does not have an NC) or on their HUI3 and NC (if the person has an NC). A random draw assigns a caregiver to the synthetic person according to proportions observed in surveys. If a caregiver is assigned, the sex, age, HUI3, out-of-pocket expenses and level of care (fewer than 7, 7 to 15, 15 to 71, more than 71 hours per week) provided by the caregiver are randomly generated according to observed data.
- If the person lives in an institution, a caregiver is assigned to them only if they had had a caregiver the last year they lived in a private household. In this case, the person is assigned a caregiver until death, but the care received is set to the lowest level (fewer than 7 hours per week) to reflect what was observed in the surveys.
Data for residents of long-term care (LTC) institutions were based on Ontario records from the Continuing Care Reporting System. Analyses were conducted and estimates provided by researchers from the University of Waterloo (interRAI).Note 24 In POHEM-Neurological, persons who enter LTC remain there until death. The probability of transitioning into an institution is assessed at the end of the calendar year. During the development phase, it was noted that current institutionalization patterns differed from historical patterns. Therefore,
- for years before 2004, transitions to LTC were not modelled;
- in 2004, some persons were transitioned “en masse” into institutions if a random number attached to them was lower than the observed percentage by sex, age group, and NC in LTC in OntarioNote 24;
- for 2005 and beyond, persons already in LTC were subject to the mortality hazards and functional health status specific to institutions; those residing in private households were subject to the possibility of transitioning into LTC according to their sex, age, functional health (HUI3), and whether they had an NC.
Despite these adjustments, values of the parameters did not correctly reproduce observed rates of transition into LTC or stays in LTC, resulting in poor projections. Therefore, a parameter was added to prevent transition into institutions. However, at any moment during the life course, the synthetic person accumulates costs related to all the categories (health sector and out-of-pocket categories); thus, costs related to LTC institutionalization are still available from the models.
Health care costs
Data on health care costs were used as input (per capita costs by cost category, age, sex) and contributed to global measures of costs (total cost per year, average cost within subpopulation). Direct health care costs were estimated from administrative health data from OntarioNote 25 and the British Columbia Ministry of Health. The health care sectors examined were: physician services; hospital costs; pharmaceutical costs; rehabilitation hospital costs; home care costs; long-term care costs; and assistive devices costs (the last three were obtained from Ontario only).
Means and standard errors of costs by sex and five-year age group were provided separately for cohorts with each of the seven NCs, and for the cohort with none of the NCs (Text table 1). For those with NCs, costs were further classified according to whether they were incurred in the first 12 months after diagnosis (incident costs) or more than a year after diagnosis (prevalent costs).
For the four health care sectors where cost estimates were available from two provinces, a weighted average was calculated as: p*(estimate from Ontario) + (1-p)*(estimate from British Columbia). Because Ontario’s population is three times as large as British Columbia’s, p was set to 0.75. Costs included two categories of out-of-pocket costs (expenses paid by an individual but not refunded by insurance or government): those of the patient and those for the caregiver.
At any time, the current cost carried by a synthetic person can be updated based on their characteristics. The cost may change for the following reasons:
- The person is diagnosed with an NC. Incident costs (specific to the sex, age and NC) are applied for one year. On the anniversary of NC incidence, prevalent costs are assigned thereafter until cure or death.
- On the person’s birthday, their costs are re-assigned based on their updated age (if needed).
To validate microsimulation models, Kopec et al.Note 26 proposed a measure with 17 criteria that consider whether the mathematical frame, collected data, code, etc. are correct and have been combined appropriately. This evidence of validity is derived from three sources:
- Model development process
- Model performance
- Consequences of model-based decisions
Kopec et al. did not recommend a specific number of criteria that should be satisfied for the model to be considered valid. However, intuitively, a high percentage of satisfied criteria indicates validity. This grid was applied to the POHEM-Neurological meta-model considered as a whole.
In general, the POHEM-Neurological meta-model meets the validation criteria for model development and model performance (Text table 2). The criteria of the third category will be evaluated through future applications.
The models have several limitations. Incidence and mortality data were obtained from only one province, but were applied nationally. Furthermore, these were administrative data. If the true incidence of NCs or the way that physicians apply the underlying administrative codes differ by province, this would not be captured in the models. Nonetheless, the algorithms used to identify NCs were validated in several provinces.
Data for health care costs come from only two provinces. Although these two provinces represent a majority of the population, data from other provinces would yield a more precise picture of incidence and costs.
The current version of the meta-model does not include risk factors (other than age and sex) for the NCs. However, inclusion of risk factors would primarily serve as points of intervention to affect future incidence of the NCs, which can now be examined in POHEM-Neurological by varying incidence and mortality rates in different scenarios.
The models do not account for the potential decreased availability of caregivers in the future (due to factors such as changing family structure). Therefore, the projections of assignment of and out-of-pocket expenses for caregivers may be questionable. However, the models focus on need for caregivers, not their availability.
In any microsimulation model, two types of uncertainty must be addressed. Type I uncertainty (Monte-Carlo errors) is captured by random generators used in the models. Large populations reduce the relative impact of random fluctuations and produce stable results for POHEM-Neurological. Type II uncertainty (parameter uncertainty) refers to imprecision in the data. For example, the data are obtained from administrative databases, surveys and analysis models, and so are subject to measure errors. This imprecision is not easily accounted for in POHEM-Neurological, but users may run different scenarios and thereby conduct sensitivity analyses.
The aim of POHEM-Neurological is to project health and economic impacts of NCs over the next 20 years. In general, this meta-model is considered to be valid. The first objective—that the NCs should be modelled with the same framework—is met: each model uses the same interface to the user, the same algorithm, the same software (ModgenNote 27), and the same programming code. The second objective—to account for the peculiarities of each of the NCs—is captured by specific sets of values of parameters. The models have been used for projections under status quo conditions but can be run with other scenarios to evaluate the future impact of some factors such as reduction of costs or modification of incidence rates. Further studies are recommended to better assess model validity and the consequences of model-based decisions.
The authors acknowledge the contribution of:
- Members of the expert National Population Health Study of Neurological Conditions Working Group on Health and Economic Modeling of Neurological Conditions which included the authors as well as: Larry Chambers (Alzheimer’s Society Canada), Celina Rayonne Chavannes (Neurological Health Charities Canada), Jonathan Chen (University of Waterloo), Alan Diener (Public Health Agency of Canada), John Hirdes (University of Waterloo), Liisa Jaakkimainen (Institute for Clinical Evaluative Science), Micaela Jantzi (University of Waterloo), Nathalie Jetté (University of Calgary), Norma Jutan (Canadian Institute for Health Information), Janice Keefe (Mount St. Vincent University), Jullian Little (University of Ottawa), Ruth Ann Marrie (University of Manitoba), Tamara Pringsheim (University of Calgary), Kim Reimer (British Columbia Ministry of Health), Michael Shevell (McGill University), Jose Tellez-Zenteno (University of Saskatoon), Karen Tu (Institute for Clinical Evaluative Science), Joan Versnel (Dalhousie University), Samuel Vezina (Mount St. Vincent University), Michael Wolfson (University of Ottawa).
- Individuals who volunteered their time for the Workshops, including the Workshop Conceptualizing the Health and Economic Impacts of Alzheimer’s Disease and other Dementias in Canada: Lynn Beattie, Debbie Benczkowski, Sandra Black, Sylvia Davidson, Julie Foley, Lowell Foley, David Harvey, and facilitator, Scott Dudgeon.
- Additional experts: Keiko Asakawa, Bill Flanagan, D Green, Patricia Lau, Xiaofen Lin, Claude Nadeau, Trang Nguyen, Maryam Oskoui, Pamela Ramage-Morin, Jason Sutherland, Ron Wall, Sharmila Vaz, Samuel Wiebe, and Ania Zycki.
- Contributors from: BC Ministry of Health, ICES, interRAI, uWaterloo.
- Past members of working groups involved in the National Population Health Study of Neurological Conditions project: Scott Dudgeon, Ken Eng, Patricia Lau, Pat McCrae, Shannon McDonald, Jillian Oderkirk, Nancy White
The authors are indebted to Deirdre Hennessy for her suggestion of the phrase “meta-model.” They thank the reviewers for their enlightening comments.