Feasibility Study on the Use of Provincial/Territorial Medicare Records for Measuring the Level of Inter-provincial and Inter-territorial Migration
3. Assessment of Medicare records and the CCTB
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.
Of all the criteria used in assessing quality, coverage is one of the most important as it impacts all other measurements. All administrative records are subject to coverage problems when they are used for estimation purposes. In fact, overall coverage is one of the greatest strengths of the health care files while it is one of the greatest weaknesses of the CCTB data in that it only covers a portion of intended population, namely children. However, by limiting the study to the coverage of children, both files are more comparable. The main issue with coverage error is whether the problems can be measured and corrected. Coverage errors are correctable if the proper steps are taken to identify the size of the coverage error and the potential biases that may be introduced in the estimation process. Not having access to all the health care records seriously impedes any chance of bias correction due to coverage error.
Coverage problems can lead to estimates that do not reflect reality or biased estimates; where the error is not evenly distributed among important sub-populations, and more importantly, populations who are more likely to migrate. Given the high stakes involved in producing unbiased results when comparing provincial estimates of migration, other elements of our assessment take on an important role.
In this section, we will also discuss aspects of completeness of the health care records by emphasizing the various populations covered by each province and how they may differ from one another. Special attention will be placed on identifying problems that may lead to uneven estimation error across jurisdictions. Issues of uniformity, consistency, timeliness, reliability, the level of details and the accuracy of these files will be discussed throughout the paper.
3.1 Total health registration coverage comparison
In discussing "coverage" we wish to make a distinction in definition as we are using the term in two different senses:
Completeness of the Medicare file (an
administrative or quality assurance concept)
- e.g. people who do not have a health card as they have opted out or never applied for one even though they were entitled to be covered (this creates undercoverage);
- e.g. there may be duplicate records for the same person; or duplicate records of people who have left the province but did not notify the province that they had left and still have a card or they have died but still have an active record on file (this creates overcoverage).
Eligibility categories for receiving
provincial health coverage (a legal or governmental definition)
- Provincial or territorial eligibility is uneven (e.g., in some cases, refugee status claimants are not covered by provincial health care but are included in Demography Division's population estimates);
- Demography Division's estimate of inter-provincial migration, includes in its population estimates, NPRs, the armed forces, Royal Canadian Mounted Police (RCMP), Canadian Security Intelligence Service (CSIS) and inmates of federal penitentiaries and their dependents that are covered by Federal health care programs and not the provinces or territories;
- Other administrative rules of certain provinces may impede the accurate measurement of the migration event (e.g., in Alberta, a person moving to Alberta and residing more than 183 days - which meets one of the permanent residency requirements – may not get covered until their spouse moves to Alberta or in the 13th month of residency in Alberta should the spouse not move.6 In other words, if the spouse has no intention of moving to Alberta, there is a waiting period of 12 months before the new resident can get medical coverage in Alberta).
Currently the provinces do not send us a running tally of their health care registrants on a regular basis. However, most of them do release this information to Health Canada for their annual report.
Table 2 compares the coverage rates of health care registration files to the population estimates from Demography Division. It is assumed that, barring the exclusions noted above, the total number of those with health coverage would be equivalent to the population estimates as Canada provides universal health care. If the ratio of the provincial/territorial registrations to our population estimates is close to one then at least the population bases are comparable. We expect because of the previously noted exclusions in coverage of the health care programs a slight underestimation from the population estimates produced by Demography Division.
[Note: We do not have age/gender or regional breakdowns and have requested these from the statistical focal points for a forthcoming analysis. These are total tallies of registrants and certain undercovered populations may be off-set by overcovered populations. Discussions on these phenomena are featured later on.]
These coverage ratios are presented here from published information and cannot be further analyzed by age, gender or sub-provincial / territorial representation. For example, one may want to see if problems of overcoverage are linked to the elderly or some other group in the population. A recommendation would be to get detailed breakdowns of population counts from all provincial/territorial health care registrations to compare to the census information.
Also, the coverage number by province may not accurately reflect the provincial /territorial records in duplication between jurisdictions. Provinces that are not being notified when a resident has left the province (so he/she still has an active health card) could explain in part ratios over one in certain areas where out-migration is higher than in-migration. Deceased individuals with a still-active health care account may also cause a ratio greater than 1.0. Not registering for a health card or just opting out could result in a ratio of less than 1.0. Differences in our estimates could also be the result of the timing as there maybe different reference periods even after adjustments are made. As you will see in the next table the population included on the health care files is actually expected to be lower than their actual resident population because of exclusionary rules. It is difficult to ascertain the size of the excluded population as they often overlap with other inclusionary rules (e.g., refugees may be excluded from certain provincial health care programs, unless they hold a work permit, then they are included).
In most cases the coverage ratio is close to one for the total population estimate. This may be an artefact of errors that cancel each other out, and does not reflect the data's accuracy. The coverage rate is the most consistent in Manitoba. This ratio indicates that, overall, the registered population as measured against the provincial/territorial total health care registration database are close to Demography Division's component method of population estimation.
Newfoundland and Labrador has the most divergent ratio averaging 1.12 between 2000 and 2006 with a ratio of 1.07 in 2006. This means that Newfoundland and Labrador has more people under their health care system than what is estimated. This discrepancy probably lies with the Newfoundland and Labrador health care registration system as in the Canada Health Act Annual Report for 2006, a footnote states that the number of registered persons in Newfoundland exceeds the number of residents. Re-registration of residents to commence in [April 1] 2006.
3.2 Consistency and completeness of health care data
Tables 3 and 4 present data on the consistency and completeness of data relative to what is covered by registrants to the provincial / territorial health care systems. These tables show the results of a 2007 questionnaire sent to the provinces and territories in order to update a similar questionnaire back in 1996.
While the data are summarized, they do point to the even application of the health care services from one province and territory to another. Moreover, they also point out the various impediments one might have to circumvent in order to get full measurement of the population's migration.
Tables 3 and 4 above indicate those areas where the provinces do not cover certain groups while the CCTB does – namely NPRs, the dependents of those in the categories of prisons, the diplomatic corps and the Armed Forces. This "coverage" difference would result in a coverage ratio of less than one. In some cases, there is also an opting out provision as in Alberta, but the number opting out is low; however, we are unable to estimate this number from the information we have.
As stated earlier, in the case of Alberta, medical coverage for new residents is only provided when both the applicant and spouse are permanently residing in Alberta or after a stay of 12 months if the spouse is not intending to move to Alberta. In the latter case, there is also a timeliness problem due to the lag-effect. This could be significant in times of economic boom as was observed in the booming economy in Alberta of recent months. This has resulted in workers moving to Alberta from other parts of Canada, leaving their spouse behind because of the increased cost and low availability of housing and not being part of the Alberta health care. To further complicate this matter is the inability to accurately measure the number of persons in this situation or the lag that may be anticipated if the spouse does plan to migrate to Alberta later within the year.
3.3 In-migrant comparisons
Although Demography Division publishes data for both in and out inter-provincial/territorial migrants, this kind of detail is not available to do a comparison using the health care files as we are presently only receiving in-migrant7 information through the new-registrants reports. It is uncertain if all the provinces could report the destination of the people who are removed from their health care programs and is it also unclear how accurate such information would be. As well, we lack detail as to a single year breakdown from the health care records for most provinces, although conceivably this information could be made available. So in comparing the CCTB data which cover those 17 and under, we have had to make adjustments to the 5-year age grouping data from the health care files.
To adjust for the under 17 age grouping, we estimated 3/5 of the 15 to 19 age group as a proxy to the 15 to 17 age group. Where the data were available by single year of age we were able to get the correct population counts for the 0 to 17 year-old population.
Therefore Table 5 presents the ratio of Medicare new resident data to preliminary CCTB for in-migrant data only (for the 0 to 17 age group).
Comparison of quarterly in-migration based on provincial and territorial health insurance data and preliminary Demography Division estimates (CCTB-based) with annual figures (as of July 1) – 0 to 17 age group
As Rémillard had previously found in his evaluation of the 1986 to 1995 period, there was a wide range of differences in the in-migration ratio values among the provinces and territories and from quarter to quarter within the same province or territory as was found for this 2001/2002 to 2005/2006 period.
The highest ratios for the 2001 to 2006 period were found in Manitoba, British Columbia and the Northwest Territories (over 1.0 in some cases), while the lowest were in New Brunswick (consistently below 1.0, ranging from 0.1 to 0.8) and Saskatchewan (from 0.4 to 1.7). Overall we note that the CCTB reports more migration than does the health care files.
The province with the worst ratios is New Brunswick. Overall, the new resident data from their health insurance registration files were consistently very low compared to the estimates from the CCTB files for the same cohort. Historically, this pattern was also true for the earlier period, 1986 to 1995. In fact, ratios between the two datasets for the quarters ranged from a high of 0.8 for the October to December 2004 period to 0.1 from October to December 2001, July to September 2003, July to September 2004 and July to September 2005. Annually, the ratio for New Brunswick is between 0.2 and 0.3. We have been in discussions with the province of New Brunswick to resolve the issue surrounding its abnormally low in-migration counts and have summarized our efforts in Annex C at the end of this document.
Saskatchewan comes in second for the worse ratios. These ratios also fluctuate widely from 0.4 to 1.7 as stated above. One important factor for Saskatchewan as to why their figures seem to be so much lower than ours has to do with their category of "unknown" – Saskatchewan does not know whether a new resident is from another province/territory or comes from outside Canada. Annex D presents these data on a quarterly basis from 2001/2002 to 2005/2006. In some cases, this unknown category represents a significant number of total in-migrants to Saskatchewan: 30% of the total in-migrants or 248 were of unknown origin for the July to September 2003 period; 18% or 333 were unknown for the July to September 2005 period.
We should point out that British Columbia had an annual ratio of 3.4 for October to December 2005 for the 0 to 17 age group – 5,899 in-migrants from British Colombia's new health card registrations from outside the province versus 1,722 in-migrants from the Demography Division estimates (the equivalent estimates for all ages were respectively, 31,444 and 9,378). The British Columbia health care registration figures are more than twice what they have been for the equivalent quarters in earlier years. More specifically, the October and November 2005 figures were higher than normal – for all ages, the average for October 2000 to 2004 was 5,102 while for October 2005 the figure was 10,736. In November, the 2000 to 2004 average was 5,466 but November 2005 had 16,435. Contact with the British Columbia Ministry of Health revealed that the large numbers in October and November 2005 were "an accumulated backlog which was processed in October and November 2005"8.
As stated previously, the ratio tends to be less than one since many of the provincial/territorial health care systems do not provide coverage to NPRs9 while Statistics Canada includes them in their inter-provincial migration and population estimates. As well, in compiling data from the provincial/territorial new health card registrations, some provinces did not provide the data for some months (e.g. Saskatchewan was missing November and December 2002; the Northwest Territories did not send us January and February 2005) leading to completeness problems.
Nation-wide, when one adds up the quarters for all provinces and territories, 75% of the quarters have a ratio of less than one indicating that the health care registration for in-migrants ages 0 to 17 as reported by the province/territory is less than the Demography Division CCTB-based in-migrant estimates. This is in contrast to the 98% to 99% coverage ratio of all health care registrations reported by the province/territory compared to the total population estimates by Demography Division.10 On the other hand, over the observed time period, the ratios have gotten closer for both quarterly and annual levels for the 0 to 17 age group with an annual national ratio of about 0.9 in 2004/2005. This may suggest that the nature of the migration captured by the CCTB may be different. The CCTB file may include temporary migration, persons planning to migrate soon after or to migrate back to their original province and so did not change their health coverage because of the long wait period.
On a quarterly basis, almost all provinces and territories for both sets of data showed the highest level of in-migrants in the July to September period; a typical higher migration period. Though there is a difference in the absolute numbers (see Table 6) between the CCTB-based estimates and the provincial/territorial health care information, both of these sources show an overall but different seasonal trend in inter-provincial flows peaking in the June to September period and generally lowest in the January to March period – those with children would tend to move when the children were not in school; thus, the period April to September tend to have more in-migration movement than the October to March period. People also are more inclined to move when the weather is warmer.
For some quarters in certain provinces such as British Columbia (July to September and October to December 2002) and New Brunswick (July to September and October to December 2004), the highest absolute numbers are sometimes reached in the October to December period, based on the Medicare records while the equivalent CCTB-based numbers are highest for the June to September period. This phenomenon could be a lag effect where most provinces and territories have a 3-month waiting period for health-care coverage and new residents may delay in notifying the health care authorities that they are now living in the province/territory.
A month by month comparison of in-migrants is presented in Annex A. In looking at the monthly graphs in Annex A, it definitely looks like there is a lag effect in many provinces. However, this is not uniformly the case throughout the year or even for some provinces. This level of information shows more volatile findings when the data are compared and it was concluded that the data were best analyzed at the quarterly level to eliminate some of the fluctuations. However, from the monthly flows it is interesting to note that some provinces report no migration at all and this was stated in our conclusion as a qualitative factor about the reliability of the source information.
As we had observed an apparent lag in the data, we forced a 3 month lag (similar to most provincial and territorial waiting periods) on the health care data to assess the impact on the differential rates. Annex B presents the same two tables only with the Medicare data being lagged by three months (e.g. figures referring to the July to September period were placed in the April to June period) to see whether there was a better fit between the two sources. The Annex also includes a count of whether there had been an improvement in the relative differential value; in other words, are more ratios closer to one. Similarly, since the health care files report fewer migrations, we noted the number of times that the differential was greater than 1. Ultimately the net absolute percent difference was compared before and after the lag.
Table 6 shows the absolute percent differences between the CCTB and health care estimates with and without the lagged data. For ease of analysis we have included the average quarterly absolute percent difference.
Absolute percent difference of quarterly in-migration based on provincial and territorial health insurance data original and lagged 3 months as compared to preliminary Demography Division estimates (CCTB-based) with annual figures (as of July 1) – 0 to 17 age group
There were consistent improvements after lagging the data in the concordance of the migration figures for the provinces of Prince Edward Island, Nova Scotia and British Columbia and a strong improvement in the ratios for Newfoundland and Labrador, Ontario, Manitoba, Saskatchewan, the Yukon and Northwest Territories. No improvements were noted in New Brunswick or Nunavut.
Interestingly, despite the better concordance due to the lagging of the data of quarterly estimates for many provinces, the annual in-migration numbers were not necessarily better. In fact they seem to be slightly further apart. While the average difference is the absolute sum of the divergences, the total takes into account the direction of differences.
Clearly, one of the advantages that the CCTB data have over the provincial sources is that the federal program is applied uniformly over all the provinces. This is not to imply that a lag does not exist in the CCTB data, but it is uniform across all jurisdictions. Moreover, CCTB data are directly affected by expenditures of public funds and the program is dutifully administered throughout the year virtually eliminating seasonal effects, with the notable exception in July as new income data come into effect for eligibility. A lag in the health care data is an indication of problems in both the timeliness and consistency of the data. Judging by the volume of migrants and where they seem to manifest themselves as compared to the CCTB, it is apparent that the lag varies in size and in duration not only from one jurisdiction to another but also at different times during the year.
A much stronger rationale for this shift (or "clumping") of migration numbers in the health care files is probably the nature of the operational cycle in processing for the different provincial/territorial health offices. The file dates are actually the dates that the data were processed.11 This date is not related to the actual residency date. In some cases, when there is an administrative backlog of unprocessed applications there is an even further skewing or clumping as to when new residents have entered the province/territory health database since the date on the file would normally be the date that the data were processed.12
We selected Manitoba as having a similar volume of in-migrants to that measured by the CCTB for the 0 to 17 age group to do our analysis on accuracy. Questions were sent to our contact in Manitoba to determine the feasibility of calculating a net migration figure since Manitoba provides us with both new registrations from other provinces plus cancellations (equivalent to out-migrants). It is the only province to do so. But trying to come up with a net migration figure is tricky due to timing issues and, no doubt, incomplete data for out-migrants considering that the lag between event and reporting is likely different. Since their ratio of all health care registrants is close to 1.0, we can assume that their file is fairly close to the CCTB measure and overall coverage is good.
Annex E presents a comparison of net migration figures between the Manitoba Health Care registration system and our own estimates based on the CCTB.
Despite an improvement in the overall ratio, there are still many inconsistencies with the data. We still have a reliability problem due to data gaps resulting from missing monthly data (See Annex G) and the lack of data from Quebec. Just recently, we have received some data from Alberta which will be included in an Addendum. Furthermore, we are missing detail as we are only receiving in-migrant and not out-migrant information by 5-year age groups (in most cases), although Manitoba does provide us with cancellation information, equivalent to out-migrants from the province; however, there is also a timing issue as well as a completeness issue with this Manitoba data as stated earlier.
6 . As per information from Client Services Branch, Alberta Health and Wellness. See Annex H and the Alberta Application for Health Care Insurance Plan Coverage, Section B, and Question 2 for the question on residency of spouse.
7 . Manitoba provides us with monthly, what they call, cancellations. These are similar or equivalent to out-migrants. It is the only province to do so. But there are issues of timing these cancellations with the timing for in-migrants to get net migration. As well, there is no doubt, under-estimation of these out-migrants. E-mail from Manitoba Health Information Management Branch.
8 . E-mail from Health Insurance B.C.
9 . According to our 2007 survey, only New Brunswick and Alberta cover all types of NPRs and most jurisdictions did not cover those claiming refugee status except for 2 provinces (Nova Scotia and Ontario) but only under certain circumstances (i.e. they have to have applied for Permanent Residence Status or have a work permit).
10 . As we did not have data for Quebec and Alberta for our evaluation, these two provinces are excluded from our analysis and our national figures.
11 . As confirmed by the Health Registration & Vital Statistics Branch of Saskatchewan Health, the data we receive has the month assigned to a new resident based more on an administrative or processing date. This is especially noticeable when there is a backlog to be processed.
12 . As previously noted in British Columbia.
- Date modified: