Comparisons between data up to 1995 and data since 1996

Starting with reference year 1996, the Survey of Labour and Income Dynamics (SLID) replaced the annual Survey of Consumer Finances (SCF) as the official source of family income in Canada. This means that data up to and including 1995 are drawn from SCF (last conducted for reference year 1997), and data for 1996 and onwards are drawn from the SLID (which was introduced in 1993).

Different surveys will produce slightly different estimates on the same topics due to a variety of factors. Every attempt was made to minimize and monitor these differences between the two income surveys, while nonetheless making some important improvements in survey practices. Before replacing the SCF series with SLID, a careful study was done on the overlapping reference years, particularly the years 1996 and 1997, as SLID only acquired its full sample size in 1996. The results of the study are contained in the Income Statistics Division research paper, A Comparison of the Results of the Survey of Labour and Income Dynamics (SLID) and the Survey of Consumer Finances (SCF) 1993-1997: Update (75F002MIE99007). All ISD research papers are available free of charge.

In short, it was found that the two surveys told essentially the same story for all of the main income concepts. It is still possible, nonetheless, that for some characteristics the data trends could reveal a “break” as a result of the change in survey. Such a break would likely appear as a noticeable upward or downward shift in a data series between the years 1995 and 1996. It represents a change in the data which is attributable to the two surveys having different samples and different methods (such as the use of tax data in the case of SLID), rather than a true change in the characteristics of the population. Users are advised to take note of the following survey differences which are known to exist and to have had an impact on the data trends at some detailed levels.

Top of Page

Better coverage of small income amounts

One notable improvement that occurred as a result of new survey techniques introduced in SLID is better coverage of small income amounts received by respondents. It has been observed in surveys conducted by questionnaire that respondents tend to forget or neglect small income amounts they received in the past. This means an underestimation of income in general, and in particular, it means that many people who received a small amount of income instead report no income at all (there are differences, however, depending on whether the income concept includes or excludes government transfers).

The use of administrative income tax files in SLID for the majority of sample respondents means that there is considerably better coverage of non-zero amounts of income, and in general, a greater number of recipients of most kinds of income. Another technique used by SLID which may have improved coverage is that, even for respondents who report income by interview instead of via their tax records, there are two chances to prompt them for income sources, and therefore a greater likelihood of capturing an amount. This is because some income concepts are touched on in the January interview and then covered in the May interview, where it is possible to remind the respondent of a positive response in January. The types of income for which such “dependent interviewing” is used are earnings (from employment or self-employment), employment insurance benefits, social assistance, and workers' compensation.

Top of Page

Detailed family types

The standard published “detailed family types” for economic families have changed in one regard. In the SCF, they are derived with reference to the “head of family”. In SLID, the same categories are used but in reference to the “major income earner”. SLID dropped the concept of head of family entirely, as it has little relevance in a modern context. But some sort of prioritization of people within a family is useful to uniquely identify the type of family, even if it is somewhat arbitrary.

The change in family concepts resulting from the transition from SCF to SLID has not affected data produced for the entire population of families consisting of two or more persons. However, for some of the detailed family types, the estimated number of families underwent a one-time increase or decrease between 1995 and 1996. Without drawing conclusions about the precise net effects of these changes, the following points can be made.

First, whereas the previous definition always gave husbands the status of head of family rather than wives, with the major income earner concept there is no distinction by sex, and it is possible for the wife to qualify. Since it still holds that wives are on average younger than husbands at least for older couples, this has caused a shift from elderly families to non-elderly families.

Second, the head of family concept gave preference to parents over their adult children and, where there is no husband-wife or parent-child relationship in the family, it gave preference to older members over younger ones. Now, younger adults are much more likely to qualify as major income earners than they did as heads of families. As a result, we see significant decreases in the number of “other elderly families” and “married couples with other relatives”, and a large increase in the number of “other non-elderly families”. (See the section “Family definitions” for the precise definitions of family types.)

Top of Page

Comparisons with previous editions

Data from different editions are not directly comparable. Every edition has some modifications done on data. The modification which is applied every year is the expression of all dollars amounts in constant dollars of the latest reference year. (See “Current dollars versus constant dollars”.)

The Survey of Labour and Income Dynamics uses estimates of the target population - which are derived independently from the survey - as benchmarks for producing survey estimates. These population estimates start with a Census and are then updated using administrative data to reflect the current population of Canada. Using these population counts reduces the sampling error and coverage bias of survey estimates. It also provides consistency of estimates across household surveys. Accurate population numbers are crucial in determining estimates from a sample survey like SLID. In order to translate the results of the survey into population estimates, each individual in the sample is assigned a weight indicating the number of persons in the population represented by that sample member.

Periodically, the weights used in the survey are updated to reflect the availability of new population benchmarks provided by a new census and new annual inter-censal estimates. When this happens, the weights are revised historically in order to maintain a consistent time series. Methodological improvements in the derivation of weights may also be implemented in a weight revision.

There was a historical weight revision in 2000 that was carried out on data back to 1980, such that figures for the entire time series changed. Traditionally, weights are derived using population benchmarks by province, age and sex. Since the 2000 weight revision, the weights in SLID also respect population benchmarks by household size and economic family size. The most recent historical weight revision for the Survey of Labour and Income Dynamics occurred with the release of data for 2003.

Top of Page

2003 historical revision

The release of the 2003 data is accompanied by a historical revision for 1990 to 2002 due to an update of the survey weights in both the Survey of Labour and Income Dynamics (SLID) and the Survey of Consumer Finances (SCF). The weighted data now take into account not only new population projections based on the 2001 Census of Population, but also valuable information on the overall distribution of wages and salaries-a major component of income-in Canada. A lot of research has been conducted on the new weighting methodology and it is felt that as a result of this, the ability of these surveys to accurately represent income levels in Canada has been given a new boost. The quality of the survey data on other topics has also been improved.

The impacts of the historical revision, which was applied to the data back to 1990, are quite similar for all years, so that the trends shown by the data remain very similar. But the actual levels of estimates do change in many situations. The estimates are in fact much closer to other reliable sources of information on similar topics.

The 2003 historical revision also incorporates revised 1992-base low income cut-offs (LICOs) resulting from a historical re-weighting of the 1992 Family Expenditure Survey.

Why we introduced new income adjustments in the weighting

Research has shown that historically, Statistics Canada's household income surveys have tended to under-represent people with very low levels of earnings or no earnings, as well as people with very high earnings, while over-representing people who are more in the middle of the earnings distribution. Average and aggregate earnings and incomes have been over-estimated as a result.

The System of National Accounts (SNA) corrected a comparable problem by applying aggregate adjustments to the estimates. In a household survey like SLID, such an approach is not possible; instead, the method for correcting the biases in estimation was to make differentiated adjustments to the weights attached to each of the respondents.

Statistics Canada over the past few years did a lot of investigation to develop the best possible options for improving the estimates through improved weighting techniques. We tested several options and evaluated their stability over several years of data, to ensure that no new biases would result from the corrections and to ensure that we chose the best possible option currently available. Finally, the evaluations included comparisons with other independent sources of information like the Census and the System of National Accounts. Apart from the time required to test and evaluate the changes, there is no particular reason why Statistics Canada introduced these changes now instead of next year or last year.

In addition to our usual adjustments to population estimates we concluded that adding the T4 administrative files (employer remittances of payroll taxes) were the best benchmark statistics available with which to adjust the weights in SLID. We believe that the quality of the T4 files is high and that historically, the quality of files is sufficient to allow this adjustment to be made back to 1990.

The improvements to survey weights during the 2000 and 2003 historical revisions have been part of a comprehensive project at Statistics Canada to improve the weighting strategies in the main annual surveys on income, expenditures, and wealth. The project is now complete for the Survey of Labour and Income Dynamics and the Survey of Consumer Finances.

Top of Page

What the new weighting methods consist of

Survey weights are the numbers assigned to each of the sample respondents so that together they sum to the target population and sub-groups of that population. To do this as accurately as possible, the weights are often adjusted to be in line with other independent sources of information. The Census of Population, with its related population projections, is the source for the benchmark demographic statistics used to adjust the survey weights. The revised weights now take into account population projections from the 2001 Census.

The additional benchmark statistics that are now used to adjust survey weights come from the annual T4 file from Canada Revenue Agency (CRA), which is a compilation of employer remittances for the purposes of payroll taxes. SLID obtains tabulated statistics from the T4. Specifically, the statistics being used concern the total number of people in each province who had earnings from paid employment during the year and the amounts they made relative to each other, called the earnings distribution. The number and earnings of self-employed people are not included. It is important to add that aggregated data are used from this file and no matching of individual information between the T4 file and the SLID and SCF data files is done.

The annual T4 files cover all employees, regardless of whether they filed a T1 tax return. The wages and salaries of employees represent a very high proportion of all income received by individuals. Although many people in the population do not have this type of income, the T4 data allow us to correct biases not only for the wage-earning population but also for the rest of the population, because it provides an accurate way of determining the presence or absence of wages.

This is not the first time that a survey has used the T4 files from CRA in order to better represent the distribution of incomes in the population. The Survey of Household Spending implemented similar weighting methods using T4 data in its 1999 historical revision.

As evidence that the additional calibration of weights has helped to better reflect the distribution of incomes in the SLID, the estimates are now more similar to related estimates from other sources. These sources are not only the T4 file, but also the T1 file (personal income tax file), the Census, and the System of National Accounts.

Top of Page

What was the impact of the revision on the estimates?

The change from the 1996 Census to the 2001 Census for population estimates and projections affected mainly the period after 1996.

The historical revision had little impact on the trends shown by SLID since 1996 and SCF up to 1995. This is because the addition of weight adjustments based on income had a similar impact for all years. The weight revision was taken back to 1990; consequently, only between the years 1989 and 1990 is there potentially a disturbance in the data series that could affect the representation of trends.

The level estimates, however, did change in many cases. The precise impacts can be seen by comparing the data that were published before the historical revision with those published in May of 2005. Note however that previously published dollar figures must be compared with their revised values using either current dollars or consistent constant dollars in both cases. Be aware also that rounding of dollar values in published tables may affect the precision of comparisons.

Finally, the SLID low income estimates changed as a result not only of the new weights in SLID, but also the new low income cut-offs with which family incomes in SLID are compared.

Top of Page

Examples of how the estimates changed

• In general, the number of people in the ten provinces with earnings from employment, also referred to as wages and salaries because it excludes self-employment, was adjusted downwards. However this is not true in all range categories of earnings: there are more people now in the lowest and highest earnings categories, but fewer in the middle of the distribution.
• Due to the predominant downward revisions in the number of persons by earnings classes and despite the upward revision in the number of high-wage earner adults, average and median wages and salaries usually decreased with the revision.
• Since wages and salaries constitute a large part of incomes, most estimates of market income, total income, and after-tax income were also adjusted downwards.
• As a consequence of lower wages and salaries, government transfers generally increased, including social assistance benefits, child benefits, and refundable sales tax credits such as the GST/HST Credit.
• The weight revisions due to the new income adjustments based on the T4 file affected estimates concerning children and adults up to age 64 relatively more than estimates concerning seniors, due to the fact that income from employment makes up a smaller portion of seniors' incomes.
• Consistent with the increases in the number of people with no earnings or very low earnings, the low income rates for persons and families (on an after-tax basis) generally underwent an upward revision in all years. Again the rates for senior-led families and individuals were less affected.
• Consistent with the increases in the number of people at the very low and very high ends of the earnings distribution, and the decrease in the number of people in the middle of the distribution, certain measures of income inequality (Gini coefficient, ratio of 5th to 1st quintile) generally increased.
• The revision also affected characteristics which are not directly related to incomes. For example, the following estimates are now closer to the estimates of the 2001 Census:
• Level of educational attainment in Canada.
• Rate of home ownership versus renting.

For more details of these impacts and more information on the new weighting methodology, please refer to the free research paper, Survey of Labour and Income Dynamics: 2003 historical revision, Statistics Canada, (forthcoming)