Revisions to 2006 to 2011 income data
Improving comparability of SLID and CIS through re-weighting
Archived Content
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.
All surveys attach a weight to each record to indicate the number of units in the population that are represented by that unit in the sample. Because differences between SLID and CIS arise largely as a result of coverage and response discrepancies, sample weighting techniques can be used to adjust SLID estimates to make them more comparable to CIS. The revisions are only done over 2006 to 2011 as these years have the best methodological information available for the adjustment.
Deriving revised weights consists of two elements. In social surveys, weighted population totals from the survey are set to equal population totals from an independent source, such as the census or an administrative data source. This process is known as “calibration”. Because there are small differences in SLID and CIS calibration methodologies, the first element in adjusting 2006 to 2011 weights is to apply the same calibration methodology used in CIS to SLID.
In the second element, SLID weights are also adjusted in a way that will reduce coverage differences and response differences between SLID and CIS. This adjustment uses information from T1FF for 2006-2011 as a new calibration source to adjust the number of persons represented at the lower-end of the income distribution in SLID. The T1FF is seen as an appropriate calibration source because it provides high quality family income information and it does not have methodological changes over this period (although as explained in the text box below, the T1FF is adjusted to make it more comparable to the survey data). Text box 1 describes this second element in more detail.
Start of text box
Using information from T1FF as a new calibration source for SLID
The following describes, in general terms, the steps involved in recalibrating SLID using information from T1FF.
In the first step, data and concepts are aligned to the greatest extent possible between SLID and T1FF;
- Populations that are excluded from SLID are also excluded from T1FF. This includes populations in the territories, on reserves and other Aboriginal settlements.
- Families are defined at the census family level in T1FF and SLIDNote 1. Since the T1FF is mainly based on the information provided on income tax returns, family estimates can only be calculated for census families and persons not in census families in T1FF.
- After-tax incomeNote 2 is calculated in the same way in the two data sources. For the purposes of this exercise, family income is defined as adjusted after-tax census family income. The adjustment is made by dividing census family income by the square root of the census family size and assigning this value to all persons in the census family. Persons not in a census family are considered a census family of size 1.
In the second step, population counts of the number of persons in low income, by province and for selected CMAs, are generated from T1FF for each year from 2006 to 2011, using the after-tax low income measure (LIM-AT) methodologyNote 3. In deriving low-income counts from T1FF, one needs to take into consideration that T1FF includes additional populations that are excluded from SLIDNote 4, and that T1FF tends to produce higher counts of persons in lower income brackets than survey or census sourcesNote 5. Thus, the low-income thresholds used in this step are adjusted accordingly to reduce the calibration totals derived from T1FFNote 6.
In the third step, sampled units in SLID are weighted such that the estimates for persons in low income in SLIDNote 7 match the calibration totals from T1FF determined in the previous steps.
End of text box
Table 1 presents high level results of this recalibration. It highlights changes in income between 2011 and 2012 using the original weights as well as the revised weights. These changes are also compared to changes in income between 2011 and 2012 from the T1FF.
| T1FF | SLID/CIS | |||||||
|---|---|---|---|---|---|---|---|---|
| 2011 | 2012 | Change | SLID 2011, unrevised | CIS 2012 | Change | SLID 2011, revised | 2011 to 2012 Change | |
| 2013 constant dollars | percent | 2013 constant dollars | percent | 2013 constant dollars | percent | |||
| Decile 1 | 13,600 | 13,900 | 2.2 | 18,600 | 18,200 | -2.2 | 18,000 | 1.1 |
| Decile 2 | 20,000 | 20,400 | 2.0 | 24,700 | 24,700 | 0.0 | 24,400 | 1.2 |
| Decile 3 | 25,500 | 25,900 | 1.6 | 30,000 | 30,300 | 1.0 | 29,900 | 1.3 |
| Decile 4 | 31,200 | 31,700 | 1.6 | 35,500 | 35,900 | 1.1 | 35,500 | 1.1 |
| Decile 5 | 37,100 | 37,700 | 1.6 | 40,800 | 41,900 | 2.7 | 40,900 | 2.4 |
| Decile 6 | 43,600 | 44,300 | 1.6 | 46,700 | 47,900 | 2.6 | 46,900 | 2.1 |
| Decile 7 | 51,200 | 52,100 | 1.8 | 53,700 | 54,900 | 2.2 | 53,900 | 1.9 |
| Decile 8 | 61,300 | 62,300 | 1.6 | 63,100 | 63,900 | 1.3 | 63,300 | 0.9 |
| Decile 9 | 78,200 | 79,600 | 1.8 | 78,300 | 79,100 | 1.0 | 78,200 | 1.2 |
|
||||||||
Estimates from T1FF refer to adjusted after-tax census family income while those of SLID and CIS refer to adjusted after-tax household incomeNote 8. The table presents upper limits of adjusted after-tax income for each decile of the respective distribution.
Although the two sets of estimates are not comparable in level (as T1FF refers to census families while SLID and CIS refer to households), the direction of change over time suggested by each source should be consistent. Since estimates from T1FF are based on a consistent methodology between 2011 and 2012, they can be used to judge the impact of methodological differences between SLID and CIS that affect the comparability of their results over the same period. If T1FF shows an increase in census family incomes between 2011 and 2012, then an increase in household incomes would also be expected.
T1FF estimates show that census families throughout the distribution experienced income gains between 2011 and 2012. In contrast, changes between 2011 SLID (unrevised) and 2012 CIS suggest otherwise. According to those estimates, households in the first decile of the distribution saw their incomes decline by 2.2% and those in the second decile remained unchanged. Income changes throughout the rest of the distribution, however, are more consistent with those from T1FF.
This result shows that inconsistencies between SLID and CIS are largely concentrated at the lower end of the distribution. Estimates that relate to families or individuals towards the middle of the distribution are more comparable.
The final panel of Table 1 presents SLID estimates for 2011 that have been derived using revised sample weights. Revised estimates for 2011 show that incomes in the bottom two deciles increased rather than decreased between 2011 and 2012.
- Date modified: