Income Research Paper Series
Improvements to the Canadian Income Survey Methodology for the 2021 Reference Year

Introduction

The Canadian Income Survey (CIS) has introduced improvements to the methods and systems used to produce income estimates with the release of its 2021 reference year estimates. Key among these improvements is the shift to using the Administrative Personal Income Masterfile (APIM) as the primary source of income estimates, while previous reference years estimates were produced using only T1 tax return information. Estimates for 2021 also include updates to income processing and improvements to the weighting methodology.

This paper begins by explaining the advantages of moving to the APIM data source rather than the T1. Then, it describes other changes in the processing system introduced at the same time, as well as the newly implemented weighting strategy. Finally, it presents the approximate net result of these changes on income estimates using data for 2019 and 2020. The changes described in this paper highlight the ways in which data quality has been improved while producing minimal impact on key CIS estimates and trends.

Administrative Personal Income Masterfile (APIM)

The Administrative Personal Income Masterfile is a comprehensive, centrally processed source of personal income data generated from tax returns and associated tax slips. The annual database contains income information for over 30 million Canadians, covering the tax-filing population and those who have not filed by the time of data production, providing a comprehensive picture of each individual’s annual income situation.

Each year, the APIM database is produced upon receipt of the Canada Revenue Agency (CRA) tax files by Statistics Canada. These files are processed centrally at Statistics Canada to ensure coherent and consistent income concepts across the agency. The foundation of the APIM is the T1 Enhanced Database, which is sourced from the T1 Personal Masterfile—a database of personal tax returns. This is then supplemented with other tax slips providing information on income and related matters that are issued for the tax year by employers, the government or other institutions, and submitted to the CRA.^Note

Certain non-taxable income sources that are not available from any CRA records, such as provincial tax credits, are estimated based on the individual’s situation (auxiliary information) and government program specifications. Other income variables may require imputation through probabilistic matching on socio-demographic characteristics, as well as any available information that correlate with the income variables targeted.

Advantages of the APIM

The primary advantage of the APIM data source is that the T1 file is supplemented with the direct use of tax slips for key credits which were previously imputed (e.g., the Goods and Services Tax (GST) Credit file, the Canada Child Benefits (CCB) file), or previously underreported income sources (e.g., smaller T4 values, scholarships and bursaries, Registered Education Savings Plan (RESP), Registered Disability Savings Plan (RDSP)).

As a result, income data based on the APIM provides more comprehensive and accurate income estimates for individuals and families than data previously based on only the T1 file, which relied more on imputation for sources not found on the T1 file.

For example, 85.1% of the records on the 2021 CIS sample linked directly to the T1 file, 8.1% did not link to a social insurance number (SIN), while 6.8% of records linked to a SIN with no associated T1 record. Of the 6.8% of records linked to a SIN with no T1 record, 81% (representing 5.5% of records) have income from one or more tax slips and 19% (representing 1.3% of records) have no tax slips. The 5.5% of records with income from one or more tax slip but no T1 have additional processing and imputation performed directly on the APIM to complete missing concepts only available on the T1.  Under the old processing environment, additional processing and imputation would have been required for the 14.9% of records that didn’t link to the T1 file. However, using the APIM data source significantly reduces the number of records that require additional processing due to the existence of the records that linked to tax slips. This reduced the percentage of records that required imputation from 14.9% to 9.4% in 2021. Additionally, the methods for imputing income data for these records were updated to take further advantage of tax slips and available auxiliary information. 

Improvements to the income processing system

In addition to updating the data source to include various tax slips in the new system, improvements were also made to the processing of certain income sources. Specifically, the processing of child benefits and social assistance were updated to provide more comprehensive and accurate data. This section will describe the changes to the derivation of these income sources, while the impacts of these changes will be presented in the final portion of the paper.

Updates to the processing of child benefit amounts

Prior to 2021, child benefit amounts in the CIS were imputed using a deterministic method based on the family situation reported in the survey and the family income indicated in the T1. As the name suggests, the Administrative Person Income Masterfile method for deriving child benefit amounts is driven by administrative data, specifically the combination of T1 and data from tax slips.

However, shifting to a 100% administrative-based method would have led to some incoherence between those expected to receive child benefits and those where child benefits were reported on their tax slips. This is due to discrepancies in family composition between the CIS and the administrative data. For example, individuals with children on the survey, who therefore appear eligible, may have zero child benefit, while conversely, individuals without children on the survey may have received child benefits. Parents with shared custody, for example, may have reported their child(ren) in the survey (or not), depending on the timing of the interview, leading to potential differences between the family composition and the administrative data.

As one of the goals of capturing child benefits is to measure the impact of the benefits on poverty rates for children and families with children, a new child benefit model was produced to edit the data and maintain a historical time series between the former and new processing system, resulting in more consistent poverty measures.

Under the new child benefit method, data are processed using the administrative data as the default values. When there are inconsistencies based on the family composition, the deterministic method is used to derive child benefit amounts.

Specifically, for families where no child benefits were expected based on their composition (i.e., unattached individuals and married or common-law couples without children), the child benefit amount, if any, is set to zero.

For expected recipients of child benefits based on family composition (i.e., married or common-law couples with children and lone-parent families), the amount on APIM (which was based on the Canada Child Benefit file amount) is used. If the APIM value of child benefits is zero, an amount is derived based on the family composition.^Note

Finally, the new child benefit model was updated so that benefit amounts are derived using the previous year’s income, which is in line with how the CRA determines the child benefit amounts, whereas those derived under the old method utilized the current year’s income as a proxy.

Updates to the processing of social assistance amounts

The T5007 (Statement of Benefits) is used to derive social assistance amounts on the APIM processing system, providing a more accurate picture for recipients of this benefit.

Under the old processing system, social assistance amounts based on the T1 tax slip were typically underreported. Therefore, the CIS question about receiving social assistance payments was used to impute an amount for individuals who did not report an amount on the T1. However, over the years, this question seems to have been misunderstood by many respondents, as many interpreted this question as other types of government transfers and wrongly answered ‘yes’ to the survey question, thus receiving a social assistance amount. This led to a higher estimate for the number of recipients of the benefit.

In addition, seniors’ supplements for some provinces that were previously included in social assistance payments are now included in “other government transfers”.

Improvements to the weighting strategy

In addition to the shift in income data sources, the 2021 reference year data also introduced an improved weighting strategy. The updates produce better CIS income estimates that are closer to the benchmarks of administrative tax files, and the new models and processes have improved functionality and reduce production time.

The details of the updates to the weighting methodology could be summarized as follows:

• The non-response adjustment was updated to include four new auxiliary variables (two major sources of income variables, household-level income decile, person-level income decile), and a new logistic regression model was used to calculate the non-response adjustment factors. The role of non-response adjustment is to reduce the non-response bias and to control the non-response variance as much as possible.
• The calibration classes were adjusted. Calibration is the process of modifying the weights so that the estimates from the survey for relevant population characteristics respect population totals from other reliable sources. Calibration is applied to correct for coverage and non-response issues. In the CIS, the weights were calibrated to match known control totals for population counts based on the 2016 Census, provided by Statistics Canada’s Centre for Demography, including the number of persons by selected geographies, age groups and sex, and economic family sizes by province. It also ensures that the weighted distribution of income (based on wages and salaries classes) in the estimates matches that of the Canadian population based on the APIM. In the new weighting methodology, household size categories, which were also previously included in the calibration classes, have been dropped and a third economic family size (3+) was added to ensure that larger families were properly represented. A few minor adjustments to the sex-age calibrations classes were also made, and the 65+ age group was removed from the number of paid employees in wages and salaries classes, because it is not the main source of income for this age group.
• For the 2021 reference year, additional control totals were included in calibration to adjust for tenure and number of recipients of the Canada Recovery Benefit (CRB) by province.
• An updated generalized calibration system was used.
• The automated treatment of outlier values was modified and expanded to analyze all records and more income variables, and to adjust the weights as needed based on the distribution of the income variables on the APIM. The goal of an influential record treatment is to decrease the impact that an influential record has on the sampling variance of the income estimate without introducing too much bias.

Impacts on the Canadian Income Survey (CIS) estimates

This section will discuss some of the key changes resulting from transitioning to the APIM processing environment. For the tables presented, estimates for four reference years are included to provide context for the transition in 2021. Estimates for the 2018 reference year were produced using the old processing system only. For the 2019 and 2020 reference years, estimates were developed using the new data source, processing system and methodology (experimental), and compared to those which were disseminated using the old (published) system. For 2021, the estimates were produced by the new APIM environment only.

It is important to point out that the updates to the data source and processing system have an impact on the number of recipients and the medians for each income source, while the updates to the weighting methodology also have an impact on the number of people by category (e.g., the number of persons by family type).

The results of this analysis should be treated as experimental, as not all elements of the new processing environment were introduced for the 2019 and 2020 reference years. The results of the analysis were used to fine tune these approaches. Therefore, the results should not be considered as replacements for the official (published) estimates for 2019 or 2020.

Note that although the following tables highlight the key impacts on the CIS estimates, there are many more differences when looking at the data at more disaggregated levels.

Modest changes to median after-tax income, market income and government transfers

The median after-tax income for economic families and persons not in an economic family at both the Canada and provincial levels show relatively small changes between the published estimates and the experimental estimates (Table 1). The differences range from a decrease of 2.0% to an increase of 3.5%, with the only statistically significant difference being for Manitoba in 2020.



Looking at the Canadian median after-tax income for selected family types (Table 2), we see that the main differences were for non-seniors not in an economic family. Differences for other family types were not statistically significant.



Median market income was about 3% higher under the new environment for both 2019 and 2020, driven by higher amounts for non-seniors not in an economic family (Table 3). For this group, the median market income was 10.3% and 11.2% higher in 2019 and 2020, respectively. This is mainly the result of the availability of the T4 and T4A forms in the new processing environment, which produces slightly higher median market income overall, and can be attributed to a higher number of recipients for wages, salaries and commissions (from the T4) and other income (from the T4A).



Couples with children saw an increase in government transfers, while lone-parent families saw a decrease under the new environment (Table 4). These differences were driven by changes in the derivation of child benefits. A subsequent section of this paper will describe the impact of child benefit changes on these family types.

Also worth noting is the increase of median government transfers for persons not in an economic family in 2019, even though the median for both sub-categories (seniors not in an economic family and non-seniors not in an economic family) barely changed. This can be explained by the fact that for persons not in an economic family, the weighting strategy changes caused the numbers of seniors to increase, while the number of non-seniors decreased. Since government transfer amounts are significantly larger for seniors than for non-seniors, an increase in the number of seniors led to increased government transfers for persons not in an economic family.



Updated weighting strategy results in increased number of families with children and unattached seniors

Updates to the weighting strategy resulted in changes to the distribution of family types between the old and new processing environments (Table 5). Worth noting are the significant increases in the number of families with children, specifically lone-parent families.

As we saw in the changes for seniors and non-seniors in Table 4, the increase in median government transfers for persons not in an economic family in 2019 can be explained by the changes in distribution of the two sub-categories for this family type (seniors not in an economic family and non-seniors not in an economic family).



The number of recipients of wages, salaries and commissions were higher for all family types. Specifically, large differences were observed for lone-parent families and seniors not in an economic family. As seen in Table 5, these two groups had the largest differences in the number of families resulting from the weighting methodology updates.

It is also worth noting that in APIM, wages and salaries are derived from both the T1 and T4, while in the old method, only the T1 was used. This could explain the overall increase in recipients of this income source.



“Other income” recipients and amounts increased under APIM

Other income consists of alimony, scholarships and bursaries, income from a Registered Education Savings Plan (RESP), income from a Registered Disability Savings Plan (RDSP) and other market income not included elsewhere. We see differences for other income recipients and medians under the new environment. The addition of the T4A slips allows the APIM to better capture bursaries, RESP and RDSP amounts, so both the number of recipients (Table 7) with other income and the amounts (Table 8) are higher. This is especially true for non-seniors not in an economic family, specifically for the younger age groups.

The only family type that saw significantly less other income recipients was lone-parent families. The main reason is that support payments for 2019 and 2020 only included amounts found on tax files under the new system, while under the old system, support payments collected through the CIS questionnaire were also included. Since support payments are usually underreported on tax files, for the 2021 reference year, support payments collected through the survey were also included, which is more in line with what was done in previous years.



In Table 8, we see that the majority of family types have higher median other income under the new environment, which is due to the previously described addition of information from the T4A slips.



Child benefits change as a result of the new method

Generally, both the number of families receiving child benefits (Table 9) and the median amounts for recipient families (Table 10) are higher under the new environment.

Lone-parent families had a higher number of recipients (12.0%) which is in line with the higher number of lone-parent families (12.0%) previously seen on Table 5.



Child benefit amounts are derived using the previous year’s income in the new environment, which led to higher amounts in 2020 than those derived previously in the old environment (which utilized the current year’s income as a proxy). In Table 10, we see that child benefit amounts were typically higher for families in 2020 under the new environment.

However, the medians were lower (although not statistically significant) for lone-parent families. This is a result of the previously described processing updates, as the administrative tax amount is retained for records where a child benefit amount was reported. So, those receiving partial child benefits would lower the median. Under the old method, these families would have received a higher deterministically calculated child benefit amount.



Fewer social assistance recipients, while amounts higher in the new environment

Social assistance is the income variable that shows the largest differences. The number of recipients is significantly lower under the new environment, which uses the T5007 slip amount. The difference is mainly attributable to the previously described frequent misinterpretation of the CIS question in the old processing environment.

Also, the large difference for seniors not in an economic family is due to provincial senior supplements that were previously included in social assistance payments now being included in the “other government transfers” income source category.



Although the number of social assistance recipients is lower under the new environment, median social assistance amounts are higher (Table 12). This is because social assistance amounts were typically higher than the imputed amounts assigned to individuals who mistakenly responded ‘yes’ to the survey question. In addition, seniors’ supplement amounts tended to be lower than social assistance amounts.



A look at poverty estimates

In this section, we will look at the impact the previously detailed changes have on the poverty estimates.

In general, poverty estimates for Canada and most provinces tend to be slightly lower under the new environment, although none of the differences were statistically significant (Table 13). However, transitioning to the use of the APIM is consistent with better estimation of small earnings amounts, and benefits like social assistance and child benefits, which in turn produces more accurate income estimates for individuals experiencing poverty.



Additionally, the poverty rate changes show relatively small differences by family type (Table 14). A slightly lower poverty rate is observed for non-seniors not in an economic family, which could be partially attributed to higher market income, specifically other income, in the new environment.

Notably, the poverty rate for persons in male-lone parent families is significantly higher in the experimental estimate for 2020. This is the result of various factors, but sampling variability due to the small number of people in that family type is likely a key contributor.



Conclusion

The 2021 reference year Canadian Income Survey estimates were produced under a new processing environment based on the Administrative Personal Income Masterfile (APIM). This paper described the key changes resulting from transitioning to the new processing system.

Overall, estimates for median after-tax income, market income, and government transfers were similar at the Canada, provincial and family type levels when comparing the new and old methods. Statistically significant differences were observed for some family types for a few income components, particularly other income and social assistance.

Likewise, the change had only a slight impact on the poverty rates, with experiments suggesting that poverty rates under the new method are only slightly lower than under the old method. At the Canada and provincial level, none of the poverty rate differences between the old and new methods were statistically significant. At the family type level, significant differences were observed for two family type categories (i.e., male lone-parent families and non-seniors not in an economic family).

In conclusion, the introduction of the new processing environment for CIS 2021 has had minimal impact on key CIS estimates and trends, while making improvements in imputation, weighting, accuracy of income components and coverage of smaller income amounts.