Historical Data Linkage Quality: The Longitudinal and International Study of Adults, and Tax Records on Labour and Income Historical Data Linkage Quality: The Longitudinal and International Study of Adults, and Tax Records on Labour and Income

by James Hemeon

Start of text box

End of text box

1. Introduction

Statistics Canada has collected administrative data for statistical purposes since its inception as the Dominion Bureau of Statistics in 1918. The linkage of survey data to administrative sources is becoming increasingly common as a means to reduce respondent burden, to replace survey questions with data that could otherwise be subject to respondent recall bias, and to collect data that a respondent may not feel comfortable disclosing during a survey interview. By nature, it can reduce the costs associated with survey collection. Statistical agencies around the world have been using administrative data to replace questionnaires for decades (Economic and Social Council, 2009).

Survey-collected data from the Longitudinal and International Study of Adults (LISA) is linked to tax and other administrative data sources for each year of survey collection. In addition, the LISA performs a historical linkage to tax files preceding the first year of LISA collection, providing a significant amount of longitudinal data with no added response burden, and at no extra collection cost. Though cross-sectional socioeconomic data, which represents a moment in time, can be extremely useful, the availability of a high-quality longitudinal dataset such as the LISA allows for the analysis of trends over the course of people’s lives, which provides additional insight for public policy decision-making.

The purpose of this paper is to explore the data quality of the LISA historical linkage data. Heisz et al. (2013), using data from a pilot study, analyzed tax data linkage rates and data accuracy, and presented the benefits of historical linkage data. The current paper will apply some of the same methods, and expand upon these findings, using the LISA dataset. More specifically, this paper will analyze the linkage rate, the degree to which it decreases as the administrative data years go back in time, and the potential of historical linkage data in analyzing phenomena that require a longitudinal data series.

2. Sample

The LISA is a sample survey, with a stratified multi-stage, multi-phase design. The sample was drawn in 2011 by selecting dwellings from 2011 Canadian Census of Population data, and is therefore a representation of the population at that time. The first LISA interviews took place in late 2011 and early 2012. For simplicity, this first collection wave is referred to as LISA 2012, and thus the LISA 2012 database is used in this study. The sample included dwellings in Canada’s ten provinces, and excluded regular members of the Canadian Forces, individuals living in institutions, and individuals living on reserves and other Aboriginal settlements in the provinces. The data includes information on respondent demographics, family and household composition, literacy numeracy and problem solving skills, education and training, health, income and wealth, and labour market participation (Statistics Canada, 2014).

The database contains data from 23,926 respondents, aged 15 years and older. The file also contains 2,943 non-respondents and 5,264 non-responding children (under the age of 15). Upon completing the LISA interview, respondents were informed of the data replacement plans to link their survey data to administrative sources - a practice referred to as “informed replacement”.

Following collection, Social Insurance Numbers (SIN) were retrieved for respondents from the tax databases of 2010 and 2011, using the respondent’s first name, last name, date of birth, sex, marital status, address and postal code (Social Insurance Numbers are not collected directly from respondents). If no direct match was found, a SIN was linked using probabilistic linkage with the aforementioned auxiliary variables.

Once a SIN was identified, the LISA data was linked with different tax files of individuals: (i) the T1 Family File (T1FF), (ii) the statement and summary of compensation paid by employers (T4 file), and (iii) the Pension Plans in Canada file^{Note 1}. Two different types of linkage were carried out: (i) a yearly linkage (renewable for each new wave of the survey) and (ii) a historical linkage of tax data for all years going back to 1982 (T1 Family File) or 2000 (T4 file, Pension Plans in Canada file).

3. Results

3.1 Linkage rate between 1982 and 2011

Data linkage fails when no linkage key can be established, or the linkage key does not find a match in the administrative data file.

As indicated above, SIN codes were retrieved for individuals using the tax databases of 2010 and 2011. If no SIN was found during this linkage process, this may be attributed to a respondent having not filed a personal tax return during those two years, or it may be due to no suitable link being found in the probabilistic linkage. For 7.5% of LISA respondents, a linkage was attempted but no SIN was found. Of these, 55.1% were 17 years of age or less, and 64.9% were 20 years of age or less. Therefore, the majority of these cases can likely be attributed to respondents being young, and not having established a need to file a tax return.

In historical data linkage, there is the additional problem that a person’s SIN may change over time. If the respondent’s SIN was not consistent over time, the linkage will fail when the SIN can no longer be found. Social Insurance Number (SIN) is a relatively stable linkage key; however, in some cases, it changes over time. For example, immigrants to Canada are assigned a temporary SIN on their arrival in Canada, and are then assigned a permanent SIN. If a SIN cannot be found for this reason, the longitudinal dataset could be missing information, since there may have been an income tax return filed during the earlier years, but it could not be associated with the respondent.

Moreover, in historical linkage there is the additional problem that a person, as we go back to older administrative files, might fall out of the administrative file because they become too young to file, or if the respondents were immigrants, they might have been living in another country, or had not yet established stable filing patterns using a permanent SIN code in Canada. The LISA sample includes respondents as young as 15 years of age (as of 2011), while the historical linkage to T1FF data includes 30 years of tax data, in the 2012 LISA release. Therefore, the availability of tax data precedes the year of birth for some LISA respondents, and precedes the year of immigration to Canada for others. For example, in 1982, 24.4% of LISA respondents were not yet born, 10.0% had not yet immigrated to Canada, and 28.6% were 20 years of age or less or had immigrated to Canada within the past 3 years. By 1997, all LISA respondents had been born, 7.5% had not yet immigrated to Canada, and 30.2% of respondents were 20 years of age or less or had immigrated within the past 3 years (see Figure 3.1-1). Therefore, a linkage is impossible, or unlikely, for a subset of the LISA sample during some years, indicating that we would expect historical linkage rates to decline as we go back in time.

A linkage rate was calculated between survey respondents and the T1FF data files for all years from 1982 to 2011, to determine the extent of its decline going back over time. Three different linkage rates were calculated, including those respondents for whom no SIN was found: (1) a gross rate (using all respondents in the sample), (2) an adjusted rate using a sample excluding respondents under 20 years of age in a given tax year, and (3) a second adjusted rate using a sample excluding respondents under 20 years of age in a given tax year, as well as immigrants landed in the three years preceding a given tax year (see Figure 3.1-2). The adjustment based on age reflects the fact that this group is less likely to produce an income tax return during a given year. The adjustment based on immigrant status reflects the fact that this group is unlikely to have filed Canadian taxes in pre-immigration years, and three years is chosen to give the immigrant respondents time to establish stable filing patterns with a permanent SIN.

Results in Figure 3.1-2 indicate that the linkage rate decreases going back in time, regardless of the sample used for the calculation. However, the most significant reduction in the linkage rate occurs when the calculation is based on a sample without exclusions, in which case it decreases significantly (1.76% per year, on average), from 90.3% in 2011 to only 39.1% in 1982. Excluding those respondents under the age of 20 in a given tax year, the rate decreases much less (0.39%, on average), from 94.8% in 2011 to 83.6% in 1982, remaining over 82% for all years. When also excluding respondents who immigrated to Canada in the 3 years prior to a given tax year, the decrease in linkage rate is small (0.12% per year, on average) from 94.8% in 2011 to 91.3% in 1982, remaining over 90% for all years. In other words, when the sample is limited to the population that is likely to file a tax return and have a constant SIN over time, the linkage rate remains high across all years.

In the LISA sample, 8.5% of respondents were not linked to tax data for any year between 1982 and 2011. This group is comprised of respondents for whom no SIN was found, and those who opted out of linkage^{Note 2}.

3.2 Linkage rate examined

To analyze whether the linkage data is representative of the sample, several linkage rates were calculated for respondents who were over the age of 20 in a given tax year and were non-immigrants or who had immigrated over 3 years prior to the given tax year (see ‘Adjusted rate 2’ in Figure 3.1-2). In addition to the overall ‘Adjusted 2’ linkage rate, linkage rates were calculated for sub-samples by age (in a given tax year), sex, immigrant status, and province of residence (as of 2011) for the tax years 1982, 1996, and 2011. Because of the slight decrease in linkage rate in 1985, as shown in Figure 3.1-2, 1985 was also included (see Figure 3.2-1 – Figure 3.2-4). Note that these rates are based on unweighted frequency counts, as the objective of the present paper is to analyze linkage quality, rather than representativeness to the population. For the total number of observations linked in 1982, 1985, 1996, and 2011, see Table 3.2-5.

The overall linkage rates for 1982, 1985, 1996, and 2011 are 91.3%, 90.3%, 92.8%, and 94.8%, respectively.

Males have a slightly lower linkage rate than females in all years, with 95.7% of females linked in 2011, compared to 93.8% of males.

The results suggest that the linkage rate generally increases with age, similar to findings by Li et al. (2006); however, the rate for the youngest respondents, while lower, is still reasonably high at 91.8%. Respondents aged 51 or above in a given tax year have the highest linkage rate of all age groups in 2011, but this rate is lower in earlier years, as the number of respondents in that age group in a given tax year decreases sharply from 8,879 in 2011 to 544 in 1982. This is to be expected, as the ‘51 or over’ age group in 1982 was aged 81 or over at the time of their LISA interview, and may have been less likely to file a tax return in 2011 or 2010 (and therefore, also less likely to find a linkage SIN).

Immigrants have a slightly higher linkage rate than non-immigrants in 2011. Respondents who resided in the provinces of Ontario and British Columbia in 2011 generally have slightly lower linkage rates, when compared to other provinces. This is particularly apparent in 1985, where the linkage rate for British Columbia was 86.7%, a 3.5% drop from 1982. The reason for this decrease is unclear. Upon analysis of the 312 total respondents who had a linkage in 1982 but a missed linkage in 1985, no trend was found among age, sex, or immigrant status.

3.3 Balanced Panels

A longitudinal dataset requires data on its sample over a period of time. A break in the continuity of data could limit its usability for researchers for some purposes. A longitudinal dataset is said to be a “balanced panel” when all observations (respondents) are present in the dataset in all periods (in the case of LISA, each year). For historical linkage, a balanced panel requires the linkage of a tax record for each year.

If a researcher required balanced panels from those who were likely to file a tax return and likely to have a constant SIN over time (see ‘Adjusted rate 2’ in Figure 3.1-2), a 30-year panel could be constructed with a 74.3% linkage rate, and would contain 6,564 respondents (using years 1982-2011). If a researcher required a 25-year panel, it could be constructed with a 78.1% linkage rate (1987-2011), containing 8,735 respondents. A 20-year panel could be constructed with an 82.1% linkage rate (1992-2011), containing 10,733 respondents. A 15-year panel could be constructed with an 84.5% linkage rate (1997-2011), containing 12,579 respondents. A 10-year panel could be constructed with an 86.7% linkage rate (2002-2011), containing 14,371 respondents. If a researcher required only a 5-year panel, it could be constructed with an 89.7% linkage rate (2007-2011), and would contain 16,568 respondents (see Appendix A). Thus, LISA can be used to create long, balanced panels of a size sufficient for many analyses.

3.4 Comparison of earnings from T1FF and T4 files

One way to verify the reliability of administrative files is to compare their data to the values in administrative files from another source.

Earnings amounts in T1FF data files and in T4 data files were compared for the period from 2000^{Note 3} to 2011. The results show that the majority of cases - 97% each year, on average - present a similar earnings situation in both the T1 Family File (T1FF) and the employer file (T4) (see Table 3.4-1). In other words, only 3% (approximately) of cases show earnings in one file without showing earnings in the other. Approximately 71% of cases show earnings in both the T1FF and T4. Another 26% of respondents had $0 earnings in the T1FF and no T4 information. The number of cases with no T1FF information and a T4 earnings value of $0 is insignificant.

The vast majority of cases show earnings in both the T1FF and T4 files, or $0 earnings in the T1FF filing and no T4 information (indicating agreement between the two files). When earnings values are reported in both the T1FF and T4 files, approximately 95% of cases show a difference of no more than one dollar between data sources (see Table 3.4-2). It should be noted that, while T4 earnings values contain cents values, T1FF earnings do not contain cents. Approximately 98% of cases show a difference of no more than one thousand dollars between data sources.

From 2000 to 2011, the difference in median employment earnings calculated from the two data sources is, on average, $116 (see Table 3.4-3). The median earnings, when present in the T1FF and T4 sources, are very similar, which suggests that the T1FF linkage data is accurate, and that the T4 is also present where expected.

The median earnings, when found in one data source only, are significantly lower than median earnings when found in both data sources. Examining this in more detail shows that the majority of values from a single source are attributed to T1FF values of $0, in which case a T4 may not be available. The majority of single source earnings values greater than $0 are attributed to T4 earnings values with no T1FF information.

3.5 Profiles of earnings by age and sex

To demonstrate the potential of LISA linked data in creating a long data series, an age-earnings profile for each sex was created for different birth cohorts.

Due to the similar earnings found when comparing the T1 Family File (T1FF) and employers’ (T4) files, the earnings used were those from the T1FF, thus providing a longer data series. The sample was divided into seven birth year groups, at five-year intervals, for which the change in employment earnings was tracked by age.

The age-earnings profiles by birth year cohort show a trend of lower earnings at the beginning of career, and a faster growth in earnings, for workers in more recent cohorts (e.g., 1971-1980 cohort versus 1961-1970 cohort). There is also a trend of higher peak earnings for workers in more recent cohorts (e.g., 1951-60 cohort versus 1941-50 cohort). These trends are similar to those found in related literature (Vijay et al., 2014; Beach and Finnie, 2004).

Most notably, females show higher career earnings progression for each successive cohort. Females aged 50-54 in the 1951-1960 cohort earned 47% more than females in the 1931-1940 cohort did when they were aged 50-54, compared to a 14% increase in earnings between the respective cohorts for males aged 50-54. These support previous findings of a greater increase in female earnings, relative to males (Williams, 2010; Suh, 2010; Blau and Kahn, 2006).

4. Conclusions

This study provides a partial assessment of the quality of administrative data from 1982 to 2011 which was linked to 2012 LISA data. In particular, linkage rates were analyzed, the data was compared across administrative sources, and the ability to use the linkage data to analyze selected phenomena requiring a longitudinal data series was assessed.

Linkage rates to administrative data were examined in multiple ways, and the results indicate that the linkage rates are high, with more than 90% of LISA respondents aged 15 and over being linked in 2011. Linkage rates to prior years were also high, especially when rates were calculated for respondents who were aged 20 and over and who had immigrated at least three years prior to the year linked. Among key demographic sub-groups, the linkage rate remains high. However, data users must consider that certain sub-groups may not have as many observations historically. For example, immigrants will only have linkage data on or after the year in which they entered Canada.

The results also suggest that the data obtained by the historical linkage produces data that is coherent across administrative data sources, and can be used for observing phenomena that require a longitudinal data series, as well as long panel datasets.

As the data is based a sample drawn in 2011, it is most appropriately used for studies describing the life-course histories of that particular cohort, as opposed to cross-sectional referencing to individual years. The linkage allows for analysis of retrospective income data that would otherwise have not been possible without 30 years of survey collection, or without introducing a significant recall bias. Furthermore, upcoming data releases will be coupled with additional years of LISA survey data, which will increase the analytical potential of the dataset.

Bibliography

Beach, C. and Finnie, R. (2004), “A Longitudinal Analysis of Earnings Change in Canada”, Analytical Studies Branch. Research Paper, Statistics Canada.
http://www.statcan.gc.ca/pub/11f0019m/11f0019m2004227-eng.pdf

Blau, Francine D.; Kahn, Lawrence M. (2006) “The US gender pay gap in the 1990s: slowing convergence”, IZA Discussion Papers, no. 2176
www.econstor.eu/dspace/bitstream/10419/34046/1/51436131X.pdf

Economic and Social Council. (2009) “Main Results Of The UNECE-UNSD Survey On The 2010 Round of Population and Housing Censuses”, Economic Commission for Europe, Conference of European Statisticians. Twelfth Meeting, 28-30 October 2009.
http://unstats.un.org/unsd/censuskb20/Attachment459.aspx

Gill, Vijay, Knowles, James, Stewart-Patterson, David. (2014). “The Buck Stops Here: Trends in Income Equality Between Generations”, Ottawa: The Conference Board of Canada.

Heisz, Andrew, Langevin, Manon, Randle, Jeffrey. (2013). “Historical data linkage of tax records on labour and income: The case of the Living in Canada Survey pilot”. Statistics Canada Catalogue no. 89-648-X (2).
http://www.statcan.gc.ca/pub/89-648-x/89-648-x2013002-eng.htm

Li, Bing, Quan, Huge, Fond, Andrew, Lu, Mingshan. (2006) “Assessing record linkage between health care and Vital Statistics databases using deterministic methods”, BioMed Central Health Services Research 2006, 6:48.
http://www.biomedcentral.com/1472-6963/6/48/

Sakshug, Joseph W., Couper, Mick P., Ofstedal, Mary B., Weir, David R. (2012) “Linking Survey and Administrative Records: Mechanisms of Consent”, Sociological Methods & Research, 41(4) 535-569.
http://smr.sagepub.com/content/41/4/535.full.pdf

Statistics Canada. (2014). LISA Detailed information for 2014 (Wave 2).
http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5144

Suh, Jingyo. (2010) “Decomposition of the Change in the Gender Wage Gap”, Research in Business and Economics Journal, 2-18.
http://www.aabri.com/manuscripts/08076.pdf

Williams, Cara. (2010) “Women in Canada: A Gender-based Statistical Report. Sixth Edition”. Economic Well-being. Statistics Canada Catalogue no. 89-503-X. p. 32-33.
http://www.statcan.gc.ca/pub/89-503-x/2010001/article/11388-eng.pdf

Appendix A

Notes