Statistics Canada - Government of Canada
Accessibility: General informationSkip all menus and go to content.Home - Statistics Canada logo Skip main menu and go to secondary menu. Français 1 of 5 Contact Us 2 of 5 Help 3 of 5 Search the website 4 of 5 Canada Site 5 of 5
Skip secondary menu and go to the module menu. The Daily 1 of 7
Census 2 of 7
Canadian Statistics 3 of 7 Community Profiles 4 of 7 Our Products and Services 5 of 7 Home 6 of 7
Other Links 7 of 7

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Skip module menu. Go to content.
Survey content Other data products About Income Trends Online catalogue About Income Trends Notes and definitions Methodology Data tables Data services More information Income Trends in Canada

Methodology

Survey universe
The sample
Data quality
Sampling errors
Standard error and coefficient of variation
Suppression rules
Quality indicators
Non sampling errors
Weighting
Cross-sectional representation in SLID
Response rates
Imputation for non-response

Survey universe

SLID is a household survey that covers all individuals in Canada, excluding residents of the Yukon, the Northwest Territories and Nunavut, residents of institutions and persons living on Indian reserves. Overall, these exclusions amount to less than three percent of the population.

The sample

The samples for SLID are selected from the monthly Labour Force Survey (LFS) and thus share the latter's sample design. The LFS sample is drawn from an area frame and is based on a stratified, multi-stage design that uses probability sampling. The total sample is composed of six independent samples, called rotation groups, because each month one sixth of the sample (or one rotation group) is replaced.

The SLID sample is composed of two panels. Each panel consists of two LFS rotation groups and includes roughly 15,000 households. A panel is surveyed for a period of six consecutive years. A new panel is introduced every three years, so two panels always overlap.

Data quality

There are two types of errors inherent in sample survey data, namely, sampling errors and non-sampling errors. The reliability of survey estimates depends on the combined impact of sampling and non-sampling errors.

Top of page



Sampling errors

Sampling errors occur because inferences about the entire population are based on information obtained from only a sample of the population. The results are usually different from those that would be obtained if information were collected from the whole population. Errors due to the extension of conclusions based on the sample to the entire population are known as sampling errors. The sample design, the variability of the population characteristics measured by the survey, and the sample size determine the magnitude of the sampling error. In addition, for a given sample design, different methods of estimation will result in sampling errors of different sizes.

Standard error and coefficient of variation

A common measure of sampling error is the standard error (SE). The standard error measures the degree of variation introduced in estimates by selecting one particular sample rather than another of the same size and design. The standard error may also be used to calculate confidence intervals associated with an estimate (Y). Confidence intervals are used to express the precision of the estimate. It has been demonstrated mathematically that, if the sampling were repeated many times, the true population value would lie within the confidence interval Y ± 2SE 95 times out of 100 and within the narrower confidence interval defined by Y ± SE, 68 times out of 100. Another important measure of sampling error is given by the coefficient of variation, which is computed as the estimated standard error as a percentage of the estimate Y (i.e., 100 × SE / Y).

To illustrate the relationship between the standard error, the confidence intervals and the coefficient of variation, let us take the following example. Suppose that the estimated average income from a given source is $10,000, and that its corresponding standard error is $200. The coefficient of variation is therefore equal to 2%. The 95% confidence interval estimated from this sample ranges from $9,600 to $10,400, i.e. $10,000 ± $400. Thus it is assumed with a 95% degree of confidence that the average income of the target population is between $9,600 and $10,400.

The bootstrap approach is used for the calculation of the standard errors of the estimates. For more information on standard errors and coefficients of variation, refer to the Statistics Canada publication, Methodology of the Canadian Labour Force Survey.

Top of page



Suppression rules

Suppression rules, or data reliability cutoffs, are currently established based on the sample size that underlies the estimate. In general, a sample size of 25 observations is required for the estimate to be published. Depending on the type of estimate, this rule can vary slightly. These rules help protect the confidentiality of survey respondents and ensure the reliability of estimates.

Table D. Suppression rules

ESTIMATE SUPPRESS IF:
Percentage, Distribution, Proportion/Shares:
  • % under the low-income cutoff (LICO)
  • Income distribution
  • Proportion of families with income=0
Denominator* sample size < 25
or
Denominator* sample size < 100 and numerator sample size < 5
Ratios:
  • female/male earnings
Numerator sample size < 25
or
Denominator sample size < 25
Quintiles (shares, means and upper income limits)
  • shares of income by quintile
  • average income by quintile
  • upper income limits
sample size in quintile/5 < 25
or
upper income limit for upper income quintile or total of quintiles
Other estimates
  • Counts
  • Mean
  • Medians
  • Gini coefficients
sample size < 25

*The denominator sample size refers to the sample size of the total estimate from which the distribution, percentage, proportion or share is derived.

Quality Indicators

Quality indicators (QI) are available for 2002 and are based on the coefficient of variation (CV) and suppression rules. The following indicators are used:

. not available for any reference period
.. not available for a specific reference period
not applicable
s value rounded to 0 (zero)
p preliminary
r revised
x suppressed to meet confidentiality requirements
A excellent (CV between 0 and 2%)
B very good (CV between 2% and 4%)
C good (CV between 4% and 8%)
D acceptable (CV between 8% and 16%)
E use with caution (CV greater than or equal to 16%)
F too unreliable to be published (suppressed)

Non-sampling errors

Non-sampling errors generally result from human errors such as inattention, misunderstanding or misinterpretation. The impact of randomly occurring errors over a large number of observations will be minimal. Errors occurring systematically can, on the other hand, have a major impact on the reliability of estimates. Considerable time and effort is invested into reducing non-sampling errors in SLID.

Non-sampling errors may arise from a variety of sources such as coverage, response, non-response and processing errors.

Coverage error arises when sampling frame units do not exactly represent the target population. Units may have been omitted from the sampling frame (under-coverage), or units not in the target population may have been included (over-coverage), or units may have been included more than once (duplicates). Under-coverage represents the most common coverage problem.

Slippage is a measure of survey coverage error. It is defined as the percentage difference between control totals (Census population projections) and weighted sample counts. Slippage rates for household surveys are generally positive because some people that should be enumerated are missed. Slippage rates have been revised back to 1996 using the 1996 Census population projections. According to the numbers in the table below, in 2002, SLID covered 84% of its target population. SLID estimation procedures use Census population projections to compensate for determined slippage.

Rates are also available upon request for sex, province and age groupings.

Table E. Slippage rates in SLID

  1996 1997 1998 1999 2000 2001 2002

Canada (%)

10.28

11.12

11.85

12.02

12.64

13.40

16.02

Response errors may be due to many factors, such as faulty questionnaire design, interviewers' or respondents' misinterpretation of questions, or respondents' faulty reporting. Great effort is invested in SLID to reduce the occurrence of response error. Measures undertaken to minimize response errors include the use of highly-skilled and well-trained interviewers, and supervision of interviewers to detect misinterpretation of instructions or problems with the questionnaire design. Response error can also be brought about by respondents who, willingly or not, provide inaccurate responses.

Income data are especially prone to misreporting, as income is a sensitive issue and includes many items with which respondents are not always familiar. To obtain more accurate information, income data for SLID are collected after the income tax “season” when respondents are more familiar with their tax records. Respondents receive information about the income interview prior to the interviewer's telephone call. This gives them time to consult documents and have information available at the time of the interview. For respondents who grant Statistics Canada permission to access their tax files (the majority of respondents), SLID collects income data directly from administrative files. This procedure reduces misreporting of income in the SLID.

Non-response errors occur to some extent in any survey for reasons such as household members being on vacation during the interview period or refusing to supply requested information, despite attempts to obtain complete response from sampled units. For these individuals, the missing data are imputed either explicitly by assigning data to each non-respondent on the basis of a similar respondent record, or implicitly by redistributing the weight of the non-respondent individual to other responding individuals. The bias introduced by non-response increases with the differences between respondent and non-respondent characteristics. Methods employed to compensate for non-response make use of information available for both respondents and non-respondents in an attempt to minimize this bias.

Processing errors can occur at various stages in the survey: data capture, editing, coding, weighting or tabulation. The computer-assisted collection method used for SLID reduces the chance of introducing capture errors because checks for consistency and completeness of the data are built into the computer application. To minimize coding, weighting or tabulation errors, diagnostic tests are carried out periodically. These tests include comparisons of results with other data sources.

Top of page



Weighting

The estimation of population characteristics from a survey is based on the premise that each sampled unit represents, in addition to itself, a certain number of unsampled units in the population. A basic survey weight is attached to each record to indicate the number of units in the population that are represented by that unit in the sample.

Two types of adjustment are then applied to the basic survey weights in order to improve the reliability of the estimates. The basic weights are first inflated to compensate for non-response. The non-response adjusted weights are then further adjusted to ensure that estimates on relevant population characteristics would respect population totals from sources other than the survey.

The population totals used for SLID are based on Statistics Canada's Demography Division population counts for different province-age-sex groups as well as counts by household and family size. In SLID, different weights apply for cross-sectional and longitudinal estimates.

Cross-sectional representation in SLID

Each longitudinal sample, or “panel” in SLID initially constitutes a representative cross-sectional sample of the population. However, because the real population changes each year, whereas by design the longitudinal sample does not, the sample must be modified to properly reflect these changes to the composition of the population. This is done by adding to the sample all new people in the population who are found to be living with the initial respondents (and likewise dropping them from the sample if they leave at later time-points).

Any original respondents who leave the target population (by moving abroad, into institutions, etc.) are given a zero weight for cross-sectional purposes. In this way, the cross-sectional sample, composed of the original respondents minus those who left the target population plus those who have entered it, is virtually fully representative of the population at each subsequent time-point. The missing group is composed of persons who have newly entered the target population and are not living with anyone who was in the target population when the most recent panel was selected. Since SLID introduces a new panel every three years, however, this group is quite small.

Response rates

High response rates are essential for the data quality of any survey and thus considerable effort is invested to encourage effective participation from SLID respondents.

Response is calculated at the household level. A household is considered to be “respondent” if at least one of its members responds to either the January or the May interview. There is the additional stipulation that the information on the household’s composition cannot be missing for more than one year.

Within a respondent household, all members are assigned identical, positive final weights, and those members (if any) who did not respond to one or both of the collection phases will have final data that is either shown as “missing” on the final database or imputed, depending on the variable.

Cross-sectional response rates, given in Table F, range between 79.1% (2002) and 85.5% (1996).

Table F. Response rates in SLID (1996-2002)

Year 1996 1997 1998 1999 2000 2001 2002

Response rate (%)

85.5

83.6

82.3

82.8

80.8

79.1

79.1

Imputation for non-response

Income data are imputed using previous years' data updated for any changes in circumstances. In the absence of previous years' data, data is imputed using the “nearest neighbour” technique, in which a respondent with certain similar characteristics becomes the “donor” for the imputed value.

Amounts received through certain government programs, such as child tax benefits, the Goods and Services Harmonized Sales Tax Credit, and the Guaranteed Income Supplement, are also derived from other information. Data obtained from the tax route are complete and do not need imputation.


Home | Search | Contact Us | Français Top of page
Date modified: 2008-10-17 Important Notices