Publications
Household Expenditures Research Paper Series
Survey of Household Spending 2008: Data Quality Indicators
Section 3
Coverage errors
Archived Content
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.
Undercoverage and overcoverage: slippage rates
Adjustment at the population and household levels
The target population was defined in the design of the survey. It is useful to go over this definition, since a good understanding of the target population is necessary in order to properly interpret the survey data. It is important to note that the Survey of Household Spending (SHS) uses the sampling frame of the Labour Force Survey (LFS).
Target population
The target population consists of individuals living in private households. It therefore excludes residents of institutions such as prisons, chronic care hospitals or senior citizens' homes, as well as members of religious orders and other groups living communally, members of the Armed Forces living in military compounds, and individuals residing permanently in hotels or rooming houses. Also excluded are foreign countries' official representatives residing in Canada and their families as well as individuals residing on Indian reserves or public lands (except for the territories). With these exclusions, the survey covers nearly 98% of the population in the ten provinces. The territories are excluded from the target population for the 2008 SHS, as they are covered only once every two years.
We did not collect data from persons temporarily living away from their families (for example, students at university) because the information would be obtained from their families if selected in the sample.
Coverage errors result from inadequate representation of the target population based on the units in the sampling frame. Some units of the target population may be omitted from the sampling frame, in which case there is undercoverage. Other units that are not in the target population may be included by error, or some units may be included more than once. These units are responsible for overcoverage.
3.1 Undercoverage and overcoverage: slippage rates
In the SHS, the sample is selected using a list of dwellings in each selected cluster. Factors contributing to undercoverage are: the omission of dwellings in the creation of the list, new dwellings that are added between the creation of the list and the interviewer's visit (mainly in developing areas), and the erroneous classification of vacant dwellings. The inclusion of dwellings that are not within the boundaries of the cluster is a source of overcoverage. Similarly, errors can occur during data collection, due to improper identification of persons as members of the selected household. These errors also contribute to undercoverage or overcoverage.
Also, as described in Section 2.4, reweighting methods are implemented to take account of nonresponse. However, when these adjustments are made, it is impossible to correct the survey weights to ensure that all subgroups within the population are well represented.
A good representation of the target population is essential to the production of realistic expenditure estimates. The sample must adequately represent the individuals in the target population as well as the distribution of households according to size.
There is generally a net undercoverage of the number of persons and the number of households in the SHS. This undercoverage is corrected by an adjustment of weights using auxiliary or reference data based on postcensal demographic estimates. The slippage rate (see Appendix A) is a measure of the percentage of difference between the estimates from these auxiliary data and the survey estimates calculated using weights not adjusted with these data.1 Slippage therefore represents the combined effect of undercoverage and imbalance in certain population subgroups created by survey nonresponse that could not be corrected at the reweighting stage.
Slippage rates by age group for the national and provincial levels are shown in Table 3.1, while slippage rates by household size, used in adjusting weights, are shown in Table 3.2. A positive rate indicates overcoverage of the number of persons or households in the survey.
Table 3.1 Slippage rates for provinces by age group
Table 3.2 Slippage rates for provinces by household size
3.2 Adjustment at the population and household levels
To correct the problem of the sample's representativeness, shown in Table 3.1, and to reduce the resulting bias, the survey data are adjusted during weighting using demographic estimates for the age groups defined in this table, for each province. For more details on the adjustment methodology, see references [1] and [5]. This adjustment reduces the bias but does not eliminate it entirely if the characteristics of the individuals omitted from the survey differ from those of individuals included for a given age group in a province.
It should also be noted that the effectiveness of the adjustment based on demographic estimates depends largely on the quality of those estimates and their accuracy in representing the target population of the survey. Demographic estimates are not error-free. They are postcensal estimates based on the population counts from the 2001 Census adjusted for net undercoverage, and they take into account recent statistics on migration, births, deaths, etc. These demographic estimates are adjusted to account for certain exclusions specific to household surveys, such as persons living in institutions. Conceptually, they differ slightly from the SHS target population in that they include persons living in non-institutional collective dwellings, such as members of groups living communally and individuals permanently residing in hotels or rooming houses. However, this difference is considered negligible, since such individuals represent less than 0.4% of the Canadian population.
To remedy the problem of the sample's representativeness in terms of the number of households by size as illustrated in Table 3.2, the survey data are adjusted using auxiliary data. By adjusting the SHS weights to reflect postcensal estimates of the number of households by size, the goal is to compensate for the bias resulting from inadequate representation of households. However, the bias will not necessarily be eliminated if characteristics of households not interviewed (i.e., missed or non-responding households) differ from those of responding households for a given household size. As in the case of demographic estimates of population, the effectiveness of the adjustment will depend on the quality of the auxiliary data on the number of households.
In addition to demographic estimates for age groups by province, two other auxiliary data sets are used during weighting to adjust the survey data and thereby improve their representativeness. The first data set is used to control for the number of children and adults in certain major cities. For the second set, counts for major categories of income from wages and salaries are used when adjusting weights to ensure a degree of consistency in income distributions between the SHS and outside sources.
Note
- The subweight, which is the survey weight adjusted for nonresponse, is used (see Appendix A).
- Date modified: