Section 2
Nonresponse

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

Response, nonresponse and vacancy rates
Nonresponse according to urbanization level
Nonresponse according to income strata
Adjustment for nonresponse

Errors due to nonresponse result from the fact that some potential respondents do not provide the necessary information or the provided information proves to be unusable. When the respondent has failed to respond to only some questions, this is referred to as partial nonresponse. In such a case, the missing data are imputed. Errors associated with imputation are described in Section 5, which deals with processing errors. In the present section, nonresponse includes collection nonresponse, which is mainly due to the inability to contact the household or to the refusal of the members of the household to participate partially or completely in the survey, as well as data collected from households that prove to be unusable.

The main impact of nonresponse on data quality is that it can introduce a bias in the estimates if the characteristics of respondents and nonrespondents differ and the difference has an impact on the characteristics studied. Nonresponse rates may easily be calculated, but they have only an indicative value with regard to data quality, since they do not allow estimation of the bias associated with the estimates. The scope of nonresponse may be considered as an indicator of the risks of bias in the estimates.

2.1 Response, nonresponse and vacancy rates

Since the units selected in the Survey of Household Spending (SHS) are dwellings, interviewers must first identify ineligible dwellings, that is, dwellings occupied by persons who are not part of the target population, as well as dwellings that no longer exist (demolished, mobile home moved or dwelling converted to business) and vacant dwellings (unoccupied, seasonal or under construction).

Among eligible dwellings, the proportion of households that did not respond to the survey is evaluated next. This is called the collection nonresponse rate. Included are households that refused to participate in the survey and households where no contact could be made with the respondents, either because they were absent or because of special circumstances (language problem, illness, death).

Again among eligible dwellings, the rate of unusable data is determined. Unusable data refers to the number of households whose questionnaires were at least partially completed but which were rejected during data processing. There are two main causes for rejection. First, when many questions on income or expenditures have been left unanswered, the questionnaire is classified as incomplete and is not used. The other source of rejection consists of questionnaires in which the difference between receipts (income and other sources of money received by the household) and disbursements (expenditures and net change in assets and liabilities) is greater than 30%. These questionnaires are also excluded from the estimate and are considered as nonresponse.

Note that all rates provided in this section are unweighted. For the 2007 Survey of Household Spending, the final response rate is 65.1%. Table 2.1-1 shows the final response rate as well as the sample size (eligible households) broken down by refusals, units not contacted, unusable data and usable data. This rate is provided at the national, provincial and territorial levels.

Table 2.1-1 Sample size and response rate (%) by province, territory and at the national level

Table 2.1-2 shows the final nonresponse rate; the collection nonresponse rate, broken down by refusals and units not contacted; and the rate of households with unusable data owing to incomplete questionnaires. The vacancy rate is also included. These rates are provided at the national, provincial and territorial level.

Note that the vacancy rates shown in tables of Section 2 include vacant dwellings (unoccupied, seasonal or under construction) as well as dwellings that no longer exist (demolished, mobile home moved or dwelling converted to business).

Table 2.1-2 Nonresponse and vacancy rates (%) by province, territory and at the national level

The final nonresponse rate in Canada is 34.9%. It is due to refusals (19.2%), to households that could not be contacted (12.1%), and finally to households for which the data were unusable (3.6%). For each province, refusals are the main cause of nonresponse, followed by units not contacted and unusable data. For Yukon, the Northwest Territories and Nunavut, the refusal rate is lower than the rate for units not contacted.

The final nonresponse rate varies from one province or territory to another. The lowest nonresponse rate, 21.4%, is observed in Nunavut, mainly owing to a very low refusal rate. The nonresponse rates are greater than 35% in Nova Scotia, Quebec and Ontario. The latter province has an especially high nonresponse rate at 43.9%. It also has one of the highest rates of units not contacted (15.7%) and the highest refusal rate (24.4%).

Vacancy rates are shown in Table 2.1-2, but it should be kept in mind that vacant dwellings do not contribute to the bias of the sample if they are correctly identified. By analysing vacancy rates, we can detect dwelling identification problems associated with the collection process. The national vacancy rate for the 2007 SHS is 14.1%.

2.2 Nonresponse according to urbanization level

Nonresponse varies according to urbanization level. The various rates at the national scale are shown by urbanization level in Table 2.2¹.

Table 2.2 Nonresponse and vacancy rates (%) by urbanization level

The final nonresponse rate generally increases with urbanization level. The urbanization category "250,000 to 499,999" has the highest rate of units not contacted (16.8%) and refusals (26.3%).

The collection nonresponse rate also tends to increase with urbanization level. There is a difference of approximately 9% between the urbanization categories "6,000 to 30,000" and "1,000,000 or more." Refusals account for more than 60% of total nonresponse at each level of urbanization, except in the "6,000 to 30,000" category, where the proportion is about 50%.

From an examination of the vacancy rate by urbanization level, it emerges that the vacancy rate in rural areas (27.2%) is at least twice that for low-population urban areas (13.7%). These low-population urban areas also have a higher vacancy rate, on average, than higher-population urban areas. This phenomenon is also observed in the Labour Force Survey (LFS) and is probably attributable to a greater number of seasonal dwellings in rural areas. This explains, among other things, why the vacancy rate is higher in the Atlantic provinces, as illustrated in Table 2.1-2, and especially in Prince Edward Island, since that province has a higher proportion of rural dwellings in the sample.

2.3 Nonresponse according to income strata

Since income information is not available for nonrespondents, it is not possible to compare nonresponse rates according to income. However, the LFS sample design, used for the SHS, was constructed in such a way that in census metropolitan areas, there are strata consisting of geographic areas with a high concentration of high-income households. While the number of high-income strata remains relatively small (51 out of a total of 1,060 strata), the comparison of response rates in this group in relation to the other strata provides relevant information on the potential effect of nonresponse (see Table 2.3).

Note that in addition to regular strata, the "Other" strata category includes the following four types of strata: strata with a high vacancy rate, high-cost strata, strata with a concentration of immigrants, and strata with a concentration of Aboriginals. Since the portion of the SHS sample allocated to the latter four strata was smaller, the results for them are not broken out in Table 2.3.

Table 2.3 Comparison of nonresponse and vacancy rates (%) in high-income strata in relation to other strata

In high-income strata, the final nonresponse rate (44.8%) is approximately 32.0% higher than in the other strata. The refusal rate for high-income strata is 27.5%, which is higher than for the other strata. High-income strata also have a higher rate of unusable data than other households, mainly owing to incomplete questionnaires.

As may be seen, the vacancy rate is lower for high-income strata than for the other strata. This phenomenon was also observed for previous surveys.

2.4 Adjustment for nonresponse

To compensate for nonresponse, the weights in the SHS are inflated by the inverse of the weighted response rate within certain predefined groups. Following the overhaul of the LFS sample design, the nonresponse adjustment groups were redefined. As was the case for previous years, these groups are defined on the basis of the different urbanization levels in each province and of sub-provincial geographic areas for Quebec, Ontario and British Columbia. Also, specific nonresponse adjustment groups were created for high-income strata. As was seen in the previous section, the number of high-income strata was increased with the introduction of the new LFS sample design. Thus, it is now possible to form nonresponse adjustment groups for high-income strata in all provinces except Prince Edward Island. Such a group cannot be formed in Prince Edward Island because that province has no high-income strata.

The weighted rates differ from the rates presented in this section, since the former take the sampling weight of each household into account. An algebraic description of the adjustment for nonresponse is provided in Appendix A.

The adjustment of weights for nonresponse serves to take account of differences in nonresponse by urbanization level (as illustrated in Section 2.2) and geographic area or by groups of high-income strata. It will serve to reduce the bias insofar as the characteristics of respondents and non-respondents are similar for a given urbanization level and geographic area or for a given group of high-income strata.

It should be noted that a nonresponse adjustment group can be combined with another group if the number of households in the group is too small or the adjustment factor is too high.

---

Note

1. Tables on nonresponse rates by urbanization level and province are available on request from the Household Survey Methods Division.