Section 2
Nonresponse

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

Response, nonresponse and vacancy rates
Nonresponse according to urbanization level
Nonresponse according to income strata
Adjustment for nonresponse

Errors due to nonresponse result from the fact that some potential respondents do not provide the necessary information or the provided information proves to be unusable. When the respondent has failed to respond to only some questions, this is referred to as partial nonresponse. In such a case, the missing data are imputed. Errors associated with imputation are described in Section 5, which deals with processing errors. In the present section, nonresponse includes collection nonresponse, which is mainly due to the inability to contact the household or to the refusal of the members of the household to participate partially or completely in the survey, as well as data collected from households that prove to be unusable.

The main impact of nonresponse on data quality is that it can introduce a bias in the estimates if the characteristics of respondents and nonrespondents differ and the difference has an impact on the characteristics studied. Nonresponse rates may easily be calculated, but they have only an indicative value with regard to data quality, since they do not allow estimation of the bias associated with the estimates. The scope of nonresponse may be considered as an indicator of the risks of bias in the estimates.

2.1 Response, nonresponse and vacancy rates

Since the units selected in the Survey of Household Spending (SHS) are dwellings, interviewers must first identify ineligible dwellings, that is, dwellings occupied by persons who are not part of the target population, as well as dwellings that no longer exist (demolished, mobile  home moved or dwelling converted to business) and vacant dwellings (unoccupied, seasonal or under construction).

Among eligible dwellings, the proportion of households that did not respond to the survey is evaluated next. This is called the collection nonresponse rate. Included are households that refused to participate in the survey and households where no contact could be made with the respondents, either because they were absent or because of special circumstances (language problem, illness, death).

Again among eligible dwellings, the rate of unusable data is determined. Unusable data refers to the number of households whose questionnaires were at least partially completed but which were rejected during data processing. There are two main causes for rejection. First, when many questions on income or expenditures have been left unanswered, the questionnaire is classified as incomplete and is not used. The other source of rejection consists of questionnaires in which the difference between receipts (income and other sources of money received by the household) and disbursements (expenditures and net change in assets and liabilities) is greater than 20%. These questionnaires are also excluded from the estimate and are considered as nonresponse.

Note that all rates provided in this section are unweighted. For the 2004 Survey of Household Spending, the final response rate is 69.2%. Table 2.1-1 shows the final response rate as well as the sample size (eligible households) broken down by refusals, no-contacts, unusable data and usable data. This rate is provided at both the national and the provincial level.

Table 2.1-1 Sample size and response rate (%) by province and at the national level

Table 2.1-2 shows the final nonresponse rate; the collection nonresponse rate, broken down by refusals and no-contacts; and the rate of unusable data, broken down into incomplete and out-of-balance questionnaires. The vacancy rate is also included. These rates are provided at the national and provincial level.

Note that the vacancy rates shown in tables of Section 2 include vacant dwellings (unoccupied, seasonal or under construction) as well as dwellings that no longer exist (demolished, mobile home moved or dwelling converted to business).

Table 2.1-2 Nonresponse and vacancy rates (%) by province and at the national level

The final nonresponse rate in Canada is 30.8%. It is due to refusals (18.4%), to households that could not be contacted (8.6%), and finally to households for which the data were unusable (3.8%). For each province, refusals are the main cause of nonresponse, followed by the households that could not be contacted and by the households for which the data were unusable.

The final nonresponse rate varies from one province to another. Saskatchewan has the lowest nonresponse rate at 21.9%. This is also the province with the lowest no-contact rate (5.8%). As may be seen, in Quebec there were no out-of-balance questionnaires. The nonresponse rates in Saskatchewan and Prince Edward Island are less than 25%, while the rates in Nova Scotia and Ontario are greater than 35%. The higher final nonresponse rate in Nova Scotia is partly attributable to a higher rate of out-of-balance questionnaires (6.3%). Ontario has an especially high nonresponse rate at 41.3%. The highest rates of no contact (10.4%) and refusal (25.0%) are also observed in Ontario.

Vacancy rates are shown in Table 2.1-2, but it should be kept in mind that vacant dwellings do not contribute to the bias of the sample if they are correctly identified. By analysing vacancy rates, we can detect dwelling identification problems associated with the collection process. The national vacancy rate for the 2004 SHS is 11.9%.

2.2 Nonresponse according to urbanization level

Nonresponse varies according to urbanization level. The various rates at the national scale are shown by urbanization level in Table 2.2¹.

Table 2.2 Nonresponse and vacancy rates (%) by urbanization level

The final nonresponse rate generally increases with urbanization level. The urbanization category "250,000 to 499,999" has the highest rates of units with no contact (11.4%), refusal (24.8%) and unusable data (7.2%). The final nonresponse rate of 43.3% for this urbanization category is due both to the nonresponse obtained during collection (36.2%), which is mostly attributable to the rate observed in Ontario (42.1%, data not shown), and to the rate of unusable data (7.2%), which is mostly attributable to the rate observed in Nova Scotia (8.5%, data not shown).

The collection nonresponse rate also tends to increase with urbanization level. There is a difference of nearly 8% between the urbanization categories "Less than 30,000" and "1,000,000 or more." Refusals account for more than 50% of total nonresponse at each level of urbanization.

From an examination of the vacancy rate by urbanization level, it emerges that the vacancy rate in rural areas (22.3%) is nearly twice that for low-population urban areas (11.7%). These areas also have a higher vacancy rate than higher-population urban areas. This phenomenon is also observed in the Labour Force Survey (LFS) and is probably attributable to a greater number of seasonal dwellings in rural areas. This explains, among other things, why the vacancy rate is higher in the Atlantic provinces, as illustrated in Table 2.1-2, and especially in Prince Edward Island, since that province has a higher proportion of rural dwellings in the sample.

2.3 Nonresponse according to income strata

Since income information is not available for nonrespondents, it is not possible to compare nonresponse rates according to income. However, the LFS sample design, used for the SHS, was constructed in such a way that in census metropolitan areas, there are strata consisting of geographic areas with a high concentration of high-income households. While the number of high-income strata remains relatively small (51 out of a total of 1,060 strata), the comparison of response rates in this group in relation to the other strata provides relevant information on the potential effect of nonresponse.

The 2004 SHS sample was selected using the new sample design for the LFS. Owing to changes that were made to stratification, especially with respect to income-based strata, and to the fact that those strata were redefined according to data from the 2001 Census, the response rates by income strata in the 2004 SHS are not directly comparable with those of previous surveys.

Under the new design, the number of high-income strata has increased. These strata consist of geographic areas with a high concentration of households with an income exceeding $125,000. They account for approximately 5% of all households in Canada. For the 2004 SHS sample, 9.9% of eligible households came from these strata, because of oversampling to obtain better representation of high-income households in the sample.

Also, under the new LFS sample design, there is no longer an apartment frame, and therefore there is no longer a frame of low-income apartment buildings such as was previously used to form low-income strata. Apartment buildings are now incorporated into the regular LFS design. Thus, there are no longer any separate low-income strata, as was the case for previous years.

Table 2.3 shows the nonresponse and vacancy rates for high-income strata in relation to other strata. Note that in addition to regular strata, the "Other" strata category includes four types of strata that were added to the new LFS sample design. These are: strata with a high vacancy rate, high-cost strata, strata with a concentration of immigrants, and strata with a concentration of Aboriginals. Since the portion of the SHS sample allocated to the latter four strata was smaller, the results for them are not broken out in Table 2.3.

Table 2.3 Comparison of nonresponse and vacancy rates (%) in high-income strata in relation to other strata

In high-income strata, the final nonresponse rate (38.9%) is approximately 30% higher than in the other strata. The refusal rate for high-income strata is 25.3%, which is higher than for the other strata. However, high-income strata and the other strata have similar rates of unusable data.

As may be seen, the vacancy rate is lower for high-income strata than for the other strata. This phenomenon was also observed for previous surveys.

2.4 Adjustment for nonresponse

To compensate for nonresponse, the weights in the SHS are inflated by the inverse of the weighted response rate within certain predefined groups. Following the overhaul of the LFS sample design, the nonresponse adjustment groups were redefined. As was the case for previous years, these groups are defined on the basis of the different urbanization levels in each province and of sub-provincial geographic areas for Quebec, Ontario and British Columbia. Also, specific nonresponse adjustment groups were created for high-income strata. As was seen in the previous section, the number of high-income strata was increased with the introduction of the new LFS sample design. Thus, it is now possible to form nonresponse adjustment groups for high-income strata in all provinces except Prince Edward Island. Such a group cannot be formed in Prince Edward Island because that province has no high-income strata.

The weighted rates differ from the rates presented in this section, since the former take the sampling weight of each household into account. An algebraic description of the adjustment for nonresponse is provided in Appendix A.

The adjustment of weights for nonresponse serves to take account of differences in nonresponse by urbanization level (as illustrated in Section 2.2) and geographic area or by groups of high-income strata. It will serve to reduce the bias insofar as the characteristics of respondents and non-respondents are similar for a given urbanization level and geographic area or for a given group of high-income strata.

It should be noted that a nonresponse adjustment group can be combined with another group if the number of households is too small or the adjustment factor is too high.

---

Note

1. Tables on nonresponse rates by urbanization level and province are available on request from the Household Survey Methods Division.