Data quality, concepts and methodology: Data accuracy

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

Many factors affect the accuracy of data produced in a survey. For example, respondents may have made errors in interpreting questions, answers may have been incorrectly entered on the questionnaires, and errors may have been introduced during the data capture or tabulation process. Every effort was made to reduce the occurrence of such errors in the survey. These efforts included: a complete verification of keyed data, validity and consistency edits, extensive follow-up with the large businesses, and consultation with selected government departments and industry associations.

In general, errors such as incomplete coverage of the universe, incorrect classification of business or government activity and inconsistencies in working definitions can be reduced if the survey is repeated at regular intervals and with sufficient frequency. In this way, the mailing list may be well maintained and the respondents will be familiar with the definitions used and the type of information required.

Incomplete coverage of the industry universe occurs when a firm in the industry is overlooked. If the reason for not including the firm is that it has been incorrectly included in another industry, this is termed a classification error. Such errors have an impact upon estimates. However, these errors are less frequent now than in the past with the adoption of the NAICS classification system (See "Data quality, concepts and methodology — Overall approach: data sources and methodology section") and see text box ("The classification of waste management services").

Assessing data accuracy

One way to assess data accuracy is to compare it to the trends of other data collected. For example, comparing the waste statistics for 2008 with those for 2006, it is apparent that there has been substantial revenue growth in the Canadian waste management industry. On a per capita basis, more non-hazardous waste was diverted during 2008 than in 2006, but approximately the same amount of non-hazardous waste was disposed in 2008 and 2006. The increase seen in the diverted waste quantity estimates are reflected in the financial and employment estimates of the business and government sectors of the industry. Furthermore, business financial data from 2008 were compared to administrative data from Statistics Canada's Business Register. Recycling estimates were also compared and validated with those of the provincial governments of Nova Scotia and Ontario.

Response rates

The overall response rate for the 2008 waste management industry surveys, based on the ratio of the number of completed and partially completed questionnaires to the total number of in-scope questionnaires, was 79% for the business sector and 93% for the government sector.

Imputation rates

Although most businesses and local governments were very co-operative in answering the survey, some could not provide all the data required in the format in which it was requested. For example, facilities operating without a weigh scale had difficulties answering questions about the weights of material collected or disposed of. In cases where values were missing from survey cells or where the respondent did not complete a questionnaire even after extensive follow-up, information was imputed.

Data reliability

Imputation rates are an indicator of data reliability. Imputation is a term that refers to the proportion of data that were not obtained directly through a survey but rather came from an administrative source or was estimated using defensible and replicable methodologies.

Imputation is necessary to "complete" the data picture when there are non or missing responses to certain questions or sets of questions.

Business sector

Employment and financial data for small firms that were not surveyed, as well as in-scope firms that did not respond, were imputed. Administrative sources such as the Statistics Canada Business Register and tax records were used to fill in the missing values.

For large firms, the imputed values were compared with values from previous years and other sources, such as annual reports and security exchange filings to ensure that the quality of the imputed values was high.

The overall imputation rate for the business financial variables was 21%.

Government sector

Historical data was used to fill in missing financial and employment values for the government sector survey. However due to the high response rate (93%) for this survey, very few values were in need of imputation.

Waste disposal and recycling

Imputation for missing values in the disposal and recycling sections involved a different set of processes. As these two sections on both the business sector survey and the government sector survey were identical, the results from the two surveys were easily combined. This made it possible to remove duplicate data and to obtain a completed response from partial responses. For example, in the case where a local government owns a landfill but contracts out its operation, both the government body and the contracted business reported for the landfill, the duplicated data were removed so that the landfill was accounted for only once. Also, each of the two respondents may not have been able to report for all aspects of the facility, but by combining responses a completed record could be obtained. To illustrate, a firm may have omitted the total quantity of waste disposed to the landfill but the municipality may have reported that value.

In cases where there were missing cell values in the completed survey forms, many of these values were obtained through an intensive period of follow-up through email or telephone calls. Any remaining values were obtained from provincial and local government contacts, industry experts and publicly available sources, such as the Internet.

The tables presented in this report cover the data that were determined to be of sufficient quality for publication at a disaggregated level. Data confidentiality considerations as well as imputation rates play a role in this assessment. Data must be released at a level where the disclosure of the identity of any respondent in any cell is not possible. In addition, the levels of imputation must remain within reasonable limits.

Data limitations

Every effort has been made to ensure that the estimates presented in this report are of both high quality and reliability. However, it is important to understand the limitations of the data presented. This knowledge will allow readers to make informed decisions before conducting further research or analysis using these estimates.

Coverage

As discussed in Section "Data quality, concepts and methodology — Overall approach: data sources and methodology– Evaluation of frame coverage", the estimates presented in this report refer only to that material entering the waste stream and do not cover any waste that may be managed on-site by a company or household. While the majority of residential waste is handled by municipalities or private businesses, and thus included in the survey coverage, it is believed that a significant quantity of non-residential waste is managed on-site by industrial generators. Also, much is transported by the generator directly to secondary processors such as pulp and paper mills, thereby bypassing entirely any firm or local government involved in waste management activities. Anecdotal evidence suggests that these practices are becoming increasingly common.

Agricultural waste is not covered by these surveys. This waste is typically managed on-site or by specialized firms that are not classified by NAICS as part of the waste management industry.

In addition, these data do not include materials that were processed for reuse and resale, (for example, wholesale of scrap metals or used clothing), nor those materials that are collected through deposit-return systems and therefore not processed at a material recovery facility.

Classification and measurement of waste flows

Improvements are constantly being sought with a view to standardize definitions of waste concepts and methods to calculate waste flows in Canada. While with each survey cycle improvements are made, some inconsistencies remain. For example, some jurisdictions consider the reuse of asphalt as recycling while other jurisdictions do not. Some include landfill cover materials in their quantity calculations and some do not.

In addition, various methods of waste measurement exist. Some facilities measure waste quantities by weight while other use volume and still others have no method of measurement. As reporting standards are agreed upon, Statistics Canada's waste management surveys will be revised appropriately.

Next | Previous