Data quality, concepts and methodology

Concepts, methodology and data quality

This publication presents the results of four monthly surveys: Steel, Primary Forms, Steel Castings and Pig Iron; Ingots and Rolled Steel products; Steel Pipe and Tubing; and Steel Wire and Specified Wire Products. These surveys measure, on a monthly basis, the quantities of selected steel products that are produced and shipped by Canadian manufacturers. The target population for these surveys includes manufacturers in Canada of primary steel, steel products, pig iron, steel pipe and tubing as well as, steel wire as defined in the Standard Classification of Goods (SCG) that report these products to the Annual Survey of Manufactures or ASM (Survey ID 2103). This means that estimates from the monthly surveys do not cover the entire universe of steel production in Canada because the ASM does not survey all manufacturers. Instead, the ASM uses administrative data to cover the small and medium-sized manufacturers. These manufacturers are not part of the four monthly surveys.

General methodology

Data are collected monthly using a mail-out / mail-back process. Respondents are mailed a questionnaire but given the option of electronic data reporting (EDR) in a common format. Received data are subject to manual and automated editing to detect errors and inconsistencies as well as to follow-up with respondents by phone or fax if necessary. Follow-up is also carried out for missing reports after a pre-specified period of time. Data are imputed when they are missing. Various confidentiality rules are applied to all data before they are released to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Under normal circumstances, data are collected, captured, edited, tabulated and published within 6-8 weeks after the reference month.

Revisions

Data may be revised to include amended information or reports from respondents that are received after the end of a collection cycle. Revisions are disseminated in subsequent periods and reflected in the CANSIM series and in the year-to-date totals in this publication.

Data accuracy

The methodology for these surveys has been designed to promote data accuracy. Since data are collected from all Canadian producers of primary steel, steel product, pig iron, steel pipe and tubing, and steel wire in the target population (see description above), the resulting estimates are not subject to sampling error. However, the results are still subject to the non-sampling errors associated with coverage, non-response, inaccurate reporting, and processing. Errors relating to coverage and non-response can be measured and are summarized in text table 1. All attempts are made to control inaccurate reporting and processing errors.

Moreover, the data are analyzed for consistency and compared with general economic conditions and industry trends using surveys such as the Steel Primary Forms Weekly survey (Survey ID 2131) and the Monthly Survey of Manufacturing (Survey ID 2101).

Coverage errors

The list of businesses included in the four steel surveys is derived from the Annual Survey of Manufactures (ASM). This leads to some under-coverage in the sub-annual surveys since it takes 15 months to compile and publish ASM survey results. This under-coverage is however, kept at a minimum by using advance information from the ASM frame, feedback from the Monthly Survey of Manufactures (MSM) and other sources such as the Steel producers association, trade journals and newspaper articles.

Non-response

Some respondents may be unable to provide data for numerous reasons (i.e. fire, theft, strike, economic hardship, etc.), while others may be late in responding. To minimize non-response, delinquent respondents are followed up rigorously by phone or FAX. Data for non-responding units are imputed using industry trend and other related information. Data are revised at a later date, if completed questionnaires are received after the end of a collection cycle.

Inaccurate response

This error may result from poor questionnaire design or an inability on the part of respondents to provide the requested information or from misinterpretation of the survey questions because of definitional difficulties. To reduce such errors the format and wording in the questionnaire is reviewed from time to time and modified based on feedback from survey respondents and data users. Respondents are also reminded of the importance of their contribution and of the accuracy of reported information.

Processing errors

These errors may occur at various stages in the processing of survey data such as data entry, verification, editing and tabulation. Data are examined for such errors using automated edits along with an analytical review by subject matter experts. Several checks are performed on the collected data to verify internal consistency and comparability over time. (see text table 1 )