Data quality, concepts and methodology: Concepts, methodology and data quality

View the most recent version.

This publication presents the results of four monthly surveys: Steel Primary Forms, Steel Castings and Pig Iron; Ingots and Rolled Steel Products; Steel Pipe and Tubing; and Steel Wire and Specified Wire Products. These surveys measure, on a monthly basis, the quantities of selected steel products that are produced and shipped by Canadian manufacturers. The target population for these surveys includes manufacturers in Canada of primary steel, steel products, pig iron, steel pipe and tubing as well as, steel wire as defined in the Standard Classification of Goods (SCG) that report these products to the Annual Survey of Manufactures and Logging or ASML (Survey ID 2103). This means that estimates from the monthly surveys do not cover the entire universe of steel producers in Canada because the ASML does not survey all businesses. Instead, the ASML uses administrative data to cover the small and medium-sized establishments. These manufacturers are not part of the four monthly surveys.

General methodology

Data are collected each month, directly from survey respondents using a mail-out / mail-back process. Data capture and preliminary editing are performed simultaneously to ensure validity of the data. Businesses from whom no response has been received or whose data may contain errors are followed-up by telephone or fax.

Missing data for the current month are imputed automatically by applying to the previous month’s value, the month-to-month change observed for the same period in the previous year, for the unit in question. However, an option exists for analysts to manually override this imputation with a better estimate based on pertinent knowledge about the industry or the business.

Various confidentiality rules are applied to all data before they are released to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent the direct or residual disclosure of identifiable data.

Direct disclosure could occur when the value in a tabulation cell is composed of a few respondents or when the cell is dominated by a few companies. Residual disclosure could occur when confidential information can be derived indirectly by piecing together information from different sources or data series.

Under normal circumstances, data are collected, captured, edited, tabulated and published within 6-8 weeks after the reference month.

Revisions

Data may be revised to include amended information or reports from respondents that are received after the end of a collection cycle. Revisions are disseminated in subsequent periods and reflected in the CANSIM series and in the tables of this publication.

Data accuracy

The methodology of these surveys has been designed to promote data accuracy. Since data are collected from all Canadian producers of primary steel, steel products, pig iron, steel pipe and tubing and steel wire within the target population, the resulting estimates are not subject to sampling error. However, the results are still subject to the non-sampling errors associated with coverage, non-response, inaccurate reporting, and processing. Errors relating to coverage and non-response can be measured and are summarized in text table 1. All attempts are made to control inaccurate reporting and processing errors.

Moreover, survey results are analyzed to ensure comparability with patterns observed in historical data series and the economic condition of the industry. Information available from other sources such as the Steel Primary Forms Weekly Survey (Survey ID 2131) and the Monthly Survey of Manufacturing or MSM (Survey ID 2101) are also used in the validation process.

Coverage error

There is a degree of under coverage (referred to as coverage error) in the survey results as there is generally a lag between the time a new business comes into existence and when it is included in the universe of these sub-annual surveys. This occurs because the list of businesses surveyed is derived from the latest available survey results for the ASML which are not available until 15 months after the reference period.

This error is kept at a minimum by also using advance information from the ASML frame, feedback from the MSM and other sources such as the Canadian Steel Producer’s Association, trade journals and newspaper articles, to identify new survey units.

Non-response error

Some respondents may be unable to provide data for numerous reasons (i.e. fire, theft, strike, economic hardship, etc.), while others may be late in responding. To minimize non-response, delinquent respondents are followed up rigorously by phone or fax. Data for non-responding units are imputed using industry trend and other related information. Data are revised at a later date, if completed questionnaires are received after the end of a collection cycle.

Inaccurate response

Inaccuracy may result from poor questionnaire design or an inability on the part of respondents to provide the requested information or from misinterpretation of the survey questions. To reduce such errors the format and wording in the questionnaire are reviewed from time to time and modified based on feedback from survey respondents and data users. Respondents are also reminded of the importance of their contribution and of the need for accurate data reporting.

Processing errors

These errors may occur at various stages in the processing of survey data such as data entry, verification, editing and tabulation. Data are examined for such errors using automated edits along with an analytical review by subject matter experts. Several checks are performed on the collected data to verify internal consistency and comparability over time.

Next