Statistics Canada Quality Guidelines
Sixth Edition – December 2019

Preface

Led by Statistics Canada, Canadians and their government have a powerful national statistical system (NSS) they can count on to access quality statistical information. The credibility of the system and the confidence in the information it produces can only be established if that information meets the country’s needs, represents the environment in which we live and which it attempts to describe, all while making sure to protect citizens’ privacy. In other words, the information must be relevant, of high quality and obtained ethically.

Quality is therefore a cultural element that is fundamental in fulfilling the mandate of Canada’s statistical system, which is to produce information. It also has a characteristic that is important to understand: it automatically deteriorates in the absence of proactive measures. This is why the methods and tools used to maintain the production of quality data must evolve as the environment around us evolves.

Such evolution has resulted in rapid changes to data acquisition methods, as well as growing awareness of the privacy concerns that this new environment brings. The Canadian statistical system is under increasing pressure to justify these activities in the context of ethics and privacy. Therefore, it is necessary that the NSS adhere to a transparent and coherent approach to addressing such issues as they pertain to the statistical business process.

In light of these realities it must face, the Canadian statistical system, under Statistics Canada’s leadership, has long provided guidance on the principles to apply in the various statistical business processes. To this end, the experiences and conclusions which constitute “best practices” have been compiled into a set of quality guidelines. The first edition of these guidelines was published in 1985; it was subsequently revised four times, in 1987, 1998, 2003 and 2009. This edition aligns with the need to update the Quality Guidelines and has three objectives: a) to provide all other data producers in the Canadian statistical system with a reference document; b) to adapt to the new reality of administrative data by covering the main statistical business processes; and c) to facilitate compliance with current quality assurance methods.

It is important to understand that the guidelines presented in this document do not all apply systematically to each data production process. Therefore, their relevance and importance must be carefully considered based on the particular requirements and constraints of each program and each of their phases.

The guidelines in this document are not intended to replace the expertise and judgment of the staff responsible for producing data. Moreover, all activities of every production process must show a concern for quality. All staff involved in statistical activities are responsible for ensuring that quality has high priority when designing and implementing statistical methods and procedures.

I would like to thank and congratulate the many experts who contributed to the preparation of this Sixth Edition of the Quality Guidelines, as well as the Quality Secretariat, which led and coordinated this effort.

Anil Arora

Chief Statistician

Abbreviations

Important definitions

Executive summary

The evolution in the world of statistics and data in recent years has created a need for transformation in both the way data are acquired and processed in the production of official statistics. To maintain a high quality level in statistical products and processes, it has become essential to look into existing methods and processes to evaluate their relevance, and modify them to preserve it. In fact, this new and fast-paced data revolution has required the development of new techniques and methods in order to sustain the ability to make informed decisions based on these data. Therefore, the new version of the Quality Guidelines recommends new guiding principles and best practices on topics such as the use of alternate data sources to traditional surveys, as well as data integration.

Given that traditional collection methods and data processing for probabilistic surveys are still supported, they are well covered in the new Quality Guidelines. With the addition of guiding principles and best practices on alternate data, as well as a stronger emphasis on ethical practices, privacy and proportionality, the sixth edition provides a better picture of today’s reality. As an example, instead of referring to data collection, the broader topic of data acquisition is discussed.

Statistics Canada has also taken advantage of this opportunity to update the format in which the information is presented by adopting a structure similar to that of the Generic Statistical Business Process Model (GSBPM). This new way of displaying the information guides the reader more easily through each and every phase and sub-process of the statistical business process.

Introduction

Statistical information and credibility of data producers

Statistical information is essential for any organized human society to function. A lack of quality data would seriously jeopardize decision-making processes, the allocation of resources and the ability of governments, businesses, institutions, and the general public to understand the country’s social and economic reality. A NSS plays a crucial role in producing and publishing statistical information.

The credibility of an organization that produces official statistics depends on a number of factors, the most important being the production of quality statistical data, cost-effectiveness, protection of personal information, confidentiality, transparency and a competent, ethical and motivated workforce with great expertise in statistical methods.

The dimensions of quality, particularly relevance, are of utmost importance for a statistical organization. If a statistical organization were unable to produce quality data, both data users and suppliers would soon lose confidence in it, making its mission more difficult to achieve. Consequently, a set of rigorous mechanisms to manage all quality matters is vital in a NSS.

Quality management in the Canadian statistical system

In the Canadian statistical system, the Quality Assurance Framework (QAF) outlines the strategies and mechanisms in place to facilitate and ensure effective quality management in all statistical programs and corporate initiatives. It includes a description of the management structure, policies and guidelines, consultation mechanisms, as well as the project delivery and management approach based on the scientific approach.

The effectiveness of this framework depends not on one single mechanism or process, but on the combined effect of many interdependent measures based on employees’ professional interests and motivation, which reinforce each other as means to meet client needs. These measures emphasize employees’ professionalism and reflect a concern for data quality. An important feature of this strategy is the synergy resulting from the many players in the programs operating within a framework of coherent processes and consistent messages.

These quality guidelines, which fall within this framework, are a companion document that describes the best practices throughout all the “stages” of the statistical business process. They are intended for members of teams responsible for developing and implementing statistical activities.

Purpose and scope of the guidelines

The main objective of the Quality Guidelines is to provide a fairly comprehensive list of guiding principles and best practices to apply during a statistical business process.

The document focuses on ensuring quality through effective and appropriate design or implementation of statistical projects or programs, from the beginning to data publication and evaluation of the process. These guidelines draw on the collective knowledge and experience of many individuals who have worked in the Canadian NSS, particularly Statistics Canada.

The document has two main sections: the first covers product quality and the second addresses the quality of processes in a statistical business activity. It is important to remember that the context in which a statistical business activity is prepared carries some constraints. Each new statistical activity not only aims to satisfy immediate information needs, but also adds information to a statistical database that can be used for a much wider range of purposes than initially identified. Therefore, it is important to ensure that the output from each activity can, to the extent possible, be combined and used with data on related subjects derived from other activities.

This implies a need to consider and respect the statistical standards on content or fields that have been put into place to achieve data coherence and harmony in the NSS. These standards include statistical frameworks (such as the System of National Accounts), statistical classification systems (such as for industry or geography), as well as other concepts and definitions that describe the statistical variables to measure. New statistical data are more useful when they can be used together with existing data.