Quality Assurance Framework
Introduction

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Statistics Canada?s mission statement ?Serving Canada with high-quality statistical information that matters? expresses the Agency?sNote 1 mandate to provide all sectors of Canadian society with access to a trusted source that serves their information needs. Statistics Canada defines the quality of its official statistics in terms of their fitness for use. Maintaining confidence in the Agency through the management and assurance of quality is essential to the success of Statistics Canada.

The Quality Assurance Framework (QAF) describes the strategies Statistics Canada has put in place to facilitate and ensure effective management of quality in all its statistical programs and organizational initiatives. Underlying these strategies are eight guiding principles.

Guiding Principles

Quality is multi-dimensional

Statistics Canada has identified six dimensions of statistical information to define its quality and evaluate its fitness for use.

Relevance reflects the degree to which statistical information meets user needs.
Accuracy reflects the degree to which statistical information correctly describes the phenomena it was designed to measure.
Timeliness refers to the delay between the end of the reference period to which statistical information pertains and the date on which the information becomes available.
Accessibility refers to the ease with which statistical information can be obtained.
Coherence reflects the degree to which statistical information is logically consistent and can be brought together with information from other sources or different time periods.
Interpretability reflects the availability of supplementary information (metadata) necessary to understand, analyze and utilize statistical information appropriately.

The six dimensions are overlapping and interrelated and achieving an appropriate level of quality in all dimensions is required, as failure in any one of them will compromise the fitness for use of an information product. Statistics Canada strives continuously to find innovative methods and data sources that can lead to achieving higher levels of quality in one or more dimensions without adversely impacting others.

Quality is relative, not absolute

Management of quality must be in conjunction with other important factors including the data needs of users and stakeholders, costs and response burden. As when managing the dimensions of quality, Statistics Canada counts on innovations, in areas such as data integration, to fulfill user needs with high quality data at lower cost and lessened response burden. It also recognizes that effective management of quality does not demand maximization of quality over all other factors. Rather, it is the result of striking an appropriate balance between the resources available to the Agency and the information needs of its data users and stakeholders. Efforts to improve the quality of official statistics take into account factors such as existing budgets, availability of specialized resources and response burden.

Every employee has a role to play in assuring quality

Statistics Canada?s management of quality reflects the principle, as stated by renowned American statistician W.E. Deming, that ?Quality comes not from inspection, but from improvement of the production process.?Note 2 That is, it is not possible to achieve quality by merely ?inspecting? a final product. Rather, quality must be built into processes from the outset. Success in assuring quality at Statistics Canada requires the sound application of knowledge and expertise by employees at all levels within the Agency ? in short, quality is ?everyone?s business? at Statistics Canada. An essential component of this strategy is recruitment and professional development programs that lead to a motivated and competent workforce.

Quality must be built in at each phase of the process

As operations at all stages can impact the quality of outputs, effective quality assurance requires measures at multiple phases of the statistical process and consideration of the impact of each phase on the process as a whole. Modelling the statistical process by dividing it into phases has proven to be an effective management tool. One such reference framework is the Generic Statistical Business Process Model, in which the principal phases are “specify needs”, “design”, “build”, “collect”, “process”, “analyze”, “disseminate” and “evaluate”. A quality management structure can be conceptualized by considering each cell of the matrix defined by these phases and the six dimensions of quality. It is important to note that effective management of quality does not necessitate similar measures at all phases.

Balancing the dimensions of quality is best achieved through a team approach

The use of multidisciplinary teams ensures that the dimensions of quality and other important factors including cost and user needs are effectively managed. Subject-matter experts bring knowledge of content, stakeholder needs and relevance while mathematical statisticians (methodologists) contribute a sound foundation in statistical methods and expertise regarding accuracy. Operations personnel have experience in collection and processing methods, are well-placed to consider practicality and efficiency, and represent field staff and respondents. Systems personnel ensure the informatics point of view is considered in design and implementation, and bring specialist knowledge of technology standards and tools. Teams are supported by a committee of senior managers who provide a forum for resolving issues and, when necessary, give guidance related to data quality trade-offs, especially with respect to timeliness and cost.

Quality assurance measures must be adapted to the specific program

At Statistics Canada, responsibility and accountability for quality assurance lie with the Agency?s statistical programs. The Agency develops and maintains quality management strategies and tools that program areas can adapt to their individual needs. Within individual programs the challenge is to achieve an appropriate level of quality by effectively balancing program objectives, evolving user and stakeholder needs, costs, response burden, and the various dimensions of quality.

Users must be informed of data quality so that they can judge whether the statistical information is appropriate for their particular use

Some dimensions of quality, such as timeliness, can be observed directly by users. However for most other dimensions, users require objective information about data quality to evaluate fitness for use. Often, the Agency is the sole source of such information. Both quantitative measures, such as coefficients of variation and response rates, and qualitative information, such as a description of sources of error, are necessary.

Quality assurance is a continuous practice

There is constant evolution of the social and economic conditions in Canada, as well as in the user and stakeholder environments. Consequently, quality is not self-sustaining and will deteriorate in the absence of regular review and refreshment. In particular a ?relevance gap? may open if official statistics do not keep pace with the changing needs of the Agency?s users and stakeholders. It is further incumbent on Statistics Canada to ensure its methods remain at the forefront of those used by national statistical offices. This is achieved through a culture that promotes a continuous search for new and innovative sources and methods.

Background to the QAF

The Statistics Canada QAF was first produced in 1997 and was updated in 2002. This 3rd edition was inspired by the generic National Quality Assurance Framework template developed by a United Nations Statistics Division Expert Group. In particular, this version expands the scope of the Statistics Canada QAF by discussing quality management in the Agency?s corporate environment and statistical programs.

In the QAF, several references are made to the organizational structure and operating procedures of Statistics Canada. An overview of the functional (reporting) structure of the Agency is given below.

  • Statistics Canada is headed by the Chief Statistician, an appointment at the deputy minister level within the Government of Canada.
  • The Chief Statistician is supported by an Executive Management Board, which consists of assistant chief statisticians who each represent one field of the Agency. A field covers a broad grouping of commitments, such as those related to economic statistics or to corporate services.
  • Fields are sub-divided into branches, and each branch is made up of divisions. Each division is led by a director.

Organization of the QAF

The QAF consists of twelve stand-alone chapters, each defined by a quality management theme. All chapters share a common three-part approach. The first, Description, introduces concepts to be discussed in the chapter, defines relevant terms and provides context and background information. Issues and factors that impact the Agency’s capacity to achieve goals related to the concepts in the chapter are discussed, including relationships to concepts described in other chapters. Assessment lists objectives essential to successful operationalization of the concepts in the chapter. Objectives are expressed in generic and qualitative terms, and represent ideals to which the Agency strives. Implementation demonstrates achievement of these objectives through references to specific Statistics Canada activities that contribute to quality, grouped by the objectives in Assessment.

Date modified: