Quality Assurance Framework
The Statistical Program

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

D. Management of input data and relations with data providers

Description

The three main sources of input data to Statistics Canada are respondents, administrative data providersNote 1 and repositories of data available to the public. Respondents, such as individuals, households, enterprises or administrations, provide data about an entity (typically themselves, but possibly an entity about which they can report) using a data collection instrument, the most common example of which is a survey questionnaire. Administrative data providers collect information about a population of interest to them for their own purposes (i.e., regulatory, financial administration, service delivery) and provide this information to Statistics Canada. The general term data available to the public refers to data that are available to any user, obtained with or without licence, and with or without payment of a fee. This includes, but is not exclusive to, open data, commercial data and web data. Statistics Canada obtains respondent and administrative data within a legal or regulatory framework that covers the Agency’s and data provider’s rights and responsibilities regarding collection and use. When obtaining data available to the public, the Agency is bound by terms and conditions that apply to all who access and use the data.

Provision of high quality input data is vital to the success of a national statistical office (NSO). Access to high-quality input data is achieved primarily through the NSO effectively managing relations with input data providers, supported by legislative or regulatory requirements and complemented by goodwill and belief in the value of quality official statistics. Another factor that contributes to the high quality of input data is the consistency in the Agency’s collection and pre-processing methods. This consistency leads to shared and standardized treatments during preliminary data processing operations.

Statistics Canada’s objective in using administrative data or data available to the public for statistical purposes is to improve the balance between relevance, data quality, response burden and cost. An important responsibility of the Agency is to justify the necessity of any new data collection activity, given its cost and the burden it imposes. In particular, when administrative data or data available to the public are of sufficient quality and adequately match the concepts being measured, they should be considered over direct collection. A further responsibility of the Agency, to both input data providers and the general public, is protecting the privacy, confidentiality and security of all data provided to it, with the provisions applying equally to survey data and administrative data.

Assessment

Statistics Canada’s efforts pertaining to obtaining and managing high-quality input data, in particular through its relations with respondents and administrative data providers, are assessed by evaluating the extent to which the Agency:

  1. justifies the necessity of any new data collection
  2. puts policies and practices in place to optimize its use of data that are already available
  3. encourages and supports respondent participation through effective communication
  4. monitors and manages response burden
  5. develops and implements effective, innovative and user-friendly collection tools and provides response options
  6. collaborates effectively with administrative data providers and other partners in the information industry
  7. provides for secure and efficient acquisition and management of administrative data through a corporate approach supported by common tools.

Implementation

Statistics Canada undertakes a broad range of specific initiatives to effectively obtain and manage the Agency’s input data and to build and maintain effective relations with input data providers. Below is a list of such initiatives, in groups that correspond to the items under Assessment.

D.1 Statistics Canada justifies the necessity of any new data collection

  • Before undertaking any new collection, determine if equivalent data are available from other sources. For example, a request to access administrative (tax) data or publicly available financial reports of corporations could be made as an alternative to collection of financial information from respondents.
    • Directive on Obtaining Administrative Data under the Statistics Act
    • Guidelines on Data Available to the Public
  • Introduce a formal review process for new data demands (i.e., increasing sample size of existing surveys or undertaking new surveys) for business and household surveys.
    • Business Response Management Committee
    • Household Survey Strategy
  • Reduce response burden through data collection and processing initiatives such as record linkage, data sharing or joint data collection activities (where surveys that collect similar or related data are integrated or combined). Study and implement innovative methods, for example electronic collection, satellite imaging and scanner data.
  • Assess the impact on privacy prior to commencing any new data collection initiative.
    • Directive on Conducting Privacy Impact Assessments
  • Limit the collection of respondent-provided information that is used exclusively for data-management or other survey-support reasons such as editing or imputation.

D.2 Statistics Canada puts policies and practices in place to optimize its use of data that are already available

  • Work actively with partners to collectively optimize the collection and sharing of data. Maintain key contact with other federal government departments and other partners including provincial focal points and industry associations. Ensure consistency through standardized protocols for objectively evaluating the quality, relevance and utility of new sources of administrative or data available to the public.
    • Policy on the Use of Administrative Data Obtained Under the Statistics Act
    • Directive on Obtaining Administrative Data Under the Statistics Act
    • Quality and Evaluation Framework for Administrative Data
  •  Keep an inventory of administrative data files available to Statistics Canada programs.
    • Administrative Data Inventory
    • Acquisition of Administrative Data
  • Promote and support the use of record linkage, small-area estimation and other techniques that bring together information from various sources.
  • Seek alternatives to traditional survey methodology and administrative microdata, in particular when direct collection is not feasible or is not cost-effective, such as for rare and hard-to-reach populations and sub-populations of large numbers of small units. Examples of such techniques include statistical modelling, proxy response and synthetic estimation.
  • Learn about innovative data gathering and treatment methods (e.g., “Big Data”) and gauge their relevance to Statistics Canada. Establish ties with information-industry partners in the private sector, academia and research institutes.

D.3 Statistics Canada encourages and supports respondent participation through effective communication

  • Instill confidence in respondents by emphasizing guarantees of privacy and protection of personal information provided by federal laws and Statistics Canada policies.
  • Provide respondents with information on Statistics Canada and the survey in which they have been asked to participate. Explain to respondents the benefits of official statistics and the importance of their participation.
  • Convey the appreciation of Statistics Canada by thanking respondents in publications and, when possible, giving back to respondents information accruing from their survey participation.
  • Maintain lines of communication by soliciting respondent feedback, responding to complaints and concerns and using the information obtained to improve programs and processes. This communication strategy includes a single point of contact for respondent inquiry and an approach where social media (including Facebook, Twitter and YouTube) are used to provide access to statistical information and to foster engagement, co-operation and information-sharing with respondents. Further, an ombudsman is available to help business survey participants by responding to their concerns.
  • Initiate outreach to promote an awareness of Statistics Canada within the Canadian public. Examples of such approaches are regular presentations by the Chief Statistician, relations with the media such that Statistics Canada results are announced to the public, and through the maintenance of a web and social media presence.

D.4 Statistics Canada monitors and manages response burden

  • Put in place controls to prevent a disproportionately high response burden on particular respondents or particular categories of respondents. This strategy entails such measures as offering periodic reporting exemptions for respondents selected for multiple surveys during a time interval, establishing upper limits for the length of time respondents spend in the sample for a specific survey, and taking respondent availability and preference into account when scheduling interviews.
  • Consider response burden in the sample allocation phase. For example, identify rare populations from the Census of Population or administrative data to avoid asking identification questions to a broad base of respondents. Response burden can also be minimized through techniques including sample rotation (e.g., Labour Force Survey rotating panel design, where one-sixth of the sample is replaced each month), coordinated allocation and use of differential sampling rates (e.g., selection probabilities proportional to size for skewed populations of enterprises).
  • Retain information to quantify response burden and set targets for its reduction over time.
  • Coordinate all data collection, irrespective of mode, through a corporate collection service. Statistics Canada uses a single portal to plan for all collection needs and an integrated collection and operations system, and achieves consistency in input files through standard methods.
    • Collection Front Door
    • Collection Planning and Research Division
    • Integrated Collection and Operations System

D.5 Statistics Canada develops and implements effective, innovative and user-friendly collection tools and provides response options

  • Use focus groups and questionnaire testing to identify impediments to response. These strategies include detecting sensibilities and social acceptability, minimizing the impact on privacy and degree of invasiveness, and detecting the impact of questionnaire length and “respondent-friendliness”.
    • Policy on the Review and Testing of Questionnaires
    • Questionnaire Design Resource Centre
  • Build and deploy an electronic data reporting infrastructure. This infrastructure includes the development of electronic questionnaire applications as part of the Corporate Business Architecture modernization, and the adoption of internet response as the default mode for the Census of Population.
    • Electronic Questionnaire Service
    • Corporate Business Architecture
  • Adapt to changes in technology and behaviour by developing applications for cellphone and tablet response, integrating methods to reach cellphone-only households and initiating the use of tablets by interviewers.
  • Where possible and appropriate (i.e., for large or complex enterprises), adapt data collection to conform to the respondent’s accounting system.

D.6 Statistics Canada collaborates effectively with administrative data providers and other partners in the information industry

  • Build and sustain good working relations with providers and custodians of administrative data. In particular, ensure contact between counterparts at a variety of levels and functions, including chief executives, program managers and operational staff. Consult with other federal government departments, provincial government departments or focal points, and industry associations. Strengthen the statistical value of administrative data by communicating Statistics Canada’s requirements in terms of coverage, conceptual definitions and timeliness. Keep abreast of organizational and operational changes at provider agencies and remind providers of Statistics Canada’s commitments to data security, confidentiality and use of data for statistical purposes only.
  • Solidify access to data by entering into data sharing agreements and by working to support terms of legislation that give the Agency access, for statistical purposes, to information of other government departments and organizations. For example, Memoranda of Understanding have been signed with Canada Revenue Agency for tax and payroll data.
    • Statistics Act
    • Directive on Obtaining Administrative Data under the Statistics Act

D.7 Statistics Canada provides for secure and efficient acquisition and management of administrative data through a corporate approach supported by common tools

  • Ensure an Agency-level coordination of external collaboration (interdepartmental, intergovernmental and international) led by the division responsible for administrative data. Coordinate and support efforts with the appropriate administrative structures, governance, resource allocation and communications plans.
    • Administrative Data Division
  • Require all projects that deal with methods or processes associated with administrative data to pass through Agency-level approval. This practice of using a corporate committee for administrative data management fosters development and implementation of consistent practices, and achieves long-term benefits including increased relevance, gains in efficiency and reduction of response burden.
    • Administrative Data Management Committee
  • Maintain a repository of information on administrative data holdings and initiatives within the Agency. This repository includes tools for use of administrative data (i.e., Metadata Explorer, Tax Data Navigator), an inventory of administrative data files that have been obtained or are in the process of being obtained by the Agency and a collection of relevant, high-level documentation on administrative data.
    • Administrative Data Inventory
  • Standardize pre-processing of administrative data. Create and maintain centralized databases of processed administrative data, including tax data. Require that “downstream” processes draw from these databases, rather than processing administrative data independently.
  • Ensure the confidentiality, security and integrity of administrative data provided to Statistics Canada. Protect administrative data with equivalent physical, technological and organizational provisions as survey data obtained directly by the Agency. Apply enhanced security measures when warranted by the sensitivity of administrative data, such as for tax data or health information. Ensure that only authorized staff has access to specific databases, controlled through a centralized system.
    • Policy on Privacy and Confidentiality
    • Information Technology Security Policy
    • Directive on the Security of Sensitive Statistical Information
    • Directive on the Transmission of Protected Information
    • Corporate Access Request System

E. Allocation and management of resources

Description

The production of most Statistics Canada outputs is supported financially through government revenues allocated to the Agency. Other resources also requiring constant management include human resources, information technology and infrastructure.

It is important to identify and harvest efficiencies in order to fund improvements and new initiatives and to provide a cushion against funding stresses. Consequently the Agency strives to minimize the resources consumed (efficiency) while ensuring that outputs are fit for use (effectiveness).

Assessment

Statistics Canada?s allocation and management of resources are assessed by evaluating the extent to which the Agency:

  1. implements effective strategies for organization and management
  2. ensures continued production of relevant, high-quality and timely information through effective long-term planning and priority-setting
  3. pursues efficiencies through centralization of resources, consolidation of services and use of generalized systems
  4. pursues efficiencies in administrative services, infrastructure and technology
  5. implements effective project management
  6. implements human resources management practices that optimize the contribution of its workforce and demonstrate its commitment to quality.

Implementation

Statistics Canada undertakes a broad range of specific initiatives to effectively allocate and manage resources in its statistical programs. Below is a list of such initiatives, in groups that correspond to the items under Assessment.

E.1 Statistics Canada implements effective strategies for organization and management

  • Corporately optimize decision-making. Decisions regarding resource management and allocation are ultimately decided by the Executive Management Board (EMB). However, various internal committees, with specialties in particular subjects, recommend decisions to the EMB. These recommendations are based on reviews that consider the Agency’s overall priorities, gaps, risks and budget.
  • Objectively evaluate the Agency’s use of resources by fully participating in external audits, scheduling regular internal evaluations and reviews to evaluate objectives in relation to cost, and producing corporate performance indicators that quantify program efficiency.
  • Maintain the capacity (staff, infrastructure) to produce custom products in response to client requests for cost-recovery work.
  • Mitigate vulnerability to funding stresses by setting aside sufficient contingency funds, in part through instilling a culture that promotes efficiency.
  • Centralize control and monitoring of financial administration, human resources management and information technology infrastructure at the Agency level.

E.2 Statistics Canada ensures continued production of relevant, high-quality and timely information through effective long-term planning and priority-setting

  • Implement a sound planning and priority-setting process at the Agency level. This process considers a long-term horizon. It works on an annual planning cycle and covers all stages of planning where financial, human resources and informatics needs are all considered together. It includes a review of corporate priorities and integrates risk management, investment planning, and evaluation into the planning process.
    • Integrated Strategic Planning Process
    • Integrated Business and Human Resources Plan
    • Information Technology Plan
  • Forecast regular and strategic investments necessary to preserve quality and continuity of statistical programs over time. This forecast covers an extended horizon (10 years) and considers the cyclical nature of specific statistical programs (e.g., five-year Census cycle, 10-year redesign cycle for certain surveys) when planning redesigns and other program modifications or enhancements. A separate fund exists for initiatives that cannot be covered by program area base budgets.
    • Continuity and Quality Maintenance Investment Plan
    • Continuity and Quality Management Fund
  • Target efficiencies by regularly conducting Agency-wide reviews of systems and practices. Statistics Canada has in place the Corporate Business Architecture which is a long-term Agency-wide review and modernization that covers the business processes, business rules, computer systems and internal organizational and physical infrastructure that Statistics Canada uses to carry out its main business of collecting, analyzing and publishing statistical information. By consolidating processes and standardizing systems where necessary, cost savings are achieved while still maintaining high standards of quality and timeliness.
    • Corporate Business Architecture
  • Communicate plans and priorities to employees and stakeholders. Besides publicly available documents, internal communication includes the Chief Statistician’s annual address and subsequent article published in the Statistics Canada electronic employee newsletter.

E.3 Statistics Canada pursues efficiencies through centralization of resources, consolidation of services and use of generalized systems

  • Limit the number of business processes and computer systems in place; that is, encourage re-use of existing systems. Foster ease of transition of employees between projects by consolidating computer applications and minimizing the number of hardware and software products (tool kits) in use.
  • Ensure efficient allocation by centralizing specialized resources and services. Examples include informatics and methodology resources, frame infrastructures for economic and social surveys and administrative data processing operations.
  • Establish single corporate service areas for statistical services to generate and capture economies of scale. Mandate their use by requiring justification of local exceptions. This plan involves common processing environments for business and social surveys, a corporate collection service and a single collection system.
    • Integrated Collection and Operation System
  • Develop, promote and support the use of a suite of Statistics Canada generalized systems and common tools.
    • Generalized Systems Resource Centre
  • Streamline data processing activities. In particular, eliminate rework (e.g., editing at multiple stages of the survey process) and pay particular attention to resource-intensive steps such as collection and manual processes such as editing and coding. For example, selective editing can be used to prioritize application of effort to where it will have the greatest impact on estimates, and responsive collection uses paradata to optimally schedule interviews.

E.4 Statistics Canada pursues efficiencies in administrative services, infrastructure and technology

  • Introduce transformation and modernization initiatives for infrastructure. This includes physical space reduction and modernization, replacement of landline telephones with cellphones and migration of department-specific email systems to a common framework.
    • Cost Effective Telephone Services Infrastructure
    • Email Transformation Initiative
  • Undertake periodic modernization of the computer network and informatics infrastructure. This includes improvements to informatics security, standardization of employee software toolkit (desktop) and migration of local processing platforms to centralized ones (e.g., SAS Grid).
    • Network Transformation Initiative
  • Centralize and modernize delivery of administrative and other support services. In particular: streamline and standardize through use of Agency-standard business processing systems for human resources management and financial administration; replace paper-based methods by electronic request and approval; and consolidate editorial and translation services.

E.5 Statistics Canada implements effective project management

  • Require the use of a standard project management process, templates and tools covering the entire project lifecycle. This process includes project “gating” (approval and monitoring) at the field level within the Agency.
    • Departmental Project Management Framework
    • Departmental Project Management Office
    • Field Planning Boards
  • Require supplemental procedures for evaluation of the informatics component of new projects.
    • Informatics Technology Architecture Committee (includes the Technical Review Committee and the Security Review Committee)
    • Integrated Security and Risk Framework
  • Facilitate project management by mapping projects to statistical process models.
  • Minimize misunderstanding and ensure project goals are respected through effective change management.
    • Changes, Issues and Risk Management Tool
    • Change Advisory Board
  • Provide tools for monitoring and evaluating project resources. Performance reports, monthly financial reporting and project dashboards are required of each program.
    • Departmental Project Management Framework

E.6 Statistics Canada implements human resources management practices that optimize the contribution of its workforce and demonstrate its commitment to quality

  • Effectively manage appointment and promotion. Centralized recruitment programs for core professional groups, generic competitions for all management levels and comprehensive “career streams” for all major groups are all standard practices which contribute to the effective use of resources.
  • Ensure Statistics Canada’s human resources meet the Agency’s needs in terms of both strength and qualification. Regular Agency-wide performance monitoring and feedback (including training needs), talent management and mentoring are ways of enhancing staff performance.
    • Training Policy and Framework
  • Anticipate future needs through effective human resources management and planning, including succession planning.
    • Integrated Business and Human Resources Plan
  • Cultivate employee motivation, professionalism and engagement. Statistics Canada motivates employees by fostering innovation, as well as by a positive and healthy workplace including physical exercise facilities, flexible work opportunities and a Wellness Centre.
    • Innovation Channel

F. Management of relations with data users and stakeholders

Description

Building and sustaining good relations with its data users and stakeholders is vital for the success of a national statistical office (NSO). Data users and stakeholders transform statistical information into knowledge, as needed for political debate, decision-making or research. When statistical information is fit for use, data users and stakeholders gain confidence in the NSO, return to it for future information and, in some instances, provide the NSO with the support it needs to carry out its work.

The focus of this chapter is activities Statistics Canada undertakes to build and sustain relations with data users and stakeholders. It is important to note that some data users and stakeholders may interact with Statistics Canada in other capacities, such as survey respondents or providers of administrative data.

Assessment

Statistics Canada?s efforts pertaining to managing its relations with data users and stakeholders are assessed by evaluating the extent to which the Agency:

  1. makes information about itself readily available
  2. improves its programs through the participation of data users and stakeholders
  3. ensures all Canadians have equitable and timely access to its data products and information releases
  4. meets the communications and support services needs of specialized groups of data users and stakeholders
  5. partners and collaborates effectively with other federal departments and agencies and with provincial, territorial and local governments.

Implementation

Statistics Canada undertakes a broad range of specific initiatives to build and maintain effective relations with its data users and stakeholders. Below is a list of such initiatives, in groups that correspond to the items under Assessment.

F.1 Statistics Canada makes information about itself readily available

F.2 Statistics Canada improves its programs through the participation of data users and stakeholders

  • Regularly engage in consultation with Canadians to gauge satisfaction, seek input and test functionality, using a variety of methods. For example, the Agency consults with: users, as part of program review and evaluation; business and industry associations and labour unions, in particular to learn about information needs and reporting preferences; not-for-profit organizations including community groups, social organizations and volunteer groups; and interested groups on particular programs (e.g., Census of Population content). When possible, the results of consultation are made available to the public.
  • Use advisory groups of experts to ensure the Agency’s programs remain relevant in light of emerging needs and trends, to verify its methods remain sound and to foster exchanges that shed light on stakeholder needs and priorities. These groups include the National Statistics Council and executive and professional advisory committees on statistical methods and in each of the major subject-matter areas.
  • Maintain extensive contacts with international, scientific and intergovernmental organizations and membership in international organizations to ensure the Agency's outputs meet international standards and that Canadian data continue to be comparable with those of other countries.

F.3 Statistics Canada ensures all Canadians have equitable and timely access to its data products and information releases

  • Publish release dates well in advance. The pre-planned release schedule is publicly available and any pre-release access is communicated transparently. Further, procedures exist for cases where data are released prematurely.
  • Require that all releases pass through a uniform dissemination service. Currently The Daily is Statistics Canada’s official release bulletin.
  • Correct errors in a structured, timely and transparent fashion.
    • Directive on Corrections to Daily Releases and Statistical Products
  • Minimize the barrier of cost by providing a set of standard products free of charge on the Statistics Canada website.
  • Produce and promote a broad range of statistical products, and define a fair and balanced policy for funding these on a cost-recovery basis. For example, research papers, analytical products, specialized tabulations, as well as the design and implementation of customized surveys, are available.

F.4 Statistics Canada meets the communications and support services needs of specialized groups of data users and stakeholders

  • Work with the media and others to maximize awareness, in particular by encouraging re-dissemination of Statistics Canada information products through use of modern information technology and an open-data licence agreement that requires no payment of fees for specific data products. This approach includes applications for smartphone and tablet access, a presence on social media (including Facebook, Twitter and YouTube), a regular blog and a facility to interact with Agency specialists.
  • Provide special and dedicated services for members of the media.
  • Facilitate researcher access to data without compromising data security and confidentiality. Apply protocols regarding external user access to statistical microdata for research purposes. These protocols apply to external researchers working at Statistics Canada’s offices and at its Research Data Centres. They also cover the use of microdata accessed externally via universities, or through remote access.
  • Put in place a commitment to serve clients in a prompt, reliable, courteous and fair manner and maintain lines of communication with them. This commitment includes a single point of contact for respondent inquiry and a follow-up response to complaints and concerns.
  • Explain the value and benefits of quality official statistics to data users and stakeholders.

F.5 Statistics Canada partners and collaborates effectively with other federal departments and agencies and with provincial, territorial and local governments

  • Maintain close bilateral relationships with key federal departments and agencies through connection at the senior management and other levels. These relationships are supported through participation of the Chief Statistician in policy and program discussions with other deputy ministers, through participation in federal government research initiatives and through inter-departmental committees. These contacts foster an awareness of respective needs and information priorities, and allow the Agency to keep abreast of emerging needs.
  • Collaborate and liaise with provincial and territorial focal points and other representatives. Determine provincial and territorial data requirements, consult on current statistical activities, harmonize provincial and territorial statistics to produce national estimates, and coordinate the dissemination of Statistics Canada’s products to provincial and territorial governments.
    • Federal-Provincial-Territorial (FPT) Consultative Council on Statistical Policy
    • Subcommittees of FPT Consultative Council on Statistical Policy
  • Provide special liaison, consultation and coordination in the areas of health, education and justice, which are primarily under provincial jurisdiction. Examples include assistance to the Board of Directors of the Canadian Institute for Health Information and participation in the Canadian Education Statistics Council and the Justice Information Council.
  • Maintain lines of communication with local and regional governments.
Date modified: