Analytical Studies: Methods and References
The System of National Quality-of-Life Statistics: Future Directions

Skip to text

Text begins

Executive summary

In response to current demands, Statistics Canada is engaged in a long-term exercise to greatly strengthen the statistics that it produces to measure the quality of life in Canada. Compared with current arrangements, these statistics will be more timely, more granular, more responsive to the needs of users, and more easily accessed.

In line with directions set out in Budget 2021, the system that is being developed will provide comprehensive, useable evidence on:

• The inter-related factors, including non-economic factors like health, housing, environment, and safety, that result in a high quality of life, including the effects of the policies and programs of all orders of government on people as they move through the course of their lives.
• Equality, showing the distribution of outcomes and opportunities across places and people, taking account the great diversity of the Canadian people and the need for analysis, and action, at the level of small population groups including those facing barriers to a high of quality life.
• Sustainable outcomes, assessing how current actions will support high quality of life in the future.

Over time, the system will also begin to provide evidence that can improve decisions about the particular social interventions that are likely to work best for specific individuals – with that information being accessible at the time when these decisions are being taken, based on analysis of what has worked best in similar situations in the past.

The resulting evidence will support government budgeting and accountability, the design of effective programs, and efficiency in their administration. Decision-making by individual citizens and service providers will be supported with practical ‘what is likely to work best’ data. The outcome will be major improvements in the operation of labour markets, health, learning and other social dimensions of life in Canada, both on average and for all population groups – and direct benefits to individuals as they make big decisions in the social, health and labour domains of life.

Unlike the existing social statistics, which are based around particular data sources such as surveys, censuses and summarized administrative records, the new system will provide far more detailed and timely information by drawing on data from multiple sources. These sources will include a much more flexible suite of surveys, finely-grained information about operation of social programs, and anonymized data about individuals that currently exist in administrative records. A multi-source system is possible as a result of the major improvements in digital information processing technology that have been introduced in recent decades, with promise of even more powerful tools and data sources on the horizon.

The foundations for this transformation are already in place and the priority on speedy implementation has increased as a result of COVID-19 and the directions outlined in Budget 2021. Many payoffs become visible over the next several years. Others will take longer to be fully realized.

One of the most visible early products will be the creation of an interactive website that will present high-level quality-of-life indicators that reflect current policy priorities, as well as a series of practical dashboards of indicators designed to support a wide range of current policy agendas for users both in and out of government. Current statistics, such as the latest unemployment rates, would be released on this site where they would be analysed along with indicators related to other dimensions of wellbeing.

The site will provide direct access to interactive tables along the lines of those on the present Statistics Canada site, except that there will be many additional tables with newly-created data, based on consultations with users, that would be drawn from different sources. This is in opposition to the present practice of presenting tables based only a particular survey or other data source.

In addition, the new techniques for combining data from multiple surveys and administrative data sources without any of the resulting data being linkable to real people opens up the possibility of developing powerful new kinds of data products for release on the web site. These are known as standard, multiple-source microdata files. These contain particularly rich data that can be used to provide a new understanding of the factors that affect the quality of people’s lives over time, including the role played by different social programs and services.

Another early payoff will be greater flexibility in responding to requests for new statistics. The interactive web site will, itself, provide far more information than is currently available. As well, many requests for new data will be met by combining existing data from different sources, without time-consuming and costly new data collection. When new data collection is needed, a flexible suite of new survey vehicles is being developed to allow fast collection of information that is finely-tuned to current needs.

Other statistical products will take longer to produce. For example, some will require developing consistent, finely-grained ways of describing the existing social, learning and health services. Such standard descriptors are needed, for example, in order to provide consistent ‘what is likely to work best’ information that will support service providers and individual citizens in making the best choices among the interventions that are open to them. In some areas, the needed information already exists. In others, much development work will be needed.

This paper describes the system as it may look in some five or so years time assuming the continuation of present plans. It does this by examining the likely future inputs, processes, outputs and outcomes of the system, and compares them with the arrangements in the traditional single-source system of social statistics that has, until recently, been the norm.

The paper also describes the main constraints and risks that will be faced as we move ahead and how these will be managed.

In terms of resource constraints, funding will come from internal reallocations and new funds outlined in Budget 2021. More powerful information technology (IT) resources will be required, but these are already being used in other applications. The biggest constraint is likely to be the shortage of people with the skills needed to produce accurate statistics from multiple sources of data, often involving large data sets and complex statistical manipulations, while ensuring privacy. New approaches will be developed to increase the size of the pool of people with these skills and to maximize the use of those skills in both Statistics Canada and in the user communities.

New ways of doing business are being proposed. The largest risk to be managed during this period of rapid change relates to the need to maintain and reinforce the trust and confidence of the users of statistics, the individual Canadians who respond to surveys and the agencies who provide the needed administrative data inputs. Managing this risk will require different kinds of outreach to explain how the new arrangements will work, how privacy and confidentiality will be assured, and how all parties will gain over time from the new approaches.

As well, the risks associated with the introduction of a new product line – i.e., the use of predictive analytics to produce ‘what is likely to work best’ evidence in support of decision-making about particular social interventions – will be mitigated by a gradual, experimental approach, working in partnership with different service providers.

In terms of social acceptance and trust, Statistics Canada has recently put in place a world-leading framework (called the Necessity and Proportionality Framework) for balancing use, privacy, and response burden. Sophisticated tools have already been developed that ensure privacy protection when linking data from multiple sources, using techniques that split out individual identifiers from the data that describe those individuals. Response burden will be reduced by lessening the demand for repetitive inputs. New emphasis will be placed on the role of Statistics Canada itself in producing statistics that are based on multiple sources of data, with built-in assurance of both quality and privacy protection, reducing past reliance on external users to carry out some of these functions. Greater use of synthetic data will simultaneously produce useful new data and assurances of anonymity.

Perhaps the most important new payoffs in terms of building acceptance and support will lie in the production of new statistics that will be directly useful, not only to researchers and policy-makers, but also to those who administer social programs and services and to individual citizens. A more immediate and transparent balance can be struck among response burden, usefulness and the protection of privacy – strengthening the implicit social contract that exists among those who provide, process and use data for statistical purposes.

A separate paper, entitled “The System of National Quality-of-Life Statistics: Conceptual Framework”, will be released as a complement to this paper that will describe the conceptual framework that underlies the new approach. It will show how the standard definitions that are used to describe the individual items of data that are collected from different sources will result in a flexible system that allows integrated analysis across all the domains that affect quality of life, including linkages with economic and environmental statistics, compatibility with the frameworks used in many academic disciplines, and comparisons with other countries.

1. Introduction and summary

Deep changes are taking place in national social statistics. This paper presents a picture of what the system might look like in some 5 or 10 years should Statistics Canada continue along its current path of rapid evolution. Possible actions in the shorter term are described, as are strategies for managing the constraints and risks that inevitably arise in making such large changes.

A supporting paper will be available shortly that describes the changes from a more technical perspective. It will describe the conceptual framework that underlies the new system and how it results in a system that is both integrated and flexible. It will show how the framework is consistent with those used in economic and environmental statistics and in different scientific disciplines.

This introductory chapter is structured as follows:

• Section 1.1 summarizes the nature of, and rationale for, these changes.
• Section 1.2 summarizes the features of the system once it becomes mature and compares it with current approaches.
• Section 1.3 explains how the constraints and risks will be managed as the system evolves.
• Section 1.4 discusses the need for flexible planning that can adjust to the introduction of new data sources and technologies.

1.1. Why change? With what payoffs?

Deep changes are taking place in the collection and use of national social statistics, which – for reasons that will be explained later – will be referred to as the ‘National System of Quality-of-Life Statistics’.

• The system is shifting from the traditional approach where data collection, and the release of the resulting statistical information, was based on single data sources, such as a survey.
• It is moving towards a system that is based on consistent data obtained from multiple sources and that will, in consequence, be able to produce integrated, timely information about the inter-related dimensions of social well-being. That information will be released in an easily understood way that shows our progress towards a better quality of life and that identifies problems that need to be addressed by policy.

The new information obtained from multiple sources is changing our understanding of how societies work, particularly as we dig down beyond broad averages and examine the diverse needs and circumstances of people in the many groups that comprise our population. It is starting to provide a far better understanding of how people change over their lifetimes, including the role played by our social institutions and programs.

Factors that are driving change

The main factor driving these changes has been increased demand for statistical information that cannot be easily met by further investments in traditional single-source source collection and processing methods. There are demands for:

• More granular, timely statistical information to support public policy – information that takes account of the many interacting determinants of quality of life, including the role played by combinations of public programs and policies over the course of the lives of individual Canadians.
• Information that reflects the diversity of the population and the inter-related dimensions of social wellbeing, taking account of the need for analysis at the level of small population subgroups.
• Predictive analytics that can directly support both individual citizens and the employees who administer social programs with information on which types of actions in the social domains of life are likely to work best in the current circumstances, based on what has worked best in the similar circumstances in the past.

This kind of information requires detailed descriptions of social programs and services. It requires detailed micro-level measures of the changes that are taking place over the lifecourses of individuals, with individuals seen in their differing social, economic, and geographic contexts.

That information cannot come only from traditional survey and census sources taken in isolation. The parallel driver of change has been the current availability of new data sources, new processing power, and new statistical methods that can be leveraged to fill these new demands in ways that are affordable, minimize response burden and protect privacy and confidentiality.

The ‘Bottom-line Q&As’ in Box 1 provide an overview of the rationale for the changes and how different audiences will benefit.

Reform is already well launched

In response to these changing demands – and to exploit the potential of new ways of doing business - the field in Statistics Canada with prime responsibility for the collection and release of health, labour and other social statistics launched a future directions exercise as part of the work on modernization that is taking place within Statistics Canada as whole.

Important first steps were taken in 2019, including a reorganization that separated responsibilities for dealing with the substantive content of the statistics from responsibilities for collecting the statistics. This was a necessary step in the shift to a multiple-source model.

COVID-19 accelerated development. Plans for improving responsiveness and flexibility in meeting the needs of users were put to immediate test with arrival of COVID-19 in 2020. The response of the statistical system in providing Canadians with the information needed to deal with COVID-19 has been astonishing, marked by innovations that would have been unbelievable only a few years ago. This was accomplished while learning how to deliver in a completely changed work environment with people working from home. These innovations not only met current demands, but also represented important steps in the desired longer-term directions.

The energy devoted to COVID-19 did, however, mean that some longer-term plans were put temporarily on hold, including the articulation and communication of a clear statement of where the social statistics system was headed. This document attempts to fill that gap.

Budget 2021 provided funds and reinforced the directions. The budget allocated funds to Statistics Canada to take concrete action along the lines initiated in 2019. Particular emphasis was placed on timely, disaggregated evidence to support policiesand programs. It also stressed the importance of measuring the multiple dimensions of quality of life and of developing ways of using these measures to shape policy-making.

The role of a paper that shows where the system is heading

Many users and partners are not aware of the huge potential for improvement in social policies and in social decision-making that will be opened up by the shift that is underway.

In the absence of a shared understanding of future directions, both inside the statistical agency and among users and partners, it seems unlikely that the frenetic pace of change in 2020 can be sustained. Yet there is still much work to do. It is important to understand that, despite the considerable recent progress, we still do not have a system of national quality-of-life statistics in the ordinary sense of the word ‘system’.

• Rather we still have a series of partly coordinated, but mainly siloed, approaches to measurement in different social domains such as education, health, justice, labour markets. There is need for greater integration with economic and environmental statistics – all of which impact on our quality of life.
• Most statistics, whether aggregated data or microdata, are still organized and accessed according to the source of that data – e.g., information from the Labour Force Survey, or from the Census, or from public use microdata based on the original source of that data.
• The tools that will hold a multi-source system together – consistent definitions that are independent of data sources, shared statements of vision and mission, coordinated outreach to users and partners, outputs that reflect the power of linking multiple, overlapping data sources – are only still partly in place.

1.2. The system of national quality-of-life statistics in summary

The chapter will end with a series of text boxes that compare the system that is being built once it is reasonably mature, perhaps after some 5 or 10 years of development, with a traditional single source system that has been the norm until recently.

Box 2 sets the context. The shift to a multi-source system in the statistical system is one dimension of a much larger shift in government towards better use of digital technology, openness, and data management. The Box shows:

• How the statistical system fits into that broader context in way that results in virtuous circles through much stronger data flows and linkages among the program and policy departments that use the statistics and produce administrative data.
• How trust and privacy protection lie at the heart of the evolution that is taking place.

Box 3 describes the outputs of the mature system, the statistics that will be produced.

• The most visible and short-term change will be the creation of an interactive website or portal that presents high-level quality-of-life indicators that reflect current policy priorities, as well as a series of more practical dashboards of indicators designed to support current policy agendas.
  • The site will provide direct access to Common Output Data Repository (CODR) tables (previously called CANSIM tables) along the lines of those on the present site, except that there will be many additional tables that are addressed to particular topics of interest – with the data drawn from many sources, including tables based on linked microdata files and including high quality synthetic variables.
  • The site will also facilitate access to a series of standard multiple-source microdata files that would be constructed by the statistical agency to shed light on analytic topics of general interest. These would be documented and anonymized.
• As well, a new output would be gradually introduced. These are databases that can create predictive analytic ‘what is likely to work best’ information that will support the decision-making of individuals and service providers as they make specific choices in the use of different types of social intervention.

Box 4 describes the inputs to the mature system, i.e., the sources of incoming data.

• There will be much heavier reliance on administrative data sources.
• Major changes will occur in the use of traditional survey and census data, particularly the use of hybrid vehicles that are designed from the outset to gather data from both survey and administrative sources.
• There will be detailed descriptions of social programs and services that can be used at the micro-level of particular interventions.
• Information will be captured about the economic, environmental, and social characteristics of the geographic areas in which the individual resides, and that information will be incorporated into individual micro-records.
• Eventually, feedback information from the new ‘what is likely to work best’ uses will provide information that will be especially useful in causal analysis of the effects of social programming.

Box 5 describes the processes that convert the inputs to outputs.

• These will make use of the sophisticated data processing and analytic tools that are now available, including techniques for data linkage that ensure privacy protection.
• A major addition will be an explicit conceptual framework to describe and integrate individual data items independent of their source. This will be an empirically-based micro-level framework, quite different from, but consistent with, the macro-level System of National Accounts that provides integration in the area of economic statistics, or other macro accounts in the environmental and social areas such as those related to health or education. As noted, a separate paper is being drafted that describes the conceptual framework.

1.3. Managing constraints and risks

Chapters 2 to 5 will elaborate on the systematic description of the future system as set out in Section 1.2 and in the boxes at the end of this chapter.

The final Chapter 6 will address the main constraints and risks that will be faced during implementation and how they can be managed.

The largest resource constraint is the relatively small number of people in both Statistics Canada and in the user community with the skills needed to make full use of the potential of systems where data is drawn from multiple sources.

The largest risk would be a failure to maintain and reinforce the trust and confidence of the users and suppliers of statistics as we move to quite new ways of doing business.

1.4. Assumptions about an uncertain future and the need for flexible planning

In the real world, the statistical system will never be mature. It will always continue to evolve and will always be messier than some hypothetical model used in planning exercises. And, of course, the future cannot be predicted with any certainty, as the COVID experience has so clearly demonstrated. For this reason, an ongoing annual planning process is envisioned. In addition to up-dated statements about longer-term directions, the planning process will also include concrete action plans covering the following three years.

This paper is a draft of the long-term directions as they might appear in the 2022 version of such a strategic plan.  It examines the system as it might exist about five or ten years into the future based on a continuation of existing reform initiatives. Given that time frame, this 2022 version assumes:

• No large organizational or governance changes will be made that would affect the operation of the system of social statistics. That could change in the future and subsequent plans will be adjusted accordingly.
• Only currently available technologies and readily available data sources are referred to in this version, although exploration and developmental work on possible future technologies and data sources will continue to be examined. However, it is likely that future versions will reflect major changes that are on the horizon.
  • For example, it is easy to imagine a future where a main source of social statistics will be portable personal devices (that combine health monitoring and GPS features) that would allow an individual to voluntarily provide anonymized information to the statistical system about their activities, contacts with others, and physical and mental wellbeing as the individual goes about the business of daily living.
  • Steps along the route of asking people to use devices to share information voluntarily have already been taken. However, there is too much uncertainty for a planning exercise to assume that this will become a mainline way of collecting data within the next several years.

• The interventions/choices that are available (content coming from the service providers).
• Which interventions/choices are likely to work best in particular cases (content coming from the statistical system).
• The same kind of ‘what is likely to work best’ information could also be provided directly to individual Canadians to help them in their decision-making.

It is included in the 2022 plan because:

• The payoffs from early action would be high on many fronts. The eventual result would be a giant step towards social policies that are truly based on evidence. As well, many more people, including individual citizens, would gain directly from such applications, increasing overall trust in the system.
• There are risks of moving in this direction, but they are manageable. Implementation would be gradual, based on pilots. Accordingly, failure on this front would not have a large, negative impact on the rest of system.

2. Outcomes: the purpose and scope of the evolving system of quality-of-life statistics

Section 2.1 discusses the first of the two outcomes identified in the logic chart in Figure 1 – supporting policies and programs.

Section 2.2 provides a similar description of the objective of supporting decision-making.

Section 2.3 proposes a possible statement of vision and mission.

2.1. Supporting policies and programs

Canada has always had a reputation for world-leading excellence in statistics. When it comes to its main traditional goal of supporting social policies and programs, the agency can build on strength.

However, there is also a long way to go to fully respond to the policy demands described in Chapter 1. This can be seen in the sidebar which provides an expanded version of the outcome that is summarized in Figure 1. While this wording is illustrative only, it captures much recent thinking in Canada and internationally.

The extended statement suggests that the mature system should be designed to improve on existing arrangements in the following areas:

• Monitoring. No capacity has been set up to monitor the various dimensions of social wellbeing in a way that is current, comparative, and authoritative. Information about inequality and sustainability risks is not as developed as are indicators of trends and levels.

• Cause and effect. The system is improving but is still weak in measuring the interplay among the multiple factors that affect wellbeing in different population groups – such as combinations of poor jobs, health, skills, living alone and housing.
• Effects of programs. Much analysis of the social effects of policies is weak since it ignores the obvious, namely that there is seldom a one-to-one relationship between a single government program (or the services provided by social institutions such as schools, hospitals, and community organizations) on an individual’s wellbeing.
  • There is good data on the direct effects of programs that provide income.
  • There is inconsistent data on the effects of programs that provide services.
  • In consequence there is virtually no information on the combined effects of income and service programs from all levels of government on the lives of individuals.

The consequences for the design of the new multi-source system are shown in the second sidebar above.

2.2. Supporting public and private decision-making

The second purpose or outcome of the statistical system is to support better decisions by a wide range of actors, public and private.

Public decision-making

The improvement in public decision-making will occur because of:

• The policy and program uses discussed above, including the use of stronger ‘what if scenarios’.
• The provision of ‘what is likely to work best’ information directly to those who deliver services within the public system. This is discussed below.

Private decision-making: by organizations

Currently, most direct uses by organizations relate to marketing. Demographic, socio-economic and health data often support the marketing functions of firms and of institutions that provide health, education, and other social services. National social statistics provide a kind of common denominator that describes the needs and characteristics of potential recipients of the products and services of those organizations. Projections of key demographic variables are particularly useful in planning initiatives in both public and private organizations.

• Often this type of information is provided by private market research firms who combine data from the national statistical system with data obtained from other sources.
• The shift to a multi-source model in the national system will provide even better ‘denominator’ statistics because of their greater granularity.

Private decision-making: by academics and researchers

Much academic research is, of course, directly or indirectly, related to the policy and program uses above. As well, a wide range of academic social research, especially research that examines the longitudinal dimensions of life, will benefit from the increased granularity and integration of the statistics produced by the multi-source system. The addition of lifecourse concepts to the micro-level conceptual framework will facilitate mixed qualitative and quantitative analysis. Many new tools of access and analysis are being developed to support external researchers (in the research data centres and through a range of virtual tools including the Data Analytics as a Service Platform initiative) and authorized researchers in other departments (e.g., Virtual Data Labs).

Private decision-making by individuals

Currently, most uses by individual Canadians are indirect. Some individual Canadians may consult the Statistics Canada website to find background information related to personal decisions. However, such direct uses are almost certainly small when compared with indirect benefits from the policy-related statistics described above. These uses will result in better policies and programs, and a more informed media. This, in turn, will have positive important effects in improving people’s lives and in the operation of institutions.

Direct uses by individuals are likely to increase. The importance of directly supporting individuals has been well recognised. For example, as seen in the sidebar, Statistics Canada’s current Modernisation Initiative highlights the importance of supporting quality decision-making by citizens.

However, the reality is that a statistical system based on single-source statistics has, in practice, provided little information that is of direct and immediate use in helping individuals make decisions.

In a multi-source system, on the other hand, a wealth of information is available to support a wide range of individual decisions using predictive analytic tools. Although this will require much development work, over the long-term, these uses could become a central outcome of the system of social statistics.

As elaborated in Box 6, this shift to serving Canadians directly responds to a long-standing aspiration in most areas of social policy, but one where little progress has been possible because of a lack of supporting statistical data.

The largest long-run payoffs will be from ‘what is likely to work best’ data. The shift to a multi-source system will make it possible to provide individual citizens and service providers with real-time information about which type of social or health intervention is likely to work best, based on what has worked best in similar circumstances in the past.

2.3. A statement of vision and mission – and a new title

Developing a statement about the purposes and scope for the multi-source system of quality-of-life statistics could help coordinate implementation activities. It could be useful in building understanding among users and partners.

A draft of such a statement of vision and mission is shown in Box 7. The statement is illustrative only and intended for planning purposes. A final version would be based on consultations and expressed with greater elegance.

Below the statement, the box provides an explanation of the factors that were taken into account in its drafting, including thoughts on the scope of Statistics Canada’s mandate in the social statistics area and the rationale for replacing the phrase social statistics with quality-of-life statistics.

3. Outputs: statistical products

This chapter deals with outputs – the statistical products of the proposed national system of quality-of-life statistics – that were identified in Figure 1.

• Section 3.1 describes the proposed interactive web site or portal, including indicators and dashboards. It discusses progress to date and the need for additional action.
• Section 3.2 describes progress and next steps related to access to aggregated data.
• Section 3.3 describes progress and further work related to access to microdata.
• Section 3.4 deals with the partnered databases that will need to be constructed to produce ‘what is likely to work best’ information tailored to individual decision-making.

3.1. Progress in building the interactive web site

Box 3 described the Quality-of-Life information on the Statistics Canada website:

• Where all the new social statistics will be available on the day of release.
• Which contained high level quality-of-life indicators and flexible dashboards of supporting indicators that would be cast at a level that is relevant to practical policy agendas.
• That would provide ways of drilling down to detailed aggregated and micro data that will be described in Sections 3.2 and 3.3.

Rationale for releasing on a common website. Continuing the present practice of using a  common website for all releases of the latest social statistics, such as the latest Labour Force Survey data or the latest census tables, has several benefits:

• It allows those who draft the press releases and associated material to situate the current statistics in the context of other social changes that are taking place – including how the current data are related to the more general story being told by the social indicators.
• It allows the media and other readers easy access to related indicators, dashboards, and supporting tabular and micro data.
• Users accessing any statistics will always get the most recent figures.
• More generally, it reinforces the shift from the traditional model to the more responsive, multi-source model.

Rationale for including systematic social indicators and policy-relevant dashboards. Box 8 describes the continued interest in high level indicators that monitor progress, and point to possible problems, in achieving a high quality of life. In part, such systems are valued because they hold promise of compensating for the highly siloed nature of most current social statistics and of the policies they support. However, Box 8 indicates that indicators are not much used in practical policy-making and proposes a better approach based on the micro-level multi-source model.

Progress to date and the need for additional action

Many of the desired features of the proposed interactive web site already exist on the current Statistics Canada web site, which is becoming increasingly powerful.

• Current releases of new data are found in the Daily section of the web site. There is often useful commentary about the wider implications of the new information, although the focus remains on the information from the particular survey or other source of the statistics. Further work is needed to develop a capacity for releasing new data in the context of a broader set of quality-of-life indicators.
• Indicators. The list of key indicators on the Daily page of the website provides easy access to important single indicators as do the key indicators shown on other pages such as the hub related to gender, diversity, and inclusion statistics. However, future work is needed to develop a systematic set of quality-of-life indicators.
  • Development work, mandated by Ministerial mandate letters and referred to in Budget 2021, has been underway with the Department of Finance to explore the creation of such indicators in the context of policy formulation and budgeting.
  • No decision has yet been made on how, and if, to proceed. However, the collaboration that has taken place means that if such indicators are adopted, they will be measurable and can be included in the proposed interactive web site.
• Dashboards. There are already a handful of dashboards on the current website that have been carefully designed to shed light on specific policy and program agendas along the lines proposed in Box 8. These include several dashboards associated with the COVID response and the Dimensions of Poverty Hub, with its 12 indicators. Future work will involve consultations that are designed to greatly increase the number of such dashboards.
• Interactivity and visualizations. The site provides increasingly powerful tools for visualizing and mapping the data, including interactive tools, and more dynamic charts and thematic mapping tools are planned. In other words, the present rate of progress is excellent. Next steps should include a provision for users to retrieve their own tailor-made dashboards.

3.2. Access to tabulated data: progress to date and next steps

Recent improvements on the Statistics Canada website, with the shift from CANSIM to CODR were major steps in the direction of flexible access to aggregated data. In addition, a plan should be developed in consultation with users to add new tables that:

• Are based on data drawn from linked microdata files and including synthetic variables. For example, Statistics Canada’s researchers produce many valuable one-time reports based on a variety of current topics using linked data. Some contain tables that would be of much interest if they were updated regularly and included on the web site.
• Complement existing tables, which are based on sources such as a particular survey, with new tables that include tabulated data related to the same subject matter but drawn from different sources, such as employment and related labour market data drawn from different surveys and administrative files. This would be one dimension of the work in developing dashboards discussed above.
• Develop ways of explaining the quality and source of linked and synthetic data and of limitations on their use. (For example, some synthetic variables or data about synthetic individuals may have been created by using methods that preclude their use in regression analysis.)
• Develop processes to ensure that new data tables do not allow the possibility of residual disclosure.

3.3. Access to microdata: progress to date and next steps

There has been much progress in recent years in using microdata for analytic purposes and in creating powerful linked data bases for research purposes as seen in Box 9. More are available in the Research Data Centres including those that use administrative data to extend the life of past longitudinal surveys.

Today, microdata can be accessed from sites that are designed primarily to support research: The Research Data Centres, the Public Use Microdata File Collection, the Data Liberation Initiative. Microdata can be analysed at a distance through Real Time Remote Access or special requests. As noted, many tools are being developed to support researchers using microdata, including the Data Analytics as a Service Platform and the Virtual Data Labs.

However, the next step – using these rich sources to create standard multi-source databases that can be accessed through an interactive web site – is not on current planning agendas.

• Statistics Canada currently maintains one such output that is made public, the database of the Social Policy Simulation Database and Model, but it is only accessed in conjunction with its supporting microsimulation model.

A new product: standard multi-source microdata files

In a multi-source system based on microdata, an important new output will be a set of standard multi-source microdata files that can be used for variety of different purposes. Responsibility for creating these is currently seen as belonging to users of the statistical system, as opposed to being a major output of the statistical system itself. In a future multi-source model:

• Statistics Canada would, based on consultations and often working in partnership, create and itself take ownership for a growing set of user-friendly microdata bases that would be designed to meet the needs of groups of users with common analytic interests. They would be anonymized and fully documented, including information about quality and sources.

  A skills shortage rationale

  There is an inherent logic in the Statistical Agency itself taking responsibility for providing access to one of its most useful outputs. This is supported by considerations related to skill shortages.

  • There are shortages of staff with the needed technical skills – both inside the statistical agency and in the user community – to create, link and use complex, multi-source microdata files.
  • The needed skills must necessarily reside in Statistics Canada where the needed record linkage and metadata tools are developed and maintained.
  • If the user community has main responsibility for constructing, documenting, and sharing linked databases, they too will need to draw on the same limited supply of skills. Progress will be slow, despite the important and welcomed improvements that are underway to provide new tools of microdata access and analysis to external researchers.
  • If the statistical agency takes responsibility for their construction and documentation, working in partnership with external researchers, the total use of scarce resources among both the creators and users of the statistics will be minimized and the potential uses of the data will be maximized.
  • Skill shortages are related to the stage of development of a technology. The final chapter discusses broader implications.
• The inputs to these standard micro databases would be drawn from a variety of sources most appropriate to the topic at hand. Synthetic variables would be included if they met a specified minimum level of quality.
• Depending on the type of use that is envisaged, some of these databases could be based on real individuals using data masking techniques to protect privacy. Others would be based on synthetic individuals. Some might be longitudinal, others cross-sectional only.
• Only data that is consistent with existing, verifiable source data would be included. In other words, from the perspective of the user, the data would look like a traditional file from a survey or administrative source – only with a great many more variables.

A consultation strategy will need to be set up determine the content of these standard databases. Early examples might draw on the linked research files in Box 9 or those in the Research Data Centres. The work could be done in partnership with the researchers involved.

3.4. Partnered ‘what is likely to work best’ databases

Box 6 identified the large potential payoffs from developing predictive tools that could assist in individual decision-making. This would be a new area for Statistics Canada, and it will be important to be clear on how this kind of tool relates to other more traditional evidence-based approaches such as ‘what if’ scenarios and ‘what works’ initiatives.

• ‘What if’ scenarios also make projections based on microdata, sometimes using microsimulation modelling tools. However, these are designed for the examination of inputs and outputs at the level of programs and policies. They can show the effects of possible program design changes on groups of individuals with different characteristics. They can provide information on, for example, program costs that result from changes in the characteristics of program participants. Statistics Canada’s Social Policy Simulation  Database and Model is a good example in the area of income support programming.
• ‘What works’ initiatives are those that provide retrospective assessments of the effectiveness of a program or intervention such as an evaluation or a clinical trial that provides information on outcomes, as well as the relationship among inputs, outputs and outcomes. They are often rigorous, based on random control techniques. Each such assessment is, however, quite limited, in that it deals with only a single program intervention taken in isolation. Many often describe only average outcomes. Much progress has also been made in systematically examining many different assessments of this kind in order to draw conclusions about which kind of intervention is likely to work best in the future. Examples are the UK What Works Network, and the assessment made by non-governmental bodies such as the Cochrane Collaboration, the Campbell Collaboration and Arnold Ventures.
• ‘What is likely to work best’ initiatives that use individual-level predictive analytics of the sort being proposed here can also produce the system-level results found in the ‘What Works’ initiatives described above.

  The data to support ‘what is likely to work best’ in Labour Market Program interventions

  ESDC’s Labour Market Program Data Platform contains rich microdata from multiple sources about the socio-economic and labour market experience of the whole population and of those who participated in different types of employment and training interventions, both before and after the intervention – as well as information about the intervention itself. It can be used to measure the outcome associated with participation in different interventions, including the subsequent:

  • Probability of employment.
  • Employment earnings.
  • Weeks in receipt of EI benefits.
  • Amount of EI benefits received.
  • Amount of Social Assistance benefits
  • Proportion of EI use.
  • Dependence on income support.
  However, they go much further by also providing real time information to inform individual decisions. This is done by providing information on which type of intervention has the highest probability of success for a specific individual, based on past experience in similar circumstances. It is based on far richer longitudinal data than that used in traditional ‘what works’ analysis and can, for example, take into account the combined impact of different programs on individuals over the course of their lives. The sidebar describes data developed by Employment and Social Development Canada (ESDC) which can be used for this purpose.

Implementation strategy

In the next several years, a realistic goal might be to:

• Identify a number of areas where ‘what is likely to work best’ information directed to individuals and services providers is likely to have early payoffs.
• Establish partnership agreements to conduct pilot studies in several of these areas.

Recent work in measuring the subsequent success of post-secondary students in specific academic programs suggests that early pilots in this area might be warranted.

The area of active labour market programming (training, job counselling, subsidized work experience, etc.) would be an obvious area to consider in establishing pilots if partnerships could be arranged with ESDC and one or two provinces:

• The ESDC predecessor department successfully piloted this approach some 20 years ago where information about the likely success of different kinds of interventions was provided to staff making referrals, at the time the referral was being made.
• The initiative was dropped when these programs were transferred to the provinces, but the techniques and data sources remain intact and used in evaluations.
• That ESDC data, described in the sidebar above, could be potentially housed in Statistics Canada, and quickly made operational in pilot settings while protecting privacy and confidentiality.

While predictive analytics are widely used in many applications, the pilots would examine their role in official statistics, particularly in calculating and explaining the level of uncertainty that is inevitable in estimating probabilities of success at the level of particular individuals.

• At minimum, where there was much uncertainty associated with the calculated probabilities, the technique could nevertheless still provide useful contextual information to the individual or counsellor in question such as descriptions of the interventions that were open to the individual and the average success of each.
• At maximum, a reliable estimate could be made of which intervention had the highest probability of success for that particular individual.

4. Processing: transforming inputs to outputs

This chapter deals with progress and next steps in the processing section of the logic model shown in Figure 1 – how the incoming data is processed into outputs. All of the processing boxes in Figure 1 will be touched, directly or indirectly, by the shift to the multi-source system. However, the main changes will be in three functions:

• Section 4.1 describes the conceptual framework that provides standard descriptions of data items, independent of the source of that data.
• Section 4.2 describes how the input data from multiple sources is managed and processed in a way that ensures quality and protects privacy – the Figure 1 box entitled Maintaining internal databases to store and manipulate the data.
• Section 4.3 describes the approach to documenting the quality of actual data in the system, as found in the Figure 1 box entitled Maintaining the Data Documentation System.

4.1. The conceptual framework

The goal is to develop a standard way of defining all the data items that are contained in the source files and that will be used in producing statistical outputs.

• In many cases, there will be information about the same characteristics from many input sources. For example, employment data is collected from the Census, from the Labour Force Survey, from tax files, from employers (e.g., the Record of Employment).
• The data from these different sources will often differ, if only slightly, for reason of reporting error, or differences in the wording of the original question that was asked, or because the administrative records were created to support somewhat different purposes.
• There is therefore a need to identify the ‘correct’ or standard data that is independent of the source of the data and that sets out the concept that is ideally needed for purposes of social analysis.
• The standard description of data items must be part of a consistent conceptual framework that supports analysis of the interrelations among all the data that are found in the database, including the factors that cause, or are associated with, changes in these characteristics over time.
• There are two such conceptual frameworks, one related to individuals and the other related to programs and institutions that provide social services and supports.

Progress to date and the need for additional action

The existing Statistics Canada policy on standards calls for:

• The use of conceptual frameworks, such as the System of National Accounts, that provide a basis for consolidating statistical information about certain sectors or dimensions of the Canadian scene.
• The use of standard names and definitions for populations, statistical units, concepts, variables, and classifications in statistical programs.
• The use of consistent collection and processing methods for the production of statistical data across surveys.

What is yet to be developed is:

• A comprehensive dictionary of standard definitions and concepts of variables that exists independent of particular data sources, definitions that are explicitly linked to the common conceptual framework.
• An explicit conceptual framework that encompasses existing definitions but that can be extended over time to create powerful new ways of integrating all the micro data in the quality-of-life statistical system.

Separate paper on the conceptual framework. A paper is being drafted that will describe the conceptual framework. It will show:

• How the characteristics of individuals will be described using stock, flow and lifecourse concepts. These are consistent with existing definitions but also allow new kinds of analysis of the interrelations among people in their social, spatial, and temporal contexts.
• How the characteristics of social programs and services will be described based on an input-process-output-outcome logic model. This will allow analysis of the effect of these programs and services on the lives of those who participated in them.

4.2. Managing and processing the data

The shift to a multi-source system requires a new approach to the management of data. Data become assets that have value in themselves, available to be used in a variety of applications, independent of their source.

• The standard definitions referred to above are an important part of the response.
• Other key dimensions are summarized in Box 5, including the sophisticated tools and mechanisms that will be needed to manage the huge amount of overlapping data in the system and to make full use of its potential in the statistical products that are produced.

Progress to date and the need for additional action

On technical matters, there will be many challenges ahead, but Statistics Canada is well positioned to take them on:

• Data management strategy. Statistics Canada has already developed a strategy for independent management of data, covering topics such as privacy and trust, IT strategies, metadata strategies and outreach. The strategy will, of course, evolve during the course of the transition to a full multi-source system, but the changes are unlikely to be fundamental.
• Methodological and analytic tools. Again, much development will work be needed to fully develop and document the technical tools that will be needed to collect and analyse data of high quality in a multi-source world. However, Statistics Canada is already a world leader in the development and use of many of these tools and the modernization initiative has further strengthened its capacity for leadership in these technical areas.
• Data linkages. The data linkage lies at the centre of the shift to a multi-use system and here as well, Statistics Canada is already well advanced. As noted earlier, it has developed:
  • A world-leading framework for balancing use, privacy, and response burden (the Necessity and Proportionality Framework).
  • Sophisticated tools for linkages that ensure privacy protection by techniques that split out individual identifiers from the data that describe those individuals (the Social Data Linkage Environment).

There will, of course, be a need for more development, including new tools for creating the linked data needed to analyze the course of people’s lives in various social, spatial contexts. However, the need has been recognized, a solid start has been made, and future progress will likely follow pathways that have already been established.

4.3. Data documentation

Section 4.1 described the need for a standard set of concepts and definitions that is independent of data sources. There will also, of course, be a continuing need to document the actual data that comes into the system from various sources. That includes information on:

• The extent to which the actual data deviates from the standard definition.
• The source of the data and the methods used in its creation.
• Its quality, which Statistics Canada defines in terms of the multiple dimensions of the ‘fitness for use’ of the data in question.

Progress to date and the need for additional action

Existing systems of documenting data (known in statistical circles as metadata) are strong and being steadily improved.

• Statistics Canada is developing a new platform that will provide seamless access to all its internal data and related metadata descriptions.
• Sophisticated ways of accessing metadata have been developed to support the increasing use of microdata, including remote access.
• Work is ongoing on several fronts to develop standards and documentation tools that will apply to all the data holdings within the federal government – as part of the broader strategy to increase data flows across the whole system.
• Work is ongoing at the international level to build comparability in the approaches to metadata used by the statistical systems of different countries.

Statistics Canada is therefore well placed technically to meet the new documentation challenges that will arise in a multi-source statistical world.

• User-friendly ways will need to document sources and ‘fitness for use’ in tables or microdata files that were created from multiple sources. New approaches may be needed to provide simple documentation related to synthetic data. Traditional footnotes to tables or separate documentation packages may no longer be sufficient.
• At a later stage, when probabilistic ‘what is likely to work best’ statistics are produced for direct use by individual Canadians, new approaches may be needed to explain, in lay language, how users should interpret those probabilities in making their own decisions. As discussed in Section 3.4, there is always uncertainty associated with causal analysis and simple ways must be found to explain the extent of this uncertainty. This may require tailormade explanations that are generated during the course of each use.

5. Inputs: the multiple sources of data

The chapter deals with the ‘input’ set of boxes in Figure 1 that describe the incoming data. The Governance, outreach and advice from experts box is discussed in Chapter 6.

• Section 5.1 describes the increased responsiveness of a multi-source system to demands for new statistics.
• Section 5.2 describes response burden and the growth of hybrid collection vehicles.
• Section 5.3 describes approaches to accessing new administrative data.
• Section 5.4 describes ways of maximizing incoming data based on geography.
• Section 5.5 describes inputs that describe social programs and social services.

5.1. Responsiveness to requests for new statistics

In the past, requests for new data (for example to meet a new policy priority), would typically be cast in terms of requests for statistics from a particular collection vehicle. Often the request would wind up in the in-baskets of staff responsible for administering a particular survey or administrative data set. If the needed data did not exist, and if the priority for obtaining it were high enough (often as demonstrated by the willingness to pay for the collection of new data), a new survey would be designed, or additional questions added to existing surveys such as the General Social Survey.

That process did work and produced important new information. However, it was typically slow and cumbersome, often taking years before the required data could be obtained.

In a mature, multi-source system, the interactive web site discussed in Chapter 3 will allow much easier access by users to existing statistical information regardless of source. Compared with today, many more requests will be able to be filled from existing data, including the expanded CODR tables and standard multi-source microdata files.

When the needed statistics are not found on the website, new approaches are being developed that will greatly improve responsiveness.

• Changes have already been made such that expertise in subject matter content and in the administration of particular collection vehicles will be located in separate organizational groups. The subject-matter content organizations are being designed to support more pro-active outreach – to better anticipate the need for new statistics and to work with clients in formulating those needs.
• Requests for data would be initially framed in terms of the content of the needed information, without reference to its source. Tools are being developed such that the first response to the request would be a rigorous search of existing data holdings, including information that could be obtained from the linkage of existing administrative and other data files.
• In order to respond to situations where new collection is still needed, new survey instruments are being put in place to maximize responsiveness. They are being designed to be flexible and fast:
  • Already in place are web panel surveys conducted as needed (currently every other month), to deal with hot topics with very quick responses. It is called Canadian Perspectives Surveys and asks a representative sample of individuals to voluntarily respond to online questions. It was piloted in 2020 and provided much important information related to COVID-19.
  • A new Omnibus Survey is under development. It will be a quarterly survey with 20,000 respondents. About a quarter of the questions will provide standard core content, including socio-economic variables. The other questions will vary depending on priorities. Some will cover social topics that are already collected in General Social Survey, but only at 5-to-7-year intervals.
• If the need cannot be met by data linkages or by these new survey vehicles, then less conventional sources will be explored. For example, Statistics Canada is currently piloting a Well-Being App where a sample of individual are asked to download a cell phone app and use it to report on their subjective feelings of well-being several times a day, along with information on what they were doing at those particular times. The results are linked to administrative tax data to provide socio-economic background information.
• If the need is truly urgent and the information cannot be collected by the representative survey instruments that are within Statistics Canada’s normal mission, non-representative means such as crowdsourcing can be used, as it was to collect information related to cannabis use and the impact of COVID-19.

5.2. Response burden and hybrid collection vehicles

Non-response rates are increasing in surveys and response burden is a serious concern. This is one of the key factors behind the increasing use of administrative records whenever possible. Longitudinal surveys have been at special risk as non-response increases during successive waves of the survey.

Hybrid collection vehicles

Part of the solution has been to replace some questions that were once asked on surveys with data obtained from administrative sources.

• For example, the census and surveys such as the Survey of Household Spending now use tax records as the source of income data rather than asking respondents to provide this information.
• The Longitudinal and International Survey of Adults draws on tax, pension, and immigration files to complement the information provided directly be respondents.

This use of administrative data to replace survey questions both reduces response burden and increases quality, since often the administrative data is more accurate than that which respondents to surveys can recall.

It is therefore likely that future multi-source systems will make much greater use of what can be called hybrid vehicles that are designed from the outset to draw on both administrative and survey data. These hybrids can draw on the strengths of both sources and will often be much more powerful than either source taken individually. As indicated in the sidebar, this opens up the possibility of more flexible survey designs.

Will response burden really diminish?

There may seem to be a conflict between the desire to reduce the burden on individual respondents to surveys and censuses and the objective of collecting much more detailed information. What is actually happening is more of a rebalancing:

• Data from individual respondents will shrink as a percentage of all data collected, given the much greater reliance on administrative data. This is particularly true for longitudinal surveys.
• As well, the data management techniques discussed in Box 2 mean that individuals will not always have to enter the same information many times in different administrative and survey applications, as seen in the previous discussion of hybrid vehicles.
• The ‘what is likely to work best’ applications, when these are developed, will require participants to provide considerably more information. However, this will be entirely voluntary and the participants will receive major direct benefit from participating. That is, response burden in these applications will not be an issue.
• On the other hand, new surveys will continue to be mounted as indicated in the preceding section and some of these will ask for quite detailed and often sensitive information. For the individuals selected to participate in these surveys, there could well be an increase in the amount of data collected. Whether this is seen as burden will depend on the respondent’s trust in the system, including an appreciation of the value of the resulting data to society.  We return to this topic in Chapter 6.

5.3. Obtaining new administrative data

Major progress. In the traditional system, obtaining administrative data for statistical purposes has typically been difficult, even though the Statistics Act provides the agency with the power to access this information.

• From the data supplier’s perspective there were often technical and cultural barriers to sharing.
• From the perspective of those within the statistical system, obtaining access to the data was often seen as a sideline to the real work of conducting surveys, censuses and working with statistical summaries of administrative data that were compiled by the holders of those files.

As shown in Box 2 in the introductory chapter, we are entering a world where priority is placed on government-wide data management, on the use of digital technologies and on open government – a world where there will be easier access to administrative files for statistical purposes, including the opportunities to shape the content of those administrative files.

In practice, huge progress has already been made in the use of linked data from multiple sources as can be seen by the examples in Box 9.

Short-term action. Despite this success, we are only at the beginning of a journey.

• Much still needs to be done as the COVID-19 experience has illustrated. A lack of shared data proved to be a major problem that delayed responses in dealing with the pandemic.
• Even when agreement to share data exists, there is often a lengthy period before it arrives.

For those who are involved in the shift to a multi-source system of national quality-of-life statistics, future action must be taken in the context of the broader data management initiatives referred to in Box 2 and, perhaps, of political-level, post-pandemic discussions about better data sharing. Next steps to supplement thesebroader initiatives could include:

• Building consensus on the priority gaps that could be filled by administrative data. This would involve consultations with stakeholders and experts whose purpose would be to identify topics where the gains from accessing new administrative data would have high payoffs, especially when combined with data from other sources. Emphasis would be placed on how the resulting new statistical information would benefit those agencies that would provide the new administrative data.
• Developing a longer-term implementation plan. Based on these consultations, strategic plans could be developed for accessing the missing data, including timetables that would be realistic from the perspective of the suppliers of the administrative data. Existing governance arrangements in areas such as education or justice could oversee the development of these plans.

5.4. Incorporating geographic information in individual records

There has been much progress in Statistics Canada’s use of geomatics and spatial visualization tools and further strengthening is underway. Basically, these tools allow statistical information to be shown on a map of a particular neighbourhood, municipality, region, health regions, or province. This becomes a particularly powerful method of showing interrelationships when data from different sources and different subjects are shown on the map in question.

There are already many initiatives related to spatial analysis and there would appear to be potential gains from additional work that incorporates these elements into analysis at the level of individuals.

5.5. Inputs describing social programs and services

The shift to the mature multi-source model will eventually result in micro-level data about social programs and services being included in the statistical system as described in the box below.

Such micro-level data about social programs and services will allow analysis of the actual difference that particular social interventions made in the lives of individuals – the kind of information that is essential for:

• Shifting to the client-centric social programming described in Box 6.
• Introducing ‘what is likely to work best’ pilots described in Section 3.4.

As noted in the box, the needed micro data already exists in the case of income transfer programs. However little progress has been made for services:

• It does exist for Labour Market Programming as shown in Section 3.4.
• Much of the needed data exists in some areas of post-secondary education and apprenticeship.
• In health, some of preconditions are being put in place through means such as the digitization of health records. COVID-19 assessments may give a push to further action.

However, it will take time to develop a comprehensive approach to describing social services using the consistent micro-level descriptors that are discussed in the Annex.

This suggests a gradual approach to development in this area, similar to the approach for ‘What is likely to works best’ pilots. The first steps would be to identify areas of low-hanging fruit and find partners willing to develop and apply these descriptors in their area of expertise.

6. Managing constraints and risks

The future directions discussed in this paper will take Statistics Canada into some quite new territory, some of it world leading. This comes with some constraints and risks during the implementation process that must be carefully managed.

• Section 6.1 discusses resource constraints, particularly those related to skills shortages.
• Section 6.2 discusses ways of ensuring understanding and support in the user community.
• Section 6.3 discusses ways of maintaining and building trust and support among the public and those who provide the input data, including individuals who respond to surveys and the program administrators who provide administrative data.

6.1. Skill shortages

In terms of resource constraints, dollar funding can be largely managed by internal reallocations, many of which have already taken place, and by the new funds that were allocated in Budget 2021.

The agency’s IT plan will need to be adjusted in light of the implementation plan. Some of the newer technologies require a lot of computing power that may require considerable lead time to put in place. However, the technology itself is not new and is being used in other applications.

The big resource challenge will therefore likely be a shortage of people with the needed skills. As already discussed, creating and using multi-source microdata for statistical purposes requires individuals with skills sets that are in short supply.

The literature on the typical life cycle of technologies, summarized in Box 10, suggests we are still at early stages of development where the technology is complex and requires high skills levels to operate. Especially in the early stages of the transition period, the best allocation of scarce resources would be for the same team to be involved in the creation and use of complex multi-source data bases (and in related projects involving complex techniques such as microsimulation and predictive analytics).

Otherwise, staff with about the same skill set will be needed in both the producer and user communities. Or, to be more realistic given the shortage of these skills, there will not be adequate skills available for either community to do a good job.

In practice, this means that:

• Statistics Canada should take the lead in most such projects as it will typically have the greatest expertise. Statistics Canada also provides a secure and trusted location in which data linkage can take place without threats to privacy. And, as suggested earlier, it makes common sense from an accountability perspective that Statistics Canada itself should take ownership for standard multi-source micro databases especially those that will become a key product of the new system of national quality-of-life statistics.

• Statistics Canada should bring in partners from the user community and academia at the outset of development. Skills would be shared during implementation, and skill base in the user community would grow as a result of their experience during the development period.

Such a model may require some adjustment to existing approaches to data linkage. As noted earlier, some existing plans assume that Statistics Canada’s role is to provide users with the tools to develop high quality, anonymized multiple source statistics. The analysis here suggests that, in many cases, it should be Statistics Canada itself that plays the lead role, in partnership with others.

Box 10 suggests that, in addition to existing recruitment and training practices, there may be a need for a specific HR strategy to support a smooth transition to the skills that will be needed as the technology matures.

6.2. Building understanding and support in the user community

A main objective in shifting to a multi-source model is to better serve users. However, most users, partners and potential funders of social statistics will be unfamiliar with techniques that will be used and of the usefulness of the new data will be created.  Concepts such as multi-source microdata files, synthetic data, predictive analytics, and lifecourse analysis will be unfamiliar to many, even among experts. A risk therefore is that Statistics Canada could be seen as moving off in unknown directions, trying to meet new needs and to anticipate the needs of existing users rather than responding to filling existing gaps and being responsive to needs that have already been formulated.

That risk will be mitigated by the early development of the interactive web site which will provide speedier access to all existing information and by the new approach to responding to current requests for information, including the new suite of flexible survey instruments, that will produce results quickly. However, these steps should be supplemented by outreach mechanisms that will allow users to work in partnership with Statistics Canada in developing practical ways of applying the new statistical evidence that will be produced.

An emphasis on outreach. Much recent attention in Statistics Canada, including in the social area, has been devoted to the issues of horizontality, partnership, and outreach. New centres have been created with a view to increasing the capacity to respond quickly to changing demands for statistics and to anticipate new demands. This emphasis on outreach is important but needs to be carefully managed. To be effective, it should be guided by the following considerations:

• Outreach should not be based solely on traditional subject-matter policy silos such as health or education. This could result in cross-cutting topics being ignored, such as equality or wellbeing. Wellbeing in different population subgroups often results from combinations of interventions from different traditional policy silos.
• Many disciplines are involved in the development of the statistical system and in its uses – and they tend to use different conceptual frameworks and terminologies.

  Governance

  The multi-source system will be designed to provide integrated statistical information that cuts across existing policy silos. Existing governance structures in areas such health, education and justice statistics are based on those silos, reflecting the differing federal and provincial roles in various areas of social policy.

  • The shift to a multi-sources system does not, however, necessitate any change in the complex, asymmetrical governance structures that have emerged in these areas over time.
  • The parties in existing governance structures will gain important new information from a multi-source system. They will have at least as much a stake in its successful development as they do in the existing system.
  • The eventual addition of ‘what is likely to work best’ applications and of the inclusion of micro descriptors of social programs will require strong partnerships with those who have responsibility for existing social and health policies and programs. If the existing governance structures did not exist, something like them would have to be invented in order to oversee these partnerships.
  One of the goals of outreach should be to use the neutral territory of statistical planning to help build bridges across these communities.
• Given the often-long time horizons in statistical planning, outreach should be with those that have long-term view as well as those concerned with current policy needs.
• The risk of fragmented outreach grows if there are many small specialized organizational groups each having independent outreach strategies.
• Outreach and consultation take much time and use skilled resources. They should never be treated as ends in themselves. They should always be directed to a particular purpose.

A balanced outreach strategy to support the evolution of the multi-source system might therefore include the following strands:

• Dashboards and web-based products. Outreach directed to developing the new dashboards that will be on the interactive website. This would be an obvious responsibility of the subject-matter divisions of Statistics Canada responsible for social statistics. It could be associated with consultations on the new content of the CODR tables and the standard multi-source microdata files.
• Development of the conceptual framework and standard definitions described in the forthcoming paper entitled “The System of National Quality-of-Life Statistics: Conceptual Framework” should be accompanied by a consultative strategy that engages many groups inside Statistics Canada and in the academic and policy communities. Discussions on the concepts to be used in measurement are close to discussions about what should be measured. That is, the statistical system can, implicitly, provide a neutral space that can foster cross-silo consensus-building around longer-term directions in the content of social policy.
• Leading-edge analysis where advice and partnership are sought on the conduct of research on high priority policy research topics that often cross sectoral boundaries and that use multi-source data in innovative ways.

  Coordinating outreach?

  Different strands of outreach from different parts of the organisation are currently coordinated by informal means, such as telephone calls among colleagues. Will the new strands require a more formal means of coordination?

  The Analytic Studies Branch already has a workplan organized along these lines.
• Improving flows of incoming data, particularly the longer-term strategy related to administrative data referred to in Section 5.3.
• Parallel consultation strands with Indigenous peoples and with groups with distinctive needs. This could build on existing outreach initiatives related to gender, diversity, and inclusion -- including with immigrants, people with disabilities, racialized minorities as well as people in remote areas.These would focus on the information needed in these communities.
• An international outreach strand could be considered since the challenge of developing a multi-source system of national quality-of-life statistics is of world-wide interest, with the potential for sharing experience

An emphasis on partnership. Many of the outreach activities above will involve close partnerships with users. In addition, as described earlier, it is proposed that particularly close partnership relationships be established in those areas that are at the leading edge from a technical perspective:

• Partnerships with researchers and analysts in other departments, in academia and think tanks in building the standard multi-source microdata bases.
• Partnerships with agencies responsible for service delivery in developing ‘what is likely to work’ pilots.

6.3. Ethical issues: building trust

Any system that makes use of data about individuals from different sources will, properly, raise alarm bells related to privacy, consent, and confidentiality. These issues are not new. The question of getting the right balance between these concerns and those related to the value of the uses of official statistics and the problems of response burden have been the subject of in-depth discussions for many years in many countries.

The privacy-related issues are part of a wider set of ethical concerns relating to the social acceptance of the boundaries of official statistics and the role of the agencies that produce them. These include the independence and neutrality of the statistical office, the transparency of its operation, trust in the quality and relevance of the statistics that are produced, the accessibility of statistical outputs, and the efficient managing and protection of information within the system.

Canada, and its statistical agency, are well-positioned on all these ethical fronts:

• The independence and neutrality of Statistics Canada, and the quality of its work, have never been in doubt. Its independence was reinforced in recent amendments to the Statistics Act.
• As noted earlier, Statistics Canada has already put in place a world-leading framework (called the Necessity and Proportionality Framework) for balancing use, privacy, and response burden.
• Again, as noted earlier, sophisticated tools have already been developed that ensure privacy protection when drawing on data from multiple sources, using techniques that split out individual identifiers from the data that describe those individuals. Linked files of identifiable individuals will not be created.
• The activities of the Statistical agency are seen as part of government-wide initiatives, described in Box 2, that emphasize the effective management of information resources including the reduction of response burden, the full use of all data collected as well as privacy and confidentiality. 

The directions outlined in this paper are designed to further strengthen trust and acceptance in the population at large and among those respondents who provide data inputs:

• Most important will be the increasing usefulness of the statistics to those who also provide the input data.
  • This will be particularly important as ‘what is likely to work best’ evidence grows, where important statistical information will be directly provided to those who supply the input data, both program administrators and individual participants.
  • More generally, disaggregated statistics that recognise the diversity of the population will be seen to be more relevant and useful to those who provide the needed inputs, including survey participants.
• New emphasis will be placed on the role of Statistics Canada itself – a secure and neutral location – in producing statistics that are based on multiple sources of data, with built-in assurance of both quality and privacy protection, reducing past reliance on external users to carry out some these functions.
• The use of synthetic data will provide even greater anonymity of the resulting statistics, particularly in applications where the data are based on synthetic individuals created by statistical manipulation techniques.
• Implementation of initiatives where there are issues surrounding quality, such as the uncertainties associated with estimating probabilities of success at the level of individuals or the use of cloud-sourcing, will be conducted only on pilot or trial basis, until its quality is assured.

As implementation proceeds, a more immediate and transparent balance will be struck among response burden, usefulness and the protection of privacy – strengthening the implicit social contract that exists among those who provide, process and use data for statistical purposes.