Publications
Population Projections for Canada (2013 to 2063), Provinces and Territories (2013 to 2038): Technical Report on Methodology and Assumptions
- 91-620-X
- Main page
- Introduction
- Chapter 1: Statistics Canada's cohort-component population projection model
- Chapter 2: Opinion Survey on Future Demographic Trends
- Chapter 3: Projection of fertility
- Chapter 4: Projection of mortality
- Chapter 5: Projection of international immigration
- Chapter 6: Projection of emigration
- Chapter 7: Projection of non-permanent residents
- Chapter 8: Projection of interprovincial migration
- Acknowledgements
- More information
- PDF version
Chapter 2: Opinion Survey on Future Demographic Trends
Archived Content
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.
By Nora Bohnert
- Introduction
- Examples from other statistical agencies
- Survey objectives and design
- Method of dissemination
- Survey Results
- Conclusion
- Acknowledgements
- References
- Notes
Introduction
The Opinion Survey on Future Demographic Trends is a new tool developed by Statistics Canada to gather inputs from the community of experts in demography and population studies about their views on future demographic trends. Although Statistics Canada already utilized some mechanisms allowing the assumptions and methods to be externally commented and reviewed, the motivation for the implementation of this tool was the desire for greater input from experts through a more formal process, the results of which could improve the plausibility and credibility of projection assumptions.Note 1
There is, in fact, strong evidence of the benefits of combining the views of a large group of informed individuals: across a number of disciplines, it is found that simply averaging the responses of many individuals can reduce error by up to 15% to 20% (Silver 2012).Note 2 Moreover, consulting with experts in the field is an optimal way to increase exposure to the literature and developments in demography. As Keyfitz (1982) argued, user confidence in forecasts comes in part from the fact that it is expected that the demographers who carried them out are knowledgeable and abreast of the population literature.
Examples from other statistical agencies
The conception of the survey started with an examination of the approaches taken by other agencies in this regard. The design and scope of the surveys utilized by other statistical agencies were found to vary depending on the associated objectives.
The British Office for National Statistics (ONS), for example, has organized an expert advisory panel with which to consult on each update of their population projections since the 2004 edition (Shaw 2008). The panel is small in size, consisting of approximately 10 scholars. Panel members respond to a short questionnaire which asks them to provide future estimates of various demographic indicators, including 67% confidence intervals. Additionally, the ONS meets with the expert panel in order to have an informed discussion about the long-term assumptions, a summary of which is eventually published.Note 3 The role of the expert panel is strictly advisory and all final decisions on the projection assumptions remain with the ONS. The ONS note that the goal of the panel consultation is not to achieve consensus, but rather to hear the full range of views on the key assumptions. The expert advisory panel is now considered to be an integral part of the Office’s assumption-setting process.
France’s statistical agency, l’Institut National de la Statistique et des Études Économiques (INSEE), has also conducted an expert survey as part of their most recent population projections (Blanpain and Chardon 2009). INSEE surveyed 22 national and international demography experts. The questionnaire asked respondents to provide estimates for low, central and high hypotheses for each component to a confidence level of 90%, and also to comment on their chosen values, for instance, how quickly they expect those values to be obtained. Respondents were also asked to what degree they agreed with the agency’s previous projection assumptions, methodology and communication of results.
A final example, quite ambitious in scope, comes from the International Institute for Applied Systems Analysis (IIASA), based out of Austria. IIASA has recently concluded a large-scale expert survey using what the agency refers to as an ‘argument-based approach’ (Lutz 2009). The objective of the questionnaire is to assess the validity and importance of a series of arguments regarding future demographic trends. It is structured around a set of major ‘forces’ which could potentially shape future levels of fertility, mortality and migration. Within each force, respondents are asked whether they think a given argument is correct or incorrect (its validity), and, if the argument did prove to be correct, how much of an impact it would have on the demographic variable (its importance), and whether that impact would be positive or negative. Invitations to participate in this exercise were sent to members of major international population associations. Around 550 demography experts from around the world submitted responses. As of the time of writing of this report, IIASA is still in the process of analyzing the results of this exercise. A modified version of this argument-based questionnaire has also been utilized by the ONS in recent years as part of their expert panel advisory process (ONS 2011).
Survey objectives and design
The approaches to expert consultation reviewed above vary in many aspects: the number of experts consulted, the backgrounds of those experts, the intensity of expert involvement as well as the topics on which experts are expected to provide input. It was therefore necessary to elaborate which objectives were most important, in terms of expert consultation, for the purposes of Statistics Canada’s National Population Projections Program. The key objectives were determined to be:
- Obtaining a range of views from Canadian demographers regarding future levels of fertility, mortality and immigration at the Canada level:
- Given that the population projections always contain several different projection scenarios (as per the agency’s Policy on Estimates with Future Reference Dates), it was considered important to obtain a variety of views from a large number of experts. In other words, arriving at a consensus of views was not the objective of this process but rather to survey the range of viewpoints held by professionals working in the field of demography.
- Notably, the core components of fertility, mortality and immigration were the focus of the questionnaire, even though there are other components of varying importance which factor into the population projections, including non-permanent residents, emigration, return emigration and interprovincial migration.Note 4 In addition to being the major components of population change, they are the components that are the most well-known and studied by external scholars.
- Obtaining both quantitative estimates and qualitative comments from respondents:
- It was considered desirable to obtain not only quantitative estimates of future demographic indicators, but perhaps more importantly, a justification from respondents as to what factors and trends they considered when forming a given estimate. Emphasis was placed on giving respondents the option to elaborate on the trends and factors that they consider most influential, in an open-ended manner and in as much detail as required. With this open-ended comment approach, respondents could also provide feedback on other aspects of the projections not directly addressed in the survey (methods, presentation of findings, other projection components such as emigration, for example) and on the survey itself, opening up the possibility of obtaining useful suggestions to improve future editions of the projections.
- Relatively low burden for respondents:
- It was determined that a relatively short questionnaire, followed by a review and discussion of the survey findings with Statistics Canada’s External Advisory Committee on Demographic Statistics and Studies,Note 5 would best meet the timelines and resources dedicated to the project while encouraging the participation of invitees by minimizing respondent burden. Thus, the survey focused on the major components of population change at the national level.
- Anonymity and confidentiality of responses:
- While not an official Statistics Canada survey, in following the policies of the Statistics Act, responses to the survey were kept strictly confidential and no information has been released that could identify individual respondents.
Survey design
Following internal testing of a preliminary version of the survey, a revised survey was finalized in March 2013. The introductory section of the survey asked respondents to provide basic information regarding their years of experience in the area of demography/population studies generally as well as self-rated expertise in fertility, mortality and migration. The main part of the survey consisted of three sections regarding fertility, mortality and immigration, respectively. In each section, a series of tables and figures demonstrating historical trends were provided for selected indicators (for example, the total fertility rate in Canada from 1921 to 2010). Respondents were asked to provide estimates of the most probable level of a given indicator 5 years and 25 years into the future (2018 and 2038),Note 6 representing short and long-term future outlooks. Additionally, respondents were asked to provide the range of the same indicator in both years that would encompass approximately 80% of possible future trends. At the end of each section, an open-ended comment box was provided for respondents to elaborate on the trends or factors considered when deciding upon the given future estimate. At the end of the questionnaire, there were two additional options for respondents to provide comments regarding a) the other projection components (emigration, non-permanent residents and interprovincial migration); and b) the production of population projections generally and the questionnaire itself.
Quantitative future estimates were asked only at the Canada level in the survey in order to keep the questionnaire to a manageable length; however, in each section, respondents were encouraged to provide any additional comments and views regarding the component at the provincial or territorial level.
Method of dissemination
In order to reach as many Canadian demographers as possible, the Demography Division worked with the executive councils of Canada’s two demography associations—the Canadian Population Society (CPS) and l'Association des démographes du Québec (ADQ)— to facilitate the dissemination of the survey to association members. In the case of l’ADQ, an invitation to participate in the survey was included in the association’s electronic bulletin, while the CPS president sent an email directly to CPS members containing the invitation to participate in the survey.
In order to facilitate ease of response for both the quantitative and open-ended qualitative sections of the questionnaire, while also allowing respondents to complete the survey electronically or by hand, the questionnaire was provided in Adobe PDF format. Respondents submitted the completed survey by sending it either through email or regular mail to the population projections team.
The invitation to participate in the survey was sent to association members in mid-March 2013. Respondents were given a period of four weeks during which completed surveys could be submitted.
Survey results
In total, 21 persons responded to at least some part of the survey. This relatively small number of respondents, considering the membership sizes of the two associations, likely reflects the fact that population projections is a fairly specialized area of study within the field of demography. Those persons that did participate were most likely those that felt confident that they were able to provide informed future estimates of various demographic indicators. This notion is confirmed by the high years of experience of the majority of respondents (Figure 2.1).
As seen in Figure 2.1, more than half of respondents (62%) had 25 or more years of experience in the field. It also appears that those who participated had high levels of expertise in population projections and in at least one of the sub-components of fertility, mortality and migration (Figure 2.2).
Despite attempts to make the survey as concise as possible and allowing respondents the option of skipping sections they felt less qualified to respond to, response rates generally declined the further one went into each section of the questionnaire, and the further in general into the questionnaire. Furthermore, more respondents provided a ‘most probable’ estimate than those who provided a corresponding range of low and high estimates. Finally, respondents were slightly more likely to provide estimates for 2018 than for 2038.
A summary of the responses to the survey are reviewed in the respective sections of the technical report, with figures displaying the median values and the overall spread of future estimates of the various demographic indicators.
Conclusion
This first venture into a formal surveying of Canadian demographers proved to be a very fruitful exercise for the National Projections Program. The persons that elected to respond to the survey held high levels of experience in the field of demography as well as expertise related to population projections and the major components of population change. The well-balanced quantitative estimates provided by respondents as well as the considerable quality and depth of the comments which accompanied estimates can attest to this fact.
The quantitative estimates received in the survey have proven extremely valuable in the assumption-building process. Equally important were the many arguments, methodologies and other evidence that were introduced to the population projections team via the open-ended comment sections of the survey. In many cases, this new information resulted in a change in approach which has undoubtedly strengthened the projection assumptions.
Feedback received from respondents indicated that the opinion survey was a welcome initiative, and there was considerable interest expressed in seeing the results of the survey. Many respondents provided comments not only related to the survey questions but also general feedback on the National Projections Program (for example, possible ways to improve access and presentation of results) and the survey itself. All of these useful suggestions have been noted and will be assessed for feasibility of implementation for the next edition of population projections.
Owing to the participation of many experts in the Canadian demography community, the survey resulted in much rich information that supports and strengthens the projection assumptions. Given the success of the pilot survey, it is anticipated that a survey of Canadian demographers will continue in some form for future editions of the projections.
Acknowledgements
The Demography Division of Statistics Canada would like to thank each of the respondents who participated in the inaugural Opinion Survey on Future Demographic Trends. We would also like to thank the executive council members of Canada’s two demography associations, the Canadian Population Society and l’Association des démographes du Québec, for their cooperation and assistance in disseminating the first edition of the Opinion Survey on Future Demographic Trends. Finally, we would like to acknowledge and thank the members of Statistics Canada’s External Advisory Committee on Demographic Statistics and Studies for their useful comments and input at various stages of the development of the survey.
References
Blanpain, N. and O. Chardon. 2009. Projections de population 2007-2060 pour la France métropolitaine: méthode et principaux résultats, Institut national de la statistique et des études économiques, Série des documents de travail, no. F1008.
Keyfitz, N. 1982. “Can knowledge improve forecasts?”, Population and Development Review, volume 8, number 44, pages 729 to 751.
Lutz, W. 2009. "Toward a Systematic, Argument-Based Approach to Defining Assumptions for Population Projections", Interim Report IR-09-037, International Institute for Applied Systems Analysis.
Office for National Statistics. 2007. "National Population Projections Expert Advisory Group, notes of meeting held on March 16, 2007".
Office for National Statistics. 2009. "2008-based National Population Projections, Chapter 11, National population projections expert advisory panel".
Office for National Statistics. 2009. "National Population Projections Expert Advisory Panel, notes of meeting held on April 1, 2009".
Office for National Statistics. 2011. “Background and Methodology”, 2010-Based National Population Projections, October 26.
Office for National Statistics. 2012. “Chapter 1: Background and Methodology”, 2010-based NPP Reference Volume, March 29.
Prommer, I. and C. Wilson. 2006. "Summary of the questionnaires on the use of expert opinions in assumption making for population projections", Bridging the micro-macro gap in population forecasting, Micmac contract no. SP23-CT-2005-006637, Work Package 3, Deliverable D14.
Shaw, C. 2008. “The National Population Projections Expert Advisory Group: Results from a questionnaire about future trends in fertility, mortality and migration”, Population Trends, volume 134, pages 42 to 54.
Silver, N. 2012. The Signal and the Noise: Why So Many Predictions Fail but Some Don’t, Penguin Press, New York, 534 pages.
Notes
- Date modified: