Other content related to Statistical methods

Skip to filters. View results.

Results

All (166)

All (166) (20 to 30 of 166 results)

21. Measuring Social Capital at the Neighbourhood Level: Experimental Estimates of Sense of Belonging to the Local Community Measured at the Census Tract Level
Articles and reports: 11-633-X2021007
Description:
Statistics Canada continues to use a variety of data sources to provide neighbourhood-level variables across an expanding set of domains, such as sociodemographic characteristics, income, services and amenities, crime, and the environment. Yet, despite these advances, information on the social aspects of neighbourhoods is still unavailable. In this paper, answers to the Canadian Community Health Survey on respondents’ sense of belonging to their local community were pooled over the four survey years from 2016 to 2019. Individual responses were aggregated up to the census tract (CT) level.
Release date: 2021-11-16
22. Construction of a Northern Market Basket Measure of poverty for Yukon and the Northwest Territories
Articles and reports: 75F0002M2021007
Description:
This discussion paper describes the proposed methodology for a Northern Market Basket Measure (MBM-N) for Yukon and the Northwest Territories, as well as identifies research which could be conducted in preparation for the 2023 review. The paper presents initial MBM-N thresholds and provides preliminary poverty estimates for reference years 2018 and 2019. A review period will follow the release of this paper, during which time Statistics Canada and Employment and Social Development Canada will welcome feedback from interested parties and work with experts, stakeholders, indigenous organizations, federal, provincial and territorial officials to validate the results.
Release date: 2021-11-12
23. Integrating Machine Learning into Coding of the 2021 Canadian Census Using fastText Archived
Articles and reports: 11-522-X202100100010
Description:
As part of processing for the 2021 Canadian Census, the write-in responses to 31 census questions must be coded. Up until, and including, 2016, this was a three stage process, including an “interactive (human) coding” step as the second stage. This human coding step is both lengthy and expensive, spanning many months and requiring the hiring and training of a large number of temporary employees. With this in mind, for 2021, this stage was either augmented with or replaced entirely by machine learning models using the "fastText" algorithm. This presentation will discuss the implementation of this algorithm and the challenges and decisions taken along the way.

Key Words: Natural Language Processing, Machine Learning, fastText, Coding

Release date: 2021-11-05
24. Statistics Netherlands and AI Archived
Articles and reports: 11-522-X202100100011
Description: The ways in which AI may affect the world of official statistics are manifold and Statistics Netherlands (CBS) is actively exploring how it can use AI within its societal role. The paper describes a number of AI-related areas where CBS is currently active: use of AI for its own statistics production and statistical R&D, the development of a national AI monitor, the support of other government bodies with expertise on fair data and fair algorithms, data sharing under safe and secure conditions, and engaging in AI-related collaborations.
Key Words: Artificial Intelligence; Official Statistics; Data Sharing; Fair Algorithms; AI monitoring; Collaboration.
Release date: 2021-11-05
25. Machine Learning Classifier Evaluation Criteria: application to price statistics Archived
Articles and reports: 11-522-X202100100012
Description: The modernization of price statistics by National Statistical Offices (NSO) such as Statistics Canada focuses on the adoption of alternative data sources that include the near-universe of all products sold in the country, a scale that requires machine learning classification of the data. The process of evaluating classifiers to select appropriate ones for production, as well as monitoring classifiers once in production, needs to be based on robust metrics to measure misclassification. As commonly utilized metrics, such as the Fß-score may not take into account key aspects applicable to prices statistics in all cases, such as unequal importance of categories, a careful consideration of the metric space is necessary to select appropriate methods to evaluate classifiers. This working paper provides insight on the metric space applicable to price statistics and proposes an operational framework to evaluate and monitor classifiers, focusing specifically on the needs of the Canadian Consumer Prices Index and demonstrating discussed metrics using a publicly available dataset.

Key Words: Consumer price index; supervised classification; evaluation metrics; taxonomy
Release date: 2021-11-05
26. Need for Speed: Using fastText (Machine Learning) to Code the Labour Force Survey Archived
Articles and reports: 11-522-X202100100013
Description: Statistics Canada’s Labour Force Survey (LFS) plays a fundamental role in the mandate of Statistics Canada. The labour market information provided by the LFS is among the most timely and important measures of the Canadian economy’s overall performance. An integral part of the LFS monthly data processing is the coding of respondent’s industry according to the North American Industrial Classification System (NAICS), occupation according to the National Occupational Classification System (NOC) and the Primary Class of Workers (PCOW). Each month, up to 20,000 records are coded manually. In 2020, Statistics Canada worked on developing Machine Learning models using fastText to code responses to the LFS questionnaire according to the three classifications mentioned previously. This article will provide an overview on the methodology developed and results obtained from a potential application of the use of fastText into the LFS coding process.
Key Words: Machine Learning; Labour Force Survey; Text classification; fastText.
Release date: 2021-11-05
27. Collaborative Data Science within Government of Canada: Development of R libraries for common tasks with Open Canada data Archived
Articles and reports: 11-522-X202100100028
Description:
Many Government of Canada groups are developing codes to process and visualize various kinds data, often duplicating each other’s efforts, with sub-optimal efficiency and limited level of code quality reviewing. This paper informally presents a working-level approach to addressing this technical problem. The idea is to collaboratively build a common repository of code and knowledgebase for use by anyone in the public sector to perform many common data science tasks, and, in doing that, help each other to master both the data science coding skills and the industry standard collaborative practices. The paper explains why R language is used as the language of choice for collaborative data science code development. It summaries R advantages and addresses its limitations, establishes the taxonomy of discussion topics of highest interested to the GC data scientists working with R, provides an overview of used collaborative platforms, and presents the results obtained to date. Even though the code knowledgebase is developed mainly in R, it is meant to be valuable also for data scientists coding in Python and other development environments. Key Words: Collaboration; Data science; Data Engineering; R; Open Government; Open Data; Open Science

Release date: 2021-10-29
28. Survey Data Integration for Regression Analysis Using Model Calibration Archived
Articles and reports: 11-522-X202100100001
Description:
We consider regression analysis in the context of data integration. To combine partial information from external sources, we employ the idea of model calibration which introduces a “working” reduced model based on the observed covariates. The working reduced model is not necessarily correctly specified but can be a useful device to incorporate the partial information from the external data. The actual implementation is based on a novel application of the empirical likelihood method. The proposed method is particularly attractive for combining information from several sources with different missing patterns. The proposed method is applied to a real data example combining survey data from Korean National Health and Nutrition Examination Survey and big data from National Health Insurance Sharing Service in Korea.
Key Words: Big data; Empirical likelihood; Measurement error models; Missing covariates.

Release date: 2021-10-15
29. Responsible use of machine learning at Statistics Canada Archived
Articles and reports: 11-522-X202100100002
Description:
A framework for the responsible use of machine learning processes has been developed at Statistics Canada. The framework includes guidelines for the responsible use of machine learning and a checklist, which are organized into four themes: respect for people, respect for data, sound methods, and sound application. All four themes work together to ensure the ethical use of both the algorithms and results of machine learning. The framework is anchored in a vision that seeks to create a modern workplace and provide direction and support to those who use machine learning techniques. It applies to all statistical programs and projects conducted by Statistics Canada that use machine learning algorithms. This includes supervised and unsupervised learning algorithms. The framework and associated guidelines will be presented first. The process of reviewing projects that use machine learning, i.e., how the framework is applied to Statistics Canada projects, will then be explained. Finally, future work to improve the framework will be described.
Keywords: Responsible machine learning, explainability, ethics

Release date: 2021-10-15
30. Predicting transitions into and out of poverty using machine learning Archived
Articles and reports: 11-522-X202100100003
Description:
The increasing size and richness of digital data allow for modeling more complex relationships and interactions, which is the strongpoint of machine learning. Here we applied gradient boosting to the Dutch system of social statistical datasets to estimate transition probabilities into and out of poverty. Individual estimates are reasonable, but the main advantages of the approach in combination with SHAP and global surrogate models are the simultaneous ranking of hundreds of features by their importance, detailed insight into their relationship with the transition probabilities, and the data-driven identification of subpopulations with relatively high and low transition probabilities. In addition, we decompose the difference in feature importance between general and subpopulation into a frequency and a feature effect. We caution for misinterpretation and discuss future directions.
Key Words: Classification; Explainability; Gradient boosting; Life event; Risk factors; SHAP decomposition.

Release date: 2021-10-15

Data (1)

Data (1) ((1 result))

1. National Population Health Survey Overview Archived
Table: 82-567-X
Description:
The National Population Health Survey (NPHS) is designed to enhance the understanding of the processes affecting health. The survey collects cross-sectional as well as longitudinal data. In 1994/95 the survey interviewed a panel of 17,276 individuals, then returned to interview them a second time in 1996/97. The response rate for these individuals was 96% in 1996/97. Data collection from the panel will continue for up to two decades. For cross-sectional purposes, data were collected for a total of 81,000 household residents in all provinces (except people on Indian reserves or on Canadian Forces bases) in 1996/97.
This overview illustrates the variety of information available by presenting data on perceived health, chronic conditions, injuries, repetitive strains, depression, smoking, alcohol consumption, physical activity, consultations with medical professionals, use of medications and use of alternative medicine.
Release date: 1998-07-29

Analysis (105)

Analysis (105) (0 to 10 of 105 results)

1. The Importance of Disaggregated Data: An Introduction (part 1)
Stats in brief: 89-20-00062024001
Description: This short video explains how it can be very effective for all levels of governments and organizations that serve communities to use disaggregated data to make evidence-informed public policy decisions. By using disaggregated data, policymakers are able to design more appropriate and effective policies that meet the needs of each diverse and unique Canadian.
Release date: 2024-07-16
2. The Importance of Disaggregated Data: An Introduction (part 2)
Stats in brief: 89-20-00062024002
Description: This short video explains how the use of disaggregated data can help policymakers to develop more targeted and effective policies by identifying the unique needs and challenges faced by different demographic groups.
Release date: 2024-07-16
3. Heterogeneous causal effects of labour market programs: A machine learning approach Archived
Articles and reports: 11-522-X202200100017
Description: In this paper, we look for presence of heterogeneity in conducting impact evaluations of the Skills Development intervention delivered under the Labour Market Development Agreements. We use linked longitudinal administrative data covering a sample of Skills Development participants from 2010 to 2017. We apply a causal machine-learning estimator as in Lechner (2019) to estimate the individualized program impacts at the finest aggregation level. These granular impacts reveal the distribution of net impacts facilitating further investigation as to what works for whom. The findings suggest statistically significant improvements in labour market outcomes for participants overall and for subgroups of policy interest.
Release date: 2024-06-28
4. Statistics Canada International Symposium Series: Proceedings
Journals and periodicals: 11-522-X
Description: Since 1984, an annual international symposium on methodological issues has been sponsored by Statistics Canada. Proceedings have been available since 1987.
Release date: 2024-06-28
5. Survey Methodology
Journals and periodicals: 12-001-X
Geography: Canada
Description: The journal publishes articles dealing with various aspects of statistical development relevant to a statistical agency, such as design issues in the context of practical constraints, use of different data sources and collection techniques, total survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves.
Release date: 2024-06-25
6. A case study of using Splink: Census duplicate matching Archived
Articles and reports: 11-522-X202200100002
Description: The authors used the Splink probabilistic linkage package developed by the UK Ministry of Justice, to link census data from England and Wales to itself to find duplicate census responses. A large gold standard of confirmed census duplicates was available meaning that the results of the Splink implementation could be quality assured. This paper describes the implementation and features of Splink, gives details of the settings and parameters that we used to tune Splink for our particular project, and gives the results that we obtained.
Release date: 2024-03-25
7. Record linkage techniques to identify 2021 Canadian Census dwellings in the new Statistical Building Register Archived
Articles and reports: 11-522-X202200100020
Description: The reconciliation of 2021 census dwellings with the new Statistical Building Register (SBgR) presented linkage challenges. The Census of Population collected information from various dwelling types. For a large proportion of the population, mailing addresses were at the centre: they were used for reaching out to people and collected as contact info. In parallel, the register environment has been evolving. The agency is transitioning from the Address Register (AR) to the SBgR holding both mailing and location addresses, while also covering non-residential buildings. The reconciliation was conducted using a combination of systems, notably the new Register Matching Engine (RME) for difficult cases. The RME holds an interesting range of sophisticated string comparators. A deterministic linkage approach was used, while incorporating some data knowledge like the entropy. Through metadata, the matching expert could also reduce the amounts of false positives and false negatives.
Release date: 2024-03-25
8. Are “immortals” an issue for survival estimates derived from Canadian Cancer Registry data?
Articles and reports: 82-003-X202301200002
Description: The validity of survival estimates from cancer registry data depends, in part, on the identification of the deaths of deceased cancer patients. People whose deaths are missed seemingly live on forever and are informally referred to as “immortals”, and their presence in registry data can result in inflated survival estimates. This study assesses the issue of immortals in the Canadian Cancer Registry (CCR) using a recently proposed method that compares the survival of long-term survivors of cancers for which “statistical” cure has been reported with that of similar people from the general population.
Release date: 2023-12-20
9. Statistical Methodology Research and Development Program Achievements
Journals and periodicals: 12-206-X
Description: This report summarizes the annual achievements of the Methodology Research and Development Program (MRDP) sponsored by the Modern Statistical Methods and Data Science Branch at Statistics Canada. This program covers research and development activities in statistical methods with potentially broad application in the agency’s statistical programs; these activities would otherwise be less likely to be carried out during the provision of regular methodology services to those programs. The MRDP also includes activities that provide support in the application of past successful developments in order to promote the use of the results of research and development work. Selected prospective research activities are also presented.
Release date: 2023-10-11
10. Construction of a Northern Market Basket Measure of poverty for Nunavut
Articles and reports: 75F0002M2022003
Description: This discussion paper describes the proposed methodology for a Northern Market Basket Measure (MBM-N) for Nunavut, as well as identifies research which could be conducted in preparation for the 2023 review. The paper presents initial MBM-N thresholds and provides preliminary poverty estimates for reference years 2018 to 2021. A review period will follow the release of this paper, during which time Statistics Canada and Employment and Social Development Canada will welcome feedback from interested parties and work with experts, stakeholders, indigenous organizations, federal, provincial and territorial officials to validate the results.
Release date: 2023-06-21

Reference (55)

Reference (55) (30 to 40 of 55 results)

31. Aboriginal Peoples Technical Report, 2006 Census Archived
Surveys and statistical programs – Documentation: 92-569-X
Description:
The 2006 Census Technical Report on Aboriginal Peoples deals with: (i) Aboriginal ancestry, (ii) Aboriginal identity, (iii) registered Indian status, and (iv) First Nation or Band membership. The report aims to inform users about the complexity of the data and any difficulties that could affect their use. It explains the conceptual framework and definitions used to gather the data, and it discusses factors that could affect data quality. The historical comparability of the data is also discussed.
Release date: 2010-02-09
32. User Guide for the Survey of Household Spending, 2008 Archived
Surveys and statistical programs – Documentation: 62F0026M2009002
Geography: Province or territory
Description:
This guide presents information of interest to users of data from the Survey of Household Spending, which gathers information on the spending habits, dwelling characteristics and household equipment of Canadian households. The survey covers private households in the 10 provinces. (The territories are surveyed every second year, starting in 1999.)
This guide includes definitions of survey terms and variables, as well as descriptions of survey methodology and data quality. One section describes the various statistics that can be created using expenditure data (e.g., budget share, market share, aggregates and medians).
Release date: 2009-12-18
33. The Water Yield for Canada As a Thirty-year Average (1971 to 2000): Concepts, Methodology and Initial Results Archived
Surveys and statistical programs – Documentation: 16-001-M2009007
Description: In this paper, we present the methodology developed by Statistics Canada to calculate the average annual water yield for Canada. Water yield, for the purposes of this paper, is defined as the amount of freshwater derived from unregulated flow (m³ s^-1) measurements for a given geographic area over a defined period of time. The methodology is applied to the 1971 to 2000 time period.
This research was conducted to fill data gaps in Statistics Canada's water statistics program. These gaps exist because estimates of freshwater flow for Canada have not been calculated regularly and have been produced using a variety of methods that do not necessarily generate comparable results. The methodology developed in this study produced results that are coherent through space and time. These results will be used in the future to investigate changes in water yield on a more disaggregated basis.

To achieve the water yield estimate a database of natural streamflow observations from 1971 to 2000 was compiled. The streamflow values were then converted to a runoff depth and interpolated using ordinary kriging to produce spatial estimates of runoff. The spatial estimates were then scaled to create a National estimate of water yield as a thirty-year average. The methodology and results were then validated using a stability analysis and several techniques involving uncertainty. The result of the methodology indicates that the thirty-year average water yield for Canada is 3435 km³.
Release date: 2009-06-01
34. Aboriginal Peoples Technical Report, 2006 Census Archived
Surveys and statistical programs – Documentation: 92-569-X2006001
Description:
The 2006 Census Technical Report on Aboriginal Peoples deals with: (i) Aboriginal ancestry, (ii) Aboriginal identity, (iii) registered Indian status, and (iv) First Nation or Band membership. The report aims to inform users about the complexity of the data and any difficulties that could affect their use. It explains the conceptual framework and definitions used to gather the data, and it discusses factors that could affect data quality. The historical comparability of the data is also discussed.

Release date: 2009-05-12
35. Feasibility Study on the Use of Provincial/Territorial Medicare Records for Measuring the Level of Inter-provincial and Inter-territorial Migration Archived
Surveys and statistical programs – Documentation: 91F0015M2008010
Geography: Canada
Description:
The objective of this study is to examine the feasibility of using provincial and territorial health care files of new registrants as an independent measure of preliminary inter-provincial and inter-territorial migration. The study aims at measuring the conceptual and quantifiable differences between this data source and our present source of the Canada Revenue Agency's Canadian Child Tax Benefit.
Criteria were established to assess the quality and appropriateness of these provincial/territorial health care records as a proxy for our migration estimates: coverage, consistency, timeliness, reliability, level of detail, uniformity and accuracy.
Based on the present analysis, the paper finds that these data do not ameliorate the estimates and would not be suitable at this time as a measure of inter-provincial/territorial migration. These Medicare data though are an important independent data source that can be used for quality evaluation.
Release date: 2009-01-13
36. Behaviour Study on the Water Quality Index of the Canadian Council of Ministers of the Environment Archived
Surveys and statistical programs – Documentation: 16-001-M2007003
Description:
The objective of the present study is to understand and explain how the Canadian Council of Ministers of the Environment (CCME) Water Quality Index (WQI) behaves, and at the same time determine its limitations to make a better use of it in the future. In order to do so, four data sets were made available to us thanks to participation of the following provinces: Newfoundland, Ontario, British Columbia and Quebec.

Release date: 2007-09-19
37. How Statistics Canada Identifies Aboriginal Peoples Archived
Surveys and statistical programs – Documentation: 12-592-X
Geography: Canada
Description:
This reference document presents an overview of the different questions used by Statistics Canada to identify Aboriginal peoples. It is divided into three parts. Part one is a brief description of the data sources and their limitations. Part 2 deals with the 2006 census questions used to identify Aboriginal peoples while Part 3 deals with the identification questions used in the Aboriginal Peoples Survey (APS) and the Aboriginal Children's Survey (ACS).
Release date: 2007-06-07
38. User Guide for the Survey of Household Spending, 2005 Archived
Surveys and statistical programs – Documentation: 62F0026M2006001
Geography: Province or territory
Description:
This guide presents information of interest to users of data from the Survey of Household Spending, which gathers information on the spending habits, dwelling characteristics and household equipment of Canadian households. The survey covers private households in the 10 provinces. (The territories are surveyed every second year, starting in 1999.)
This guide includes definitions of survey terms and variables, as well as descriptions of survey methodology and data quality. One section describes the various statistics that can be created using expenditure data (e.g., budget share, market share, aggregates and medians).
Release date: 2006-12-12
39. The new culture of the Culture Statistics Program Archived
Notices and consultations: 87-004-X20030039213
Description:
The Culture Statistics Program (CSP) has been Statistic Canada's chief source for analysis of the culture sector since the program's inception in 1972 and this role will continue. However, the CSP is making substantial changes to the way it collects culture data and, in effect, the data themselves. This article is intended to inform users of these data, of the scope of these upcoming changes and how the CSP is managing the challenges presented by this transition.
Release date: 2006-06-12
40. User Guide for the Survey of Household Spending, 2004 Archived
Surveys and statistical programs – Documentation: 62F0026M2005007
Geography: Province or territory
Description:
This guide presents information of interest to users of data from the Survey of Household Spending, which gathers information on the spending habits, dwelling characteristics and household equipment of Canadian households. The survey covers private households in the 10 provinces. (The territories are surveyed every second year, starting in 1999.)
This guide includes definitions of survey terms and variables, as well as descriptions of survey methodology and data quality. One section describes the various statistics that can be created using expenditure data (e.g., budget share, market share, aggregates and medians).
Release date: 2005-12-12

Report a problem or mistake on this page

Date modified:: 2024-09-19

Other content related to Statistical methods

Filter results by

Keyword(s)

Type

Geography

Survey or statistical program

Content

Results

All (166) (20 to 30 of 166 results)

Data (1) ((1 result))

Analysis (105) (0 to 10 of 105 results)

Reference (55) (30 to 40 of 55 results)

Other content related to Statistical methods

Filter results by

Keyword(s)

Type

Geography

Survey or statistical program

Content

Results

All (166) (20 to 30 of 166 results)

Data (1) ((1 result))

Analysis (105) (0 to 10 of 105 results)

Reference (55) (30 to 40 of 55 results)

How do I use the filters and the search box?

How do I refine my search?

How does the search work?

How are the results ordered?

How are the results ordered?