Analysis

Description:

The Canadian Census of Construction (COC) uses a complex plan for sampling small businesses (those having a gross income of less than $750,000). Stratified samples are drawn from overlapping frames. Two subsamples are selected independently from one of the samples, and more detailed information is collected on the businesses in the subsamples. There are two possible methods of estimating totals for the variables collected in the subsamples. The first approach is to determine weights based on sampling rates. A number of different weights must be used. The second approach is to impute values to the businesses included in the sample but not in the subsamples. This approach creates a complete “rectangular” sample file, and a single weight may then be used to produce estimates for the population. This “large-scale imputation” technique is presently applied for the Census of Construction. The purpose of the study is to compare the figures obtained using various estimation techniques with the estimates produced by means of large-scale imputation.

Description:

In the presence of unit nonresponse, two types of variables can sometimes be observed for units in the “intended” sample s, namely, (a) variables used to estimate the response mechanism (the response probabilities), (b) variables (here called co-variates) that explain the variable of interest, in the usual regression theory sense. This paper, based on Särndal and Swensson (1985 a, b), discusses nonresponse adjusted estimators with and without explicit involvement of co-variates. We conclude that the presence of strong co-variates in an estimator induces several favourable properties. Among other things, estimators making use of co-variates are considerably more resistant to nonresponse bias. We discuss the calculation of standard error and valid confidence intervals for estimators involving co-variates. The structure of the standard error is examined and discussed.

Description:

The procedure of subsampling the nonrespondents suggested by Hansen and Hurwitz (1946) is considered. Post-stratification prior to the subsampling is examined. For the mean of a characteristic of interest, ratio estimators suitable for different practical situations are proposed and their merits are examined. Suitable ratio estimators are also suggested for the situations in which the Hard-Core are present.

Description:

Missing survey data occur because of total nonresponse and item nonresponse. The standard way to attempt to compensate for total nonresponse is by some form of weighting adjustment, whereas item nonresponses are handled by some form of imputation. This paper reviews methods of weighting adjustment and imputation and discusses their properties.

Description:

In this paper, different types of response/nonresponse and associated measures such as rates are provided and discussed together with their implications on both estimation and administrative procedures. The missing data problems lead to inconsistent terminology related to nonresponse such as completion rates, eligibility rates, contact rates, and refusal rates, many of which can be defined in different ways. In addition, there are item nonresponse rates as well as characteristic response rates. Depending on the uses, the rates may be weighted or unweighted.

Description:

Using the optimal estimating functions for survey sampling estimation (Godambe and Thompson 1986), we obtain some optimality results for nonresponse situations in survey sampling.

Description:

Multiple imputation is a technique for handling survey nonresponse that replaces each missing value created by nonresponse by a vector of possible values that reflect uncertainty about which values to impute. A simple example and brief overview of the underlying theory are used to introduce the general procedure.

Description:

Statistics Canada has undertaken a project to develop a generalized edit and imputation system, the intent of which is to meet the processing requirements of most of its surveys. The various approaches to imputation for item non-response, which have been proposed, will be discussed. Important issues related to the implementation of these proposals into a generalized setting will also be addressed.

Description:

The analysis of survey data becomes difficult in the presence of incomplete responses. By the use of the maximum likelihood method, estimators for the parameters of interest and test statistics can be generated. In this paper the maximum likelihood estimators are given for the case where the data is considered missing at random. A method for imputing the missing values is considered along with the problem of estimating the change points in the mean. Possible extensions of the results to structured covariances and to non-randomly incomplete data are also proposed.

Description:

For periodic business surveys which are conducted on a monthly, quarterly or annual basis, the data for responding units must be edited and the data for non-responding units must be imputed. This paper reports on methods which can be used for editing and imputing data. The editing is comprised of consistency and statistical edits. The imputation is done for both total non-response and partial non-response.

Description:

Recent economic trends, including expanding globalization of trade (as evidenced by new trade agreements), volatility in market exchange rates, and greater interest in international comparisons of real income and productivity are generating increased interest in intercountry comparisons. These comparisons are made in real terms derived from purchasing power parities (PPPs). In Canada, a particularly important relationship with the United States focusses attention on US/Canada comparative price and volume measures. This article includes updated annual bilateral volume indexes of real gross domestic product (GDP) per capita and its components for the United States, compared with Canada, and the associated PPPs.

Description: This discussion summarizes the interesting new findings around measurement errors in opt-in surveys by Kennedy, Mercer and Lau (KML). While KML enlighten readers about “bogus responding” and possible patterns in them, this discussion suggests combining these new-found results with other avenues of research in nonprobability sampling, such as improvement of representativeness.

Description: Kennedy, Mercer, and Lau explore misreporting by respondents in non-probability samples and discover a new feature, namely that of deliberate misreporting of demographic characteristics. This finding suggests that the “arms race” between researchers and those determined to disrupt the practice of social science is not over and researchers need to account for such respondents if using high-quality probability surveys to help reduce error in non-probability samples.

Description: Nonprobability samples are quick and low-cost and have become popular for some types of survey research. Kennedy, Mercer and Lau examine data quality issues associated with opt-in nonprobability samples frequently used in the United States. They show that the estimates from these samples have serious problems that go beyond representativeness. A total survey error perspective is important for evaluating all types of surveys.

Description: Statistical approaches developed for nonprobability samples generally focus on nonrandom selection as the primary reason survey respondents might differ systematically from the target population. Well-established theory states that in these instances, by conditioning on the necessary auxiliary variables, selection can be rendered ignorable and survey estimates will be free of bias. But this logic rests on the assumption that measurement error is nonexistent or small. In this study we test this assumption in two ways. First, we use a large benchmarking study to identify subgroups for which errors in commercial, online nonprobability samples are especially large in ways that are unlikely due to selection effects. Then we present a follow-up study examining one cause of the large errors: bogus responding (i.e., survey answers that are fraudulent, mischievous or otherwise insincere). We find that bogus responding, particularly among respondents identifying as young or Hispanic, is a significant and widespread problem in commercial, online nonprobability samples, at least in the United States. This research highlights the need for statisticians working with commercial nonprobability samples to address bogus responding and issues of representativeness – not just the latter.

Description: This paper is an introduction to the special issue on the use of nonprobability samples featuring three papers that were presented at the 29^th Morris Hansen Lecture by Courtney Kennedy, Yan Li and Jean-François Beaumont.

Description: The Canadian Income Survey (CIS) has introduced improvements to the methods and data sources used to produce income and poverty estimates with the release of its 2022 reference year estimates. Foremost among these improvements is a significant increase in the sample size for a large subset of the CIS content. The weighting methodology was also improved and the target population of the CIS was changed from persons aged 16 years and over to persons aged 15 years and over. This paper describes the changes made and presents the approximate net result of these changes on the income estimates and data quality of the CIS using 2021 data. The changes described in this paper highlight the ways in which data quality has been improved while having little impact on key CIS estimates and trends.

Description: This study applies small area estimation (SAE) and a new geographic concept called Self-contained Labor Area (SLA) to the Canadian Survey on Business Conditions (CSBC) with a focus on remote work opportunities in rural labor markets. Through SAE modelling, we estimate the proportions of businesses, classified by general industrial sector (service providers and goods producers), that would primarily offer remote work opportunities to their workforce.

Description: Record linkage aims at identifying record pairs related to the same unit and observed in two different data sets, say A and B. Fellegi and Sunter (1969) suggest each record pair is tested whether generated from the set of matched or unmatched pairs. The decision function consists of the ratio between m(y) and u(y),probabilities of observing a comparison y of a set of k>3 key identifying variables in a record pair under the assumptions that the pair is a match or a non-match, respectively. These parameters are usually estimated by means of the EM algorithm using as data the comparisons on all the pairs of the Cartesian product ?=A×B. These observations (on the comparisons and on the pairs status as match or non-match) are assumed as generated independently of other pairs, assumption characterizing most of the literature on record linkage and implemented in software tools (e.g. RELAIS, Cibella et al. 2012). On the contrary, comparisons y and matching status in ? are deterministically dependent. As a result, estimates on m(y) and u(y) based on the EM algorithm are usually bad. This fact jeopardizes the effective application of the Fellegi-Sunter method, as well as automatic computation of quality measures and possibility to apply efficient methods for model estimation on linked data (e.g. regression functions), as in Chambers et al. (2015). We propose to explore ? by a set of samples, each one drawn so to preserve independence of comparisons among the selected record pairs. Simulations are encouraging.

Description: The authors used the Splink probabilistic linkage package developed by the UK Ministry of Justice, to link census data from England and Wales to itself to find duplicate census responses. A large gold standard of confirmed census duplicates was available meaning that the results of the Splink implementation could be quality assured. This paper describes the implementation and features of Splink, gives details of the settings and parameters that we used to tune Splink for our particular project, and gives the results that we obtained.

Description: Estimation at fine levels of aggregation is necessary to better describe society. Small area estimation model-based approaches that combine sparse survey data with rich data from auxiliary sources have been proven useful to improve the reliability of estimates for small domains. Considered here is a scenario where small area model-based estimates, produced at a given aggregation level, needed to be disaggregated to better describe the social structure at finer levels. For this scenario, an allocation method was developed to implement the disaggregation, overcoming challenges associated with data availability and model development at such fine levels. The method is applied to adult literacy and numeracy estimation at the county-by-group-level, using data from the U.S. Program for the International Assessment of Adult Competencies. In this application the groups are defined in terms of age or education, but the method could be applied to estimation of other equity-deserving groups.

Description:

Statistics Canada is engaged in the "Information System for Science and Technology Project" to develop useful indicators of activity and a framework to tie them together into a coherent picture of science and technology (S&T) in Canada. The working papers series is used to publish results of the different initiatives conducted within this project. The data are related to the activities, linkages and outcomes of S&T. Several key areas are covered such as: innovation, technology diffusion, human resources in S&T and interrelations between different actors involved in S&T. This series also presents data tabulations taken from regular surveys on research and development (R&D) and S&T and made possible by the project.

Description:

This publication shows readers how to design and conduct a census or sample survey. It explains basic survey concepts and provides information on how to create efficient and high quality surveys. It is aimed at those involved in planning, conducting or managing a survey and at students of survey design courses.

This book contains the following information:

-how to plan and manage a survey;-how to formulate the survey objectives and design a questionnaire; -things to consider when determining a sample design (choosing between a sample or a census, defining the survey population, choosing a survey frame, identifying possible sources of survey error); -choosing a method of collection (self-enumeration, personal interviews or telephone interviews; computer-assisted versus paper-based questionnaires); -organizing and conducting data collection operations;-determining the sample size, allocating the sample across strata and selecting the sample; -methods of point estimation and variance estimation, and data analysis; -the use of administrative data, particularly during the design and estimation phases-how to process the data (which consists of all data handling activities between collection and estimation) and use quality control and quality assurance measures to minimize and control errors during various survey steps; and-disclosure control and data dissemination.

This publication also includes a case study that illustrates the steps in developing a household survey, using the methods and principles presented in the book. This publication was previously only available in print format and originally published in 2003.

Description:

Beginning in late 2006, the Social and Aboriginal Statistics Division of Statistics Canada embarked on the process of review of questions used in the Census and in surveys to produce data about Aboriginal peoples (North American Indian, Métis and Inuit). This process is essential to ensure that Aboriginal identification questions are valid measures of contemporary Aboriginal identification, in all its complexity. Questions reviewed included the following (from the Census 2B questionnaire):- the Ethnic origin / Aboriginal ancestry question;- the Aboriginal identity question;- the Treaty / Registered Indian question; and- the Indian band / First Nation Membership question.

Additional testing was conducted on Census questions with potential Aboriginal response options: the population group question (also known as visible minorities), and the Religion question. The review process to date has involved two major steps: regional discussions with data users and stakeholders, and qualitative testing. The regional discussions with over 350 users of Aboriginal data across Canada were held in early 2007 to examine the four questions used on the Census and other surveys of Statistics Canada. Data users included National Aboriginal organizations, Aboriginal Provincial and Territorial Organizations, Federal, Provincial and local governments, researchers and Aboriginal service organizations. User feedback showed that main areas of concern were data quality, undercoverage, the wording of questions, and the importance of comparability over time.

Description:

This report presents details on the data sources and methods underlying the air quality indicators as they were reported in Canadian Environmental Sustainability Indicators, 2007 (16-251-XWE). The air quality indicators focus on human exposure to ground-level ozone and fine particulate matter.

Details on the indicators reported after 2007 can be found on Environment Canada's site: &&www.ec.gc.ca/indicateurs-indicators/

Description:

This report presents details on the data sources and methods underlying the freshwater quality indicator as it was reported in the Canadian Environmental Sustainability Indicators, 2007 (16-251-XWE). The national freshwater quality indicator provides an overall measure of the suitability of water bodies to support aquatic life in selected monitoring sites in Canada.

Details on this indicator reported after 2007 can be found on Environment Canada's site: www.ec.gc.ca/indicateurs-indicators/

Description:

This report summarizes the main issues raised in these meetings. Four questions used to identify Aboriginal people from the Census and surveys were considered in the discussions.Statistics Canada regularly reviews the questions used on the Census and other surveys to ensure that the resulting data are representative of the population. As a first step in the process to review the questions used to produce data about First Nations, Inuit and Métis populations, regional discussions were held with more than 350 users of Aboriginal data in over 40 locations across Canada during the winter, spring and early summer of 2007.

This report summarizes the main issues raised in these meetings. Four questions used to identify Aboriginal people from the Census and surveys were considered in the discussions.

Description:

This feasibility report provides a blueprint for improving data on fraud in Canada through a survey of businesses and through amendments to the Uniform Crime Reporting (UCR) Survey. Presently, national information on fraud is based on official crime statistics reported by police services to the Uniform Crime Reporting Survey. These data, however, do not reflect the true nature and extent of fraud in Canada due to under-reporting of fraud by individuals and businesses, and due to inconsistencies in the way frauds are counted within the UCR Survey. This feasibility report concludes that a better measurement of fraud in Canada could be obtained through a survey of businesses. The report presents the information priorities of government departments, law enforcement and the private sector with respect to the issue of fraud and makes recommendations on how a survey of businesses could help fulfill these information needs.

To respond to information priorities, the study recommends surveying the following types of business establishments: banks, payment companies (i.e. credit card and debit card companies), selected retailers, property and casualty insurance carriers, health and disability insurance carriers and selected manufacturers. The report makes recommendations regarding survey methodology and questionnaire content, and provides estimates for timeframes and cost.

The report also recommends changes to the UCR Survey in order to improve the way in which incidents are counted and to render the data collected more relevant with respect to the information priorities raised by government, law enforcement and the private sector during the feasibility study.

Description:

This document summarizes the results of content analyses of the 2004 Census Test. The first section briefly explains the context of the content analyses by describing the nature of the sample, its limitations and the strategies used to evaluate data quality. The second section provides an overview of the results for questions that have not changed since the 2001 Census by describing the similarities between 2001 and 2004 distributions and non-response rates. The third section evaluates data quality of new census questions or questions that have changed substantially: same-sex married couples, ethnic origins, levels of schooling, location where highest diploma was obtained, school attendance, permission to access income tax files, and permission to make personal data publicly available 92 years after the census. The last section summarizes the overall results for questions whose content was coded and evaluated as part of the 2004 test, namely industry, occupation and place of work variables.

Description: The Economic Analysis Methodology Paper Series circulates information on definitions employed, standards used, procedures followed and evaluations of the quality of the economic statistics produced by the System of National Accounts (SNA). These papers can be downloaded free at www.statcan.gc.ca.

Description:

This report describes sampling and weighting procedures used in the 2001 Census. It reviews the history of these procedures in Canadian censuses, provides operational and theoretical justifications for them, and presents the results of the evaluation studies of these procedures.