Weighting and estimation

Results

All (638)

All (638) (20 to 30 of 638 results)

21. On the use of machine learning methods for the treatment of unit nonresponse in surveys
Articles and reports: 12-001-X202500100003
Description: In recent years, there has been a significant interest in machine learning in national statistical offices. Thanks to their flexibility, these methods may prove useful at the nonresponse treatment stage. In this article, we conduct an empirical investigation in order to compare several machine learning procedures in terms of bias and efficiency. In addition to the classical machine learning procedures, we assess the performance of ensemble approaches that make use of different machine learning procedures to produce a set of weights adjusted for nonresponse.
Release date: 2025-06-30
22. Mean squared prediction error estimators of the empirical best linear unbiased predictor of a small area mean under a semi-parametric Fay-Herriot model
Articles and reports: 12-001-X202500100005
Description: In this paper, we derive a second-order unbiased (or nearly unbiased) mean squared prediction error (MSPE) estimator of the empirical best linear unbiased predictor (EBLUP) of a small area mean for a semi-parametric extension to the well-known Fay-Herriot model. Specifically, we derive our MSPE estimator essentially assuming certain moment conditions on both the sampling errors and random effects distributions. The normality-based Prasad-Rao MSPE estimator has a surprising robustness property in that it remains second-order unbiased under the non-normality of random effects when a simple Prasad-Rao method-of-moments estimator is used for the variance component and the sampling error distribution is normal. We show that the normality-based MSPE estimator is no longer second-order unbiased when the sampling error distribution has non-zero kurtosis or when the Fay-Herriot moment method is used to estimate the variance component, even when the sampling error distribution is normal. Interestingly, when the simple method-of moments estimator is used for the variance component, our proposed MSPE estimator does not require the estimation of kurtosis of the random effects. Results of a simulation study on the accuracy of the proposed MSPE estimator, under non-normality of both sampling and random effects distributions, are also presented.
Release date: 2025-06-30
23. sCHAID: A tool for constructing nonresponse adjustment cells under a design-based framework
Articles and reports: 12-001-X202500100006
Description: Survey practitioners have increasingly embraced the benefits of modern machine learning techniques, including classification and regression tree algorithms, in the development of nonresponse adjustments. These methods, which do not require a predefined functional relationship between outcomes and predictors, offer a practical means of conducting variable selection and deriving interpretable structures that link response propensity with explanatory variables. However, when applying these algorithms to survey data, it is common to overlook crucial factors like sampling weights, as well as sample design features such as stratification and clustering. To bridge this shortcoming, we propose an extension of the Chi-square Automatic Interaction Detector (CHAID) approach, and we describe the design-based asymptotic properties of the resulting “survey CHAID” (sCHAID) method. To facilitate the practical use of sCHAID, we incorporate a Rao-Scott correction into the splitting criterion, accounting for the survey design. Using data from the U.S. American Community Survey, we illustrate the use of the method and evaluate its performance through comparisons with existing weighted and unweighted algorithms.
Release date: 2025-06-30
24. Model-assisted calibration estimation using generalized entropy calibration in survey sampling
Articles and reports: 12-001-X202500100007
Description: We introduce a novel approach to model-assisted calibration estimation in survey sampling using generalized entropy. The method builds upon recent work by Kwon, Kim and Qiu (2024) and extends it to a model-assisted framework. Unlike traditional calibration techniques, this approach employs a generalized entropy function as the objective for optimization and incorporates a debiasing calibration constraint to ensure design consistency. The proposed estimator is shown to be asymptotically equivalent to an augmented generalized regression (GREG) estimator. It allows for unequal model variance, potentially improving efficiency when the sampling design is informative. The paper presents both design-based and model-based justifications for the method, along with asymptotic properties and variance estimation techniques. Computational aspects are discussed, including an unconstrained optimization approach that facilitates implementation, especially for high-dimensional auxiliary variables. The method’s performance is evaluated through a simulation study, demonstrating its effectiveness in improving estimation efficiency, particularly when the sampling design is informative.
Release date: 2025-06-30
25. Use of nonprobability samples for official statistics, state of the art
Articles and reports: 12-001-X202500100008
Description: Tightened budgets, continuing decrease of response rates in traditional probability surveys and increasing pressure by users for more timely data, has stimulated research on the use of nonprobability sample data, such as administrative records, web scraping, mobile phone data and voluntary internet surveys, for inference on finite population parameters like means and totals. These data are often easier, faster and cheaper to collect than traditional probability samples. However, a major concern with the use of this kind of data for official statistics is their nonrepresentativeness due to possible selection bias, which if not accounted for properly, could bias the inference. In this article, we review and discuss methods considered in the literature to deal with this problem and propose new methods, distinguishing between methods based on integration of the nonprobability sample with an appropriate probability sample, and methods that base the inference solely on the nonprobability sample. Empirical illustrations, based on simulated data are provided.
Release date: 2025-06-30
26. Authors’ response to comments on “Trends and directions in sample survey theory and methods”
Articles and reports: 12-001-X202500100010
Description: The discussants highlight promising research topics for improving the quality and granularity of estimates from surveys. We agree that continued research is needed to evaluate models used for inference, and suggest development of measures of model dependence.
Release date: 2025-06-30
27. Comments by M. Giovanna Ranalli on “Trends and directions in sample survey theory and methods”
Articles and reports: 12-001-X202500100011
Description: This discussion examines some advancements in survey design and estimation, inspired by the comprehensive appraisal of Professors Jon Rao and Sharon Lohr on current trends in the field. It delves into three specific areas: balanced sampling, calibration, and small area estimation. Probabilistic balanced sampling methods, such as the cube method and penalized balanced sampling, are explored, with an emphasis on addressing emerging challenges, including extensions to linear mixed models, nonparametric regression models, and spatially balanced designs. Calibration is discussed using a modular framework that incorporates modern regression techniques, and highlights innovative uses of model calibration for data editing and causal inference. Small area estimation is considered in the context of latent variable modeling and data integration, emphasizing its role when the variable(s) of interest cannot be measured either directly or without error. Applications in integrating probability and non-probability data and conducting causal analysis at local level are also discussed.
Release date: 2025-06-30
28. Comments by Jean D. Opsomer, Daifeng Han and Medha Uppala on “Trends and directions in sample survey theory and methods”
Articles and reports: 12-001-X202500100012
Description: In this discussion, we complement the excellent overview by Profs. Lohr and Rao with some additional topics. The first topic is a call for more recognition of the central role of modeling in survey estimation. The second is a brief discussion of the use of partial frame information in survey design. Finally, we draw the attention to recent increases of synthetic methods, in particular, multilevel regression and poststratification (MRP) in small area estimation applications.
Release date: 2025-06-30
29. Trends and directions in sample survey theory and methods
Articles and reports: 12-001-X202500100014
Description: Rao (1999) summarized trends in sample survey theory and methods at the turn of the millenium. We provide an updated discussion of some current trends in survey design and estimation methods for the 50th anniversary of Survey Methodology. Recent innovations in survey design include research on anticipating nonsampling errors at the design stage and development of balanced and adaptive sampling designs to take advantage of detailed sampling frame information or data gathered during the survey process. Nonparametric and machine learning methods are increasingly used for data editing as well as for model-assisted estimation and nonresponse adjustments. Small area models have been expanded to incorporate spatial and time series information, increase the flexibility and robustness of the linking and variance models, benchmark to large-area direct estimators, and (for unit level models) account for informative sampling designs. The increasing availability of large administrative datasets, sensor and satellite data, and convenience samples has spurred research on how to use these sources - on their own and when integrated with probability samples. We conclude by discussing some frontiers for survey research.
Release date: 2025-06-30
30. Bayesian predictive inference of a finite population mean without specifying the relation between the study variable and the covariates
Articles and reports: 12-001-X202400200004
Description: While we avoid specifying the parametric relationship between the study variable and covariates, we illustrate the advantage of including a spatial component to better account for the covariates in our models to make Bayesian predictive inference. We treat each unique covariate combination as an individual stratum, then we use small area estimation techniques to make inference about the finite population mean of the continuous response variable. The two spatial models used are the conditional autoregressive and simple conditional autoregressive models. We include the spatial effects by creating the adjacency matrix via the Mahalanobis distance between covariates. We also show how to incorporate survey weights into the spatial models when dealing with probability survey data. We compare the results of two non-spatial models including the Scott-Smith model and the Battese, Harter, and Fuller model to the spatial models. We illustrate the comparison between the aforementioned models with an application using BMI data from eight counties in California. Our goal is to have neighboring strata yield similar predictions, and to increase the difference between strata that are not neighbors. Ultimately, using the spatial models shows less global pooling compared to the non-spatial models, which was the desired outcome.
Release date: 2024-12-20

Data (0)

Data (0) (0 results)

No content available at this time.

Analysis (610)

Analysis (610) (600 to 610 of 610 results)

601. Some estimators of population totals from simple random samples containing large units Archived
Articles and reports: 12-001-X197700100006
Description: The problem considered is the estimation of population total of some characteristic from a simple random sample containing a few large or extreme observations. The effect of these large units in the sample is to distort the estimate of the population total. It is therefore important to correct the weights for such units or deflate their values at the estimation stage once they have been sampled and identified as unusually large units. In this paper, three estimators which alter the usual sampling weights have been considered. The efficiencies of these estimators have been worked out in terms of the ratio of the variance of the usual estimator of the population total to the mean square error of these estimators. An empirical study of these estimators is also discussed.
Release date: 1977-06-20
602. Stratification index: Methodology and analysis Archived
Articles and reports: 12-001-X197600200002
Description: To obtain estimates of means or totals for a universe, a sample of units is often drawn to represent the universe and these units are then surveyed. One of the most important procedures used in the selection of the units is that of stratification, whereby the universe is split up into strata and independent samples of units are drawn from each stratum. A stratification index is developed to indicate the approximate fractional reduction in the sampling variance from that which would result if no stratification were undertaken. Also the methodology is extended to examine the effect of stratification on the sampling variance at different levels of stratification through the concept of a summary index. The stratification index is also extended to the case of ratio estimates using independent source data to re-weight the sample data. The index has been applied to the Canadian Labour Force Survey (LFS), a typical multi-stage stratified sample where ratio estimation, using projected age-sex population estimates is applied and empirical data are presented and analyzed.
Release date: 1976-12-13
603. The estimation of total variance in the 1976 Census Archived
Articles and reports: 12-001-X197600200004
Description: Published reports for the 1976 Census will include estimates of Total Variance as indicators of the reliability of the figures in these reports. In order to obtain these estimates of Total Variance, an Interpenetrating Design Experiment was incorporated into the collection methods for a sample of enumeration areas. In this paper we derive the formula for Total Variance in terms of variances due to sampling, correlated response and simple response. We then show how the Total Variance, and its components, can be estimated from the design and we give the estimators that will be used for the 1976 Census. The estimates of sampling and correlated response variance are unbiased but the simple response variance estimate is not.
Release date: 1976-12-13
604. Raking ratio estimators Archived
Articles and reports: 12-001-X197600100003
Description: This paper presents large sample results for the bias and variance of raking-ratio estimators for up to four iterations. Estimators of the bias and variance are also presented. An expression for the asymptotic covariance matrix of the maximum likelihood estimators of the cell proportions in a two-way table with known marginals is also given.
Release date: 1976-06-14
605. On the improvement of sample survey estimates Archived
Articles and reports: 12-001-X197500254830
Description: This paper focuses on the improvement of sample survey estimates in the particular situation where the survey sample, or part of it, is included in a larger sample from which auxiliary information is available. The properties of a method of estimation - sometimes applied in specific circumstances - are investigated and the limitations of its application are found. The application of the method to rotation designs in continuing surveys is more closely studied in the context of composite estimation.
Release date: 1975-12-15
606. On a ratio estimate with post-stratified weighting Archived
Articles and reports: 12-001-X197500254832
Description: A ratio estimate based on an auxiliary variable is considered for the case when the sample is post-stratified using information on another auxiliary variable. The variance of the ratio estimate is derived by the method of linearization [3,4]. An application to subprovincial estimation in the Canadian Labour Force Survey is discussed.
Release date: 1975-12-15
607. Measurement of response errors in Censuses and sample surveys Archived
Articles and reports: 12-001-X197500254824
Description:
Madow [1968] has proposed a two-phase sampling scheme under which response bias can be eliminated from sample surveys by obtaining “true” values for a subsample of the original sample. Often in cases of Censuses or ongoing surveys, the subsample data are not used to correct the main survey estimates but to assess their reliability. The main purpose of this paper is to present methods by which reliability estimates can be obtained when true values can be determined for a subsample of units.
Release date: 1975-12-15
608. The development of an automated estimation system Archived
Articles and reports: 12-001-X197500100001
Description: Although a survey is designed to satisfy a specific set of survey constraints, some steps involved in designing a survey, such as stratification, sample allocation and sample selection are common to all surveys. The steps involved in the creation of survey design systems are to identify, develop and implement common methods and procedures for such stages which, when taken together, constitute a survey design. The paper describes some methodological considerations in the development of an automated system for three methods of ratio estimation.
Release date: 1975-06-16
609. Some estimators for domain totals Archived
Articles and reports: 12-001-X197500100004
Description: A major concern in large scale surveys is the problem of sub-population estimation (domain estimation). This paper presents a study of four estimators for estimating domain totals. The domain considered in the study is an area type of domain, that is, a domain consisting of a combination of a certain number of area units belonging to different strata. This paper uses some actual data and some fictitious data to compare variances and mean square errors of the four estimators.
Release date: 1975-06-16
610. Components of variance model in multi-stage stratified samples Archived
Articles and reports: 12-001-X197500100007
Description: There are several multi-stage sample designs in various countries, such as the Current Population Survey in U.S.A., Labour Survey in Sweden, and the General Household Survey in United Kingdom. From each survey, estimated totals of Employed, Unemployed, and other characteristics may be obtained.
The Canadian Labour Force Survey is a monthly household survey in which the dwelling is the ultimate unit of sampling requiring two to four stages of selection. Each province is split up into strata and sampling units at various stages so that the sampling variance contains up to four components of variance whose actual formulae and estimation formulae are derived, utilizing those formerly derived by Yates and Grundy [12]. Ratio estimation is employed and the formulas are modified accordingly. To analyze the components of variance, it is necessary to express them in terms of components of sampling ratios and the sizes of sampling units at the various stages at provincial and national levels and approximate variance functions are thus derived.
Release date: 1975-06-16

Reference (28)

Reference (28) (0 to 10 of 28 results)

1. A Framework for Projecting Housing Use Associated with Immigrants and Non-Permanent Residents
Surveys and statistical programs – Documentation: 11-633-X2026002
Description: Recent changes in Canada’s immigration levels have heightened interest in understanding how immigration affects housing demand. This article develops a methodological framework for projecting housing use associated with permanent residents (PRs) and non-permanent residents (NPRs) under alternative immigration scenarios. The framework applies observed per capita housing use rates from the Census of Population to estimate incremental housing use by tenure over time.
Release date: 2026-04-24
2. Technical Guide on Demographic Estimates at Statistics Canada
Surveys and statistical programs – Documentation: 91-528-X
Description: The Technical Guide on Demographic Estimates at Statistics Canada provides detailed descriptions of the most current data sources and methods used by the Centre for demography at Statistics Canada to produce demographic estimates as part of the Demographic estimates program. They comprise postcensal and intercensal population estimates; base population; births and deaths; immigrants; emigrants; returning emigrants; non-permanent residents; interprovincial migration; subprovincial estimates of population and intraprovincial migration; population estimates by age and gender; and census family estimates. A glossary of commonly used terms is available at the end of the guide.
Release date: 2025-12-17
3. Sampling and Weighting Technical Report, Census of Population
Surveys and statistical programs – Documentation: 98-306-X
Description:
This report describes sampling, weighting and estimation procedures used in the Census of Population. It provides operational and theoretical justifications for them, and presents the results of the evaluations of these procedures.
Release date: 2023-10-04
4. Effective Income Tax and Transfer Rates: Technical Reference Note
Notices and consultations: 75F0002M2019006
Description:
In 2018, Statistics Canada released two new data tables with estimates of effective tax and transfer rates for individual tax filers and census families. These estimates are derived from the Longitudinal Administrative Databank. This publication provides a detailed description of the methods used to derive the estimates of effective tax and transfer rates.
Release date: 2019-04-16
5. Revisions to 2006 to 2011 income data Archived
Surveys and statistical programs – Documentation: 75F0002M2015003
Description:
This note discusses revised income estimates from the Survey of Labour and Income Dynamics (SLID). These revisions to the SLID estimates make it possible to compare results from the Canadian Income Survey (CIS) to earlier years. The revisions address the issue of methodology differences between SLID and CIS.
Release date: 2015-12-17
6. Methodology for measuring the underground economy by province and territory Archived
Surveys and statistical programs – Documentation: 13-605-X201500414166
Description:
Estimates of the underground economy by province and territory for the period 2007 to 2012 are now available for the first time. The objective of this technical note is to explain how the methodology employed to derive upper-bound estimates of the underground economy for the provinces and territories differs from that used to derive national estimates.
Release date: 2015-04-29
7. Sampling and Weighting Technical Report, National Household Survey, 2011 Archived
Surveys and statistical programs – Documentation: 99-002-X2011001
Description:
This report describes sampling and weighting procedures used in the 2011 National Household Survey. It provides operational and theoretical justifications for them, and presents the results of the evaluation studies of these procedures.
Release date: 2015-01-28
8. Sampling and Weighting Technical Report, National Household Survey
Surveys and statistical programs – Documentation: 99-002-X
Description: This report describes sampling and weighting procedures used in the 2011 National Household Survey. It provides operational and theoretical justifications for them, and presents the results of the evaluation studies of these procedures.
Release date: 2015-01-28
9. 2006 Census Technical Report: Sampling and Weighting
Surveys and statistical programs – Documentation: 92-568-X
Description:
This report describes sampling and weighting procedures used in the 2006 Census. It reviews the history of these procedures in Canadian censuses, provides operational and theoretical justifications for them, and presents the results of the evaluation studies of these procedures.
Release date: 2009-08-11
10. Improvements in 2006 to the Labour Force Survey (LFS) Archived
Surveys and statistical programs – Documentation: 71F0031X2006003
Description:
This paper introduces and explains modifications made to the Labour Force Survey estimates in January 2006. Some of these modifications include changes to the population estimates, improvements to the public and private sector estimates and historical updates to several small Census Agglomerations (CA).
Release date: 2006-01-25