Survey Methodology

In the age of big data, nonprobability surveys are becoming increasingly abundant. Data integration techniques involving both probability and nonprobability surveys are being extensively used for providing improved estimates for finite population estimation. While much of the existing research has focused on mitigating selection bias in nonprobability surveys, the issue of measurement error within these surveys remains relatively unexplored. Statistical methods devised with the purpose of reducing selection bias are appropriate for reliable estimation, only under the assumption of accuracy of survey responses. Motivated by a recent case study of Kennedy, Mercer and Lau (2024), our research addresses bias from both measurement and sampling errors in nonprobability surveys. In this article, we propose a new data integration method that uses multiple probability and nonprobability surveys and leverages machine learning models to construct a composite estimator. The proposed composite estimator integrates probability and nonprobability surveys, when both contain response variables of interest. We analyze the performance of this estimator in comparison to an existing composite estimator in literature, analytically as well as empirically, using multiple survey data from Kennedy et al. (2024). Finally, we identify conditions under which the proposed estimator outperforms estimators based solely on probability surveys.

We propose small area estimators of general indicators in off-census years, which avoid the use of deprecated census microdata, but are nearly optimal in census years. The procedure is based on replacing the obsolete census file with a larger unit-level survey that adequately covers the areas of interest and contains the values of useful auxiliary variables. However, the minimal data requirement of the proposed method is a single survey with microdata on the target variable and suitable auxiliary variables for the period of interest. We also develop an estimator of the mean squared error (MSE) that accounts for the uncertainty introduced by the large survey used to replace the census of auxiliary information. Our empirical results indicate that the proposed predictors perform clearly better than the alternative predictors when census data are outdated, and are very close to optimal ones when census data are correct. They also illustrate that the proposed total MSE estimator corrects for the bias of purely model-based MSE estimators that do not account for the large survey uncertainty.

We construct a hybrid Bayesian method, which includes a differentially private mechanism, to mask Census county totals for a U.S. state on acreage of a commodity. We use surrogates for data collected at the farm level from a past U.S. Census of Agriculture to illustrate our procedure. We use two Bayesian small area models (parametric and mixture) to accommodate the smaller counties with fewer farms and some counties with large acres. In these models, the Laplace distribution provides a differentially private mechanism. In pre-processing, we also incorporate the Census weights to form the observed total acreage, a scaling factor to the Laplace mechanism for each county, a square-root transformation of the observed total acreage to avoid negative masked estimates especially for small counties, and the p-percent rule and the 3+ rule to partition the counties into suppressed counties, non-sensitive counties and sensitive counties. Because of difficulties in specifying and tuning the privacy budget (an unknown parameter), to balance security and utility, we specify a prior for the privacy budget, where the values are not specified, and the Gibbs sampler is used to fit the hierarchical Bayesian models. In post-processing, we use Bayesian predictive inference to obtain masked county acreages, and this includes a benchmarking so that the masked state total matches the observed state total. As a measure of reliability of the Bayesian procedure, we use the posterior coefficients of variation for the masked posterior means of the counties. As a measure of utility, we use the absolute relative errors for the individual counties, together with other global measures. For the sensitive counties, there are some differences between the two small area models but both are much better than an individual area model; the mixture model being the best compromise for security and utility.

With the exception of two-phase sampling, the standard variance approximation of the generalized regression (GREG) estimator assumes that the population totals in the weighting scheme are observed without error. If the weighting model of the GREG estimator contains population totals that are observed with measurement error sources other than the sampling error of first-phase estimates, then this uncertainty will be ignored by the variance approximation of the GREG estimator. This paper proposes a variance approximation for the GREG estimator that accounts for additional uncertainty arising from measurement error in one or more of the population totals used in the weighting scheme. This approach has been developed for, and is being applied to, the Dutch Labour Force Survey (DLFS). The monthly publications of the DLFS are obtained with a time series model, which corrects for rotation group bias and discontinuities caused by major redesigns and the loss of face-to-face interviews during COVID-19. The GREG estimates for the quarterly figures are benchmarked to the average of the monthly publications to enforce numerical consistency between monthly and quarterly publication tables. The standard variance approximation of the GREG estimator assumes that these population totals are observed without error. This results in an underestimation of the variance of the GREG estimator. The variance approximation proposed in this paper results in more realistic standard errors for the quarterly GREG estimates.

Combining estimates from independent surveys via inverse-variance weights can lead to negative bias when unknown variances are estimated and the target variable is non-negative and positively skewed. In such cases, strong positive correlations typically arise between the estimators and their corresponding variance estimators, causing standard linear combinations with inverse-variance weights to exhibit negative bias. We introduce a strikingly simple method to reduce bias: replace the standard weight with the ratio of the estimator to the variance estimator. Under a linear model linking the two, we show that the new ratio-weighted estimator is approximately unbiased, whereas the conventional inverse-variance combination exhibits downward bias. Through simulations, we demonstrate that the new method brings both the bias and the mean squared error closer to the optimum for a wide range of different target variables. As our method uses only standardly reported summary statistics, it can be immediately adopted to reduce this widespread bias and improve the reliability of scientific findings in various fields.

This paper introduces an innovative and intuitive finite population sampling method that has been developed using a unique graphical framework. In this approach, first-order inclusion probabilities are represented as bars on a two-dimensional graph. By manipulating the positions of these bars, researchers can create a wide range of different sampling designs. This graphical visualization of sampling designs facilitates the exploration of alternative designs and may simplify certain aspects of the implementation compared to traditional mathematical algorithms. This novel approach holds significant promise for tackling complex challenges in sampling, such as achieving an optimal design. By applying a version of the greedy best-first search algorithm to this graphical approach, the potential for integrating intelligent algorithms into finite population sampling is demonstrated.

National statistical institutes operate sample coordination systems to spread the response burden in business surveys. Despite the applied sample coordination and monitoring the response burden, some businesses might still be heavily sampled within a short period. This may lead to a peaking response burden for individual businesses, which could affect response rates and response quality. This paper proposes a new sample coordination method based on Adapted Spatially Correlated Poisson (ASCP) sampling that focuses on businesses with a high response burden. The effects on the response burden will be evaluated in two simulation studies and compared with a stratified approach, a pragmatic method in which sampling fractions are manually adjusted and with the baseline method of ignoring the response burden. For the simulations, real-world scenarios and data from Statistics Netherlands are used. The first simulation study considers a practical situation in which a given sample is adjusted with the aim to avoid the occurrence of businesses with a peaking response burden. The second simulation study analyzes the longer-term effects of the different sample coordination methods and focuses both on the reduction and spread of the response burden. The advantages and disadvantages of the different methods will be explained and discussed in detail, and recommendations for applying these methods at national statistical institutes and other survey agencies will be given.

We introduce a general framework for constructing master samples that preserve desirable design properties across panels. The core procedure is to order an initial probability sample. Since the final sequence must be robust to a uniform random rotation, we define and minimize an objective that aggregates panel-level performance across all possible circular panels. A final random rotation is applied to ensure design validity. The framework is flexible with respect to the choice of design criteria, such as spatial balance or marginal balance, and can be implemented efficiently using simulated annealing to obtain high-quality approximate solutions. By construction, the approach supports both positive and negative sample coordination for spatially balanced, marginally balanced, and doubly balanced samples. The method’s versatility is demonstrated through three applications: constructing a master sample with spatially balanced panels, marginally balanced panels, and doubly balanced panels.

Confidence intervals are very often constructed based on a probability distribution that uses a certain number of degrees of freedom as a parameter. This is the case with the Student and the modified Wilson confidence intervals, discussed in this article, which use quantiles from the Student distribution where the number of degrees of freedom is generally unknown. For the length of a confidence interval to be representative of the reliability of an estimate, the actual coverage rate must match the nominal rate. To that end, the number of degrees of freedom in the probability distribution used in practice to calculate the confidence interval must be estimated as precisely as possible. An approximate rule is often used, although it tends to overestimate the actual number of degrees of freedom. In this article, a more precise version of degrees of freedom, derived from the Satterthwaite approximation, is obtained in the context of the Canadian Census of Population. The sampling design is equivalent to a simple random design without replacement, cluster-stratified, and the variance estimation method is an adaptation of the balanced repeated replication method. An explicit expression of the degrees of freedom is obtained under these conditions, enabling the factors influencing them to be identified. For comparison, the degree of freedom formula is also established for the conventional variance estimator. A simulation study shows that using this version of degrees of freedom corrects the undercoverage problem observed with the approximate rule, showing the importance of accurately assessing this number.

We test the notion that a quasi-probabilistic method of selecting individuals within households (last birthday, LB) draws in a different sample compared to a non-probabilistic approach that selects respondents according to known parameters on age and gender (frequency matching, FM). With data from an original field experiment, we evaluate fieldwork efficiency (time and completed cases), economy (cost), success in recruiting a representative sample, and differences across a set of attitudinal and behavioral measures. We find that the FM approach performs better on efficiency and cost and achieves a comparable sample; importantly, this comparability extends across measures of personality traits and public opinion. With appropriate caveats, we conclude that researchers’ choice of selection methods should be guided by both theoretical benefits and practical tradeoffs.

Probability-proportional-to-size sampling is widely used by national statistical offices. Here population units are selected with probabilities proportional to an auxiliary variable. Variance formulas in such designs require both first- and second-order inclusion probabilities. The computation of second-order inclusion probabilities is particularly challenging for large populations, and has been the subject of extensive research. This article presents some new exact and approximation formulas for second-order inclusion probabilities in randomized systematic sampling with unequal probabilities and without replacement.

Survey data typically have missing values due to unit and item nonresponse. Sometimes, survey organizations know the marginal distributions of certain categorical variables in the target population. As shown in previous work, survey organizations can leverage these distributions in multiple imputation for nonignorable unit nonresponse, generating imputations that result in plausible completed-data estimates for the variables with known margins. However, this prior work does not use the design weights for unit nonrespondents. We extend this previous work to utilize the design weights for all sampled units. We illustrate the approach using simulation studies.

Wayne A. Fuller is a leading figure in statistics whose career at Iowa State University (ISU) began in 1959; he is now Distinguished Professor Emeritus in Statistics and Economics. This article briefly recounts his early life and training in agricultural economics at ISU and highlights influential contributions spanning time series analysis, measurement error models, and survey sampling. It documents his impact through seminal textbooks, methodological advances such as the Dickey-Fuller test and regression estimation, sustained work on major operational surveys (e.g., the National Resources Inventory), and mentorship of many graduate students. The article includes an interview conducted on May 20th, 2025, at Professor Fuller’s home.