Model-assisted sample design is minimax for model-based prediction
Section 1. Introduction

• Previous
• Next

Model-based inference about finite population totals relies on an assumed model and usually does not make reference to the sampling plan. Probability sampling, where every unit $i$ has a known probability of selection ${\pi }_i\mathrm{<mo>></mo>}0,$ is not strictly necessary, but is often used anyway, because it “eliminates conscious and unconscious bias” (Valliant, Dever and Kreuter, 2013, page 310) and ensures the non-informativeness of sampling which is required for most model-based procedures (Chambers and Clark, 2012, page 12). Särndal, Swensson and Wretman (1992, page 534) note that “proponents of model-based inference advocate randomized selection of the sample as a safeguard against selection bias, but the randomization probabilities play no role in the inference”. See also Lohr (2010, page 263), Chambers and Clark (2012, page 92) and Scott, Brewer and Ho (1978) who suggest probability sample designs for model-based predictions. For a review of the model-based approach, see also Valliant, Dorman and Royall (2000).

Model-assisted inference (e.g., Särndal et al., 1992) is an alternative approach, where estimators are design-unbiased (at least asymptotically), that is, unbiased over repeated probability sampling from any fixed population. Subject to this constraint, they minimize the anticipated variance (AV), which is the variance over both repeated realisations of the population from a model and repeated probability sampling. In models with independent errors, the lowest possible AV amongst such estimators (for any given probability sample design) is the Godambe-Joshi lower bound (GJLB) (Godambe and Joshi, 1965). The lower bound is asymptotically achieved for linear models by the well known generalized regression estimator.

Model-assisted designs are probability sample designs which are intended to minimize the AV of the generalized regression estimator (or, equivalently, to minimize the GJLB). These AV-optimal sample designs have been derived for model-assisted inference. In particular, the sample design which minimizes the GJLB for fixed expected sample size for models with independence has probability proportional to the square root of the model error variance for each unit $\left({\sigma }_i\right)$ (e.g., Särndal et al., 1992); this will be called a PP $\sigma$ design. The PPC $\sigma$ design is a generalization allowing for unequal unit costs (Steel and Clark, 2014). There are no analogous results on optimal probability sampling for model-based prediction, except under strong conditions, a gap that this paper partially fills. Isaki and Fuller (1982) suggested the design-estimation strategy of using the PP $\sigma$ design for model-based prediction, showing that this design is optimal when selection probabilities and their squares are in the column space of the matrix of covariates. This condition comes at a price, as will be seen in the simulation in this paper.

Optimal non-probability samples have been derived for model-based best linear unbiased predictors (BLUPs) under linear models. These tend to be somewhat extreme designs, where the units with the largest, or the largest and smallest, values of auxiliary variables are chosen (e.g., Royall, 1970). Robust model-based balanced designs have been developed, where one or more sample moments of auxiliary variables are equal to the corresponding population moments (Royall and Herson, 1973), while “over-balanced designs” meet a different constraint on the sample moments (Scott et al., 1978). Another balanced design was proposed by Kott (1986). These designs are robust to families of polynomial alternatives to a working linear model. They are not probability designs, although probability designs have been proposed in order to approximately meet balancing constraints (Valliant et al., 2000, Section 3.4). Exactly balanced probability designs have also been proposed (Tillé, 2006). The choice of balancing or over-balancing strategy depends on which set of polynomial alternatives is postulated. In another non-probability approach, Welsh and Wiens (2013) find the sample which minimizes the maximum model-based variance in a neighbourhood of a working model.

This article derives an asymptotic upper bound for the AV of the BLUP under probability sampling. The AV is the most relevant quantity for probability sample design even in the model-based framework, because averaging over all possible samples is appropriate in advance of sample selection. The bound is applicable to any probability sample design, and is over the space of possible covariate sets. This is useful for sample design in practice, because the precise model to be used is not decided until after data have been collected. For example, some design variables might not be included in the model if the sample data suggests that they have little relevance for the variable whose total is being estimated, but this would not be clear prior to surveying. Or splines might be used, with the number and placement of knots guided by the sample data. It turns out that the upper bound is the GJLB. This implies that model-assisted designs, such as PP $\sigma$ and PPC $\sigma ,$ are minimax strategies for model-based estimation. The upper bound is an equality when the model has a particular property, which is satisfied when the model is sufficiently rich and includes all design variables.

Other researchers have considered the relationship between the BLUP and the model-assisted generalized regression estimator, including conditions under which these two estimators are identical (e.g., Isaki and Fuller, 1982; Tam, 1988) and modifications to the BLUP so that it is equivalent to a generalized regression estimator at the expense of its optimality under the model (e.g., Brewer, Hanif and Tam, 1988; Brewer, 1999; Nedyalkova and Tillé, 2008). The results here are new because:

• Existing results do not cater for situations where both: the surveyor wants to use the BLUP because it is model-optimal, and the BLUP and the generalized regression estimators are not equal. The case where the two estimators are equal is shown here to be, in a particular sense, the worst case for the BLUP.
• An expression for the AV of the BLUP is derived and shown explicitly to be less than or equal to the upper bound. The result seems intuitively reasonable, given that the GJLB is attained by the design-consistent generalized regression estimator, whereas the BLUP is not subject to the constraint of design-based consistency. However, it is not at all obvious from the expression for the BLUP’s AV that the upper bound applies, so it is useful to have an explicit result.
• The interpretation is made that the upper bound is over the space of possible choices for the model covariates $x.$ Thus, the upper bound is relevant when the sample designer is unsure what model will ultimately be adopted once data have been collected.

Section 2 contains the key theoretical results. Section 3 confirms and illustrates the main result in a simulation study with a variable of interest $Y$ and two auxiliary variables: $x_1$ (continuous) and $x_2$ (binary). The expected value of $Y$ conditional on these variables is defined by a linear and a sinusoidal term in $x_1.$ It does not depend on $x_2.$ The probabilities of selection are a function of both $x_1$ and $x_2.$ BLUPs are calculated based on the model with lowest Bayesian Information Criterion (BIC) from a set including the simple linear model in $x_1$ and splines in $x_1$ of various degrees, both with and without $x_2.$ The ratio of the simulation prediction mean squared error (MSE) of the BLUP to the GJLB is either less than or equal to 1 or just above 1 across a range of scenarios. Section 4 is a discussion.

Much of the literature comparing model-assisted and model-based estimators and inference has focussed on bias due to mis-specified models when either (a) the mean function is incorrect, or (b) some design variables are inappropriately excluded. See for example Hansen, Madow and Tepping (1983) and the reworking of their simulation study in Valliant et al. (2000, Section 3.4). The simulation in Section 3 considers (a) and (b) to some extent, but this isn’t the main focus of the paper. The aim here is to see whether a committed model-based statistician can use the GJLB as an upper bound for the AV for sample design purposes, rather than to adjudicate between model-based and design-based inference. It is assumed that a sufficiently good model can be identified using the sample data; this process would be aided by a design which minimizes the maximal AV over the space of all linear models. A PP $\sigma$ or PPC $\sigma$ design is recommended when there is considerable uncertainty over the form of the final model.

• Previous
• Article main page
• Next