Are probability surveys bound to disappear for the production of official statistics?
Section 2. Background

• Previous
• Next

One of the first steps to meet information needs is to define the target population for which that information is sought. We denote this target population by $U.$ Then, it is necessary to define the parameters of interest, i.e. what it is desired to know about the target population. In practice, it is often desired to estimate many parameters. To simplify the discussion, we suppose that only one parameter is of interest: the total of the variable $y,$ $\theta ={\displaystyle {\sum }_{k\in U}{y}_k},$ where $y_k$ is the value of the variable $y$ for unit $k$ of the population $U.$ We use $Y$ to denote the vector containing the values $y_k$ for $k\in U.$ Lastly, a set of procedures must be established for the estimation of the parameter $\theta$ while taking into account various factors, such as the available budget, the respondent burden, the desired precision, etc. During this process, it is necessary to identify the data sources that will be used – probabilistic or not – and a statistical inference framework that will allow for assessing the properties of the estimates produced, such as bias and variance.

The above sequence, which starts with defining the target population and parameters of interest, followed by the data sources and estimation procedures, is consistent with the proposal by Citro (2014). She suggests that national statistical agencies first determine the information needs along with potential users. Next, they can work at identifying the data source(s) that will meet those needs while preserving an acceptable quality of estimates, keeping costs within the established budget and controlling for respondent burden. It seems preferable to avoid the reverse procedure, however tempting it is, of first identifying available data sources and then artificially determining the needs based on what can be produced by these sources. In general, this kind of procedure cannot adequately meet users’ actual needs.

We assume that we have access to data from a non-probability source (e.g., administrative data, web survey data, etc.). Values are observed for a few variables, including a variable $y^{*}\text{},$ for all units of a subset of $U,$ denoted as $s_{\text{NP}}.$ The variable $y^{*}$ is not necessarily equal to $y$ because of conceptual differences and/or measurement errors. At least, it is hoped that there is a strong association between the two variables. We denote the inclusion indicator in $s_{\text{NP}}$ as ${\delta }_k;$ in other words, ${\delta }_k=1$ if unit $k$ is in $s_{\text{NP}}$ and ${\delta }_k=0,$ otherwise. The vector of the inclusion indicators ${\delta }_k$ for $k\in U$ is denoted by $\delta .$

Data from a probability survey may also be available. In that case, a sample $s_P$  of the population $U$  is randomly selected with probability $p\left(s_P|Z\right).$  The matrix $Z$  contains information available on the sampling frame that is used to define the sampling design, such as stratum identifiers for each unit of the population. The sample inclusion indicators, $I_k,$   $k\in U,$ are defined as follows: $I_k=1$  if unit $k$  is selected in the sample $s_P;$  otherwise, $I_k=0.$  We use $I$  to denote the vector containing the sample inclusion indicators for $k\in U.$  The probability that unit $k$  of the population $U$  is chosen in the sample is denoted by ${\pi }_k=E\left(I_k|Z\right).$  Most of the time it is known or can be approximated. We assume that ${\pi }_k>0,$   $k\in U.$  For each unit $k\in s_P,$  the values of certain variables are collected, which may or may not include the variable $y.$

We use $\Omega$ to denote the set of all the auxiliary data used to make inferences. Among other things, $\Omega$ includes the design information, $Z,$ if a probability sample is used, and potentially other auxiliary variables such as calibration variables, matching variables or explanatory variables of a model (see Sections 3 and 4). The inclusion indicator ${\delta }_k$ can also be used as an auxiliary variable either for stratifying the population or for calibration (see Section 3). The vector $\delta$ can thus be included in $Z$ and $\Omega .$

The following two assumptions are used throughout the article:

Assumption 1: $I$ is independent of $\Omega$ and $Y$ after conditioning on $Z.$

Assumption 2: $\delta$ and $I$ are independent after conditioning on $\Omega$ and $Y.$

Assumption 1 implies that the values of the variables included in $\Omega$ and $Y$ are not affected by whether or not a unit is included in the sample $s_P.$ This is implicit in the literature on probability surveys and results from the very definition of the sampling design, which depends only on $Z.$ Assumption 2 is automatically satisfied if the non-probability source (and thus $\delta )$ is available prior to selecting the probability sample. Note that if ${\delta }_k$ is used as an auxiliary variable to stratify the population, then $\delta$ is included in $\Omega$ and assumption 2 is still satisfied. It will not be satisfied if being selected in $s_P$ impacts the provision of data to the non-probability source. For example, being selected in $s_P$ (and contacted) could be an indirect reminder for the selected individual to fill out forms required by the government (non-probability source). It can be expected that assumptions 1 and 2 are satisfied in most cases.

The union of $\Omega ,$ $\delta ,$ $I$ and $Y$ contains all the information used for making inferences. The various approaches to inference set out in Sections 3 and 4 differ in what they treat as fixed and what they treat as random. For example, in the design-based approach to inference, everything is considered fixed except for the vector $I;$ in other words, design-based inferences are conditional on $\Omega ,$ $Y$ and $\delta .$ To simplify the notation, we use ${\Omega }_P$ to denote the union of $\Omega ,$ $Y$ and $\delta .$ Thus, design expectations are denoted as $E\left(\cdot |{\Omega }_P\right)$ rather than $E\left(\cdot |\Omega ,Y,\delta \right).$ In the design-based approach to inference, an estimator $\widehat{\theta }$ of $\theta$ is usually chosen so that the design bias, $E\left(\widehat{\theta }-\theta |{\Omega }_P\right),$ is zero or negligible. Under assumptions 1 and 2, we note that $E\left(I_k|{\Omega }_P\right)=E\left(I_k|Z\right)={\pi }_k.$ For estimating the total $\theta ={\displaystyle {\sum }_{k\in U}{y}_k},$ an estimator of the form $\widehat{\theta }={\displaystyle {\sum }_{k\in s_P}{w}_k{y}_k}$ is frequently used, where $w_k$ is a survey weight for unit $k.$ The standard basic weight is $w_k={\pi }_k^{-1}.$ This weight ensures that the estimator $\widehat{\theta }$ is exactly design-unbiased for $\theta .$ The basic weight can then be modified using calibration techniques (e.g., Deville and Särndal, 1992; Haziza and Beaumont, 2017). The advantage of this approach is its non-parametric nature: no model assumption is needed for making valid inferences about the population because the first two design moments are controlled by the statistician and are usually known. Yet, the approach is not free of assumptions, for example to ensure the consistency and asymptotic normality of estimators, but it does not require any parametric model.

In practice, non-response is often observed in probability surveys as well as other non-sampling errors. Non-response of some sample units is often viewed as an additional phase of sampling that is not controlled by the statistician. In other words, the non-response mechanism is not known, unlike the sampling design. Assuming an adequate model for the non-response mechanism, estimators with little or no bias can be obtained, for example by weighting the responding units by the inverse of their estimated response probability. However, this requires careful modelling of the response indicators. In the rest of this paper, we ignore non-sampling errors and assume that the estimates from the probability survey are not biased or, at least, that their bias is small compared to the bias of the estimates from the non-probability source alone. This assumption may not always be satisfied in practice, but it is reasonable in many contexts (see Brick, 2011), especially in large surveys conducted by national statistical agencies.

The acquisition of data from non-probability sources is generally inexpensive compared to the cost of collecting data from a probability survey. Therefore, they would ideally be used to replace data from a probability survey. This data replacement is valid only if $y_k^{*}\approx y_k,$ $k\in U.$ This assumption will not be satisfied with all non-probability data sources, but may be realistic with some administrative data sources. In Sections 3 and 4, we will differentiate the methods based on the assumption that $y_k^{*}=y_k$ from the methods not requiring this assumption. Several methods described in Sections 3 and 4 are also reviewed in the upcoming article by Rao (2020) that was presented to Statistics Canada in summer 2018.

• Previous
• Article main page
• Next