On combining independent probability samples
Section 2. Combining separate estimates

• Previous
• Next

We assume that we have $k\ge 2$ estimators, ${\widehat{Y}}_1,{\widehat{Y}}_2,\dots ,{\widehat{Y}}_k$ of a population total $Y,$ resulting from $k$ independent samples from the same population. Our options greatly depend on what information is available. If we have estimates and corresponding variance estimates, then a linear combination based on weights calculated from estimated variances may be an interesting option. We could also weight the estimators with respect to sample size, if available, but that is known to be far from optimal in some situations. We recall the theory for an optimal linear combination of independent unbiased estimators. The linear combination of ${\widehat{Y}}_1,{\widehat{Y}}_2,\dots ,{\widehat{Y}}_k$ with the smallest variance is

$${\widehat{Y}}_L\mathrm{<mo>=</mo>}{\alpha }_1{\widehat{Y}}_1+{\alpha }_2{\widehat{Y}}_2+\mathrm{...}+{\alpha }_k{\widehat{Y}}_k\mathrm{,}$$

where

$${\alpha }_i=\frac{1/V_i\left({\widehat{Y}}_i\right)}{{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>\; 1}}^{k}1/V_j\left({\widehat{Y}}_j\right)}}$$

are positive weights that sum to 1. The variance of ${\widehat{Y}}_L$ is

$$V\left({\widehat{Y}}_L\right)\mathrm{<mo>=</mo>}\frac{1}{{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>\; 1}}^{k}1/V_j\left({\widehat{Y}}_j\right)}}\mathrm{.}$$

It is common that variance estimates are used in place of the unknown variances when calculating the $\alpha$ -weights, see Cochran and Carroll (1953) and Cochran (1954). If the variance estimators are consistent, that approach will asymptotically provide the optimal weighting. Moreover, under the assumption that the variance estimators are independent of the estimators ${\widehat{Y}}_1,{\widehat{Y}}_2,\dots ,{\widehat{Y}}_k,$ the resulting estimator

$${\widehat{Y}}_L^{*}\mathrm{<mo>=</mo>}{\widehat{\alpha }}_1{\widehat{Y}}_1+{\widehat{\alpha }}_2{\widehat{Y}}_2+\mathrm{...}+{\widehat{\alpha }}_k{\widehat{Y}}_k\mathrm{,}$$

is unbiased and its variance depends only on the variance of ${\widehat{Y}}_L$ and the MSEs of the ${\widehat{\alpha }}_i’s,$ see Rubin and Weisberg (1974). However, as we soon will illustrate, the assumption of independence is likely to be violated in many sampling applications. In case of positive correlations between the estimators and their variance estimators, we will on average put more weight on small estimates because they tend to have smaller estimated variances. Thus the combined estimator (using weights based on estimated variances) will be negatively biased and the negative bias can increase as the number of independent surveys we combine increases, see Example 1. The opposite holds as well, in case of negative correlation, but that is likely a rarer situation in sampling applications.

Example 1: A very simplistic example that illustrates that the bias can increase as the number of independent surveys we combine increase. Let the unbiased estimator $\widehat{Y}$  for one sample take the values 1 or 2 with equal probabilities and let the variance estimator take values $c$  times the estimator (perfectly correlated) and let it be unbiased $\left(c=1/6\right).$  Clearly the expected value of $\widehat{Y}$  is 1.5. Next, we consider the linear combination of two independent estimators $\left({\widehat{Y}}_1\mathrm{,}{\widehat{Y}}_2\right)$  of the same type as $\widehat{Y}$  using estimated variances. The pair $\left({\widehat{Y}}_1\mathrm{,}{\widehat{Y}}_2\right)$  has the following four possible outcomes (1,1), (1,2), (2,1), (2,2), each with probability 1/4. The corresponding outcomes for the linear combination ${\widehat{Y}}_L^{*}$  with estimated variances are 1, 4/3, 4/3, 2 with expectation $17/12\approx \mathrm{1.4167.}$  It is negatively biased. If a third independent estimator of the same type is added we have the eight outcomes (1,1,1), (1,1,2), (1,2,1), (1,2,2), (2,1,1), (2,1,2), (2,2,1), (2,2,2), each with equal probability 1/8. The corresponding outcomes for ${\widehat{Y}}_L^{*}$  are 1, 6/5, 6/5, 3/2, 6/5, 3/2, 3/2, 2, with expectation $111/80\mathrm{<mo>=</mo>\; 1.3875.}$  It is even more negatively biased, and the bias continues to grow as more independent estimators of the same type are added in the combination.

2.1  Why positive correlation between estimator and variance estimator is common in sampling applications

The issue of positive correlation between the estimator of a total and its variance estimator has previously been noticed by e.g., Gregoire and Schabenberger (1999) when sampling skewed biological populations, but we show that a high correlation may appear in more general sampling applications. Assume that the target variable is non-negative and that $y_i\mathrm{<mo>></mo>\; 0}$ for exactly $N^{\prime }$ units. The proportion of non-zero (positive) $y_i’s$ is denoted by $p\mathrm{<mo>=</mo>}{N}^{\prime }/N.$ This is a very common situation in sampling and we get such a target variable if we estimate a domain total $(y_i\mathrm{<mo>=</mo>\; 0}$ outside of the domain) or if only a subset of the population has the property of interest.

The design-based unbiased Horvitz-Thompson (HT) estimator is given by

$$\widehat{Y}\mathrm{<mo>=</mo>}{\displaystyle \sum _{i\in S}}\frac{y_i}{{\pi }_i}\mathrm{,}$$

where $S$ denotes the random set of sampled units and ${\pi }_i\mathrm{<mo>=</mo>}\mathrm{Pr}\left(i\in S\right).$ Under fixed size designs the variance of $\widehat{Y}$ is

$$V\left(\widehat{Y}\right)\mathrm{<mo>=</mo>}-\frac{1}{2}{\displaystyle \sum _{i\mathrm{<mo>=</mo>\; 1}}^N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^N}\left({\pi }_{ij}-{\pi }_i{\pi }_j\right){\left(\frac{y_i}{{\pi }_i}-\frac{y_j}{{\pi }_j}\right)}^2\mathrm{,}$$

where ${\pi }_{ij}\mathrm{<mo>=</mo>}\mathrm{Pr}\left(i\in S\mathrm{,}j\in S\right)$ is the second order inclusion probability. The corresponding variance estimator is

$$\widehat{V}\left(\widehat{Y}\right)\mathrm{<mo>=</mo>}-\frac{1}{2}{\displaystyle \sum _{i\in S}}{\displaystyle \sum _{j\in S}}\frac{{\pi }_{ij}-{\pi }_i{\pi }_j}{{\pi }_{ij}}{\left(\frac{y_i}{{\pi }_i}-\frac{y_j}{{\pi }_j}\right)}^2\mathrm{.}$$

Provided that all ${\pi }_{ij}$ are strictly positive, it follows that the variance estimator is an unbiased estimator of  $V\left(\widehat{Y}\right).$

The number of non-zero $y_i’s$ in $S$ (and hence in $\widehat{Y})$ is here denoted by $n^{\prime }$ and it will usually be a random number. It can be shown that the number of non-zero elements in $\widehat{V}\left(\widehat{Y}\right)$ is approximately proportional to $n^{\prime }$ if $p$ is small, which indicates that there might be a strong correlation between $\widehat{Y}$ and $\widehat{V}\left(\widehat{Y}\right)$ in general if $p$ is small. To show that the number of non-zero terms in $\widehat{V}\left(\widehat{Y}\right)$ is approximately proportional to $n^{\prime }$ we look at three cases, where the third case is the most general. 

Case 1: Assume that all the non-zero $y_i/{\pi }_i’s$ are different, i.e., $y_i/{\pi }_i\ne y_j/{\pi }_j$ for $i\ne j,$ and ${\pi }_{ij}\ne {\pi }_i{\pi }_j$ for all $i\mathrm{,}j.$ The double sum in $\widehat{V}\left(\widehat{Y}\right)$ then contains $2n^{\prime }\left(n-n^{\prime }\right)$ non-zero terms of the form

$$\frac{{\pi }_{ij}-{\pi }_i{\pi }_j}{{\pi }_{ij}}{\left(\frac{y_k}{{\pi }_k}\right)}^2\mathrm{,}$$

where $k$ is equal to $i$ or $j$ and $i\ne j.$ There are $n^{\prime }\left(n^{\prime }-1\right)$ non-zero terms of the form

$$\frac{{\pi }_{ij}-{\pi }_i{\pi }_j}{{\pi }_{ij}}{\left(\frac{y_i}{{\pi }_i}-\frac{y_j}{{\pi }_j}\right)}^2\mathrm{,}$$

where $i\ne j.$ In total the number of non-zero terms is $n^{\prime }\left(2n-n^{\prime }-1\right).$ If $n$ is fairly large and $p$ is small, then $n^{\prime }<<n$ and roughly we have $n^{\prime }\left(2n-n^{\prime }-1\right)\approx 2n^{\prime }n.$ The number of non-zero terms is approximately proportional to $n^{\prime }.$

Case 2: Assume that all the non-zero $y_i/{\pi }_i’s$ are equal, e.g., $y_i$ is an indicator variable and ${\pi }_i\mathrm{<mo>=</mo>}n/N,$ and ${\pi }_{ij}\ne {\pi }_i{\pi }_j$ for all $i\mathrm{,}j.$ Then the double sum in $\widehat{V}\left(\widehat{Y}\right)$ contain $2n^{\prime }\left(n-n^{\prime }\right)$ non-zero terms of the form

$$\frac{{\pi }_{ij}-{\pi }_i{\pi }_j}{{\pi }_{ij}}{\left(\frac{y_k}{{\pi }_k}\right)}^2\mathrm{,}$$

where $k$ is equal to $i$ or $j$ and $i\ne j.$ If $n$ is fairly large and $p$ is small, then $n^{\prime }<<n$ and roughly we have $2n^{\prime }\left(n-n^{\prime }\right)\approx 2n^{\prime }n.$ Thus, the number of non-zero terms is still approximately proportional to $n^{\prime }.$

Case 3: If some of the non-zero $y_i/{\pi }_i’s$ are equal and the rest are different, then the number of non-zero terms will be between $2n^{\prime }\left(n-n^{\prime }\right)$ (case 2) and $n^{\prime }\left(2n-n^{\prime }-1\right)$ (case 1). Thus, the number of non-zero terms in $\widehat{V}\left(\widehat{Y}\right)$ is always approximately proportional to $n^{\prime }$ if $p$ is small.

If ${\pi }_{ij}\le {\pi }_i{\pi }_j$ for all $i\ne j,$ then all non-zero terms are positive. This condition holds e.g., for simple random sampling (SRS) and high entropy unequal probability designs such as Conditional Poisson, Sampford and Pareto. More discussion about entropy of sampling designs can be found in e.g., Grafström (2010). The average size of the positive terms in $\widehat{V}\left(\widehat{Y}\right),$ or $\widehat{Y},$ is not likely to depend much on $n^{\prime }.$ Thus, if $\widehat{Y}$ contains $n^{\prime }$ positive terms, and $\widehat{V}\left(\widehat{Y}\right)$ contains a number of positive terms that is proportional to $n^{\prime },$ their sizes are mainly determined by $n^{\prime }.$ A high relative variance in $n^{\prime }$ can cause a high correlation between $\widehat{Y}$ and $\widehat{V}\left(\widehat{Y}\right),$ see Example 2.

Commonly used designs can produce a high relative variance for $n^{\prime }.$ If we do simple random sampling without replacement we get $n^{\prime }\sim \text{Hyp}\left(N\mathrm{,}{N}^{\prime }\mathrm{,}n\right)$ and $V\left(n^{\prime }\right)/E\left(n^{\prime }\right)\mathrm{<mo>=</mo>}\left(1-p\right)\left(N-n\right)/\left(N-1\right)\approx \left(1-p\right)\left(1-n/N\right),$ which means that we need a large $p$ or a large sample fraction $n/N$ in order to achieve a small relative variance for $n^{\prime }.$ In many applications we will have a rather small $p$ and a small sampling fraction $n/N$ and, thus, for many designs (that do not use prior information which can explain to some extent if $y_i\ne 0$ or not) there will be a high relative variance for $n^{\prime }.$ To illustrate the magnitude of the resulting correlation between the estimator and its variance estimator an example for simple random sampling without replacement follows.

Example 2: For this example we first simulate a population of size $N\mathrm{<mo>=</mo>}\text{1,000}$  where $N^{\prime }\mathrm{<mo>=</mo>\; 100},$  i.e., $p\mathrm{<mo>=</mo>}\text{0}\text{.1}.$  The 100 non-zero $y-$values are simulated from $N\left(\mu \mathrm{,}{\sigma }^2\right)$  with $\mu \mathrm{<mo>=</mo>\; 10}$  and $\sigma \mathrm{<mo>=</mo>\; 2.}$  We select samples of size $n\mathrm{<mo>=</mo>\; 200}$  with simple random sampling, so ${\pi }_i\mathrm{<mo>=</mo>}n/N$  and ${\pi }_{ij}\mathrm{<mo>=</mo>}n\left(n-1\right)/\left(N\left(N-1\right)\right)$  for $i\ne j.$  The observed correlation between $\widehat{Y}$  and $\widehat{V}\left(\widehat{Y}\right)$  was 0.974 for ${10}^6$  samples, see Figure 2.1 for the first 1,000 observations of $\left(\widehat{Y}\mathrm{,}\widehat{V}\left(\widehat{Y}\right)\right).$  If we increase $p$  to 0.3, the correlation is still above 0.9. The results remain unchanged if the ratio $\sigma /\mu$  remains unchanged, e.g., we get the same correlations if $\mu \mathrm{<mo>=</mo>\; 100}$  and $\sigma \mathrm{<mo>=</mo>\; 20.}$

Now, assume we have access to more than one sample for the estimation of $Y.$ As previously noted, with high positive correlations between the estimators and their corresponding variance estimators there is a risk of severe bias if we use a linear combination with estimated variances. The interest of using combined information may be the largest for small domains or rare properties, in which case the problem of high correlation is the most likely. Next, we turn to alternative options for using combined information from multiple samples.

• Previous
• Article main page
• Next