A note on Wilson coverage intervals for proportions estimated from complex samples
Section 2. The extension

• Previous
• Next

It is not hard to generalize Wilson coverage intervals (also called “score intervals”) to complex survey data. See, for example, Kott and Carr (1997). As with the Wilson itself, one simply solves this equation for the true proportion $P:$

$$\frac{{\left(p-P\right)}^2}{\left[\frac{P(1-P)}{n*}\right]}\le z_{1-\alpha /2}^2,(2.1)$$

where $p$ is a consistent estimator for $P$ under probability-sampling theory, and $z_{1-\alpha /2}$ is the Normal $z-$ score for $\left(1-\alpha /2\right)$ given the goal is to produce a $\left(1-\alpha \right)\%$ coverage interval $(\alpha$ is often set at 0.05). The missing piece to equation (2.1) is $n*,$ the so-called “effective sample size”, which in the standard Wilson formulation is the sample size $n.$ In our more general context, $n*=p\left(1-p\right)/\mathrm{var}\left(p\right),$ where $\mathrm{var}\left(p\right)$ is a consistent estimator for the variance of $p,\text{Var}\left(p\right).$

In order to calculate $n*,$ we need both $p\left(1-p\right),$ and $\text{var}\left(p\right)$ to be positive. In addition, let us assume that $1/n*=O_P\left(1/n^a\right)$ for some positive $a\le 1,$ $p-P=O_P\left(1/\sqrt{n*}\right),$ $0<\text{Var}\left(p\right)=O\left(1/n*\right),$ and $\mathrm{var}\left(p\right)/\text{Var}\left(p\right)$ is $1+O_P\left(1/\sqrt{n*}\right).$ Note that the last three are always true under simple random sampling with replacement so long as $P\left(1-P\right)\ge B>0.$

Dropping $O_P\left(1/{\left[n*\right]}^{3/2}\right)$ terms, but allowing $p\left(1-p\right)$ to be small (effectively $o_p(1)),$ one can derive this Wilson-like interval for $P$ from equation (2.1):

$$\begin{array}{l}p+\frac{1-2p}{n*}\frac{z_{1-\alpha /2}^2}{2}-z_{1-\alpha /2}{\left(\frac{p\left(1-p\right)}{n*}+\frac{z_{1-\alpha /2}^2}{4{\left(n*\right)}^2}\right)}^{1/2}\le P\hfill \\ \le p+\frac{1-2p}{n*}\frac{z_{1-\alpha /2}^2}{2}+z_{1-\alpha /2}{\left(\frac{p\left(1-p\right)}{n*}+\frac{z_{1-\alpha /2}^2}{4{\left(n*\right)}^2}\right)}^{1/2}.(2.2)\hfill \end{array}$$

We can call this the “complex-sampling Wilson coverage interval”. WesVar (2007) computes a variant of this interval that does not drop $O_P\left({1/\left[n*\right]}^{3/2}\right)$ terms. It is dropped here because other terms of that size will be dropped later in this note.

If it is reasonable to drop $O_P\left({1/\left[n*\right]}^{3/2}\right)$ terms in deriving equation (2.2), one can also safely ignore the difference between $1/n$ and $1/\left(n-1\right).$ Under simple random sampling without replacement, $n*=n/\left(1-f\right)\left(\text{or}\left(n-1\right)/\left(1-f\right)\right)$ where $f$ is the sampling fraction. When $f$ is very small, the distinction between with and without replacement sampling can be ignored.

Observe that under simple random sampling with replacement, the denominator of the pivotal appearing on the left-hand side of equation (2.1) has no variance at all. By contrast, the denominator in the traditional Wald pivotal, $\mathrm{var}\left(p\right)=p\left(1-p\right)/\left(n-1\right),$ can have considerable variance, especially when $p$ or $1-p$ is small. That is why Wilson intervals have superior performance under simple random sampling, whether with or without replacement.

That superiority carries over to complex sampling (see, for example, Kott, Andersson and Nerman, 2001), where the pivotal’s denominator is

$$\begin{array}{ll}\frac{P\left(1-P\right)}{n*}=\text{var}\left(p\right)\frac{P\left(1-P\right)}{p\left(1-p\right)}\hfill & =\text{var}\left(p\right)\left[1-\frac{\left(p-P\right)-\left(p^2-P^2\right)}{p\left(1-p\right)}\right]\hfill \\ \hfill & =\text{var}\left(p\right)\left[1-\frac{\left(p-P\right)-\left(p-P\right)\left(p+P\right)}{p\left(1-p\right)}\right]\hfill \\ \hfill & =\text{var}\left(p\right)-\frac{1-2P}{n*}\left(p-P\right)+{\text{O}}_P\left(1/{\left[n*\right]}^2\right),\hfill \end{array}$$

which is likely to have less variance than $\mathrm{var}\left(p\right)$ in most applications. For an intuition into why this is so, observe that a putative variance estimator of the form ${\mathrm{var}}_1\left(p\right)=\mathrm{var}\left(p\right)-b\left(p-P\right)$ is minimized when $b=\text{Cov}\left[\mathrm{var}\left(p\right),p\right]/\text{Var}\left(p\right).$ Under simple random sampling, whether with or without replacement, $b$ is exactly $\left(1-2P\right)/n*\text{}.$

Although the minimizing $b$  is not exactly equal to $\left(1-2P\right)/n*\text{},$ under more complex sampling designs, the optimal $b$ is likely to be closer to $\left(1-2P\right)/n*$ than to 0. It is thus not surprising that the variance of $\mathrm{var}\left(p\right)-\left[\left(1-2P\right)/n*\right]\left(p-P\right)$ will usually be less than the variance of $\mathrm{var}\left(p\right).$ Nevertheless, a slight improvement on the complex-sampling Wilson coverage interval can be made by replacing $n*$ in equation (2.2) by

$$\tilde{n}=\left[\left(1-2p\right)\mathrm{var}\left(p\right)\right]/\mathrm{cov}\left[\mathrm{var}\left(p\right),p\right]$$

when $\mathrm{cov}\left[\mathrm{var}\left(p\right),p\right],$ a consistent estimator for $\text{Cov}\left[\mathrm{var}\left(p\right),p\right],$ exists (see Kott et al., 2001).

As with the standard Wilson, the center of the complex-sample Wilson interval in equation (2.2) is slightly different from $p$ when $p$  is not $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.:$

$$C=p+\frac{1-2p}{n*}\frac{z_{1-\alpha /2}}{2}.$$

Its length $L$ appears longer than the Wald’s:

$$L=z_{1-\alpha /2}{\left(\frac{p\left(1-p\right)}{n*}+\frac{z_{1-\alpha /2}^2}{4{\left(n*\right)}^2}\right)}^{1/2}>z_{1-\alpha /2}{\left(\frac{p\left(1-p\right)}{n*}\right)}^{1/2}\text{}.$$

When $P\left(1-P\right)\ge B>0,$ however,

$$\begin{array}{ll}{\left(\frac{p\left(1-p\right)}{n*}+\frac{z_{1-\alpha /2}^2}{4{\left(n*\right)}^2}\right)}^{1/2}\hfill & ={\left(\frac{p\left(1-p\right)}{n*}\right)}^{1/2}{\left(1+\frac{\frac{1}{4}{z}_{1-\alpha /2}^2}{n*p\left(1-p\right)}\right)}^{1/2}\hfill \\ \hfill & ={\left(\frac{p\left(1-p\right)}{n*}\right)}^{1/2}+{\text{o}}_p\left(\frac{1}{n*}\right).(2.3)\hfill \end{array}$$

• Previous
• Article main page
• Next