5 The weighted Dirichlet posterior

Jeremy Strief and Glen Meeden

Previous | Next

It is often the case that weights are attached to data in public use files. These weights are then used by researchers to make point and interval estimates of population parameters. We shall see that the stepwise Bayes weights introduced here can often be used in standard frequentist formulas to estimate parameters of interest just as the usual weights are. We will use our weights to define the Weighted Dirichlet posterior (WDP) and show that it gives an alternative way to compute point and interval estimates for a variety of population quantities.

Let the $w_j’\text{s}$ be a set of weights defined by equation (4.1) with ${\mu }_j=w_j/N.$ Consider the Dirichlet distribution over the simplex defined by the vector $n\mu =\left(n{\mu }_1,\dots ,n{\mu }_n\right)$ as an alternative posterior distribution for $p=\left(p_1,\dots ,p_n\right)$ when using the observed sample to generate complete simulated copies of the population. We will call this posterior the weighted Dirichlet posterior (WDP). Note the WDP is a looser version of the CPP. Under the CPP every complete copy of the population will satisfy the constraints; however, under the WDP, only the average of all the simulated populations will satisfy the constraints. It is easy to see that under the WDP

$E\left({\displaystyle \sum _{i=1}^n{p}_i{y}_i}\right)={\displaystyle \sum _{i=1}^n{\mu }_i{y}_i={\overline{y}}_{bw}}$ (5.1)

and

$$\begin{array}{cc}\hfill V\left({\displaystyle \sum _{i=1}^n{p}_i{y}_i}\right)& ={\displaystyle \sum _{i=1}^n{y}_i^{2}V\left(p_i\right)+\underset{i<j}{{\displaystyle \sum }{\displaystyle \sum }}}{y}_i{y}_j\text{Cov}\left(p_i,p_j\right)\hfill \\ & ={\displaystyle \sum _{i=1}^n\frac{n{\mu }_i\left(n-n{\mu }_i\right)y_i^2}{n^2\left(n+1\right)}}-2\underset{i<j}{{\displaystyle \sum }{\displaystyle \sum }}\frac{n{\mu }_{i}n{\mu }_j{y}_i{y}_j}{n^2\left(n+1\right)}\text{ (5.2)}\hfill \\ & =\frac{1}{n+1}\left({\displaystyle \sum _{i=1}^n{\mu }_i}\left(1-{\mu }_i\right)y_i^2+2\underset{i<j}{{\displaystyle \sum }{\displaystyle \sum }}{\mu }_{i}n{\mu }_j{y}_i{y}_j\right)\hfill \\ & =\frac{1}{n+1}\left({\displaystyle \sum _{i=1}^n{\mu }_i{y}_i^2-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{i=1}^n{\mu }_i{\mu }_j{y}_i{y}_j}}}\right)\hfill \\ & =\frac{1}{n+1}{\sigma }_{bw}^2\hfill \end{array}$$

where ${\overline{y}}_{bw}$ and ${\sigma }_{bw}^2$were defined in equation (4.2).

From this we see that when estimating the population mean, simulating from the WDP is equivalent to using the sample and their weights to construct the best guess for the population. In particular, when the weights are all equal the WDP is just the Polya posterior.

There are two main reasons for introducing the WDP. The first is that as the number of constraints used increases the approximate 0.95 credible intervals based on the CPP become too short and contain the true parameter value less than 95% of the time. This happens because with a large number of constraints the CPP does not allow enough variability in the simulated complete copies of the population which it generates. The second reason is that simulating from the WDP is much easier that simulating from the CPP. Now it would be possible to simulated from the constrained WDP in such a way that all the constraints would be satisfied but this involves as much effort as simulating from the CPP. Moreover, we believe that this would yield approximate 0.95 credible intervals which have poor frequentist coverage properties because they are too short.

Now suppose our set of weights is the reciprocals of the inclusion probabilities from the sampling design. Let $W={\displaystyle {\sum }_{i=1}^n{w}_i}.$ For most samples this value will not be equal to $N$but often is is quite close. Again we can construct our best guess for the population based on the weights. The mean and variance of this population will be

${\overline{y}}_{dw}={\displaystyle \sum _{i=1}^n\frac{w_i}{W}{y}_i}$ and ${\sigma }_{dw}^2={\displaystyle \sum _{i=1}^n\frac{w_i}{W}}{\left(y_i-{\overline{y}}_{dw}\right)}^2.$(5.3)

If we use ${\overline{y}}_{dw}$ as an estimate of the unknown population mean then an unbiased estimate of its variance depends on the joint inclusion probabilities of the units in the sample. Since these are often difficult to obtain, what has been recommended in practice (Särndal, Swensson and Wretman 1992) is to assume the sampling was done with replacement even when that is not the case. Then the resulting approximate estimate of variance for ${\overline{y}}_{dw}$is

$$\begin{array}{cc}\hfill {\widehat{V}}_d\left({\overline{y}}_{dw}\right)& =\frac{1}{n\left(n-1\right)}{\displaystyle \sum _{i=1}^n{\left(n\frac{w_i}{W}{y}_i-{\overline{y}}_{dw}\right)}^2}\text{ (5.4)}\hfill \\ & =\frac{{\sigma }_{dw}^2+{\gamma }_{dw}}{n-1}\hfill \end{array}$$

where the second line follows from some simple algebra and where

${\gamma }_{dw}={\displaystyle \sum _{i=1}^n\frac{w_i}{W}{y}_i^2\left(n\frac{w_i}{W}-1\right).}$ (5.5)

Note that when the design is simple random sampling with or without replacement and $N=nk$ then ${\gamma }_{dw}=0.$In this case, the estimate of variance in (5.4) is essentially equivalent to the variance in equation (5.2).

In situations where the Horvitz-Thompson estimator makes sense, calculations have shown that ${\gamma }_{dw}$ tends to be negative. This suggests that in such situations intervals based on the WDP will tend to be conservative. However calculations also show that ${\gamma }_{dw}$term tends to be positive in situations where the Horvitz-Thompson estimator is not appropriate. We will see in such cases that the usual approximation can work poorly and intervals based on the WDP can have better frequentist properties.

Previous | Next