3. Weighted finite population Bayesian bootstrap

Qi Dong, Michael R. Elliott and Trivellore E. Raghunathan

Previous | Next

3.1  Finite Population Bayesian Bootstrap (FPBB)

Assume that the (scalar) population elements $Y_i,i=1,\dots ,N$ are exchangeable and can take on $K\le N$ possible values $\left(b_1,\dots ,b_K\right);$ thus $Y_i|\theta ~\text{MULTI}\left(1;{\theta }_1,\dots ,{\theta }_K\right).$ Further assuming a conjugate Dirichlet prior for $\theta ~\text{DIR}\left({\alpha }_1,\dots ,{\alpha }_K\right)$ yields (Ghosh and Meeden 1983)

$$\begin{array}{cc}\hfill P\left(Y_{\text{nob}}|y\right)& =P\left(b_1^{\text{nob}}=N_1-n_1,\dots ,b_K^{\text{nob}}=N_K-n_K|b_1^{\text{obs}}=n_1,\dots ,b_K^{\text{obs}}=n_K\right)\hfill \\ & =\frac{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^{1}p\left(Y_{\text{nob}}|y,\theta \right)p\left(y|\theta \right)p\left(\theta \right)d}{\theta }_1\dots d{\theta }_K}}{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^{1}p\left(y|\theta \right)p\left(\theta \right)d}{\theta }_1\dots d{\theta }_K}}\hfill \\ & =\frac{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^{1}p\left(Y_{\text{nob}}|\theta \right)p\left(y|\theta \right)p\left(\theta \right)}d{\theta }_1\dots d{\theta }_K}}{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^{1}p\left(y|\theta \right)p\left(\theta \right)}d{\theta }_1\dots d{\theta }_K}}\text{ (3}\text{.1) }\hfill \\ & =\frac{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^1{\displaystyle {\prod }_{i=1}^K{\theta }_i^{N_i-n_i}}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{n_i}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{{\alpha }_i-1}d{\theta }_1\dots d{\theta }_K}}}{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^1{\displaystyle {\prod }_{i=1}^K{\theta }_i^{n_i}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{{\alpha }_i-1}d{\theta }_1\dots d{\theta }_K}}}}\hfill \\ & =\frac{\left({\displaystyle {\prod }_{i=1}^K\Gamma \left(N_i+{\alpha }_i\right)/\Gamma \left({\alpha }_i\right)}\right)/\left(\Gamma \left(N+{\alpha }_0\right)/\Gamma \left({\alpha }_0\right)\right)}{{\displaystyle {\prod }_{i=1}^K\Gamma \left(n_i+{\alpha }_i\right)}/\Gamma \left(n+{\alpha }_0\right)}\hfill \end{array}$$

where ${\alpha }_0={\displaystyle {\sum }_{i=1}^K{\alpha }_i},$ ${\displaystyle {\sum }_{i=1}^K{N}_i=N},$ and $n_1,\dots ,n_K$ refers to the number of distinct values we observe from our sample $y=\left(y_1,\dots ,y_n\right),$ ${\displaystyle {\sum }_{i=1}^K{n}_i=n}.$ If ${\alpha }_i\equiv 0$ then $p\left(Y_{\text{nob}}|y\right)$ reduces to

$$\left({\displaystyle \prod _{i=1}^K\Gamma \left(N_i\right)/\Gamma \left(n_i\right)}\right)/\left(\Gamma \left(N\right)/\Gamma \left(n\right)\right).$$

To ease implementation, Lo (1988) proposed making draws from the FPBB posterior predictive distribution using a "Pólya urn scheme� procedure. Suppose an urn contains $n$ balls, each of which have a distinct real number label $b_i,i=1,\dots ,K.$ A Pólya sample of size $m$ is selected by first selecting a ball at random from the urn and returning the selected ball into the urn, then putting one same ball into the urn and repeating this process until $m$ balls have been selected. It can be shown that the probability of getting $m_i$ balls of type $b_i$ is given by

$$p\left(b_1=m_1,\dots ,b_K=m_K\right)=\frac{{\displaystyle {\prod }_{i=1}^k\Gamma \left(n_i+m_i\right)}/\Gamma \left(n_i\right)}{\Gamma \left(n+m\right)/\Gamma \left(n\right)}\text{ (3}\text{.2)}$$

where $n_i$ is the number of balls of type $b_i$ originally in the urn. The distribution of the counts of type $b_i$ is invariant under any permutation of the draws. Note that this corresponds directly to the posterior probability of a total of $\left(m_1,\dots ,m_K\right)$ elements of type $\left(b_1,\dots ,b_K\right)$ in a population, given that $\left(n_1,\dots ,n_K\right)$ elements were observed in a (simple random) sample of size ${\displaystyle {\sum }_{i=1}^K{n}_i=n}.$ Hence a FPBB replicate sample can be drawn from this Pólya posterior using the following steps:

Step 1. Draw a Pólya sample of size $m=N-n,$ denoted by $\left(y_1^{*},\dots ,y_{N-n}^{*}\right)$ from the urn $\left\{y_1,\dots ,y_n\right\};$ by (3.2), with $m_k=N_k-n_k$ draws of value $b_k^{\text{obs}}$ for $k=1,\dots ,K,$ this corresponds to a draw of $P\left(Y_{\text{nob}}|y\right)$ from (3.1).

Step 2. Form the FPBB population $y_1,\dots ,y_n,y_1^{\text{*}},\dots ,y_{N-n}^{\text{*}}.$

3.2 FPBB with unequal probabilities of selection

Cohen (1997) extended the FPBB procedure to adjust for the unequal probabilities of selection. Assume $\left(y_1,\dots ,y_n\right)$ is a sample from a finite population $\left(Y_1,\dots ,Y_N\right)$ with design weights $\left(w_1,\dots ,w_n\right),$ where

$$w_i=\frac{1}{P\left(I_i=1\right)}$$

and $I$ is the sampling indicator. The procedure has two steps:

Step 1. Draw a sample of size $N-n,$ denoted by $\left(y_1^{\text{*}},\dots ,y_{N-n}^{\text{*}}\right),$ by drawing $y_k^{\text{*}}$ from $\left(y_1,\dots ,y_n\right)$ in such a way that $y_i$ is selected with probability

$$\frac{w_i-1+l_{i,k-1}\text{*}\left(N-n\right)/n}{N-n+\left(k-1\right)\text{*}\left(N-n\right)/n},$$

where $w_i$ is the weight of unit $i$ and $l_{i,k-1}$ is the number of bootstrap selections of $y_i$ among $y_1^{\text{*}},\dots ,y_{k-1}^{\text{*}}.$ (The function wtpolyap in the R package polypost can be used to obtain draws from a weighted Pólya urn.)

Step 2. Form the FPBB population $y_1,\dots ,y_n,y_1^{\text{*}},\dots ,y_{N-n}^{\text{*}}.$

Although Cohen (1997) did not provide theoretical proof for this procedure, it can be obtained as a straightforward extension of the standard FPBB and Pólya urn equivalency described in Section 3.1. First, we determine the posterior distribution of the FPBB sample with unequal probabilities of selection implied by the weighted FPBB procedure. The multinomial likelihood based on our weighted sample is given by

$$p\left(y_{\text{obs}}|\theta \right)={\displaystyle \prod _{i=1}^K{\theta }_i^{w_i^{*}}},$$

where

$$w_i^{*}=\left(\frac{n}{N-n}\right){\displaystyle \sum _{j=1}^{n}I\left(y_j=b_i\right)\left(w_j-1\right)}$$

is the sum of the design weights minus one across all sampled elements with value $b_i,i=1,\dots ,K,$ normalized to sum to $n.$ (Note that this removes subjects sampled with weights equal to one $–$ "certainty sample� elements $–$ from the likelihood, as they have no chance to be part of the unobserved portion of the population, and thus contribute no information about these unobserved elements.) Assuming an improper Dirichlet prior $p\left(\theta \right)={\displaystyle {\prod }_{i=1}^k{\theta }_i^{-1}},$ the weighted finite population Bayesian bootstrap posterior is given by

$$\begin{array}{cc}\hfill P\left(Y_{\text{nob}}|y,w\right)& =P\left(b_1^{\text{nob}}=r_1,\dots ,b_K^{\text{nob}}=r_K|w_1^{*},\dots ,w_K^{*}\right)\hfill \\ & =\frac{{\displaystyle {\int }_0^1\dots {\displaystyle {\int }_0^{1}p\left(Y_{\text{nob}}|\theta \right)p\left(y|\theta \right)p\left(\theta \right)}d{\theta }_1\dots d{\theta }_K}}{{\displaystyle {\int }_0^1\cdots {\displaystyle {\int }_0^{1}p\left(y|\theta \right)p\left(\theta \right)}d{\theta }_1\dots d{\theta }_K}}\hfill \\ & =\frac{{\displaystyle {\int }_0^1\cdots {\displaystyle {\int }_0^1{\displaystyle {\prod }_{i=1}^K{\theta }_i^{r_i}}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{w_i^{*}}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{-1}d{\theta }_1\cdots d{\theta }_K}}}{{\displaystyle {\int }_0^1\cdots {\displaystyle {\int }_0^1{\displaystyle {\prod }_{i=1}^K{\theta }_i^{w_i^{*}}}{\displaystyle {\prod }_{i=1}^K{\theta }_i^{-1}d{\theta }_1\cdots d{\theta }_K}}}}\text{ (3}\text{.3)}\hfill \\ & ={\displaystyle \prod _{i=1}^K\frac{\Gamma \left(w_i^{*}+r_i\right)}{\Gamma \left(w_i^{*}\right)}}/\frac{\Gamma \left(N\right)}{\Gamma \left(n\right)}\hfill \end{array}$$

since ${\displaystyle {\sum }_{j=1}^n{r}_i}=N-n$ and ${\displaystyle {\sum }_{j=1}^n{w}_i^{*}}=n.$

Next, we show the distribution of samples obtained from the unequal probability of selection Pólya Urn scheme of Cohen (1997) is equal to the posterior distribution of the FPBB sample with unequal probabilities of selection. Given the observed data, the probability that we draw $N-n$ balls and that the first $r_1$ balls have value $b_1$ through the last $r_k$ balls have value $b_k$ is:

$$\begin{array}{c}P\left(b_1=r_1,\dots ,b_K=r_K\right)=\frac{w_1^{*}}{n}\times \frac{w_1^{*}+1}{n+1}\dots \times \frac{w_1^{*}+r_1-1}{n+r_1-1}\times \dots \times \frac{w_K^{*}}{n+{\displaystyle {\sum }_{i=1}^{k-1}{r}_i}}\times \dots \times \frac{w_K^{*}+r_K-1}{n+{\displaystyle {\sum }_{i=1}^k{r}_i-1}}\\ ={\displaystyle \prod _{i=1}^K\frac{\Gamma \left(w_i^{*}+r_i\right)}{\Gamma \left(w_i^{*}\right)}}/\frac{\Gamma \left(N\right)}{\Gamma \left(n\right)}\end{array}$$

where the first equality follows from the fact the distribution of the counts of type $b_i$ is invariant under any permutation of the draws, as in the unweighted setting, and the second equality from the identity $\Gamma \left(x\right)=\left(x-1\right)\Gamma \left(x\right)$ for $x>0.$ Thus, noting that

$$\frac{w_i-1+l_{i,k-1}*\left(N-n\right)/n}{N-n+\left(k-1\right)*\left(N-n\right)/n}=\frac{w_i^{*}+l_{i,k-1}}{n+\left(k-1\right)},$$

a draw from the unequal probability of selection Pólya Urn scheme yields a draw from $P\left(Y_{\text{nob}}|y,w\right)$ in (3.3).

Previous | Next