Bayesian pooling for analyzing categorical data from small areas
Section 2. Hierarchical Bayesian models

• Previous
• Next

2.1   Parametric models

For a hierarchical Bayesian baseline model, we consider an $I\times K$ contingency table, where $n_{ik}$ indicates the $k^{\text{th}}$ response in the $i^{\text{th}}$ area, for $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I\mathrm{,}k\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,K.$ Let ${\pi }_{ik}$ denote the corresponding proportion for each cell. Then, we assume that

$$n_i|{\pi }_i\stackrel{\text{iid}}{{\displaystyle ~}}\text{Multinominal}\left(n_{i\mathrm{.}}\mathrm{,}{\pi }_i\right)\mathrm{,}(2.1)$$

where $n_i\mathrm{<mo>=</mo>}\left(n_{i1}\mathrm{,}\dots ,n_{iK}\right)$ for $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I,$ is a vector of responses, $n_{i\mathrm{.}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^K{n}_{ik}}$ is the sum of the responses in area $i,$ and ${\pi }_i\mathrm{<mo>=</mo>}\left({\pi }_{i1}\mathrm{,}\dots ,{\pi }_{iK}\right),$ $0\le {\pi }_{ik}\le 1,$ ${\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^K{\pi }_{ik}=1},$ is the proportion vector for each area. The model does not allow any pooling and is denoted as a baseline to compare our models. The parameters ${\pi }_i$ are independent and do not share a common effect. That is, the areas are unrelated.

There are five categories of parametric pooling models, classified according to the priors of the proportion vectors in a multinomial distribution. First, the four prior distributions for parametric Bayesian inferences are given as follows:

where $\mu \mathrm{<mo>=</mo>}{\left({\mu }_1\mathrm{,}\dots ,{\mu }_K\right)}^{\prime },$ $0\le {\mu }_k\le 1,$ ${\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^K{\mu }_k\mathrm{<mo>=</mo>}1}$ and $\tau \mathrm{<mo>></mo>}0$ are the hyperparameters of the Dirichlet distribution. We further assume that $\pi \left(\mu \mathrm{,}\tau \right)\mathrm{<mo>=</mo>}\left(K-1\right)\mathrm{<mo>!</mo>}/{\left(1+\tau \right)}^2,$ a shrinkage prior. We note that Yin and Nandram (2020a, b) place a Dirichlet process on the sampling process to accomodate gaps, outliers and ties in survey data, see also Nandram and Yin (2016a, b) for additional discussion of the Dirichlet process. The $\varphi ~\text{uniform}\left(1/2,1\right),$ $\varphi >1/2$ means that more weight is attached to adaptive pooling.

Model 1 is a no-pooling model that estimates the parameter without any data sharing from other areas. Model 2, on the other hand, is a complete pooling model that estimates the parameter while treating the different areas as one. When conducting parameter estimations on a small area with a small number of data using Model 1, the estimation may face the small area problem, as the parameter is estimated by relying on insufficient data. Although the complete pooling model alleviates this issue, it faces problems of its own: Overshrinkage and individual areas cannot be discerned. Hence, this paper introduces various pooling approaches to find a model that delivers better estimates. In Model 3, the adaptive pooling model introduced by Nandram, Zhou and Kim (2019), all areas share the same hyperparameters; hence, they share their area data information as well. This is an indirect complete pooling method that preserves some area variation but, in general, assumes all areas to have identical traits; see also Nandram (1998). This creates variations in estimating the hyperparameters. Thus, we propose the restricted pooling model, which alters Model 3 by removing data-sharing information between distinct areas. In this new model, distinct areas use their local data to estimate parameters, and areas with similar traits share information through pooling based on the same hyperparameter, thereby improving the estimation. This model, however, assigns the same hyperparameter to areas with similar traits, which may lead to the overshrinkage problem when smoothing in the category occurs. To mitigate this issue, we propose Model 5, the global-local pooling model, see Dunson (2009). The global-local pooling model pools information in data among areas with similar traits but also preserves each variation in the category through the local effect model, thereby reducing the smoothing in the categorical effect. Indeed, Model 5 is flexible and robust.

The fifth prior distribution used for parametric Bayesian inferences is called global $–$ local pooling. In this case, we use different notation for the proportion vector of each area. Let $p_i,$ for $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I,$ denote the corresponding cell proportion vector in the $i^{\text{th}}$ area. We assume that

$$n_i|p_i\stackrel{\text{ind}}{{\displaystyle ~}}\text{Multinomial}\left(n_{i\mathrm{.}}\mathrm{,}{p}_i\right)\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I\mathrm{,}(2.2)$$

where $p_i\mathrm{<mo>=</mo>}{\left(e^{\theta +{\eta }_{i1}}/\left(1+{\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^{K-1}{e}^{\theta +{\eta }_{ik}}}\right),\dots ,e^{\theta +{\eta }_{i\left(K-1\right)}}/\left(1+{\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^{K-1}{e}^{\theta +{\eta }_{ik}}}\right)\mathrm{,}1/\left(1+{\displaystyle {\sum }_{k\mathrm{<mo>=</mo>\; 1}}^{K-1}{e}^{\theta +{\eta }_{ik}}}\right)\right)}^{\prime }.$ Here, $p_i$ is composed of two components, namely, $\theta$ and ${\eta }_{ik},$ and $\theta$ reflects the basic probability that brought all the areas together. The global-local effect is reflected in the component ${\eta }_i\mathrm{<mo>=</mo>}{\left({\eta }_{i1}\mathrm{,}\dots ,{\eta }_{i\left(K-1\right)}\right)}^{\prime }.$ Specifically,

$${\eta }_i\stackrel{\text{iid}}{{\displaystyle ~}}{I}_{\left(z_i=0\right)}{\displaystyle \prod _{k\mathrm{<mo>=</mo>\; 1}}^{K-1}\text{N}\left(\mathrm{0,}{\sigma }^2\right)}+I_{\left(z_i=1\right)}{\displaystyle \prod _{k\mathrm{<mo>=</mo>\; 1}}^{K-1}\text{N}\left(\mathrm{0,}{\sigma }_k^2\right)}\mathrm{,}(2.3)$$

where $\text{N}\left(\mathrm{0,}{\sigma }^2\right)$ is the normal distribution of the global parameter ${\sigma }^2,$ $\text{N}\left(\mathrm{0,}{\sigma }_k^2\right)$ is the normal distribution of the local parameters ${\sigma }_k^2$ for each category, where each area is denoted by a different index. Then, $z_i,$ $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I,$ follows a Bernoulli distribution with a hyperparameter $\varphi ,$ which adjusts the proportion between the global and local effects. Thus, if we need to focus on the global effect, the prior for $\varphi$ is set using the uniform distribution on $\left(1/2,1\right).$ Specifically, we assume that

$$z_i|\varphi \stackrel{\text{iid}}{{\displaystyle ~}}\text{Bernoulli}\left(\varphi \right)\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I\mathrm{,}$$

$$\varphi ~\text{Uniform}\left(\frac{1}{2},1\right)\mathrm{,}$$

$$\pi \left(\text{}\theta \text{}\right)\mathrm{<mo>=</mo>}\frac{1}{\pi \left(1+{\theta }^2\right)}\mathrm{,}\text{a Cauchy prior,}$$

$$\pi \left({\sigma }^2\mathrm{,}{\sigma }_1^2\mathrm{,}\dots ,{\sigma }_{K-1}^2\right)\mathrm{<mo>=</mo>}\frac{1}{{\left(1+{\sigma }^2\right)}^2}{\displaystyle \prod _{k\mathrm{<mo>=</mo>\; 1}}^{K-1}\frac{1}{{\left(1+{\sigma }_k^2\right)}^2}}\mathrm{,}$$

where $-\infty <{\eta }_{ik}<\infty ,$ $k\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,K-1,$ ${\sigma }^2>0,$ and ${\sigma }_k^2>0,$ $k\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,K-1.$ We have used the Cauchy prior for $\theta$ and shrinkage priors for variance components. The shrinkage priors are similar to the half Cauchy prior and they are mathematically more convenient when we make transformations to $\left(\mathrm{0,}1\right).$

The models proposed in this paper are based on the adaptive pooling model (Nandram, Zhou and Kim, 2019), which applies the principle of assigning the same subscripts of parameter in prior distribution to similar experiments (Malec and Sedransk, 1992) to categorical data. In particular, the no pooling model and complete pooling model represent two extreme cases of adaptive pooling, with parameters $\mu$ and $\tau .$ On the other hand, the restricted pooling model has a pooling principle such as adaptive pooling model, but the model also reflects uncertainty through the weighting parameter $\varphi .$ However, the same parameter in the prior distribution used to pool such data frequently results in an overshrinkage problem. To compensate for this problem, we propose the global-local pooling model. The effects of the parameters are separated into global and local effects in this model. As a result, we propose two new models, restricted pooling model and global-local pooling model, and we compare these models with existing ones.

2.2   Nonparametric models

We consider the DP prior for ${\pi }_i$ of (2.1) in Section 2.1. The prior structure is as follows:

$$\begin{array}{l}{\pi }_i|G\stackrel{\text{iid}}{{\displaystyle ~}}G\\ G~\text{DP}\left(\alpha \mathrm{,}{G}_0\right)\mathrm{,}\end{array}$$

where $G_0$ is the base distribution and the positive real number, $\alpha ,$ is the concentration parameter in the DP prior. The model is specified by the structure of the base distribution. We note here that Yin and Nandram (2020a, b) used a DP on the sampling process to accommodate gaps, outliers and ties in survey data. First, we define the model using two prior distributions, as follows:

We assume $\pi \left(\mu \mathrm{,}\tau \right)\mathrm{<mo>=</mo>}\left(K-1\right)\mathrm{<mo>!</mo>}/{\mathrm{<mo\; stretchy="false">(</mo>\; 1}+\tau )}^2,$ $\pi \left(\text{}\alpha \text{}\right)\propto {1/\left(1+\alpha \right)}^2,$ and $\varphi ~\text{uniform}\left(1/2,1\right).$ In Models 6 and 7, we use a stict-breaking process for the DP prior (Sethuraman, 1994).

The last model is a nonparametric version of (2.2) in Section 2.1, used for global-local pooling. Here, ${\eta }_i$ are used to construct the nonparametric Bayesian setting, as follows:

$${\eta }_i\stackrel{\text{iid}}{{\displaystyle ~}}{I}_{\left(z_i=0\right)}{G}_0+I_{\left(z_i=1\right)}{\displaystyle \prod _{k\mathrm{<mo>=</mo>\; 1}}^{K-1}{G}_k}$$

$$G_0~\text{DP}\left({\alpha }_0\mathrm{,}\text{MVN}\left(0\mathrm{,}{\sigma }^{2}I\right)\right)\mathrm{,}{G}_k~\text{DP}\left({\alpha }_1\mathrm{,}N\left(\mathrm{0,}{\sigma }_k^2\right)\right)\mathrm{,}$$

where ${\eta }_i\mathrm{<mo>=</mo>}{\left({\eta }_{i1}\mathrm{,}\dots ,{\eta }_{i\left(K-1\right)}\right)}^{\prime },$ $z_i~\text{Bernouilli}\left(\varphi \right),$ $\pi \left(\text{}\theta \text{}\right)\propto 1,$ $\varphi ~\text{uniform}\left(1/2,1\right),$ $\pi \left({\sigma }^2\mathrm{,}{\sigma }_1^2\mathrm{,}\dots \mathrm{,}{\sigma }_{K-1}^2\right)\mathrm{<mo>=</mo>}1/\left\{{\left(1+{\sigma }^2\right)}^2{\displaystyle {\prod }_{k\mathrm{<mo>=</mo>\; 1}}^{K-1}{\left(1+{\sigma }_k^2\right)}^2}\right\},$ for $i\mathrm{<mo>=</mo>}\mathrm{1,}\dots ,I.$ This is model (A.1), nonparametric global-local pooling.

The distribution of ${\eta }_i$ involves a mixture of global and local pooling areas. While global pooling is conducted according to the same principle as the aforementioned nonparametric models, the Dirichlet process prior, where the normal distribution is the base distribution for each cell, is independently defined, thereby alleviating the overshrinkage problem that could arise owing to global pooling. Here, $z_i,$ which represents the weight of the model, follows the Bernouilli distribution with $\varphi$ greater than $1/2$ as its parameter, and $\varphi$ follows the uniform distribution. Hence, the local pooling area is weighted so that it is defined as the form that can better alleviate the degree of shrinkage. In addition, to ensure simplicity of the model, the heavy tail and noninformative characteristic of $\theta$ distribution follow the improper prior which is identical to, but simpler than, the previous parametric model and the posterior distribution is presented properly.

• Previous
• Article main page
• Next