Bayesian inference for multinomial data from small areas incorporating uncertainty about order restriction
Section 2. Hierarchical multinomial Dirichlet

• Previous
• Next

In this section, we present a brief review of Multinomial-Dirichlet model and its extensions with the order restriction. To study the association between bone mineral density and body mass index (BMI) from several U.S. counties, Nandram, Kim and Zhou (2019) provided a clear discussion of the general hierarchical multinomial Dirichlet model and their methodology for small area estimation. Let $n_{ij}$ be the cell counts, which are numbers in each category $j$ for each area $i,$ ${\theta }_{ij}$ be the corresponding cell probabilities, $i\mathrm{<mo>=</mo>}\mathrm{1,}\mathrm{2,}\dots \mathrm{,}I,$ $j\mathrm{<mo>=</mo>}\mathrm{1,}\mathrm{2,}\dots \mathrm{,}K,$ and the total number for each area $i$ is $n_{i\mathrm{.}}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>1}}^K{n}_{ij}}.$ The general hierarchical multinomial Dirichlet model is

$$n_i|{\theta }_i\stackrel{\text{ind}}{{\displaystyle ~}}\text{Multinomial}\left(n_{i.},{\theta }_i\right),n_i\mathrm{<mo>=</mo>}\left(n_{i1}\mathrm{,}\dots \mathrm{,}{n}_{iK}\right)\mathrm{,}$$

$${\theta }_i|\mu ,\tau \stackrel{\text{ind}}{{\displaystyle ~}}\text{Dirichlet}\left(\mu \tau \right)\mathrm{,}{\theta }_i\mathrm{<mo>=</mo>}\left({\theta }_{i1}\mathrm{,}\dots \mathrm{,}{\theta }_{ik}\right)\mathrm{,}$$

$$\pi \left(\mu \mathrm{,}\tau \right)\mathrm{<mo>=</mo>}\frac{\left(K-1\right)!}{{\left(1+\tau \right)}^2}\mathrm{,}$$

where hyper-parameters $\mu \mathrm{<mo>=</mo>}\left({\mu }_1\mathrm{,}\dots \mathrm{,}{\mu }_K\right),$ ${\mu }_j>\mathrm{0,}{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K{\mu }_j\mathrm{<mo>=</mo>}1}\mathrm{,}\tau >0.$

They suggest the non-information prior which will be easy to reparameterize. Without any prior information, they take $\mu$ and $\tau$ to be independent, $E\left({\theta }_{ij}\right)\mathrm{<mo>=</mo>}{\mu }_j,$ ${\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K{\mu }_j\mathrm{<mo>=</mo>}1}.$ As an interpretation of hyper-parameters, $\mu$ are related to cell means and $\tau$ is related to a prior sample size. This model features stratification and hyper-parameters to pool information from different strata together.

This hierarchical multinomial Dirichlet model is a convenient starting point for small area estimation. For convenience, we denote it as $M_1$ model for the future discussion.

2.1 Hierarchical multinomial Dirichlet model with order restrictions

Chen and Nandram (2019) incorporate the order restriction into the Bayesian hierarchical multinomial Dirichlet model. Letting $n_{ij}$ be the cell counts, ${\theta }_{ij}$ the corresponding cell probabilities, $i\mathrm{<mo>=</mo>}\mathrm{1,}\mathrm{2,}\dots \mathrm{,}I,$ $j\mathrm{<mo>=</mo>}\mathrm{1,}\mathrm{2,}\dots \mathrm{,}K,$ $n_{i.}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K{n}_{ij}}$ and we believe the mode of ${\theta }_i’\text{s}$ is ${\theta }_{im}\mathrm{,}1\le m\le K.$

Specifically, they assume

$$n_i|{\theta }_i\stackrel{\text{ind}}{{\displaystyle ~}}\text{Multinomial}\left(n_{i.},{\theta }_i\right)\mathrm{,}{\theta }_i\in C\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots \mathrm{,}I\mathrm{,}$$

where $C\mathrm{<mo>=</mo>}\left\{{\theta }_i\mathrm{<mo>:</mo>}{\theta }_{i1}\le \dots \le {\theta }_{im}\ge \dots \ge {\theta }_{iK}\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots \mathrm{,}I\right\},$ and assume the modal position $m$ in $C$ is known.

At the second stage they assume

$${\theta }_i|\mu ,\tau \stackrel{\text{ind}}{{\displaystyle ~}}\text{Dirichlet}\left(\mu \tau \right)\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots \mathrm{,}I\mathrm{,}$$

$$\pi \left(\mu \mathrm{,}\tau \right)\mathrm{<mo>=</mo>}\frac{K\left(m-1\right)!\mathrm{}\left(K-m\right)!\mathrm{}}{{\left(1+\tau \right)}^2}\mathrm{,}{\mu }_j>\mathrm{0,}{\displaystyle \sum _{j\mathrm{<mo>=</mo>1}}^K{\mu }_j\mathrm{<mo>=</mo>1}}\mathrm{,}\mu \in C_{\mu }\mathrm{.}$$

Since $E\left({\theta }_{ij}\right)\mathrm{<mo>=</mo>}{\mu }_j,$ $\mu$ should have the same order restriction as ${\theta }_i,$ which is $\mu \in C_{\mu },$

$$C_{\mu }\mathrm{<mo>=</mo>}\left\{\mu \mathrm{<mo>:</mo>}{\mu }_1\le \dots \le {\mu }_m\ge \dots \ge {\mu }_K\right\}\mathrm{,}$$

and we assume the modal position $m$ in $C_{\mu }$ is known.

A posteriori ${\theta }_i|\mu \mathrm{,}\tau \mathrm{,}{n}_i\stackrel{\text{ind}}{{\displaystyle ~}}\text{Dirichlet}\left(n_i+\mu \tau \right),$ ${\theta }_i\in C_i\mathrm{,}i\mathrm{<mo>=</mo>}\mathrm{1,}\dots \mathrm{,}I,$ where

$$f_{{\theta }_i|\mu \mathrm{,}\tau \mathrm{,}n}\mathrm{<mo>=</mo>}\frac{\frac{\Gamma \left[{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K\left(n_{ij}+{\mu }_j\tau \right)}\right]}{{\displaystyle {\prod }_{j\mathrm{<mo>=</mo>}1}^K\Gamma \left(n_{ij}+{\mu }_j\tau \right)}}{\displaystyle {\prod }_{j\mathrm{<mo>=</mo>}1}^K{\theta }_{ij}^{n_{ij}+{\mu }_j\tau -1}}}{C\left(n_i+\mu \tau \right)}\mathrm{,}$$

where

$$C\left(n_i+\mu \tau \right)\mathrm{<mo>=</mo>}{\displaystyle {\int }_{{\theta }_i\in C}\frac{\Gamma \left[{\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K\left(n_{ij}+{\mu }_j\tau \right)}\right]}{{\displaystyle {\prod }_{j\mathrm{<mo>=</mo>1}}^K\Gamma \left(n_{ij}+{\mu }_j\tau \right)}}}{\displaystyle \prod _{j\mathrm{<mo>=</mo>1}}^K{\theta }_{ij}^{n_{ij}+{\mu }_j\tau -1}d{\theta }_i}\mathrm{.}$$

In our BMI data application, there are five categories of BMI. We only interest in the normal and overweight BMI level. We use model $M_2$ represent the model with order restrictions and its modal position is at the second, which is normal weight. Model $M_3$ represents the model with order restrictions and its modal position is at the third, which is overweight weight. $M_2$ and $M_3$ are the same hierarchical multinomial Dirichlet model, but with different order restrictions.

The joint posterior density of $M_2$ or $M_3$ is

$$\pi \left(\theta \mathrm{,}\mu \mathrm{,}\tau |n\right)\propto {\displaystyle \prod _{i\mathrm{<mo>=</mo>1}}^I\left\{{\displaystyle \prod _{j\mathrm{<mo>=</mo>1}}^K{\theta }_{ij}^{n_{ij}}\frac{1}{D\left(\mu \tau \right)C\left(\mu \tau \right)}{\displaystyle \prod _{j\mathrm{<mo>=</mo>1}}^K{\theta }_{ij}^{{\mu }_j\tau -1}}}\right\}}\frac{K\left(m-1\right)!\mathrm{}\left(K-m\right)!\mathrm{}}{{\left(1+\tau \right)}^2}\mathrm{,}$$

where

$$D\left(\mu \tau \right)\mathrm{<mo>=</mo>}\frac{{\displaystyle {\prod }_{j\mathrm{<mo>=</mo>}1}^K\Gamma \left({\mu }_j\tau \right)}}{\Gamma \left({\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K{\mu }_j\tau }\right)}$$

is the normalization constant of Dirichlet distribution,

$$C\left(\mu \tau \right)\mathrm{<mo>=</mo>}{\displaystyle {\int }_{{\theta }_i\in C}\frac{\Gamma \left({\displaystyle {\sum }_{j\mathrm{<mo>=</mo>}1}^K{\mu }_j\tau }\right)}{{\displaystyle {\prod }_{j\mathrm{<mo>=</mo>}1}^K\Gamma \left({\mu }_j\tau \right)}}}{\displaystyle \prod _{j\mathrm{<mo>=</mo>1}}^K{\theta }_{ij}^{{\mu }_j\tau -1}d{\theta }_i}\mathrm{,}$$

is the normalization constant of the truncated Dirichlet distribution, $\theta \in C\mathrm{,}\mu \in C_{\mu }.$

Nandram (1998) showed how to generate samples from model $M_1.$ In fact, using the griddy Gibbs sampler, it can be done easier than the method in Nandram (1998). Chen and Nandram (2019) present sampling methods for $\mu$ and $\theta$ with order restrictions from the joint posterior distribution of model $M_2$ and $M_3,$ as in Appendix A.1 and Appendix A.2.

Gelfand, Dey and Chang (1992) used predictive distributions to address the issues of model adequacy and model selection. They proposed the conditional predictive ordinate for the model determination. The conditional predictive ordinate (CPO) is based on leave-one-out cross validation. CPO estimates the probability of observing $n_i$ in the future if after having already observed $n_{\left(i\right)}.$ The sum of the log CPO’s is an estimator for the log marginal likelihood. The “best” model amongst competing models have the largest LPML.

Chen and Nandram (2021) presented a method to compute the conditional predictive ordinate (CPO) and LPML as a Bayesian model selection criteria. In Appendix A.3, we have improved estimation to integrate out the order-restricted $\theta ,$ and the estimated CPO of $M_2$ and $M_3$ are

where ${\theta }_i^{\mathrm{<mo\; stretchy="false">(</mo>}{h}^{\prime }\mathrm{<mo\; stretchy="false">)</mo>}}~\text{Dirichlet}\left(n_i+{\mu }^{\left(\text{}h\text{}\right)}{\tau }^{\left(\text{}h\text{}\right)}\right)$ with order restriction, ${\mu }^{\left(\text{}h\text{}\right)}$ and ${\tau }^{\left(\text{}h\text{}\right)}$ are the posterior samples from the joint posterior density.

• Previous
• Article main page
• Next