Multiple imputation of missing values in household data with structural zeros
Section 2. Review of the NDPMPM model

Table of contents

Hu et al. (2018) present the NDPMPM model including motivation for how it can preserve associations across variables and account for structural zeros. Here, we summarize the model without detailed motivations, referring the reader to Hu et al. (2018) for more information. We begin with notation needed to understand the model and the Gibbs sampler, assuming complete data. The presentation closely follows that in Hu et al. (2018).

2.1 Notation and model specification

Suppose the data contain $n$ households. Each household $i =1, \dots, n$ contains $n_{i}$ individuals, so that there are $\sum_{i =1}^{n} n_{i} = N$ individuals in the data. Let $X_{i k} \in {1, \dots, d_{k}}$ be the value of categorical variable $k$ for household $i,$ which is assumed to be identical for all $n_{i}$ individuals in household $i,$ where $k = p + 1, \dots, p + q .$ Let $X_{i j k} \in {1, \dots, d_{k}}$ be the value of categorical variable $k$ for person $j$ in household $i,$ where $j =1, \dots, n_{i}$ and $k =1, \dots, p .$ Let $X_{i} = (X_{i (p + 1)}, \dots, X_{i (p + q)}, X_{i 11}, \dots, X_{i n_{i} p})$ include all household-level and individual-level variables for the $n_{i}$ individuals in household $i .$

Let $H$ be the set of all household sizes that are possible in the population. For all $h \in H,$ let $C_{h}$ represent the set of all combinations of individual-level and household-level variables for households of size $h$ , including impossible combinations; that is, $C_{h} = \prod_{k = p + 1}^{p + q} {1, \dots, d_{k}} \prod_{j =1}^{h} \prod_{k =1}^{p} {1, \dots, d_{k}} .$ Let $S_{h} \subset C_{h}$ represent the set of impossible combinations, i.e., those that are structural zeros, for households of size $h .$ These include combinations of variables within any individual, e.g., a three year old person cannot be a spouse, or across individuals in the same household, e.g., a person cannot be older than his biological parents. Let $C = \cup_{h \in H} C_{h}$ and $S = \cup_{h \in H} S_{h} .$

Although the NDPMPM model we use restricts the support of $X_{i}$ to $C - S,$ it is helpful for understanding the model to begin with no restrictions on the support of $X_{i} .$ Each household $i$ belongs to one of $F$ classes representing latent household types. For $i =1, \dots, n,$ let $G_{i} \in {1, \dots, F}$ indicate the household class for household $i .$ Let $π_{g} = \Pr (G_{i} = g)$ be the probability that household $i$ belongs to class $g .$ Within any class, all household-level variables follow independent, multinomial distributions. For any $k \in {p + 1, \dots, p + q}$ and any $c \in {1, \dots, d_{k}},$ let $λ_{g c}^{(k)} = \Pr (X_{i k} = c | G_{i} = g)$ for any class $g,$ where $λ_{g c}^{(k)}$ is the same value for every household in class $g .$ Let $π = {π_{1}, \dots, π_{F}},$ and $λ = {λ_{g c}^{(k)} : c =1, \dots, d_{k}; k = p + 1, \dots, p + q; g =1, \dots, F} .$

Within each household class, each individual belongs to one of $S$ individual-level latent classes. For $i =1, \dots, n$ and $j =1, \dots, n_{i},$ let $M_{i j}$ represent the individual-level latent class of individual $j$ in household $i .$ Let $ω_{g m} = \Pr (M_{i j} = m | G_{i} = g)$ be the probability that individual $j$ in household $i$ belongs to individual-level class $m$ nested within household-level class $g .$ Within any individual-level class, all individual-level variables follow independent, multinomial distributions. For any $k \in {1, \dots, p}$ and any $c \in {1, \dots, d_{k}},$ let $ϕ_{g m c}^{(k)} = \Pr (X_{i j k} = c | (G_{i}, M_{i j}) = (g, m))$ for the class pair $(g, m),$ where $ϕ_{g m c}^{(k)}$ is the same value for every individual in the class pair $(g, m) .$ Let $ω = {ω_{g m} : g =1, \dots, F; m =1, \dots, S},$ and $ϕ = {ϕ_{g m c}^{(k)} : c =1, \dots, d_{k}; k =1, \dots, p; m =1, \dots, S; g =1, \dots, F} .$

For purposes of the Gibbs sampler in Section 2.2, it is useful to distinguish values of $X_{i}$ that satisfy all the structural zero constraints from those that do not. Let the superscript $“ 1 ”$ indicate that a random variable has support only on $C - S .$ For example, $X_{i}^{1}$ represents data for a household with values restricted only on $C - S,$ i.e., not an impossible household, whereas $X_{i}$ represents data for a household with any values in $C .$ Let $X^{1}$ be the observed data comprising $n$ households, that is, a realization of $(X_{1}^{1}, \dots, X_{n}^{1}) .$ The kernel of the NDPMPM, $\Pr (X^{1} | θ),$ is

$L (X^{1} | θ)= \prod_{i =1}^{n} \sum_{h \in H} 1 {n_{i} = h} 1 {X_{i}^{1} \notin S_{h}} [\sum_{g =1}^{F} π_{g} \prod_{k = p + 1}^{p + q} λ_{g X_{i k}^{1}}^{(k)} \prod_{j =1}^{h} \sum_{m =1}^{S} ω_{g m} \prod_{k =1}^{p} ϕ_{g m X_{i j k}^{1}}^{(k)}], (2.1)$

where $θ$ includes all the parameters, and $1 {.}$ equals one when the condition inside the ${}$ is true and equals zero otherwise.

For all $h \in H,$ let $n_{1 h} = \sum_{i =1}^{n} 1 {n_{i} = h}$ be the number of households of size $h$ in $X^{1}$ and $π_{0 h} (θ) = \Pr (X_{i} \in S_{h} | θ) .$ As stated in Hu et al. (2018), the normalizing constant in the likelihood in (2.1) is $\prod_{h \in H} {(1 - π_{0 h} (θ))}^{n_{1 h}} .$ Therefore, the posterior distribution is

$\Pr (θ | X^{1} , T (S)) \propto \Pr (X^{1} | θ) \Pr (θ) = \frac{1}{\prod_{h \in H} {(1 - π_{0 h} (θ))}^{n_{1 h}}} L (X^{1} | θ) \Pr (θ) (2.2)$

where $T (S)$ emphasizes that the density is for the NDPMPM with support restricted to $C - S .$

The likelihood in (2.1) can be written as a generative model of the form

$\begin{array}{l} X_{i k} | G_{i}, λ \sim & Discrete (λ_{G_{i} 1}^{(k)}, \dots, λ_{G_{i} d_{k}}^{(k)}) \\ \forall i =1, \dots, n and k = p + 1, \dots, p + q (2.3) \end{array}$

$\begin{array}{l} X_{i j k} | G_{i}, M_{i j}, ϕ, n_{i} \sim & Discrete (ϕ_{G_{i} M_{i j} 1}^{(k)}, \dots, ϕ_{G_{i} M_{i j} d_{k}}^{(k)}) \\ \forall i =1, \dots, n, j =1, \dots, n_{i} and k =1, \dots, p (2.4) \end{array}$

$\begin{array}{l} G_{i} | π \sim & Discrete (π_{1}, \dots, π_{F}) \\ \forall i =1, \dots, n (2.5) \end{array}$

$\begin{array}{l} M_{i j} | G_{i}, ω, n_{i} \sim & Discrete (ω_{G_{i} 1}, \dots, ω_{G_{i} S}) \\ \forall i =1, \dots, n and j =1, \dots, n_{i} (2.6) \end{array}$

where the Discrete distribution refers to the multinomial distribution with sample size equal to one. We restrict the support of each $X_{i}$ to ensure the model assigns zero probability to all combinations in $S$ as desired. The model in (2.3) to (2.6) can be used without restricting the support to $C - S .$ This ignores all structural zeros. While not appropriate for the joint distribution of household data, this model turns out to useful for the Gibbs sampler. We refer to the generative model in (2.3) to (2.6) with support on all of $C$ as the untruncated NDPMPM. For contrast, we call the model in (2.1) the truncated NDPMPM.

For prior distributions, we follow the recommendations of Hu et al. (2018). We use independent uniform Dirichlet distributions as priors for $λ$ and $ϕ,$ and the truncated stick-breaking representation of the Dirichlet process as priors for $π$ and $ω$ (Sethuraman, 1994; Dunson and Xing, 2009; Si and Reiter, 2013; Manrique-Vallier and Reiter, 2014),

$λ_{g}^{(k)} = (λ_{g 1}^{(k)}, \dots, λ_{g d_{k}}^{(k)}) \sim Dirichlet (1, \dots, 1) (2.7)$

$ϕ_{g m}^{(k)} = (ϕ_{g m 1}^{(k)}, \dots, ϕ_{g m d_{k}}^{(k)}) \sim Dirichlet (1, \dots, 1) (2.8)$

$π_{g} = u_{g} \prod_{f < g} (1 - u_{f}) for g =1, \dots, F (2.9)$

$u_{g} \sim Beta (1, α) for g =1, \dots, F - 1, u_{F} =1 (2.10)$

$α \sim Gamma (0 .25, 0 .25) (2.11)$

$ω_{g m} = v_{g m} \prod_{s < m} (1 - v_{g s}) for m =1, \dots S (2.12)$

$v_{g m} \sim Beta (1, β_{g}) for m =1, \dots, S - 1, v_{g S} =1 (2.13)$

$β_{g} \sim Gamma (0 .25, 0 .25) . (2.14)$

We set the parameters for the Dirichlet distributions in (2.7) and (2.8) to $1_{d_{k}}$ (a $d_{k} -$ dimensional vector of ones) and the parameters for the Gamma distributions in (2.11) and (2.14) to 0.25 to represent vague prior specifications. We also set $β_{g} = β$ for computational expedience. For further discussion on prior specifications, see Hu et al. (2018).

Conceptually, the latent household-level classes can be interpreted as clusters of households with similar compositions, e.g., households with children or households in which no one is related. Similarly, the latent individual-level classes can be interpreted as clusters of individuals with similar characteristics, e.g., older male spouses or young female children. However, for purposes of imputation, we do not care much about interpreting the classes, as they serve mainly to induce dependence across variables and individuals in the joint distribution.

It is important to select $F$ and $S$ to be large enough to ensure accurate estimation of the joint distribution. However, we also do not want to make $F$ and $S$ so large as to produce many empty classes in the model estimation. Allowing many empty classes increases computational running time without any corresponding increase in estimation accuracy. This can be especially problematic in the Gibbs sampler for the truncated NDPMPM, as these empty classes can introduce mass in regions of the space where impossible combinations are likely to be generated. This slows down the convergence of the Gibbs sampler.

We therefore recommend following the strategy in Hu et al. (2018) when setting $(F , S) .$ Analysts can start with moderate values for both, say between 10 and 15, in initial tuning runs. After convergence, analysts examine posterior samples of the latent classes to check how many individual-level and household-level latent classes are occupied. Such posterior predictive checks can provide evidence for the case that larger values for $F$ and $S$ are needed. If the numbers of occupied household-level classes hits $F ,$ we suggest increasing $F .$ If the number of occupied individual-level classes hits $S,$ we suggest increasing $F$ first but then increasing $S,$ possibly in addition to $F ,$ if increasing $F$ alone does not suffice. When posterior predictive checks do not provide evidence that larger values of $F$ and $S$ are needed, analysts need not increase the number of classes, as doing so is not expected to improve the accuracy of the estimation. We note that similar logic is used in other mixture model contexts (Walker, 2007; Si and Reiter, 2013; Manrique-Vallier and Reiter, 2014; Murray and Reiter, 2016).

2.2 MCMC sampler for the NDPMPM

Hu et al. (2018) use a data augmentation strategy (Manrique-Vallier and Reiter, 2014) to estimate the posterior distribution in (2.2). They assume that the observed data $X^{1} ,$ which includes only feasible households, is a subset from a hypothetical sample $X$ of $(n + n_{0})$ households directly generated from the untruncated NDPMPM. That is, $X$ is generated on the support $C$ where all combinations are possible and structural zeros rules are not enforced, but we only observe the sample of $n$ households $X^{1}$ that satisfy the structural zero rules and do not observe the sample of $n_{0}$ households $X^{0} = X - X^{1}$ that fail the rules.

We use the strategy of Hu et al. (2018) and augment the data as follows. For each $h \in H,$ we simulate $X$ from the untruncated NDPMPM, stopping when the number of simulated feasible households in $X$ directly matches $n_{1 h}$ for all $h \in H .$ We replace the simulated feasible households in $X$ with $X^{1} ,$ thus, assuming that $X$ already contains $X^{1}$ and we only need to generate the part $X^{0}$ that fall in $S .$ Given a draw of $X,$ we draw $θ$ from posterior distribution defined by the untruncated NDPMPM, treating $X$ as the observed data. This posterior distribution can be estimated using a blocked Gibbs sampler (Ishwaran and James, 2001; Si and Reiter, 2013).

We now present the full MCMC sampler for fitting the truncated NDPMPM. Let $G^{0}$ and $M^{0}$ be vectors of the latent class membership indicators for the households in $X^{0}$ and $n_{0 h}$ be the number of households of size $h$ in $X^{0} ,$ with $n_{0} = \sum_{h} n_{0 h} .$ In each full conditional, let “ $-$ ” represent conditioning on all other variables and parameters in the model. At each MCMC iteration, we do the following steps.

S1. Set $X^{0} = G^{0} = M^{0} = \emptyset .$ For each $h \in H,$ repeat the following:

Set $t_{0} =0$ and $t_{1} =0.$
Sample $G_{i}^{0} \in {1, \dots, F} \sim Discrete (π_{1}^{* *} , \dots, π_{F}^{* *})$ where $π_{g}^{* *} \propto λ_{g h}^{(k)} π_{g}$ and $k$ is the index for the household-level variable “household size”.
For $j =1, \dots, h,$ sample $M_{i j}^{0} \in {1, \dots, S} \sim Discrete (ω_{G_{i}^{0} 1}, \dots, ω_{G_{i}^{0} S}) .$
Set $X_{i k}^{0} = h,$ where $X_{i k}^{0}$ corresponds to the variable for household size. Sample the remaining household-level and individual-level values using the likelihoods in (2.3) and (2.4). Set the household’s simulated value to $X_{i}^{0} .$
If $X_{i}^{0} \in S_{h},$ let $t_{0} = t_{0} + 1,$ $X^{0} = X^{0} \cup X_{i}^{0},$ $G^{0} = G^{0} \cup G_{i}^{0}$ and $M^{0} = M^{0} \cup {M_{i 1}^{0}, \dots, M_{i h}^{0}} .$ Otherwise set $t_{1} = t_{1} + 1.$
If $t_{1} < n_{1 h},$ return to step (b). Otherwise, set $n_{0 h} = t_{0} .$

S2. For observations in $X^{1} ,$

Sample $G_{i} \in {1, \dots, F} \sim Discrete (π_{1}^{*}, \dots, π_{F}^{*})$ for $i =1, \dots, n,$ where

$π_{g}^{*} = \Pr (G_{i} = g | -) = \frac{π_{g} [\prod_{k = p + 1}^{q} λ_{g X_{i k}^{1}}^{(k)} (\prod_{j =1}^{n_{i}} \sum_{m =1}^{S} ω_{g m} \prod_{k =1}^{p} ϕ_{g m X_{i j k}^{1}}^{(k)})]}{\sum_{f =1}^{F} π_{f} [\prod_{k = p + 1}^{q} λ_{f X_{i k}^{1}}^{(k)} (\prod_{j =1}^{n_{i}} \sum_{m =1}^{S} ω_{g m} \prod_{k =1}^{p} ϕ_{f m X_{i j k}^{1}}^{(k)})]}$

for

g =1, \dots, F ​ .

Set

G_{i}^{1} = G_{i} .

Sample $M_{i j} \in {1, \dots, S} \sim Discrete (ω_{G_{i}^{1} 1}^{*}, \dots, ω_{G_{i}^{1} S}^{*})$ for $i =1, \dots, n$ and $j =1, \dots, n_{i},$ where

$ω_{G_{i}^{1} m}^{*} = \Pr (M_{i j} = m | -) = \frac{ω_{G_{i}^{1} m} \prod_{k =1}^{p} ϕ_{G_{i}^{1} m X_{i j k}^{1}}^{(k)}}{\sum_{s =1}^{S} ω_{G_{i}^{1} s} \prod_{k =1}^{p} ϕ_{G_{i}^{1} s X_{i j k}^{1}}^{(k)}}$

for

m =1, \dots, S .

Set

M_{i j}^{1} = M_{i j} .

S3. Set $u_{F} =1.$ Sample

$u_{g} | - \sim Beta (1 + U_{g}, α + \sum_{f = g + 1}^{F} U_{f}), π_{g} = u_{g} \prod_{f < g} (1 - u_{f})$

where

$U_{g} = \sum_{i =1}^{n} 1 (G_{i}^{1} = g) + \sum_{i =1}^{n_{0}} 1 (G_{i}^{0} = g)$

for

g =1, \dots, F - 1.

S4. Set $v_{g M} =1$ for $g =1, \dots, F .$ Sample

$v_{g m} | - \sim Beta (1 + V_{g m}, β + \sum_{s = m + 1}^{S} V_{g s}), ω_{g m} = v_{g m} \prod_{s < m} (1 - v_{g s})$

where

$V_{g m} = \sum_{i =1}^{n} 1 (M_{i j}^{1} = m, G_{i}^{1} = g) + \sum_{i =1}^{n_{0}} 1 (M_{i j}^{0} = m, G_{i}^{0} = g)$

for

m =1, \dots, S - 1

and

g =1, \dots, F .

S5. Sample

$λ_{g}^{(k)} | - \sim Dirichlet (1 + η_{g 1}^{(k)}, \dots, 1 + η_{g d_{k}}^{(k)})$

where

$η_{g c}^{(k)} = \sum_{i | G_{i}^{1} = g}^{n} 1 (X_{i k}^{1} = c) + \sum_{i | G_{i}^{0} = g}^{n_{0}} 1 (X_{i k}^{0} = c)$

for

g =1, \dots, F

and

k = p + 1, \dots, q .

S6. Sample

$ϕ_{g m}^{(k)} | - \sim Dirichlet (1 + ν_{g m 1}^{(k)}, \dots, 1 + ν_{g m d_{k}}^{(k)})$

where

$ν_{g m c}^{(k)} = \sum_{i, j | \begin{matrix} G_{i}^{1} = g, M_{i j}^{1} = m \end{matrix}}^{n} 1 (X_{i j k}^{1} = c) + \sum_{i, j | \begin{matrix} G_{i}^{0} = g, M_{i j}^{0} = m \end{matrix}}^{n_{0}} 1 (X_{i j k}^{0} = c)$

for

g =1, \dots, F,

m =1, \dots, S

and

k =1, \dots, p .

S7. Sample

$α | - \sim Gamma (a_{α} + F - 1, b_{α} - \sum_{g =1}^{F - 1} log (1 - u_{g})) .$

S8. Sample

$β | - \sim Gamma (a_{β} + F \times (S - 1), b_{β} - \sum_{m =1}^{S - 1} \sum_{g =1}^{F} log (1 - v_{g m})) .$

This Gibbs sampler is implemented in the R software package “NestedCategBayesImpute” (Wang, Akande, Hu, Reiter and Barrientos, 2016). The software can be used to generate synthetic versions of the original data, but it requires all data to be complete.

ISSN : 1492-0921

Editorial policy

Survey Methodology publishes articles dealing with various aspects of statistical development relevant to a statistical agency, such as design issues in the context of practical constraints, use of different data sources and collection techniques, total survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves. All papers will be refereed. However, the authors retain full responsibility for the contents of their papers and opinions expressed are not necessarily those of the Editorial Board or of Statistics Canada.

Submission of Manuscripts

Survey Methodology is published twice a year in electronic format. Authors are invited to submit their articles in English or French in electronic form, preferably in Word to the Editor, (statcan.smj-rte.statcan@canada.ca, Statistics Canada, 150 Tunney’s Pasture Driveway, Ottawa, Ontario, Canada, K1A 0T6). For formatting instructions, please see the guidelines provided in the journal and on the web site (www.statcan.gc.ca/SurveyMethodology).

Note of appreciation

Canada owes the success of its statistical system to a long-standing partnership between Statistics Canada, the citizens of Canada, its businesses, governments and other institutions. Accurate and timely statistical information could not be produced without their continued co-operation and goodwill.

Standards of service to the public

Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner. To this end, the Agency has developed standards of service which its employees observe in serving its clients.

Copyright

Published by authority of the Minister responsible for Statistics Canada.

Catalogue No. 12-001-X

Frequency: Semi-annual

Ottawa

Report a problem on this page

Is something not working? Is there information outdated? Can't find what you're looking for?

Please contact us and let us know how we can help you.

Privacy notice

Date modified:: 2019-07-04

Language selection

Search and menus

Search

Multiple imputation of missing values in household data with structural zeros
Section 2. Review of the NDPMPM model

2.1 Notation and model specification

2.2 MCMC sampler for the NDPMPM

Multiple imputation of missing values in household data with structural zeros Section 2. Review of the NDPMPM model

2.1 Notation and model specification

2.2 MCMC sampler for the NDPMPM

Editorial policy

Submission of Manuscripts

Note of appreciation

Standards of service to the public

Copyright

Multiple imputation of missing values in household data with structural zeros
Section 2. Review of the NDPMPM model