Conditional calibration and the sage statistician
Section 5. The Bayesian posterior distribution of $Q$

• Previous
• Next

The Bayesian approach differs from the Neymanian approach (and from Fisher’s fiducial approach) by formulating the problem so that a real conditional probability distribution for the estimand $Q$ can be calculated, using the laws of probability theory to condition on the fact that the observed data equals $Y^{*}-$ this distribution is called the posterior distribution of $Q,$ that is, posterior after seeing $Y=Y^{*}.$ To conduct this activity formally, $Q$ must be a random variable, and thus $Q$ needs to have a “starting” probability distribution, called its prior distribution, meaning prior to seeing any data; in the context of our setup, this prior distribution is a distribution over the possible local estimands, that is, a set of $K$ probabilities (summing to one), one probability for each possible Truth. This prior distribution is essentially a set of $K$ weights $\left\{W_k,k=1,\dots ,K\right\}$ reflecting the prior beliefs of experts that each of the $K$ possible local estimands is the correct one. The Neymanian frequentist has no use for such weights over the set of possible Truths, because the 95% is supposed to hold for any set of weights, and thus for each possible Truth (i.e., for all $K$ point mass prior distributions).

Now comes the part of the argument that hints at a departure from Neyman’s 1970’s claim to me that conditional inference is too difficult. In the context of the simulation just described, and admitting some Bayesian or fiducial logic, when confronted with actual observed data set $Y^{\text{*}},$ attention should be focused on the parts of the simulation where the generated $Y_{jk}$ equals $Y^{\text{*}};$ the other $Y_{jk}$ can be ignored (at least in the context of the idealized description here, where $J$ is essentially infinite) because, to be fully Bayesian, we want to condition on $Y$ equaling $Y^{\text{*}}.$

In fact, let us use the simulation itself to describe the Bayesian posterior distribution of $Q,$ i.e., the distribution of $Q$ conditioning on the fact that $Y=Y^{\text{*}}.$ Let $M_k^{\text{*}}$ be the proportion of the $J$ values of $Y_{jk}$ that match $Y^{\text{*}},$ for $k=1,\dots ,K;$ that is, for truth $T_k,$ $M_k^{\text{*}}$ is the proportion of the generated data sets from truth $T_k$ that match the actual data set $Y^{\text{*}}.$ For example, if $M_k^{\text{*}}$ is zero, then the a priori possible truth $T_k$ could not be the actual truth because it could not have generated observed data $Y^{\text{*}}.$ The posterior probability that the estimand $Q$ equals $Q_k,$ the local value of $\tilde{Q}$ for Truth $T_k,$ is the weighted average of the proportions, $M_k^{\text{*}},$ weighted by $W_k,$ the prior probability that $T_k$ is the correct truth. Here, this weighted average of proportions is generally labeled ${\pi }_k,$ where ${\pi }_k$ for the observed data $Y^{*}$ is labelled ${\pi }_k^{*}$ and equals $M_k^{\text{*}}{W}_k/{\displaystyle {\sum }_{k^{\prime }=1}^K[M_{k^{\prime }}^{\text{*}}{W}_{k^{\prime }}];}$ we could call ${\pi }_k^{*}$ the estimated ability of Truth $T_k$ to match observed data $Y^{\text{*}}.$ This description of the posterior distribution of $Q$ using simulation is from Rubin (1984); see Figure 5.1.

There are objections to this approach. First, where do the prior weights $W_k$ come from and who are the experts providing these weights? Perhaps we should find some way to avoid using these potentially overly subjective prior weights? Second, perhaps the requirement for exact equality between a generated data set $Y_{jk}$ and the observed data set $Y^{\text{*}}$ should be relaxed in some way so that a generated $Y_{jk}$ does not have to equal $Y^{\text{*}}$ exactly but only “look like” it came from the same distribution as did $Y^{\text{*}},$ and so match $Y^{\text{*}}$ in some way?

More on this second point first, which is clearly important when trying to conduct an actual simulation like this idealized one with a finite budget. The approximate equality between generated data $Y$ and observed data $Y^{\text{*}}$ can be achieved in situations with low-dimensional sufficient statistics, because only those statistics have to match. But this idea of generated data being “close to” observed data $Y^{\text{*}}$ is the basis of all work using this description of the posterior distribution to conduct “ABC” $–$  Approximate Bayesian Computation, apparently first described in the paragraph in Figure 5.1 (https://en.wikipedia.org/wiki/Approximate_Bayesian_computation, Tavare, Balding, Griffiths and Donnelly, 1997). We simply assume at this point that we have chosen some such metric to define the function $M_k,$ and use it to define the ability of Truth $T_k$ to generate data sets that match the observed data, $Y^{\text{*}}.$

• Previous
• Article main page
• Next