Two local diagnostics to evaluate the efficiency of the empirical best predictor under the Fay-Herriot model
Section 7. Conclusion

• Previous

Users of small area estimates are usually interested in only one domain. Therefore, they seek a quality indicator that applies to their domain and not an overall indicator. The design MSE of small area estimators is a conceptually attractive quality indicator since it conditions on the unexplained local effect. However, it is known that design-unbiased estimators of the design MSE are generally unstable when the domain sample size is small. To circumvent this problem, we proposed two diagnostics that are intended to identify domains where the design MSE of the direct estimator is smaller than that of the EB estimator. Our simulation results seem promising and allow us to envision the implementation of a useful indicator for choosing between the direct and EB estimators for a particular domain. In future research, it would be interesting to evaluate the efficiency of a hybrid estimator that would leverage these diagnostics.

Appendix

A. Proof of equivalence between equations (4.1) and (4.2)

Using equation (4.1) and the conditional distribution of $v_i$ given in Section 4.1, we have:

                            $$\begin{array}{ll}{D}_{1i}\hfill & \mathrm{<mo>=</mo>}\text{Prob}\left(-v_{L\mathrm{,}i}\le v_i\le v_{L\mathrm{,}i}|\text{Z}\mathrm{,}{\widehat{\theta }}_i\right)\hfill \\ \hfill & \mathrm{<mo>=</mo>}\text{Prob}\left(\frac{-v_{L\mathrm{,}i}-{\sigma }_v\sqrt{{\gamma }_i}{\epsilon }_i}{{\sigma }_v\sqrt{1-{\gamma }_i}}\le \frac{v_i-{\sigma }_v\sqrt{{\gamma }_i}{\epsilon }_i}{{\sigma }_v\sqrt{1-{\gamma }_i}}\le \frac{v_{L\mathrm{,}i}-{\sigma }_v\sqrt{{\gamma }_i}{\epsilon }_i}{{\sigma }_v\sqrt{1-{\gamma }_i}}|\text{Z}\mathrm{,}{\widehat{\theta }}_i\right)\mathrm{.}\hfill \end{array}$$

Replacing $v_{L\mathrm{,}i}$ with ${\sigma }_v\sqrt{\frac{1+{\gamma }_i}{{\gamma }_i}}$ results in:

$$\begin{array}{ll}{D}_{1i}\hfill & \mathrm{<mo>=</mo>}\text{Prob}\left(\frac{-\sqrt{(1+{\gamma }_i)/{\gamma }_i}-\sqrt{{\gamma }_i}{\epsilon }_i}{\sqrt{1-{\gamma }_i}}\le \frac{v_i-{\sigma }_v\sqrt{{\gamma }_i}{\epsilon }_i}{{\sigma }_v\sqrt{1-{\gamma }_i}}\le \frac{\sqrt{(1+{\gamma }_i)/{\gamma }_i}-\sqrt{{\gamma }_i}{\epsilon }_i}{\sqrt{1-{\gamma }_i}}|\text{Z}\mathrm{,}{\widehat{\theta }}_i\right)\hfill \\ \hfill & \mathrm{<mo>=</mo>}\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left(-{\epsilon }_i+\frac{\sqrt{1+{\gamma }_i}}{{\gamma }_i}\right)\right\}-\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left(-{\epsilon }_i-\sqrt{\frac{1+{\gamma }_i}{{\gamma }_i}}\right)\right\}\mathrm{.}\hfill \end{array}$$

Since for any value $t,$ we have $\Phi \left(\text{}t\text{}\right)\mathrm{<mo>=</mo>}1-\Phi \left(\text{}-t\text{}\right)$ then

                                 $$D_{1i}\mathrm{<mo>=</mo>}\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left({\epsilon }_i+\frac{\sqrt{1+{\gamma }_i}}{{\gamma }_i}\right)\right\}-\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left({\epsilon }_i-\frac{\sqrt{1+{\gamma }_i}}{{\gamma }_i}\right)\right\}\mathrm{.}$$

We notice that $D_{1i}$ is a symmetric function of ${\epsilon }_i$ around 0, i.e. $D_{1i}\left({\epsilon }_i\right)\mathrm{<mo>=</mo>}{D}_{1i}\left(-{\epsilon }_i\right)\mathrm{.}$ Therefore, we can rewrite $D_{1i}$ as in equation (4.2):

                              $$D_{1i}\mathrm{<mo>=</mo>}\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left(\left|{\epsilon }_i\right|+\frac{\sqrt{1+{\gamma }_i}}{{\gamma }_i}\right)\right\}-\Phi \left\{\sqrt{\frac{{\gamma }_i}{1-{\gamma }_i}}\left(\left|{\epsilon }_i\right|-\frac{\sqrt{1+{\gamma }_i}}{{\gamma }_i}\right)\right\}\mathrm{.}$$

B. $p$ -value associated with the test statistic $\left|{\epsilon }_i\right|$

First, recall that $P_i\left(v_i\right)\mathrm{<mo>=</mo>}\text{Prob}\left(\left|{\epsilon }_i\right|\mathrm{<mo>></mo>}|{\epsilon }_{\text{obs}\mathrm{,}i}||\Omega \mathrm{<mo>;</mo>}{v}_i\right).$ We define the $p$ -value as the maximum of $P_i\left(v_{L\mathrm{,}i}\right)$ and $P_i\left(-v_{L\mathrm{,}i}\right).$ Since ${\tau }_i\mathrm{<mo>=</mo>}\frac{\mathrm{<mo>|</mo>}{\epsilon }_{\text{obs}\mathrm{,}i}\mathrm{<mo>|</mo>}-\sqrt{1+{\gamma }_i}}{\sqrt{1-{\gamma }_i}},$ we can then write:

                                          $$\begin{array}{ll}{P}_i\left(v_i\right)\hfill & \mathrm{<mo>=</mo>}\text{Prob}\left(\left|{\epsilon }_i\right|\mathrm{<mo>></mo>}\sqrt{1+{\gamma }_i}+\sqrt{1-{\gamma }_i}{\tau }_i|\Omega \mathrm{<mo>;</mo>}{v}_i\right)\hfill \\ \hfill & \mathrm{<mo>=</mo>}\text{Prob}\left({\epsilon }_i\mathrm{<mo>></mo>}\sqrt{1+{\gamma }_i}+\sqrt{1-{\gamma }_i}{\tau }_i|\Omega \mathrm{<mo>;</mo>}{v}_i\right)\hfill \\ \hfill & +\text{Prob}\left({\epsilon }_i\mathrm{<mo><</mo>}-\sqrt{1+{\gamma }_i}-\sqrt{1-{\gamma }_i}{\tau }_i|\Omega \mathrm{<mo>;</mo>}{v}_i\right)\mathrm{.}\hfill \end{array}$$

Using the standardized error distribution (4.3), we obtain:

                             $$\begin{array}{ll}{P}_i\left(v_i\right)\hfill & \mathrm{<mo>=</mo>}\text{Prob}\left(\frac{{\epsilon }_i-v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}\mathrm{<mo>></mo>}\frac{\sqrt{1+{\gamma }_i}-v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}+{\tau }_i|\Omega \mathrm{<mo>;</mo>}{v}_i\right)\hfill \\ \hfill & +\text{Prob}\left(\frac{{\epsilon }_i-v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}\mathrm{<mo><</mo>}\frac{-\sqrt{1+{\gamma }_i}-v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}-{\tau }_i|\Omega \mathrm{<mo>;</mo>}{v}_i\right)\hfill \\ \hfill & \mathrm{<mo>=</mo>}\Phi \left(\frac{-\sqrt{1+{\gamma }_i}+v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}-{\tau }_i\right)\hfill \\ \hfill & +\Phi \left(\frac{-\sqrt{1+{\gamma }_i}-v_i\sqrt{{\gamma }_i}/{\sigma }_v}{\sqrt{1-{\gamma }_i}}-{\tau }_i\right)\mathrm{.}\hfill \end{array}$$

Using the expression $v_{L\mathrm{,}i}\mathrm{<mo>=</mo>}{\sigma }_v\sqrt{\frac{1+{\gamma }_i}{{\gamma }_i}},$ we have:

                                                $$\begin{array}{ll}{P}_i\left(v_i\right)\hfill & \mathrm{<mo>=</mo>}\Phi \left(-{\tau }_i+\frac{\sqrt{1+{\gamma }_i}}{\sqrt{1-{\gamma }_i}}\left[\frac{v_i}{v_{L\mathrm{,}i}}-1\right]\right)\hfill \\ \hfill & +\Phi \left(-{\tau }_i+\frac{\sqrt{1+{\gamma }_i}}{\sqrt{1-{\gamma }_i}}\left[-\frac{v_i}{v_{L\mathrm{,}i}}-1\right]\right)\mathrm{.}\hfill \end{array}$$

Under the null hypothesis $H_0,$ $v_i\mathrm{<mo>=</mo>}{v}_{L\mathrm{,}i}$ or $v_i\mathrm{<mo>=</mo>}-v_{L\mathrm{,}i}$ and in both cases the above equation reduces to:

                                        $$P_i\left(v_{L\mathrm{,}i}\right)\mathrm{<mo>=</mo>}{P}_i\left(-v_{L\mathrm{,}i}\right)\mathrm{<mo>=</mo>}\Phi \left(-{\tau }_i\right)+\Phi \left(-{\tau }_i-2\frac{\sqrt{1+{\gamma }_i}}{\sqrt{1-{\gamma }_i}}\right)\mathrm{.}$$

We will now show that if we reject $H_0$ (with a threshold smaller than 0.5 such as 0.1) then we would reject even more strongly the null hypothesis $H_0^{*}\text{}\mathrm{<mo>:</mo>}\left|v_i\right|\mathrm{<mo>=</mo>}{v}_i^{*}$ for any value $0\le v_i^{*}\mathrm{<mo><</mo>}{v}_{L\mathrm{,}i}.$ First, if ${\tau }_i\le 0,$ i.e., $\left|{\epsilon }_{\text{obs}\mathrm{,}i}\right|\le \sqrt{(1+{\gamma }_i)},$ we observe that $P_i(v_{L\mathrm{,}i})\mathrm{<mo>=</mo>}{P}_i(-v_{L\mathrm{,}i})\ge \text{0}\text{.5}$ and we never reject the null hypothesis $H_0.$ Second, if ${\tau }_i\mathrm{<mo>></mo>}0,$ we can easily show that the function $P_i\left(v_i\right)$ is increasing in $v_i$ over the interval $\left[\mathrm{0,}{v}_{L\mathrm{,}i}\right].$ We also note that it is a function of $v_i$ that is symmetrical around $v_i\mathrm{<mo>=</mo>}0$ since $P_i(v_i)\mathrm{<mo>=</mo>}{P}_i(-v_i).$ Consequently, $P_i(v_i)$ is decreasing on the interval $\left[-v_{L\mathrm{,}i}\mathrm{,}0\right],$ is minimum when $v_i\mathrm{<mo>=</mo>}0$ and maximum when $v_i\mathrm{<mo>=</mo>}{v}_{L\mathrm{,}i}$ and $v_i\mathrm{<mo>=</mo>}-v_{L\mathrm{,}i}.$ Therefore, when $\left|v_i\right|\mathrm{<mo><</mo>}{v}_{L\mathrm{,}i},$ we have:

                                          $$P_i\left(v_i\right)\mathrm{<mo><</mo>}{P}_i\left(v_{L\mathrm{,}i}\right)\mathrm{<mo>=</mo>}\Phi \left(-{\tau }_i\right)+\Phi \left(-{\tau }_i-2\frac{\sqrt{1+{\gamma }_i}}{\sqrt{1-{\gamma }_i}}\right)\mathrm{.}$$

References

Fay, R.E., and Herriot, R.A. (1979). Estimation of income from small places: An application of James-Stein procedures to census data. Journal of the American Statistical Association, 74, 269-277.

Hidiroglou, M.A., Beaumont, J.-F. and Yung, W. (2019). Development of a small area estimation system at Statistics Canada. Survey Methodology, 45, 1, 101-126. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2019001/article/00009-eng.pdf.

Pfeffermann, D., and Ben-Hur, D. (2019). Estimation of randomisation mean square error in small area estimation. International Statistical Review, 87, S1, S31-S49.

Rao, J.N.K., and Molina, I. (2015). Small Area Estimation. John Wiley & Sons, Inc., Hoboken, New Jersey.

Rao, J.N.K., Rubin-Bleuer, S. and Estevao, V.M. (2018). Measuring uncertainty associated with model-based small area estimators. Survey Methodology, 44, 2, 151-166. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2018002/article/54958-eng.pdf.

Rivest, L.-P., and Belmonte, E. (2000). A conditional mean squared error of small area estimators. Survey Methodology, 26, 1, 67-78. Paper available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2000001/article/5179-eng.pdf.

• Previous
• Article main page