7. Discussion

Cyril Favre Martinoz, David Haziza and Jean-François Beaumont

Previous

This paper outlined a proposed method for determining the threshold for winsorized estimators. This method has the advantage of being simple to apply in practice and can be used for sampling designs with unequal probabilities. We also proposed a calibration method that satisfies a consistency relation between the domain-level winsorized estimates and a population-level winsorized estimate. Although we applied the method in the case of winsorized estimators, it can be used with any type of robust estimator.

Acknowledgements

The authors are grateful to an associate editor and two reviewers for their comments and suggestions, which substantially improved the quality of this paper. David Haziza's research was funded by a grant from the Natural Sciences and Engineering Research Council of Canada.

Appendix

We want to show that there exists a solution to the equation

$$-\Delta \left(K\right)={\displaystyle \sum _{j\in S}{a}_j}\max \left(\mathrm{0,}{d}_j{y}_j-K\right)=\frac{{\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}}{2}=\widehat{t}-{\widehat{t}}_R$$

under the conditions ${\pi }_{ij}-{\pi }_i{\pi }_j\le 0$ and $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\ge 0.$

First, we arrange the units in order from the smallest value of $b_i=d_i{y}_i\mathrm{,}i\in S\mathrm{,}$ to the largest, so that unit 1 has the smallest value of $b_i$ and unit $n$ the largest value. We begin by considering the case of $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)=0.$ We have to solve the equation $-\Delta \left(K\right)=0,$ and we can easily see that this equation is satisfied for all $K\ge b_n.$

We now turn to the case of  $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\text{>}0.$ We note first that the function $-\Delta \left(K\right)$ is continuous and piecewise linear for $0\le K\le b_n.$ The pieces are defined by the intervals $[b_{j-1}\mathrm{,}{b}_j[\mathrm{,}j=1,\mathrm{...,}n,$ where $b_0=0.$ We also note that $-\Delta \left(0\right)={\displaystyle {\sum }_{j=m}^n{a}_j{b}_j}\text{>}0,$ where $m$ is the smallest index such that $b_m\ge 0.$ By the intermediate value theorem, there is a solution to equation (4.7) if we can show that

$$-\Delta \left(b_n\right)=0\text{<}\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\le -\Delta \left(0\right)={\displaystyle \sum _{j=m}^n{a}_j{b}_j}\mathrm{.}(\text{A}\text{.1})$$

The first inequality follows directly from the condition $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\text{>}0.$ To prove the second inequality, we first note that $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\le {\widehat{B}}_{\text{max}}.$ If we use the estimator of the conditional bias (2.2) and the condition ${\pi }_{ij}-{\pi }_i{\pi }_j\le 0,$ we observe that ${\widehat{B}}_{\text{max}}\le \left(d_k-1\right)y_k,$ index $k$ being associated with the unit that has the largest estimated conditional bias. For the Dalén-Tambay winsorized estimator, the last inequality can be rewritten as ${\widehat{B}}_{\text{max}}\le a_k{b}_k.$ It follows that $a_k{b}_k\le -\Delta \left(0\right)={\displaystyle {\sum }_{j=m}^n{a}_j{b}_j},$ which completes the proof that there is a solution to equation (4.7). For the standard winsorized estimator, we can also easily show that ${\widehat{B}}_{\text{max}}\le a_k{b}_k$ and therefore that a solution exists. In addition, if the $y_i\mathrm{,}i\in S\mathrm{,}$ are all positive, the function $-\Delta \left(K\right)$ is monotonically decreasing for $0\le K\le b_n$ and the solution is unique.

To find the solution $K_{\text{opt}},$ we find the largest index $l$ such that $-\Delta \left(b_l\right)\ge \frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right),$ for $l\le n.$ The solution can then be calculated by linear interpolation between points $b_l$ and $b_{l+1};$ that is,

$$K_{\text{opt}}=b_l\frac{\Delta \left(b_{l+1}\right)-\Delta \left(K_{\text{opt}}\right)}{\Delta \left(b_{l+1}\right)-\Delta \left(b_l\right)}+b_{l+1}\frac{\Delta \left(K_{\text{opt}}\right)-\Delta \left(b_l\right)}{\Delta \left(b_{l+1}\right)-\Delta \left(b_l\right)}\mathrm{,}$$

where $\Delta \left(K_{\text{opt}}\right)=-\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right).$

References

Beaumont, J.-F., Haziza, D. and Ruiz-Gazen, A. (2013). A unified approach to robust estimation in finite population sampling. Biometrika, 100, 555-569.

Berger, Y.G. (1998). Rate of convergence for asymptotic variance of the Horvitz-Thompson estimator. Journal of Statistical Planning and Inference, 74, 149-168.

Clark, R.G. (1995). Winsorization methods in sample surveys. Masters Thesis, Department of Statistics, Australian National University.

Dalén, J. (1987). Practical estimators of a population total which reduce the impact of large observations. R and D Report. Statistics Sweden.

Datta, G.S., Gosh, M., Steorts, R. and Maple, J. (2011). Bayesian benchmarking with applications to small area estimation. Test, 20, 574-588.

Deville, J.-C., and Särndal, C.-E. (1992). Calibration estimators in survey sampling. Journal of the American Statistical Association, 87, 376-382.

Fattorini, L. (2006). Applying the Horvitz-Thompson criterion in complex designs: A computer-intensive perspective for estimating inclusion probabilities. Biometrika, 93, 269-278.

Haziza, D., Mecatti, F. and Rao, J.N.K. (2008). Evaluation of some approximate variance estimators under the Rao-Sampford unequal probability sampling design. Metron, 66, 91-108.

Kokic, P.N., and Bell, P.A. (1994). Optimal Winsorizing cutoffs for a stratified finite population estimator. Journal of Official Statistics, 10, 419-435.

Moreno-Rebollo, J.L., Muñoz-Reyez, A.M. and Muñoz-Pichardo, J.M. (1999). Influence diagnostics in survey sampling: Conditional bias. Biometrika, 86, 923-928.

Moreno-Rebollo, J.L., Muñoz-Reyez, A.M., Jimenez-Gamero, M.D. and Muñoz-Pichardo, J. (2002). Influence diagnostics in survey sampling: Estimating the conditional bias. Metrika, 55, 209-214.

Rivest, L.-P. (1994). Statistical properties of Winsorized means for skewed distributions. Biometrika, 81, 373-383.

Rivest, L.-P., and Hidiroglou, M. (2004). Outlier treatment for disaggregated estimates. Proceedings of the Survey Research Methods Section, American Statistical Association, Alexandria, Virginia, 4248-4256.

Rivest, L.-P., and Hurtubise, D. (1995). On Searls' Winsorized mean for skewed populations. Survey Methodology, 21, 2, 107-116.

Tambay, J.-L. (1988). An integrated approach for the treatment of outliers in sub-annual surveys. Proceedings of the Survey Research Methods Section, American Statistical Association, Alexandria, Virginie, 229-234.

Thompson, M.E., and Wu, C. (2008). Simulation-based randomized systematic PPS sampling under substitution of units. Survey Methodology, 34, 1, 3-10.

You, Y., Rao, J.N.K. and Dick, P. (2004). Benchmarking hierarchical Bayes small area estimators in the Canadian census undercoverage estimation. Statistics in Transition, 6, 631-640.

Previous