Improved Horvitz-Thompson estimator in survey sampling
Section 1. Introduction
The Horvitz-Thompson (HT) estimator proposed by Horvitz and Thompson (1952) is widely used in survey sampling. It has also been applied to other fields such as functional data analysis (Cardot and Josserand, 2011) and the treatment effect (Rosenbaum, 2002). The HT estimator is an unbiased estimator constructed via inverse probability weighting. However, when the inclusion probabilities are highly heterogeneous, i.e., inclusion probabilities of some units are relatively tiny, the variance of the HT estimator becomes large due to inverse probability weighting. In this paper, we propose an improved Horvitz-Thompson (IHT) estimator to address this problem.
Our approach is to use a hard-threshold for the first-order inclusion probabilities. Specifically, we carefully choose an inclusion probability as the threshold. The inclusion probabilities that are smaller than the threshold are replaced by the threshold, while the others remain unchanged. In this way, we obtain the modified inclusion probabilities, and construct an estimator based on the modified inclusion probabilities through inverse probability weighting. We call this estimator the IHT estimator. This method looks very easy but is more efficient than the HT estimator. This hard-threshold approach can be explained as a shrinkage method. Shrinkage is very commonly used in statistics, such as ridge regression (Hoerl and Kennard, 1970) and high-dimensional statistics (Tibshirani, 1996). In this paper, we use it to reduce the negative effect of highly heterogeneous inclusion probabilities. Similar to other shrinkage methods, our approach introduces a bias, which is proved to be very small, but reduces the variance to a larger extent, so it improves the estimation efficiency. We will theoretically and numerically show the improvement from using the modified inclusion probabilities. In addition to the population total estimator, we also extend this strategy to the ratio estimator, and accordingly, an improved ratio estimator is obtained.
The remainder of the paper is organized as follows. Section 2 introduces the HT estimator and shows its drawback. Section 3 proposes our modified inclusion probabilities and the resultant IHT estimator. We also provide the IHT estimator’s properties, and theoretically compare it with the HT estimator in this section. Section 4 extends our idea to obtain an improved ratio estimator and shows that this modification is efficient. Section 5 presents numerical evidence from simulations and a real data analysis. Section 6 concludes. Proofs of theoretical results are given in the Appendix.
- Date modified: