4. Application to winsorized estimators

Cyril Favre Martinoz, David Haziza and Jean-François Beaumont

Previous | Next

Estimator (3.5) can be written in alternative forms, which can make it easier to implement in some cases. We consider the winsorized form. This form has been widely studied in the literature. As mentioned in Section 1, standard winsorization is distinguished from Dalén-Tambay winsorization.

Standard winsorization involves decreasing the value of units that are above a particular threshold, taking their weight into account. Let ${\tilde{y}}_i$ be the value of variable $y$ for unit $i$ after winsorization. We have

$${\tilde{y}}_i=\{\begin{array}{ll}{y}_i\hfill & \text{if }{d}_i{y}_i\le K\hfill \\ \frac{K}{d_i}\hfill & \text{if }{d}_i{y}_i\text{>}K\hfill \end{array}(4.1)$$

where $K\text{>}0$ is the winsorization threshold. The standard winsorized estimator of the total $t$ is given by

$$\begin{array}{ll}{\widehat{t}}_s\hfill & ={\displaystyle \sum _{i\in S}{d}_i{\tilde{y}}_i}(4.2)\hfill \\ \hfill & =\widehat{t}+\Delta \left(K\right),\hfill \end{array}$$

where

$$\Delta \left(K\right)=-{\displaystyle \sum _{i\in S}}\max \left(\mathrm{0,}{d}_i{y}_i-K\right)\mathrm{.}$$

Hence, the estimator (4.2) can be written in the form (3.1). An alternative is to express ${\widehat{t}}_s$ as a weighted sum of the initial values using modified weights:

$${\widehat{t}}_s={\displaystyle \sum _{i\in S}{\tilde{d}}_i{y}_i}\mathrm{,}$$

where

$${\tilde{d}}_i=d_i\frac{\text{min}\left(y_i\mathrm{,}\frac{K}{d_i}\right)}{y_i}\mathrm{.}(4.3)$$

If $\text{min}\left(y_i\mathrm{,}K/d_i\right)=y_i$ (that is, if unit $i$ is not influential), then ${\tilde{d}}_i=d_i.$ Thus, the weight of a non-influential unit is not modified. In contrast, the modified weight of an influential unit is less than $d_i$ and may even be less than 1. It is worth noting that a unit with a value of $y_i=0$ presents no particular problems, since its contribution to the estimated total, ${\widehat{t}}_s\mathrm{,}$ is zero. In this case, an arbitrary value can be assigned to the modified weight ${\tilde{d}}_i\mathrm{.}$

In the case of Dalén-Tambay winsorization, the values of the variable of interest after winsorization are defined by

$${\tilde{y}}_i=\{\begin{array}{ll}{y}_i\hfill & \text{if }{d}_i{y}_i\le K\hfill \\ \frac{K}{d_i}+\frac{1}{d_i}\left(y_i-\frac{K}{d_i}\right)\hfill & \text{if }{d}_i{y}_i\text{>}K\hfill \end{array}.(4.4)$$

This leads to the winsorized estimator of the total $t_y:$

$$\begin{array}{ll}{\widehat{t}}_{\text{DT}}\hfill & ={\displaystyle \sum _{i\in S}{d}_i{\tilde{y}}_i}(4.5)\hfill \\ \hfill & =\widehat{t}+\Delta \left(K\right)\mathrm{,}\hfill \end{array}$$

where

$$\Delta \left(K\right)=-{\displaystyle \sum _{i\in S}}\frac{\left(d_i-1\right)}{d_i}\max \left(\mathrm{0,}{d}_i{y}_i-K\right)\mathrm{.}$$

Estimator (4.5) can also be written in the form (3.1). As in the case of ${\widehat{t}}_s,$ an alternative is to express ${\widehat{t}}_{\text{DT}}$ as a weighted sum of the initial values using modified weights:

$${\widehat{t}}_{\text{DT}}={\displaystyle \sum _{i\in S}{\tilde{d}}_i{y}_i}\mathrm{,}$$

where

$${\tilde{d}}_i=1+\left(d_i-1\right)\frac{\text{min}\left(y_i\mathrm{,}\frac{K}{d_i}\right)}{y_i}\mathrm{.}(4.6)$$

As in the case of the standard winsorized estimator, the weight of a non-influential unit is not modified. Unlike standard winsorization, Dalén-Tambay winsorization guarantees that the modified weights will not be less than 1. Once again, a unit with a value of $y_i=0$ presents no particular problems, since its contribution to the estimated total, ${\widehat{t}}_{\text{DT}}\mathrm{,}$ is zero. In this case, an arbitrary value can be assigned to the modified weight ${\tilde{d}}_i\mathrm{.}$

Since the standard and Dalén-Tambay winsorized estimators are of the form (3.1), the optimal constant $K_{\text{opt}}$ that minimizes (3.2) is obtained by solving

$$\Delta \left(K\right)=-\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)$$

or

$${\displaystyle \sum _{j\in S}{a}_j\max \left(\mathrm{0,}{d}_j{y}_j-K\right)}=\frac{{\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}}{2}\mathrm{,}(4.7)$$

where $a_j=1$ in the case of ${\widehat{t}}_s$ and $a_j=\left(d_j-1\right)/d_j$ in the case of ${\widehat{t}}_{\text{DT}}.$ It is shown in the Appendix that a solution to equation (4.7) exists under the following conditions:

1. ${\pi }_{ij}-{\pi }_i{\pi }_j\le 0;\text{ and}$
2. $\frac{1}{2}\left({\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}\right)\ge 0.$

Condition 1 is satisfied for most one-stage designs used in practice, such as stratified simple random sampling and Poisson sampling. Condition 2 implies that ${\widehat{t}}_R$ must be less than or equal to $\widehat{t},$ since by construction, a winsorized estimator cannot be greater than the Horvitz-Thompson estimator. It is generally expected that Condition 2 will be satisfied in most skewed populations encountered in business surveys and social surveys. It is also shown in the Appendix that the solution to equation (4.7) is unique if the above conditions are met and if $y_i\ge 0$ for $i\in S.$ The Appendix contains a brief description of an algorithm for finding the solution to equation (4.7).

It should be noted that while the value $K_{\text{opt}}$ is different for each type of winsorized estimator used, the resulting robust estimators are identical. In other words, we have

$${\widehat{t}}_s\left(K_{\text{opt}}\right)={\widehat{t}}_{\text{DT}}\left(K_{\text{opt}}\right)={\widehat{t}}_R=\widehat{t}-\frac{{\widehat{B}}_{\text{min}}+{\widehat{B}}_{\text{max}}}{2}\mathrm{.}(4.8)$$

To compare the influence of each population unit with respect to the (non-robust) expansion estimator, $\widehat{t}\mathrm{,}$ and its robust version (4.8), we carried out a simulation study. For that purpose, we generated two populations, each of size $N=100.$ One population was generated according to a normal distribution with mean 4,108 and standard deviation 1,500, and the other was generated according to a lognormal distribution with mean 4,108 and standard deviation 7,373. From each population we selected $M=\text{500,000}$ samples according to two sampling designs: (i) a simple random sampling without-replacement design of size $n=10,$ and (ii) a Bernoulli design of expected size $n=10.$ First, we calculated the conditional bias of the Horvitz-Thompson estimator for a simple random sampling without-replacement design, given in (2.3) and for a Bernoulli design, given in (2.4). Note that the conditional bias of the Horvitz-Thompson estimator does not have to be approximated by simulation since all the population parameters are known. The conditional bias associated with unit $i$ of the robust estimator given in (3.3) was approximated as follows: Out of the 500,000 selected samples, we identified those which contained unit $i.$ In each of these samples, we calculated the error, ${\widehat{t}}_R-t.$ Finally, we calculated the average value of ${\widehat{t}}_R-t$ over all the samples containing unit $i.$

The results for the simple random sampling without-replacement design for the normal and lognormal distributions are shown in Figures 4.1 (a) and 4.1 (b) respectively. The results for the Bernoulli sampling design for the normal and lognormal distributions are shown in Figures 4.1 (c) and 4.1 (d) respectively. In each figure, the absolute value of the conditional bias of ${\widehat{t}}_R$ is shown in relation to the absolute value of the conditional bias of $\widehat{t}$ for each population unit. The units above the first bisectrix have a conditional bias associated with ${\widehat{t}}_R$ whose absolute value is greater than that of the conditional bias associated with $\widehat{t}.$ Looking first at the results for simple random sampling without replacement, we see that the behaviour of the absolute value of the conditional bias of ${\widehat{t}}_R$ is similar to that of the absolute value of the conditional bias of $\widehat{t},$ which indicates that the influence of the units is not altered significantly after robustification of the expansion estimator. This result is not surprising since the population does not contain any highly influential units. In the case of the lognormal distribution, we see that the influence of the values that have a high conditional bias associated with $\widehat{t}$ has been reduced significantly. On the other hand, we note that for the majority of the data, the conditional bias of ${\widehat{t}}_R$ is slightly higher than that of $\widehat{t}.$ Turning to the results for Bernoulli sampling, we see that in the case of the normal population, the influence of most units has been reduced, since the absolute value of the conditional bias of ${\widehat{t}}_R$ is significantly lower than the absolute value of the conditional bias of $\widehat{t}.$ In the case of the lognormal distribution, the results are similar to those obtained with simple random sampling without replacement for the same distribution.

Figure 4.1 Absolute value of the conditional biases of the robust and non-robust estimators

Description for Figure 4.1

Previous | Next