Criteria for choosing between calibration weighting and survey weighting
Section 4. Simulation study

• Previous
• Next

In order to evaluate the ${\widehat{\mathrm{Weff}}}_S$ criterion (3.4), so that we can determine whether to use calibration weights or sampling weights, we conducted a series of simulations using data observed for a population of 5,800 cottage-industry units. We considered six calibration variables, from which several variables of interest $Y_i$ were generated, with consideration for linear regression models, while accounting for the strength of the link between the variables of interest and the calibration variables through the choice of residual variance in the regression models. Furthermore, to study the impact of the heteroskedasticity of the model residuals on the results obtained for criterion ${\widehat{\text{Weff}}}_S,$ we also considered the case where the variables of interest are generated using models with heteroskedastic residuals.

For the purposes of these simulations, we selected 10,000 samples using a simple random sampling design (SRSD), with three sample sizes: 100, 200 and 400 cottage-industry units, to study the impact of the sample size on the results obtained. Across the 10,000 samples selected, we calculated the following indicators:

• ${\text{MSE}}_{\text{Cal}}\text{:}$ the AMSE for the calibration estimator, the expression of which is given by (2.5) and where $E\left(w_{kS\text{},C}|I_k=1\right)$ and $V_k$ are determined respectively by the mean and the variance of weights $w_{kS,C}$ considering all of the selected samples containing unit $k.$

• ${\widetilde{\text{MSE}}}_{\text{HT}}\text{:}$ approximation (2.10) of the AMSE for the HT estimator. ${\widetilde{\text{MSE}}}_{\text{HT}}$ corresponds to ${\text{MSE}}_{\text{HT}}$ (AMSE (2.7) for the HT estimator) that we were able to calculate in these simulations since the samples were selected using SRSD.

• Weff: the theoretical value of the Weff calculated using (3.1) and defined by the ratio of ${\text{MSE}}_{\text{Cal}}$ and ${\widetilde{\text{MSE}}}_{\text{HT}}\text{.}$

• ${\overline{\widehat{\text{MSE}}}}_{\text{Cal}}\text{:}$ the simulation mean for the ${\widehat{\text{MSE}}}_{\text{Cal}}$ estimator of ${\text{MSE}}_{\text{Cal}}$ where

$${\overline{\widehat{\text{MSE}}}}_{\text{Cal}}=\frac{1}{\text{10,000}}{\displaystyle \sum _{s=1}^{\text{10,000}}\left({\displaystyle \sum _{k\in s}{d}_k{\widehat{\sigma }}_k^2\left[\frac{{\left(w_{ks\text{},C}-d_k\right)}^2}{d_k}+{\widehat{R}}_{ks}^2\left(d_k-1\right)+{\left({\widehat{R}}_{ks}-1\right)}^2\right]}\right)}.$$

• ${\overline{\widehat{\text{MSE}}}}_{\text{HT}}\text{:}$ the simulation mean for the ${\widehat{\widetilde{\text{MSE}}}}_{\text{HT}}$ estimator of ${\widetilde{\text{MSE}}}_{\text{HT}}$ where

$${\overline{\widehat{\text{MSE}}}}_{\text{HT}}=\frac{1}{\text{10,000}}{\displaystyle \sum _{s=1}^{\text{10,000}}\left({\widehat{V}}_{\text{Approx}\text{},s}+{\displaystyle \sum _{k\in s}{d}_k{\widehat{\sigma }}_k^2\left(d_k-1\right)}\right)}.$$

• $\overline{\widehat{\text{Weff}}}\text{:}$ the simulation mean for the ${\widehat{\text{Weff}}}_S$ estimator (3.4) of Weff.

• $\text{MSE}({\widehat{\text{Weff}}}_S):$ the MSE of ${\widehat{\text{Weff}}}_S$ simulations defined by

$$\text{MSE}\left({\widehat{\text{Weff}}}_S\right)=\frac{1}{\text{10,000}}{\displaystyle \sum _{s=1}^{\text{10,000}}{\left({\widehat{\text{Weff}}}_s-\text{Weff}\right)}^2}.$$

The simulation results for heteroskedastic regression models are presented in Table 4.1 below, while the results for homoskedastic models are given in Table A.1 in the appendix.

Hence, the simulation results show that the Weff criterion proposed to measure the impact of using calibration weights helps us to identify situations where calibration weighting should not be used, i.e., when the variable of interest is weakly correlated with the calibration variables $\left(R^2<\text{0}\text{.20}\right).$ Furthermore, the ${\widehat{\text{Weff}}}_S$ estimator (3.4) proposed to estimate the Weff criterion proved to be an effective estimator, recording the same performances, regardless of the strength of the link between the variable of interest and the calibration variables. Heteroskedastic residuals for regression models, representing the link between the variable of interest and the calibration variables, had little impact on the performances of the Weff criterion and the ${\widehat{\text{Weff}}}_S$ estimator. We also noted a lack of impact in using approximation (2.8) for the variance under design ${\displaystyle {\sum }_{k\in S}{d}_k{x}_k^{\prime }\beta }$ since the impact of the deviation between the AMSE for the HT estimator $\left({\text{MSE}}_{\text{HT}}\right)$ and its approximation ${\widetilde{\text{MSE}}}_{\text{HT}}$ (2.10) was negligible in the results for the Weff criterion. This was predictable since the design being considered was a SRSD.

• Previous
• Article main page
• Next