Variance estimation in multi-phase calibration
Section 4. A simulation study

• Previous
• Next

The main objective of our analysis in this paper was to provide a consistent estimator for the variance of multi-phase calibrated estimators that holds for any number of phases of calibration. A simulation study could thus be executed to compare the innovative estimator with others found in the literature. As generally no alternative estimators are found for schemes with three or more phases $\left(p\ge 3\right)$ we conducted the comparison mainly for the most investigated case of two phases. Another study was preformed for $p\mathrm{<mo>=</mo>\; 3}$ to evaluate the deviation of the proposed estimator from the true simulated value. The studies are described here in general terms. They meant basically to demonstrate the relevancy of the proposed estimator, its concurrence with the “boundary condition” of two phases and its potential for designs with more than two phases. An extensive study to characterize the efficiency of the proposed estimator as a function of the design parameters such as the sampling rates, the choice of calibrating variables and their correlation with $y,$ etc., is left for future research.

An estimation process of two-phase calibration was applied to data from a recent survey on career and mobility of Doctorate Holders’ (DHs). As there exists no frame of DHs, data about higher education was extracted from a recent population census. However, only a sample $S_1$ that constitutes one fifth of the households enumerated in the census were given an elaborated questionnaire that includes also questions about higher education. A subsample $S_2$ from $S_1$ was drawn for the DHs survey in which a further elaborated questionnaire was given to those who were in fact DHs. Thus, a two-phase calibration scheme to estimate characteristics of DHs was in order. The first phase calibrated the joint variables of $S_1$ and $S_2$ to estimated totals computed from $S_1.$ In the second phase, demographic data of $S_1$ was calibrated to the known totals from the full population register $U.$ We conducted a simulation study on that data where the survey data served in our study as the true population. One thousand samples (realizations) $\left\{u\text{}\mathrm{,}{s}_1\text{}\mathrm{,}{s}_2\right\}$ of sizes $N\mathrm{<mo>=</mo>}$  1,000, $n_1\mathrm{<mo>=</mo>}$  200, $n_2\mathrm{<mo>=</mo>}$  50 were randomly drawn from the dataset $S_2$ of DHs. To each sample we applied the same process of two-phase calibration utilizing the estimator given by (3.7) with equation (3.6) as a presentation of the calibrated weights ${\tilde{w}}_2,$ and its variance estimator given by (3.11) as a special case of (3.8). As already indicated, when $p\mathrm{<mo>=</mo>\; 2}$ the estimates $\widehat{Y}\mathrm{<mo>=</mo>}{\tilde{w}}_2^{ \prime }y$ are identical either under the new presentation or under the conventional one used so far in the literature, Särndal et al. (1992) so our focus was on the variance estimators (3.10) and (3.11) computed according to the two different methods. A typical pattern of the comparison between the two variance estimators in this special case of two-phase calibration is presented in Figure 4.1. It can be seen that although the fundamental difference between the two variance estimators, in most realizations, the difference between their estimates is quite small. Though on a certain one it can reach up to 20%. For that particular variable shown in the figure, the mean value of both estimators for the variance was very similar, namely, 54.17² and 54.65², while the true value in the simulation data was 54.46². Even the variance of their standard deviation estimator, namely, 5.73² versus 5.93² were almost the same for that variable. These results are reported in Table 4.1. The favorable characteristic of the proposed estimator stands out in the $5^{\text{th}}$ column. Contrary to the conventional estimator where the two terms of the variance estimator are of the same order of magnitude, the $2^{\text{nd}}$ term of (3.11) constitutes over 99% of the variance, with a variation of less than two percent between all 1,000 realizations. We discussed the explanation to this phenomenon in 3.2. The outcomes reported above repeated themselves for all variables studied and we found it irrelevant at this point to present other variables or investigate this specific data or the special case of two-phase calibration any further.

The similarity in estimates of the two variance estimators in the case of two phases is reassuring but a comparison in three or more phases could not be preformed because an alternative estimator to the variance does not exist. A replication method for two-phase stratified sampling was proposed by Kim et al. (2006) and a sketch for a generalization for a three-phase case is briefly outlined but with no explicit formulation or simulation results. In our simulation we added a third calibration phase using some variables, expertise field related, common with the second phase sample of DHs and conducted the study in the same manner as with the two-phase case. The simulation study has again demonstrated an excellent estimation for the variance of a three-phase calibrated estimator for all variables $Y$ examined and all different sets of calibrating variables in all phases. Rapid convergence rates of the variance estimator are displayed even for very small sample sizes such as $n\mathrm{<mo>=</mo>}$  25 or lower at the third phase. Some results for various design parameters are reported in Table 4.2. As portrayed earlier the simulation was performed over a population size of 1,000 so the first three designs have overall weight of $w^{*}\mathrm{<mo>=</mo>}$  40 and the next three of $w^{*}\mathrm{<mo>=</mo>}$ 20. So, as expected, the variance of the calibrated estimator for the first three designs is generally higher, although it also depends on the sample sizes of the $1^{\text{st}}$ and $2^{\text{nd}}$ phases as shown for example in the artificial case number 4 which depicts a generally impractical scenario. The relative biases $\frac{E\left({\widehat{\text{Std}}}_P\right)}{\text{Std}}-1$ are close to zero for all designs investigated and the 95% Confidence Interval (CI) coverages were also estimated and found to be mostly conservative and close to their nominal levels. The standard deviation of ${\widehat{\text{Std}}}_P$ are roughly about 5% - 10% of its value as presented in column 7.

• Previous
• Article main page
• Next