Criteria for choosing between calibration weighting and survey weighting
Section 5. Conclusion
In this paper, we have proposed a new criterion
for measuring the impact of using
calibration weights to estimate the total for a variable of interest.
This criterion can be calculated for each variable of interest to determine
whether it is better to use a set of calibration
weights or sampling weights to estimate the total for the variable. The
proposed criterion has the benefit of taking into account the two main aspects
that influence the precision of a total estimator: bias due to the use of
calibration weights and the quality of the linear regression model that
represents the link between the variable of interest and the calibration
variables. Therefore, this criterion can be seen as a
measurement of the threshold where the gain in the variance obtained with the
calibration estimator exceeds the loss in bias due to the use of calibration
weights rather than sampling weights. The simulations conducted to evaluate the proposed criterion showed that
this criterion does indeed identify, for a given variable of interest,
situations where it is best to use calibration weights, i.e., when the variable
of interest is sufficiently correlated with the calibration variables.
It is important to note that the role of this
criterion is not to introduce a new weighting system to replace calibration
weighting or sample weighting. It is used solely to identify which of the two
weighting systems would be best to use for a given variable of interest, which
is very useful for practitioners, particularly in the case of surveys that
cover different subjects, such as omnibus surveys. However, it would be
interesting to study the possibility of producing a unique new weighting system
for all survey variables, based on this criterion, while taking into account
the advantages of both calibration weights and sampling weights. Finally, it
should be noted that the proposed criterion requires a linear relationship
between the variables of interest and the calibration variables, and the
robustness of the criterion is worth investigating.
Acknowledgements
We would like to thank the reviewers for their
thorough work, which helped to improve the results presented in this paper.
Appendix
Simulations results for homoskedastic residual models
Table A.1
(Homoskedastic populations):
Simulation results for the
criterion, by sample size and degree of the
link between the variables of interest and the calibration variables
Table summary
This table displays the results of (Homoskedastic populations): Simulation results for the
criterion Variables of interest, Y1 , Y2 , Y3 , Y4 , Y5 and Y6 (appearing as column headers).
|
Variables of interest |
| Y1 |
Y2 |
Y3 |
Y4 |
Y5 |
Y6 |
| (R2 = 0.01) |
(R2 = 0.10) |
(R2 = 0.20) |
(R2 = 0.50) |
(R2 = 0.75) |
(R2 = 0.98) |
| n = 100 |
(107) |
30,150.81 |
9,298.14 |
1,492.16 |
177.42 |
56.54 |
3.58 |
|
(107) |
27,162.87 |
8,530.43 |
1,477.41 |
326.93 |
207.72 |
160.37 |
|
(107) |
27,162.82 |
8,530.40 |
1,477.39 |
326.90 |
207.69 |
160.34 |
|
1.11 |
1.09 |
1.01 |
0.54 |
0.27 |
0.02 |
|
(107) |
31,523.63 |
9,775.29 |
1,565.31 |
192.17 |
61.49 |
3.90 |
|
(107) |
29,024.17 |
9,128.96 |
1,573.25 |
338.45 |
211.87 |
160.75 |
|
1.09 |
1.07 |
1.00 |
0.58 |
0.30 |
0.02 |
|
0.020 |
0.021 |
0.021 |
0.016 |
0.007 |
0.00008 |
| n = 200 |
(107) |
14,277.16 |
4,441.79 |
732.99 |
83.44 |
26.59 |
1.68 |
|
(107) |
13,343.16 |
4,190.39 |
725.75 |
160.60 |
102.04 |
78.78 |
|
(107) |
13,343.14 |
4,190.37 |
725.73 |
160.58 |
102.02 |
78.77 |
|
1.07 |
1.06 |
1.01 |
0.52 |
0.26 |
0.02 |
|
(107) |
14,195.90 |
4,398.60 |
753.49 |
86.72 |
27.69 |
1.75 |
|
(107) |
13,795.17 |
4,336.28 |
748.77 |
163.53 |
102.90 |
78.84 |
|
1.06 |
1.05 |
1.01 |
0.53 |
0.27 |
0.02 |
|
0.003 |
0.003 |
0.004 |
0.005 |
0.002 |
0.00002 |
| n = 400 |
(107) |
9,086.04 |
2,826.00 |
470.43 |
53.96 |
17.20 |
1.09 |
|
(107) |
8,736.60 |
2,743.71 |
475.19 |
105.15 |
66.81 |
51.58 |
|
(107) |
8,736.58 |
2,743.69 |
475.18 |
105.14 |
66.80 |
51.57 |
|
1.04 |
1.03 |
0.99 |
0.51 |
0.26 |
0.02 |
|
(107) |
9,178.88 |
2,894.26 |
478.67 |
55.38 |
17.65 |
1.12 |
|
(107) |
8,946.42 |
2,833.29 |
485.09 |
106.41 |
67.21 |
51.57 |
|
1.03 |
1.02 |
0.98 |
0.52 |
0.27 |
0.02 |
|
0.001 |
0.001 |
0.002 |
0.003 |
0.002 |
0.00001 |
References
Deville, J.-C., and Särndal, C.-E. (1992). Calibration
estimators in survey sampling. Journal of the American Statistical
Association, 87, 376-382.
Deville, J.-C., and Tillé, Y. (2005). Variance
approximation under balanced sampling. Journal
of Statistical Planning and Inference, 128, 411-425.
Hájek, J. (1981). Sampling
from a Finite Population. New York: Marcel Dekker.
Henry, K.A., and Valliant, R. (2015). A design effect
measure for calibration weighting in single-stage samples. Survey Methodology, 41, 2, 315-331. Paper available at https://www150.statcan.gc.ca/n1/fr/pub/12-001-x/2015002/article/14236-eng.pdf.
Horvitz, D., and Thompson, D. (1952). A generalization
of sampling without replacement from a finite universe. Journal of the
American Statistical Association, 47, 663-685.
Matei, A., and Tillé, Y. (2005). Evaluation of variance
approximations and estimators in maximum entropy sampling with unequal
probability and fixed sample size. Journal
of Official Statistics, 21(4), 2005, 543-570.
Nedyalkova, D., and Tillé, Y. (2008). Optimal sampling
and estimation strategies under linear model. Biometrika, 95, 521-537.
Tirari,
M.H.T. (2003). Estimation d’un total pour les plans de sondage à taille fixe et
équilibrés. Thesis report.
ISSN : 1492-0921
Editorial policy
Survey Methodology publishes articles dealing with various aspects of statistical development relevant to a statistical agency, such as design issues in the context of practical constraints, use of different data sources and collection techniques, total survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves. All papers will be refereed. However, the authors retain full responsibility for the contents of their papers and opinions expressed are not necessarily those of the Editorial Board or of Statistics Canada.
Submission of Manuscripts
Survey Methodology is published twice a year in electronic format. Authors are invited to submit their articles in English or French in electronic form, preferably in Word to the Editor, (statcan.smj-rte.statcan@canada.ca, Statistics Canada, 150 Tunney’s Pasture Driveway, Ottawa, Ontario, Canada, K1A 0T6). For formatting instructions, please see the guidelines provided in the journal and on the web site (www.statcan.gc.ca/SurveyMethodology).
Note of appreciation
Canada owes the success of its statistical system to a long-standing partnership between Statistics Canada, the citizens of Canada, its businesses, governments and other institutions. Accurate and timely statistical information could not be produced without their continued co-operation and goodwill.
Standards of service to the public
Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner. To this end, the Agency has developed standards of service which its employees observe in serving its clients.
Copyright
Published by authority of the Minister responsible for Statistics Canada.
© Her Majesty the Queen in Right of Canada as represented by the Minister of Industry, 2018
Use of this publication is governed by the Statistics Canada Open Licence Agreement.
Catalogue No. 12-001-X
Frequency: Semi-annual
Ottawa
