7 Conclusions and future work
Marco Ballin and Giulio Barcaroli
For any given multipurpose and multidomain sample survey, the optimal stratification of the sampling frame can be determined together with the optimal sample size and allocation of units among strata, by means of a combined use of the Bethel algorithm (or, more generally, of a NLP solver) for the determination of the minimum sample size required to satisfy precision constraints, and of the genetic algorithm for the exploration of the universe of potential stratifications, rigorously generated accordingly to the theory of partitions. The information required is nearly the same as the one required by the allocation problem: desired precision on estimates of total (or means) of target variables, and information regarding the distributions of each target variable in population strata. Initial stratification should be considered at the most detailed level (atomic stratification), i.e. the one determined by the Cartesian product of values of all available stratification variables.
The complete exploration of the set of all possible stratifications is in practical cases computationally prohibitive. The use of the genetic algorithm permits to explore the space of solutions in a very efficient manner. By carefully tuning the execution parameters, it is possible to determine the optimal solution, or at least a solution likely to be not far from the optimal one.
The application of this algorithm to two different surveys (the 2003 Italian Farm Structure Survey and the 2010 Monthly milk and milk products) shows that the obtained solutions are much better, in terms of sample efficiency, than the ones manually produced by expert methodologists (in Istat, the algorithm has been applied to three more surveys: "Economic outcomes of agricultural holdings�, "Structure and production of main wooden cultivations�, "Survey on forecasting of some herbal crops sowing�).
In all the cases reported, it has been possible to calculate the values required as input to our algorithm (in particular: means and standard deviations of the target variables in the different atomic strata), because of the availability of related values for each unit in the frames. In more realistic situations, this kind of information is not directly available. Instead, we could use estimates produced by alternative sources: administrative data, other surveys, or previous rounds of the same survey, or even hypothesis (usually conservative) on the variability of target variables within the strata. Accordingly to Rivest (2002) it is also possible to model target variables assuming auxiliary variables s as explanatory variables, in order to estimate means and standard deviations on the basis of predicted values of s. Of course, the less "direct� is the information on the target variables, the less robust is the proposed method, because of the uncertainty caused by the use of proxy information, or model-based predictions.
Another limit affecting this approach still lies in the handling of continuous auxiliary variables. In our approach, we simply suggest to transform them into categorical ones, in order to be considered in the determination of the universe of all possible stratifications of the sampling frame. A first element for future work is in giving indications on how to transform these variables in order to get the best from them. A second one is in the fact that some of the strata contained in the optimal solution may be characterized by non contiguous values of the transformed continuous variables, or of the categorical ordinal variables, which is something odd that should not be allowed: this could be prevented by imposing constraints on the generation of candidate solutions.
References
Baillargeon, S., and Rivest, L.-P. (2009). A general algorithm for univariate stratification. International Statistical Review, 77, 3, 331-344.
Baillargeon, S., and Rivest, L.-P. (2011). The construction of stratified designs in R with the package stratification. Survey Methodology, 37, 1, 53-65.
Barcaroli, G., Pagliuca, D. and Willighagen, E. (2013a). SamplingStrata: Optimal stratification of sampling frames for multipurpose sampling surveys. R package version 1.0-1. http://cran.r-project.org/web/packages/SamplingStrata/index.html.
Barcaroli, G. (2013b). Optimization of sampling strata with the SamplingStrata package. http://cran.r-project.org/web/packages/SamplingStrata/vignettes/SamplingStrataVignette.pdf.
Benedetti, R., Espa, G. and Lafratta, G. (2008). A tree-based approach to forming strata in multipurpose business surveys. Survey Methodology, 34, 2, 195-203.
Bethel, J. (1985). An optimum allocation algorithm for multivariate surveys. American Statistical Proceedings of the Survey Research Methods Section, 209-212.
Bethel, J. (1989). Sample allocation in multivariate surveys. Survey Methodology, 15, 1, 47-57.
Chromy, J.B. (1987). Design optimization with multiple objectives. Proceedings of the American Statistical Association Section on Survey Research Methods 1987, 194-199.
Cochran, W.G. (1977). Sampling Techniques. New York: John Wiley & Sons, Inc.
Dalenius, T., and Hodges, J.L. (1959). Minimum variance stratification. Journal of American Statistical Association, 54, 88-101.
Day, C.D. (2006). Application of an evolutionary algorithm to multivariate optimal allocation in stratified sampling designs. Proceedings of the American Statistical Association Section on Survey Research Methods 2006 [CD-ROM].
Day, C.D. (2010). A multi-objective evolutionary algorithm for multivariate optimal allocation. Section on Survey Research Methods - JSM 2010, 3351-3358.
Díaz-García, J.A., and Cortez, L.U. (2008). Multi-objective optimisation for optimum allocation in multivariate stratified sampling. Survey Methodology, 34, 2, 215-222.
Gunning, P., and Horgan, J.M. (2004). A new algorithm for the construction of stratum boundaries in skewed populations. Survey Methodology, 30, 2, 159-166.
Hankin, R.K.S., and West, L.J. (2007). Set Partitions in R. Journal Of Statistical Software, Code Snippet 2. December 2007, 23, http://www.jstatsoft.org/.
Hankin, R.K.S. (2011). Partitions: Additive partitions of integers. R package version 1.9-19. http://cran.r-project.org/web/packages/partitions/index.html.
Hartigan, J.A., and Wong, M.A. (1979). A k-means clustering algorithm. Applied Statistics, 28, 100-108.
Hidiroglou, M.A. (1986). The costruction of self-representing stratum of large units in survey design. The American Statistician, 40, 27-31.
Huddleston, H.F., Claypool, P.L. and Hocking, R.R. (1970). Optimal sample allocation to strata using convex programming. Applied Statistics, 19, 273-278.
Keskintürk, T., and Er, S. (2007). A genetic algorithm approach to determine stratum boundaries and sample sizes of each stratum in stratified sampling. Computational Statistics and Data Analysis, 15 September 2007, 52, 1, 53-67.
Khan, M.G.M., Nand, N. and Ahmad, N. (2008). Determining the optimum strata boundary points using dynamic programming. Survey Methodology, 34, 2, 205-214.
Khan, M.G.M., Maiti, T. and Ahsan, M.J. (2010). An optimal multivariate stratified sampling design using auxiliary information: An integer solution using goal programming approach. Journal of Official Statistics, 26, 4, 695-708.
Kish, L. (1976). Optima and proxima in linear sample designs. Journal of the Royal Statistical Society, Series A, 159, 80-95.
Kozak, M., Verma, M.R. and Zieliński, A. (2007). Modern approach to optimum stratification: Review and perspectives. Statistics in Transition, 8(2), 223-250.
Kozak, M., Zieliński, A. and Singh, S. (2008). Stratified two-stage sampling in domains: Sample allocation between domains, strata, and sampling stages. Statistics & Probability Letter, June 2008, 78, 8, 970-974.
Kozak, M., and Wang, H.Y. (2010). On stochastic optimization in sample allocation among strata. Metron - International Journal of Statistics, LXVIII, 1, 95-103.
Lavallée, P., and Hidiroglou, M.A. (1988). On the stratification of skewed populations. Survey Methodology, 14, 1, 33-43.
Rivest, L.-P. (2002). A generalization of the Lavallée and Hidiroglou algorithm for stratification in business surveys. Survey Methodology, 28, 2, 191-198.
Schmitt, L.M. (2001). Theory of genetic algorithms. Theoretical Computer Science, 259, 1-61.
Schmitt, L.M. (2004). Theory of genetic algorithms II: Models for genetic operators over the string-tensor representation of populations and convergence to global optima for arbitrary fitness function under scaling. Theoretical Computer Science, 310, 181-231.
Singh, R. (1971). Approximately optimum stratification on the auxiliary variables. Journal of the American Statistical Association, 66, 829-833.
Stokes, L., and Plummer, J. (2004). Using spreadsheet solvers in sample design. Computational Statistics & Data Analysis, 44, 527-546.
Vose, M.D. (1999). The Simple Genetic Algorithm: Foundations and Theory, MIT Press, Cambridge, MA.
Willighagen, E. (2012). Genalg: R Based Genetic Algorithm. R package version 0.1.1. http://cran.r-project.org/web/packages/genalg/index.html.
- Date modified: