A simulated annealing algorithm for joint stratification and sample allocation
Section 7. Comparison with the continuous method in SamplingStrata

• Previous
• Next

We also compared the SAA with the traditional genetic algorithm which Ballin and Barcaroli (2020) have applied to partition continuous strata. We used the target variables outlined in Table 6.1 above as both the continuous target and auxiliary variables (for clarity we outline them again in Table 7.1 below) along with the precision constraints outlined in Table A.1 (the Appendix). In practice, the target variable would not be exactly equal to the auxiliary variable though it is common for the auxiliary variable to be an imperfect version (for example an out-of-date or a related variable) available on the sampling frame. We invite the reader to consider this when reviewing the results of the comparisons below. It is also worth noting that initial solutions were created for both algorithms using the k-means method. Details on the training of hyperparameters for these experiments also can be found in the Appendix.

The attained sample sizes are compared in Table 7.2 below where the sample size for the SAA is expressed as a ratio of the TGA. After the hyperparameters were fine-tuned (see Section A.6) the resulting sample sizes are comparable.

Table 7.3 compares the execution times for the set of hyperparameters that found the sample sizes for each algorithm in Table 7.2 above, as well as the total execution times taken to train that set of hyperparameters.

These results indicate a significantly lower execution time for the SAA for the attained solution quality. The computational efficiency gained by delta evaluation in the training of the recommended hyperparameters is also evident in the total execution times. For the American Community Survey, 2015 experiment significantly more solutions were generated by the SAA than the TGA as a result of the given hyperparameters and this impacts the execution and total execution times (see also Table 7.4). Table 7.4 compares the number of solutions generated by the traditional genetic algorithm with the simulated annealing algorithm.

In all cases except for Kiva Loans and the American Community Survey, 2015 the SAA has generated fewer solutions. The low number of solutions generated by both algorithms for the US Census, 2000 experiment may indicate that the initial k-means solution was near the global minimum. The American Community Survey, 2015 results indicate that the SAA generated significantly more solutions to get to a comparable sample size with the TGA. As we are moving, predominantly, $q=1$ atomic strata between strata such changes in this case had limited impact on solution quality from one solution to the next. However, the gains achieved by delta evaluation meant that more solutions were evaluated per second leading to a more complete search and a lower sample size being attained.

For these experiments, the TGA took longer to find a comparable sample size in all cases. As pointed out in O’Luing et al. (2019), traditional genetic algorithms are not as efficient for grouping problems as the grouping genetic algorithm because solutions tend to have a great deal of redundancy. We would, therefore, propose that the GGA be applied also to continuous strata. On the basis of the above analysis, and the performance of SAAs in local search generally speaking along with the added gains in efficiency from delta evaluation, we would also propose that the SAA be considered as an alternative to the traditional genetic algorithm.

• Previous
• Article main page
• Next