Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force Survey 5. DiscussionAdaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force Survey 5. Discussion

We constructed a multi-mode optimization problem that extends the framework of adaptive survey designs to mixed-mode survey (re)designs. This framework is especially useful when it is anticipated that method effects due to a change of mode design may impact the comparability and accuracy of statistics. To our best knowledge, this is the first research attempt of its kind and can be used as a basis for minimizing method effects subject to costs and other constraints.

In the optimization model, we included three quality criteria, one cost criterion and one logistical criterion. The quality criteria were the numbers of respondents in sampling strata, which acts as a surrogate for precision, the absolute adjusted overall method effect, which is the level shift caused by the design relative to the benchmark design and may be viewed as comparability in time, and the maximal absolute difference in method effects over important subpopulations, which may be viewed as comparability over population domains. The cost criterion is the total budget of the survey. The logistic criterion is the sample size, which needs to be limited in order to avoid a quick depletion of the sampling frame. The third quality criterion, the maximal absolute difference in subpopulation method effects, is nonlinear in the decision variables (the strategy allocation probabilities) and makes the optimization problem computationally complex. Although this criterion complicates the problem, it is a useful constraint that is often put forward by survey analysts and users. In regular redesigns, this criterion is often not considered and the Dutch LFS mixed-mode design leads to relatively large differences in method effects between subpopulations. Clearly, some of the criteria may be omitted and other quality, cost or logistical criteria may be added. In a follow-up on this research at Statistics Netherlands, various other criteria, mostly logistical, are considered.

In the optimization model, the focus was on maximizing quality, reflected by comparability in time, subject to cost constraints and other constraints on quality and logistics. The objective of the optimization may, however, be changed and each of the constraints could function as the objective. For instance, one may minimize cost subject to quality and logistical constraints. One may also take a wider approach and perform several optimizations for different budget and quality levels in order to derive an informative multidimensional view on which a decision can be based.

Our attempt must be seen as merely a first step towards adaptive mixed-mode survey designs. There are various methodological and practical issues that need to be resolved. First, our approach is suited for surveys with only a few key statistics. For each of these statistics, an optimization can be performed and a weighted decision can be made. When a survey has a wide range of statistics, such an approach is not feasible. Second, the optimization leans heavily on the accuracy of its input parameters, i.e., estimated response probabilities, registered-telephone probabilities, cost parameters and mode effects in this case. It is important to assess the sensitivity of the optimization results to the accuracy of these parameters. It may be hypothesized that the objective function is relatively smooth with respect to these parameters, however, it is still important to perform sensitivity analyses. Third, it is essential to consider the sampling variation of the realized quality and costs of the optimized design when multiple waves of a survey are conducted. Such variation may be large and downsize the value of a precise optimization. Fourth, once nonlinear criteria are added to the problem, one has to rely on advanced solvers in statistical software. Even when using such solvers, convergence to global optimum is usually not assured and one has to be satisfied with local optima. For this reason, it is important to choose a useful set of starting points, including starting points that correspond to current designs. The practical issues concern the number of population strata, the number of strategies and the coordination to other surveys. Although survey administration systems and tools may support adaptive survey designs, such designs are harder to monitor and analyze. Furthermore, the tailoring of survey modes affects the size and form of interviewer workloads; interviewers may get only a specific range of subpopulations.

An important aspect of adaptive survey designs is the use of estimates for all kinds of input parameters such as response propensities, variable costs per sample unit and method effects between designs. Such estimates may not be readily available and there may only be weak historic survey data to support estimation. There are then four options: search for similar surveys that have historic support, be modest and restrictive in the choice of design features, perform a transitional period in which pilot studies and parallel runs are conducted, and develop a framework for learning and updating of parameters. In particular, designs with $W e b$ as one of the modes may still lack historic support for estimation in many countries, see, e.g., Mohorko, de Leeuw and Hox (2013). We also note that input parameters may gradually change in time, so that continuous updating will be needed. However, all of this is no different from a non-adaptive survey, except that now estimates are needed for relevant subpopulations instead of the overall population alone. Finally, we note that optimized adaptive designs, like optimized non-adaptive designs, provide an average, expected quality and costs. Due to sampling variation, the realized quality and costs will vary and unforeseen events may lead to deviations. Hence, monitoring and reacting to unforeseen events remain necessary.

Future research needs to address robustness of adaptive survey designs and should investigate other quality, cost and logistical criteria. It is also important that this study is replicated in order to evaluate whether the investment in terms of additional data collection and in terms of explicit optimization is worth the effort. The ultimate goal of this research is a data collection design strategy that allows for learning and updating optimization input parameters and that supports effective and efficient cost-benefit analyses in mixed-mode (re)designs. A Bayesian approach seems most promising for this purpose.

Acknowledgements

The authors would like to thank dr. Sandjai Bhulai (VU University Amsterdam) for his constructive comments on the mathematical framework presented in the current paper. The authors also thank Boukje Janssen (CBS) and Martijn Souren (CBS) for processing the raw field data for analysis and Joep Burger (CBS) for his comments that helped improve this paper.

Appendix A

Estimates of input parameters

In Section 4.4, we explain the estimation of input parameters for strategies that are observed only partially in the parallel runs. Here, we give the estimates for the response propensities, telephone registration propensities, variable costs per sample unit and adjusted method effects. Standard errors for all parameters were estimated using bootstrap resampling.

Table A2 presents the estimated response propensities $ρ (s, g)$ from available data and their corresponding standard errors. Table A1 shows the estimated propensity for a registered phone $λ (g) .$

Table A1
Estimated propensities for registered phone for group $g \in G$ with the corresponding standard errors given in brackets
Table summary
This table displays the results of Estimated propensities for registered phone for group XXXX with the corresponding standard errors given in brackets. The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$G$	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$	$g_{7}$	$g_{8}$	$g_{9}$
$λ (g)$	38.1%	76.4%	30.2%	22.4%	60.0%	38.9%	32.0%	53.4%	62.4%
$λ (g)$	(0.9)	(1.6)	(2.0)	(2.2)	(1.1)	(0.7)	(1.3)	(0.6)	(1.2)

Table A2
Estimated response propensities per strategy $s$ and group $g$ with the corresponding standard errors given in brackets
Table summary
This table displays the results of Estimated response propensities per strategy XXXX and group XXXX with the corresponding standard errors given in brackets . The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$ρ (s, g)$	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$	$g_{7}$	$g_{8}$	$g_{9}$
$W e b$	23.2%	23.6%	15.5%	10.8%	27.9%	27.7%	17.5%	36.7%	22.4%
$W e b$	(0.3)	(0.6)	(0.6)	(0.6)	(0.4)	(0.2)	(0.5)	(0.2)	(0.5)
$T e l 2$	12.2%	31.4%	8.5%	4.7%	19.7%	13.3%	7.2%	18.1%	21.2%
$T e l 2$	(0.5)	(1.1)	(0.8)	(0.8)	(0.6)	(0.4)	(0.5)	(0.4)	(0.8)
$T e l 2 +$	20.8%	41.3%	15.2%	8.6%	31.1%	23.8%	14.3%	33.3%	37.5%
$T e l 2 +$	(0.6)	(1.1)	(1.0)	(1.0)	(0.7)	(0.5)	(0.7)	(0.5)	(0.9)
$F 2 F 3$	43.5%	53.5%	42.2%	34.1%	45.1%	45.3%	35.9%	46.7%	54.6%
$F 2 F 3$	(1.5)	(1.7)	(2.4)	(2.4)	(1.1)	(0.9)	(1.5)	(0.7)	(1.4)
$F 2 F 3 +$	52.4%	58.3%	51.0%	41.2%	51.2%	54.9%	46.0%	56.8%	61.4%
$F 2 F 3 +$	(1.3)	(1.6)	(2.5)	(2.2)	(1.1)	(0.8)	(1.4)	(0.7)	(1.3)
$W e b \to T e l 2$	28.3%	41.0%	20.2%	13.9%	36.3%	34.0%	20.8%	44.5%	23.1%
$W e b \to T e l 2$	(0.4)	(0.8)	(0.7)	(0.8)	(0.4)	(0.3)	(0.5)	(0.3)	(0.5)
$W e b \to T e l 2 +$	32.8%	48.4%	23.8%	17.5%	42.1%	41.1%	25.8%	52.1%	24.4%
$W e b \to T e l 2 +$	(0.4)	(0.7)	(0.8)	(0.9)	(0.5)	(0.3)	(0.6)	(0.3)	(0.5)
$W e b \to F 2 F 3$	46.3%	57.7%	38.6%	32.7%	50.0%	51.0%	39.3%	58.9%	50.0%
$W e b \to F 2 F 3$	(0.5)	(1.0)	(1.0)	(1.0)	(0.6)	(0.4)	(0.7)	(0.4)	(0.5)
$W e b \to F 2 F 3 +$	49.8%	58.3%	43.4%	36.6%	52.6%	54.7%	44.3%	62.0%	54.2%
$W e b \to F 2 F 3 +$	(0.5)	(0.9)	(0.9)	(0.9)	(0.5)	(0.4)	(0.6)	(0.4)	(0.5)

For the method effect $D (s, g),$ two benchmarks were selected after consultation with practitioners, i.e., ${BM}_{1} = {\bar{y}}_{F 2 F 3 +}$ and ${BM}_{2} = 1 / 3 * ({\bar{y}}_{W e b} + {\bar{y}}_{T e l 2 +} + {\bar{y}}_{F 2 F 3 +}),$ where ${\bar{y}}_{mode}$ represents the average unemployment rate estimated via the indicated survey mode. Tables A3 and A4 present the estimated method effects against the two benchmarks including their standard errors.

The estimates for the variable costs per sample unit plus estimated standard errors are given in Table A5. The costs are expressed relative to the $F 2 F 3 +$ strategy, which is set at one.

Table A3
Estimated method effects against benchmark ${BM}_{1} = {\bar{y}}_{F 2 F 3 +}$ with the corresponding standard errors given in brackets
Table summary
This table displays the results of Estimated method effects against benchmark XXXX with the corresponding standard errors given in brackets. The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$D^{{BM}_{1}} (s, g)$	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$	$g_{7}$	$g_{8}$	$g_{9}$
$W e b$	1.5%	0.0%	-2.3%	-4.5%	0.9%	-0.4%	-2.2%	0.6%	-0.4%
$W e b$	(1.0)	(0.5)	(1.5)	(3.1)	(0.7)	(0.4)	(1.5)	(0.5)	(0.6)
$T e l 2$	-0.2%	-0.1%	-2.6%	-6.8%	-1.0%	-0.9%	-1.1%	0.2%	-1.3%
$T e l 2$	(0.7)	(0.1)	(0.9)	(1.8)	(0.4)	(0.3)	(1.1)	(0.4)	(0.4)
$T e l 2 +$	-0.1%	-0.1%	-2.3%	-4.9%	-0.6%	-1.0%	-0.8%	-0.2%	-1.2%
$T e l 2 +$	(0.7)	(0.1)	(0.8)	(1.7)	(0.4)	(0.3)	(1.0)	(0.3)	(0.4)
$F 2 F 3$	-0.5%	-0.1%	0.0%	0.7%	-0.1%	0.0%	0.5%	0.3%	0.1%
$F 2 F 3$	(0.3)	(0.1)	(0.4)	(0.6)	(0.1)	(0.1)	(0.3)	(0.1)	(0.1)
$F 2 F 3 +$	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%
$F 2 F 3 +$	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)
$W e b \to T e l 2$	0.9%	0.0%	-2.4%	-3.4%	-0.1%	-0.7%	-4.4%	0.9%	-0.7%
$W e b \to T e l 2$	(1.0)	(0.4)	(1.5)	(3.7)	(0.6)	(0.5)	(1.9)	(0.5)	(0.6)
$W e b \to T e l 2 +$	0.9%	-0.1%	-3.7%	-1.7%	0.5%	-0.7%	-3.0%	0.6%	-0.4%
$W e b \to T e l 2 +$	(0.9)	(0.3)	(1.4)	(3.2)	(0.7)	(0.4)	(1.4)	(0.5)	(0.6)
$W e b \to F 2 F 3$	0.7%	0.0%	-1.2%	-1.6%	0.6%	-0.3%	-1.0%	0.5%	-0.2%
$W e b \to F 2 F 3$	(0.6)	(0.3)	(0.8)	(1.4)	(0.5)	(0.3)	(0.8)	(0.3)	(0.3)
$W e b \to F 2 F 3 +$	0.9%	0.0%	-1.2%	-2.0%	0.6%	-0.3%	-1.2%	0.4%	-0.2%
$W e b \to F 2 F 3 +$	(0.6)	(0.3)	(0.8)	(1.4)	(0.5)	(0.3)	(0.8)	(0.3)	(0.3)

Table A4
Estimated method effects against benchmark ${BM}_{2} = 1 / 3 * ({\bar{y}}_{W e b} + {\bar{y}}_{T e l 2 +} + {\bar{y}}_{F 2 F 3 +})$ with the corresponding standard errors given in brackets
Table summary
This table displays the results of Estimated method effects against benchmark XXXX with the corresponding standard errors given in brackets. The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$D^{{BM}_{2}} (s, g)$	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$	$g_{7}$	$g_{8}$	$g_{9}$
$W e b$	1.0%	0.1%	-0.8%	-1.4%	0.8%	0.1%	-1.2%	0.5%	0.1%
$W e b$	(0.5)	(0.3)	(0.9)	(1.8)	(0.4)	(0.2)	(0.8)	(0.2)	(0.3)
$T e l 2$	-0.6%	-0.1%	-1.0%	-3.7%	-1.2%	-0.5%	-0.1%	0.1%	-0.8%
$T e l 2$	(0.3)	(0.2)	(0.6)	(1.4)	(0.2)	(0.2)	(0.8)	(0.2)	(0.2)
$T e l 2 +$	-0.6%	-0.1%	-0.8%	-1.7%	-0.7%	-0.5%	0.2%	-0.3%	-0.6%
$T e l 2 +$	(0.2)	(0.2)	(0.5)	(1.0)	(0.2)	(0.1)	(0.5)	(0.1)	(0.2)
$F 2 F 3$	-1.0%	-0.1%	1.6%	3.8%	-0.2%	0.5%	1.5%	0.2%	0.6%
$F 2 F 3$	(0.7)	(0.2)	(0.8)	(1.6)	(0.4)	(0.2)	(0.8)	(0.3)	(0.3)
$F 2 F 3 +$	-0.5%	0.0%	1.6%	3.1%	-0.1%	0.5%	1.0%	-0.1%	0.5%
$F 2 F 3 +$	(0.5)	(0.2)	(0.7)	(1.4)	(0.4)	(0.2)	(0.7)	(0.3)	(0.3)
$W e b \to T e l 2$	0.4%	0.0%	-0.9%	-0.3%	-0.2%	-0.2%	-3.4%	0.7%	-0.1%
$W e b \to T e l 2$	(0.5)	(0.3)	(1.0)	(2.9)	(0.4)	(0.3)	(1.5)	(0.3)	(0.4)
$W e b \to T e l 2 +$	0.5%	0.0%	-2.1%	1.5%	0.4%	-0.2%	-2.0%	0.5%	0.1%
$W e b \to T e l 2 +$	(0.4)	(0.2)	(0.8)	(2.0)	(0.4)	(0.2)	(0.8)	(0.2)	(0.3)
$W e b \to F 2 F 3$	0.3%	0.0%	0.4%	1.5%	0.5%	0.2%	0.0%	0.4%	0.3%
$W e b \to F 2 F 3$	(0.2)	(0.1)	(0.3)	(0.6)	(0.2)	(0.1)	(0.3)	(0.1)	(0.1)
$W e b \to F 2 F 3 +$	0.4%	0.0%	0.4%	1.1%	0.5%	0.2%	-0.2%	0.3%	0.3%
$W e b \to F 2 F 3 +$	(0.1)	(0.1)	(0.3)	(0.5)	(0.2)	(0.1)	(0.3)	(0.1)	(0.1)

Table A5
Estimated relative unit costs (in euros) per strategy $s$ and group $g$ with the corresponding standard errors given in brackets
Table summary
This table displays the results of Estimated relative unit costs (in euros) per strategy XXXX and group XXXX with the corresponding standard errors given in brackets . The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$c (s, g)$	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$	$g_{7}$	$g_{8}$	$g_{9}$
$W e b$	0.03	0.04	0.04	0.03	0.04	0.03	0.03	0.03	0.03
$W e b$	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)
$T e l 2$	0.11	0.15	0.10	0.09	0.13	0.11	0.09	0.12	0.14
$T e l 2$	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.0)	(0.1)
$T e l 2 +$	0.13	0.17	0.11	0.10	0.15	0.14	0.11	0.16	0.20
$T e l 2 +$	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.2)
$F 2 F 3$	0.84	0.89	0.83	0.82	0.86	0.84	0.81	0.84	0.89
$F 2 F 3$	(0.4)	(0.5)	(0.5)	(0.8)	(0.3)	(0.2)	(0.5)	(0.2)	(0.5)
$F 2 F 3 +$	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
$F 2 F 3 +$	(0.6)	(0.6)	(0.7)	(1.1)	(0.4)	(0.3)	(0.6)	(0.2)	(0.5)
$W e b \to T e l 2$	0.08	0.11	0.09	0.09	0.09	0.08	0.08	0.07	0.07
$W e b \to T e l 2$	(0.0)	(0.1)	(0.1)	(0.1)	(0.0)	(0.0)	(0.0)	(0.0)	(0.0)
$W e b \to T e l 2 +$	0.09	0.12	0.10	0.10	0.10	0.09	0.09	0.08	0.07
$W e b \to T e l 2 +$	(0.1)	(0.1)	(0.1)	(0.1)	(0.1)	(0.0)	(0.1)	(0.0)	(0.0)
$W e b \to F 2 F 3$	0.60	0.66	0.64	0.70	0.59	0.56	0.65	0.51	0.61
$W e b \to F 2 F 3$	(0.3)	(0.7)	(0.6)	(0.8)	(0.4)	(0.3)	(0.5)	(0.2)	(0.4)
$W e b \to F 2 F 3 +$	0.71	0.71	0.80	0.84	0.73	0.68	0.81	0.62	0.71
$W e b \to F 2 F 3 +$	(0.4)	(0.7)	(0.9)	(1.2)	(0.6)	(0.4)	(0.8)	(0.3)	(0.6)

Appendix B

Overview optimization results

In Section 4.5 we illustrate our approach to solve the multi-mode optimization problem for a range of input parameters. Tables B1 and B2 give a brief overview of the optimization results.

Table B1
Overview optimization results linear programming formulation - minimize costs
Table summary
This table displays the results of Overview optimization results linear programming formulation - minimize costs. The information is grouped by Sample
size (appearing as row headers), Objective
value, Benchmark, Method
effect, Max
difference
in mode
effects and Response
rate (appearing as column headers).
Sample size $(S_{\max})$	Objective value $(\min costs)$	Benchmark	Method effect $({\bar{D}}^{BM})$	Max difference in mode effects $(M)$	Response rate
9,500	123,748.50	${BM}_{1}$	0.16%	2.06%	48.0%
9,500	123,748.50	${BM}_{2}$	0.29%	3.31%	48.0%
11,000	88,408.95	${BM}_{1}$	0.05%	5.97%	39.9%
11,000	88,408.95	${BM}_{2}$	0.19%	2.98%	39.9%
12,500	82,270.72	${BM}_{1}$	0.08%	5.97%	36.9%
12,500	82,270.72	${BM}_{2}$	0.21%	2.98%	36.9%
15,000	74,350.44	${BM}_{1}$	0.12%	5.97%	29.4%
15,000	74,350.44	${BM}_{2}$	0.25%	2.39%	29.4%

Table B2
Overview optimization results nonlinear problem - minimize average method effect in LFS
Table summary
This table displays the results of Overview optimization results nonlinear problem - minimize average method effect in LFS. The information is grouped by XXXX (appearing as row headers), XXXX (appearing as column headers).
$S_{max}$	$B$	$BM$	$M$	${\bar{D}}^{BM}$	$M$	${\bar{D}}^{BM}$	$M$	${\bar{D}}^{BM}$
9,500	160,000	${BM}_{1}$	1%	0.155%	0.5%	Infeasible	0.25%	Infeasible
	160,000	${BM}_{2}$	1%	0.170%	0.5%	Infeasible	0.25%	Infeasible
	170,000	${BM}_{1}$	1%	0.131%	0.5%	Infeasible	0.25%	Infeasible
	170,000	${BM}_{2}$	1%	0.170%	0.5%	Infeasible	0.25%	Infeasible
	180,000	${BM}_{1}$	1%	0.100%	0.5%	Infeasible	0.25%	Infeasible
	180,000	${BM}_{2}$	1%	0.170%	0.5%	Infeasible	0.25%	Infeasible
12,000	160,000	${BM}_{1}$	1%	0.097%	0.5%	0.119%	0.25%	0.123%
	160,000	${BM}_{2}$	1%	0.046%	0.5%	0.046%	0.25%	0.046%
	170,000	${BM}_{1}$	1%	0.076%	0.5%	0.093%	0.25%	0.101%
	170,000	${BM}_{2}$	1%	0.036%	0.5%	0.036%	0.25%	0.036%
	180,000	${BM}_{1}$	1%	0.009%	0.5%	0.058%	0.25%	0.095%
	180,000	${BM}_{2}$	1%	0.014%	0.5%	0.014%	0.25%	0.014%
15,000	160,000	${BM}_{1}$	1%	0.051%	0.5%	0.094%	0.25%	0.112%
	160,000	${BM}_{2}$	1%	0.006%	0.5%	0.006%	0.25%	0.006%
	170,000	${BM}_{1}$	1%	0.020%	0.5%	0.080%	0.25%	0.097%
	170,000	${BM}_{2}$	1%	0.004%	0.5%	0.004%	0.25%	0.004%
	180,000	${BM}_{1}$	1%	0.005%	0.5%	0.058%	0.25%	0.095%
	180,000	${BM}_{2}$	1%	0.000%	0.5%	0.000%	0.25%	0.000%

References

Birgin, E.G., and Martinez, J.M. (2008). Improving ultimate convergence of an augmented lagrangian method. Optimization Methods and Software , 23, 177-195.

Blanke, K., and Luiten, A. (2012). ESSnet project on data collection for social survey using multi modes (dcss). Paper for the UNECE Conference of European Statistics, Oct 31 - Nov 2, Geneva, Switzerland.

Buelens, B., and van den Brakel, J. (2014). Measurement error calibration in mixed-mode surveys. Forthcoming in Sociological Methods and Research.

Calinescu, M., and Schouten, B. (2013a). Adaptive survey designs that account for nonresponse and measurement error. Disscussion paper, Statistics Netherlands.

Calinescu, M., and Schouten, B. (2013b). Adaptive survey designs to minimize mode effects a case study on the dutch labour force survey. Disscussion paper, Statistics Netherlands.

Conn, A.R., Gould, N.I.M. and Toint, P.L. (1991). A globally convergent augmented lagrangian algorithm for optimization with general constraints and simple bounds. SIAM Journal on Numerical Analysis , 28, 545-572.

de Leeuw, E.D. (2005). To mix or not to mix data collection modes in surveys. Journal of Offcial Statistics , 21, 233-255.

Dillman, D., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J. and Messes, B. (2009). Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (ivr) and the internet. Social Science Research , 38, 1-18.

Fan, W., and Yan, Z. (2010). Factors affecting response rates of the web survey: a systematic review. Computers in Human Behavior , 26, 132-139.

Heeringa, S., and Groves, R. (2006). Responsive design for household surveys: tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society: Series A (Statistics in Society) , 169(3).

Jäckle, A., Roberts, C. and Lynn, P. (2010). Assessing the effect of data collection mode on measurement. International Statistical Review , 78, 3-20.

Johnson, S.G. (2013). The nlopt nonlinear-optimization package. Available online at http://ab-initio.mit.edu/nlopt.

Klausch, T., Hox, J. and Schouten, B. (2013a). Assessing the mode-dependency of sample selectivity across the survey response process. Discussion paper, Statistics Netherlands.

Klausch, T., Hox, J. and Schouten, B. (2013b). Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociological Methods & Research , 42(3), 227-263.

Klausch, L., Hox, J. and Schouten, B. (2014). Evaluating bias in sequential mixed-mode surveys against single- and hybrid-mode benchmarks the case of the crime victimization survey. Discussion paper, Statistics Netherlands.

Kreuter, F. (2013). Improving Surveys with Process and Paradata . John Wiley & Sons, Inc.

Mohorko, A., de Leeuw, E.D. and Hox, J. (2013). Internet coverage and coverage bias in Europe: Developments across countries over time. Journal of Offcial Statistics , 29, 609-622.

Powell, M.J.D. (1998). Direct search algorithms for optimization calculations. Acta Numerica , 7, 287-336.

Roy, J.S. (2007). Stochastic optimization - scipy project. Available online at http://js2007.free.fr/.

Schouten, B., Calinescu, M. and Luiten, A. (2013a). Optimizing quality of response through adaptive survey designs. Survey Methodology , 39, 1, 29-58.

Schouten, B., van den Brakel, J., Buelens, B., van der Laan, J., Burger, J. and Klausch, T. (2013b). Disentangling mode-specific selection and measurement bias in social surveys. Social Science Research , 42, 1555-1570.

Suzer-Gurtekin, Z. (2013). Investigating the Bias Properties of Alternative Statistical Inference Methods in Mixed-Mode Surveys . Ph.D. thesis, University of Michigan.

Svanberg, K. (2002). A class of globally convergent optimization methods based on conservative convex separable approximations. SIAM Journal on Optimization , 12, 555-573.

Vannieuwenhuyze, J. (2013). Mixed-Mode Data Collection: Basic Concepts and Analysis of Mode Effects . Ph.D. thesis, Katholieke Universiteit Leuven.

Wagner, J. (2008). Adaptive Survey Design to Reduce Nonresponse Bias . Ph.D. thesis, University of Michigan.

Editorial policy

Survey Methodology publishes articles dealing with various aspects of statistical development relevant to a statistical agency, such as design issues in the context of practical constraints, use of different data sources and collection techniques, total survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves. All papers will be refereed. However, the authors retain full responsibility for the contents of their papers and opinions expressed are not necessarily those of the Editorial Board or of Statistics Canada.

Submission of Manuscripts

Survey Methodology is published twice a year in electronic format. Authors are invited to submit their articles in English or French in electronic form, preferably in Word to the Editor, (statcan.smj-rte.statcan@canada.ca, Statistics Canada, 150 Tunney’s Pasture Driveway, Ottawa, Ontario, Canada, K1A 0T6). For formatting instructions, please see the guidelines provided in the journal and on the web site (www.statcan.gc.ca/SurveyMethodology).

Note of appreciation

Canada owes the success of its statistical system to a long-standing partnership between Statistics Canada, the citizens of Canada, its businesses, governments and other institutions. Accurate and timely statistical information could not be produced without their continued co-operation and goodwill.

Standards of service to the public

Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner. To this end, the Agency has developed standards of service which its employees observe in serving its clients.

Copyright

Published by authority of the Minister responsible for Statistics Canada.

Catalogue no. 12-001-X

Frequency: semi-annual

Ottawa

Date modified:: 2017-09-20

Language selection

Search and menus

Search