5. Examples of cost models

David G. Steel and Robert Graham Clark

Previous | Next

The key quantities determining the usefulness of the unit cost data are $C_b$ and $C_c.$ Optimal designs using unequal cost information are not very common, so there is relatively little literature on the typical values of these measures. Unequal costs may be driven by a variety of factors, including mode effects, geography and willingness to respond, and literature on these issues is helpful to give a rough idea of cost models that may apply in practice.

One reason why unequal per-unit costs may arise is the use of mixed mode interviewing. Different respondents may respond using different modes of collection, for example computer-assisted personal or telephone interviewing, mail or web questionnaires, or face to face interviewer (Dillman, Smyth and Christian 2009). This may be done to reduce cost or to improve response rate, however care must be taken that the approach does not introduce bias due to mode effects. Mode effects may consist of selection effects (which are generally not a problem) and measurement effects (which typically lead to bias), and the two are often hard to disentangle (Vannieuwenhuyze, Loosveldt and Molenbergs 2012). Cost savings from the use of mixed modes could potentially be magnified by incorporating mode costs into the sample design as described in this paper. Groves (1989, p. 538) compares per-respondent costs of telephone interviewing ($38.00) and personal interviewing ($84.90) of the general population. If the preference of all units on a frame was known, and half preferred each mode, this would imply $C_c=\mathrm{0.38.}$ Greenlaw and Brown-Welty (2009) compared paper and web surveys, and found per-respondent costs of $4.78 and $0.64, respectively, in a survey of members of a professional association. In a mixed mode option, two thirds of respondents opted for the web option. If preferences are known in advance, then $C_c=\mathrm{0.76.}$

Another reason for varying costs is that some respondents are more difficult to recruit than others, requiring more visits or reminders. Groves and Heeringa (2006, Section 2.2) trialled a survey where interviewers classified non-respondents from the first approach as either likely or unlikely to respond. In subsequent follow-up, the first group had a response rate of 73.7% compared to 38.5% for the second group. This suggests that the per-respondent cost for the second group would be at least 1.9 times higher than the first group. (In fact, the ratio would be higher, because more follow-up attempts would be made for the difficult group.) If 50% of respondents are in both groups, then $C_c=\mathrm{0.31.}$

Geography is another source of differential costs in interviewer surveys. In the Australian Labour Force Survey, costs have been modelled as having a per-block component and a per-dwelling component (Hicks 2001, Table 4.2.1 in Section 4.2) depending on the type of area (15 types were defined). Assuming a constant 10 dwellings sampled per block, the net per-dwelling costs range from $4.98 in Inner City Sydney and Melbourne to $6.71 in Sparse and Indigenous areas. While this is a significant difference in costs across area types, the great majority of the population are in three area types (settled area, outer growth and large town) where per-dwelling costs vary only between $5.71 and $6.07. As a result, $C_c$ is estimated at a very small 0.054.

Table 5.1 shows the approximate percentage improvement in the anticipated variance from using estimated cost information for different values of $C_c$ and $C_b,$ some suggested by these examples. Negative values indicate that the design is less efficient than ignoring costs altogether. The table suggests that cost information is only worthwhile provided there is a fair variation in the unit costs, otherwise the benefit is very small, and can be erased when there is even small imprecision in the estimated costs. Mixed mode surveys have the most potential for exploiting varying unit costs in sample design, but the possibility of measurement bias would need to be carefully assessed in any such approach, using methods such as those in Vannieuwenhuyze, Loosveldt and Molenberghs (2010), Vannieuwenhuyze et al. (2012), Vannieuwenhuyze and Loosveldt (2013) and Schouten, Brakel, Buelens, Laan and Klaus (2013). It might even be possible to incorporate mode effects (or uncertainty about mode effects) into the optimal design via the variance model, and this may be the topic of future research. The findings made in this paper suggest that such an approach is worth considering.

Table 5.1
Percentage improvement in anticipated variance from using estimated cost information compared to no cost information
Table summary
This table displays the results of Percentage improvement in anticipated variance from using estimated cost information compared to no cost information. The information is grouped by Coefficient of Variation of Unit Costs
$\left(C_c\right)$ (%) (appearing as row headers), Possible scenario and Coefficient of Variation of Error Factor
$\left(C_b\right)$ (%) (appearing as column headers).

Coefficient of Variation of Unit Costs $\left(C_c\right)$ (%)	Possible scenario	Coefficient of Variation of Error Factor $\left(C_b\right)$ (%)
		0	10	25	50
5		0.1	-0.2	-1.5	-6.2
10	Interviewer travel due to remoteness	0.2	0.0	-1.3	-6.0
20		1.0	0.7	-0.6	-5.2
30	Response propensity	2.2	2.0	0.7	-3.9
40	Mixed mode (phone/personal int.)	3.8	3.6	2.3	-2.2
50		5.9	5.6	4.4	0.0
75	Mixed mode (paper/web self-complete)	12.3	12.1	11.0	6.8

Previous | Next