The promise and challenge of pushing respondents to the Web in mixed-mode surveys
Section 5. A promising future, but difficult challenges remain
5.1 Reasons for optimism
The development and deployment of web-push methodologies for survey data collection during the past decade provide reasons for optimism that higher quality survey data collection can be accomplished. That optimism arises less from the excitement over a specific approach of contacting people and convincing them to respond on the web than it does from a combination of considerations.
Address-based sampling now provides excellent household coverage and is conducive to the use of respondent selection procedures. Substantial proportions of survey populations can be contacted by one mode (mail) and encouraged to respond by another (web, or telephone). Survey sponsors not known to the recipient of the request can be legitimized through mail contact in ways that cannot be accomplished with email requests that go mostly unread or voice telephone requests that go mostly unanswered.
Postal contact also allows the sending of small token incentives with the request, thus providing motivation for making the transition from letter to computer and entering a URL (Uniform Resource Locator) and password. Multiple mail contacts provide the opportunity to offer more complete explanations of why a survey is being conducted and how the results will be used. Sending a paper questionnaire alternative in a later contact not only increases responses rates significantly, but brings in types of households not represented well among the initial internet responses. Several studies have also shown that the ability of web-push designs to bring in from half to three-fourths of all respondents quickly over the Internet, depending upon the sample frame and modes of contact, can reduce survey costs.
When email addresses are available for sample units, as is now the case for some survey populations, email augmentation (i.e., the sending of a quick email follow-up to the initial postal request to provide an electronic link that makes it easier for the recipient to respond over the internet) has been shown to improve web response considerably. Similarly, when telephone numbers are available, a telephone augmentation can be effective for improving response. The concept of using such contacts to augment previous mail contacts encourages surveyors to think not just about stand-alone contacts, but how each contact becomes part of an overall response strategy.
As shown by the American Community Survey, The Canadian Census, National Science Foundation, and Nexant studies, multiple modes of contact and response provides the potential for achieving response rates that many survey sponsors thought were no longer possible. The ability to approach people with repeated requests to respond and to do so in different modes improves survey response more than any single mode of contact and/or response.
In addition, relying to a great extent on self-administration (internet and mail) achieves a better cultural fit with people than does a voice telephone conversation, which is increasingly out of sync with routine communication behavior that places great emphasis on texting and email. Also, changes in questionnaire construction methods from using different question structures and wording in each mode in the spirit of creating what’s best for each mode through unified mode constructions assists in avoiding measurement differences across survey modes.
Over time, it seems likely that an increasing proportion of adults will be willing and able to respond to surveys over the Internet. Thus, web-push data collection procedures seem consistent with other societal trends that favor the internet over other forms of communications.
The promise of web-push survey methods stems from its ability to reduce survey error from coverage and survey nonresponse. In addition, our greater understanding of how visual vs. aural construction of questionnaires affects answers and the use of unified mode construction methods makes it possible to reduce measurement differences and error. It seems likely that the number of surveys using web-push methods is just beginning.
- Date modified: