The promise and challenge of pushing respondents to the Web in mixed-mode surveys
Section 2. Why Web-push data collection is needed
Fundamentally, making contact with households or individuals by one mode, such as mail or telephone, to request that they respond by another mode, is not an ideal data collection procedure. There is bound to be some friction between receiving a postal letter or phone call and then having to go to a different response mode. The switch by itself is likely to take a toll on response rates. Thus, it is not surprising that difficulties in conducting single mode telephone and e-mail/web surveys are the fundamental reason for seeking an alternative.
2.1 The declining effectiveness of telephone surveys
In the mid-20th century, most methodologists considered face-to-face interviews as the only acceptable means for conducting sample surveys (e.g., Parten 1950; Kerlinger 1965). In addition, sampling and surveying households was slow and costly, and therefore limited mostly to conducting national and other large area surveys.
Although telephone surveys had been used occasionally to support data collection (Nathan 2001), development of the telephone as a sole means of collecting survey responses did not occur until the early 1970’s, a process described in detail by Nathan (2001).The first three books on methods for conducting telephone surveys appeared in rapid succession, developing marketing (Blankenship 1977), state and special population (Dillman 1978) and national (Groves and Kahn 1979) population survey perspectives. The use of telephone data collection methods advanced rapidly because of the expanding presence of telephone in households and development of the Mitofsky-Waxberg procedure for using random digit dialing methods of selecting households. In addition the declining costs for long distance calling resulted in RDD voice telephone surveys replacing most in-person interviewing (Dillman 2005).
Between 1997 and 2012 the Pew Research Center (2012), a major conductor of social surveys by telephone in the United States, reported declines in RDD response rates from 35 percent to about 9 percent. More recently, Dutwin and Lavrakas (2016) conducted an analysis of telephone response rates for nine organizations. They found that landline response rates declined from 15.7 percent in 2008 to 9.3 percent in 2015, while cell phone response rates declined during this period from 11.6 to 7.0 percent. They also reported that this roughly 40% decline in response is less the result of an increase in refusals than it is an increase in no answers and answering machines of 10 percentage points for landlines and 24 percentage points for cell phones.
However, these results present only the tip of the iceberg with regard to what is happening to telephone. The telephone has changed from being a household device, or landline, shared by all household members to a wireless individually possessed instrument, easily transportable from place-to-place. In the United States, half of all households and 60% of those with children are now wireless only (Blumberg and Luke 2017). At the same time, the presence of cell and/or landline phones in households has reached an all-time high of at least 95% in most European countries (Mohorko, de Leeuw and Hox 2013) and 97% in U.S. households (Blumberg and Luke 2017). One implication of the increased proportion of wireless phones is that household sampling has become much more difficult. It is possible to include mobile numbers in RDD sample frames. However, it has also become necessary to devote precious interview minutes to ascertaining a range of information including number and type of phones in a household in order to determine household selection probabilities.
In addition, one needs to learn whether the person who answers the phone is an adult, and select an appropriate respondent. Also, the landline “inconvenient time problem” of the respondent being interrupted while, for example, fixing dinner and not having time to talk has been expanded to needing to find out if a person who answers the phone is driving a car or engaged in another task where safety emerges as a serious issue. The inclusion of such items takes away from the ability to ask other questions in phone interviews for which considerable pressure exists to keep the length to only a few minutes. In sum, a major effect of the changes in how telephones are owned, regulated, and used has made its use for important data collection efforts, increasingly difficult.
Landline and cell phones jointly face a larger challenge. Fewer and fewer people engage in voice conversations by telephone. This is a huge change from the time when essential communication for business discussions, maintaining social relationships, and coordinating daily activities in a timely way, were done mostly by voice telephone. Email and texting have largely replaced that use. Talking over the phone with a survey interviewer is increasingly out of sync with other aspects of people’s daily lives.
Answering machines now take most incoming voice calls on both landline and cell phones. Not answering one’s phone is no longer considered rude. Desired calls from children and other close relatives may be assigned a special ringtone to draw the call recipient’s attention. Phone calls from specific numbers can also be blocked, or, on smartphones, swiped away. In addition, both landline and cell telephone numbers are now transportable across type of phone and area codes in the United States and different federal rules apply to automatic dialing of phones.
Yet, another emerging problem with telephones is that the repeated contacts necessary for achieving reasonable response rates for all types of phones are becoming less effective. Increasingly, telephone interviewers have only one chance, lasting just a few seconds, to persuade people to be interviewed. The appearance of the originating telephone number and/or source on caller identification screens makes it increasingly likely that follow-up phone calls can be avoided. Also, the plethora of marketing and fundraising calls has produced an environment in which fewer people are willing to answer the telephone, let-alone be interviewed. An additional challenge associated with cell phones is that their use has brought with them a greater likelihood that requests to be surveyed come when the call recipient is in the midst of business and work activities that are not conducive to people taking time to be interviewed.
The decrease in RDD telephone interviews was slowed down for awhile by research that has shown intensive callbacks to increase response rates does not improve the accuracy of results (Keeter, Miller, Kohut, Groves and Presser 2000) and other research that has suggested that occurrence of non-response error (the differences between respondents and non-respondents) is not closely related to response rates (Groves and Peytcheta 2008). The telephone’s continued use was also encouraged by the great investment that organizations have in telephone hardware, software, and specialized personnel, many of whom had not done other types of survey data collection. However, the continued decline of telephone response rates in recent years noted by Dutwin and Lavrakas (2016) and measurement concerns have reduced the credibility of doing stand-alone telephone surveys.
2.2 The slower than anticipated emergence of email/web only surveys
In the mid 1990’s when telephone response rates were starting their persistent decline, Internet surveying, the expected replacement, was beginning its rapid development (Dillman 2000, Chapter 11). Yet, two decades later, its use for general population surveys remains limited.
Household Internet penetration in the U.S. and many other developed nations now exceeds 85%, which is higher than for telephone when the rapid development of surveys by telephone occurred in the early 1970’s (Nathan 2001). The lack of Internet in some households (e.g., 41% of U.S. adults 65+ and 26% of individuals with only a high school or less education) remains a concern (Anderson and Perrin 2016), but each year sees that becoming less of a problem. Internet use skills are now fundamental to the educational process, to organizational operations, and to accessing consumer services. Yet, the barriers to obtaining web responses for household surveys when using only email contacts remains huge.
There is no household or general population sampling algorithm for email addresses that will provide a known non-zero chance of being selected for survey participation, as calling random numbers has provided for telephone surveys. Email addresses do not exist in standard formats as is the case for our 10 digit telephone numbers that identify an area code, exchange, and the 10,000 number possibilities in each exchange. People within households are also likely to have multiple email addresses so that the probabilities of reaching specific households or other sample units cannot be calculated. In addition, some of the population that are most computer literate young adults have developed a reputation for minimizing their use of traditional email systems. Instead they focus heavily on Facebook, Snapchat, and other instant messaging applications for connecting with friends and acquaintances.
In addition, web response rates for random samples of existing email addresses are likely to be as low or lower than those achieved for today’s telephone surveys (Lozar et al. 2008). And, they are likely to include disproportionately high numbers of individuals who are younger and better educated, despite the fact that many younger people rely on other ways of connecting electronically that make them only occasional users of traditional email. People’s computer inboxes are typically a crowded space, with unsolicited and unwanted emails being more prevalent than the number of unwanted telephone calls once was. In addition, emails are often scanned and deleted based solely upon source or after reading only a few beginning words of the accompanying message.
Changing computer technologies are also contributing to web survey nonresponse. Smartphones that fit in one’s purse, handbag, or pocket, now have far more computer power than desktop computers had when internet surveying began (e.g., Friedman 2016). Their constant presence with people has led to these devices being used as the first responders for scanning and discarding unwanted requests. Some users may defer answering survey requests until they get to a laptop or desktop with a full-scale keyboard. However, for some individuals, smartphones are now the dominant, or even only, device for responding to all emails.
When our dominant survey mode was telephone, interviewers could usually focus the respondent’s attention on survey questions and guide that person through the interview. On desktops, laptops, and now tablets that are used in a person’s office or home, considerable mental concentration by the respondent can often be achieved. In the smartphone era when people are as likely to be on the move from one place to another, concentration on survey content seems somewhat less likely to be achieved. It is evident that the proportion of surveys completed over smartphones is increasing as a proportion of all web completions (Couper, Antoun and Mavletova 2017). However, there appears to be no evidence that the smartphone delivery of web survey requests is increasing total survey response, and may in fact be lowering it. In addition, breakoff rates are much higher for smartphones than desktops and laptops.
Concern about the consequences of attempting to answer an electronic survey is another factor limiting the potential effectiveness of email/Internet surveying. The ease and low cost of sending out massive numbers of email survey requests, has increased the likelihood that people receive requests from organizations that they know nothing about. In addition, considerable fear exists that such requests may be originating from sources that are imitating legitimate sponsors, and attempting to deliver malware, ransomware, and/or collecting data that can be used for other nefarious purposes. Thus, people who are willing and able to respond to legitimate web surveys may be unwilling to take that risk. For many, the internet is a scary place where a “consumer beware” climate prevails.
For all of these reasons, it is hardly surprising that low cost email contact/web response surveys have not become the method of choice for conducting random surveys of the general public needed for public policy purposes. Even if the challenge of drawing probability samples could be solved, multiple issues including computer technologies, the circumstances in which potential respondents encounter survey requests, and mistrust about who is requesting a survey request and how data might be used, are limiting its ability to replace the telephone.
- Date modified: