5 Conclusion and discussion
Roos Haer and Nadine Meidert
One of the biggest quests of Web survey designers is to get respondents to sign on to the survey site and to keep respondents motivated to complete the survey once they have begun. However, most designers have looked into the lay-out and the design of the Web survey itself, and how this affects breakoff rates. In their quest, they have paid less attention to the important role that the welcome screen plays. This initial screen of a Web survey turns the invitee into a respondent and influences their first impression of the survey. In addition, empirical research has determined that most respondents breakoff after this initial page.
The purpose of this study was to ascertain some of the factors related to the design of the welcome screen for Web surveys affecting response rates in the electronic environment. To examine this influence, we embedded a 2x2x2 design a Web survey. The findings suggest that the lay-out of the welcome screen plays an important role in communicating to the respondent. The longer the expected announced length on the welcome screen and the more emphasis is placed on the explanation of the respondent's privacy rights, the fewer respondents started and completed the Web survey. However, background color did not have a statistical significant influence on the level of breakoff rates on the welcome screen. Only an impact during the Web survey itself was observed but as there is no significant impact on the overall breakoff rate this design feature is not regarded as relevant. Overall, based on these results we can state some more practical implications, which may help to improve the Web survey practice: (1) Keep the Web survey as short as possible. (2) Use elaborate pretests to determine reliable information concerning the time necessary to complete the survey. (3) Providing privacy rights is an important element of the welcome screen, but most respondents prefer a short description of these rights and do not want to spend too much time reading them. An appropriate way of fulfilling these wishes is providing respondents a link to a more detailed description of these rights.
One important limitation of this study is that it was conducted on a sample of a very specific population – first-year university students. It is highly likely that the topic of the Web survey, i.e. the choice of study, is highly salient among students compared to other survey topics and this might increase the general level of response. Web survey research on populations other than students should then also determine whether the presented results are robust. In addition, it would be interesting to determine the precise effect of the emphasis on privacy rights on response rates: is it determined by the amount of words devoted to this topic, or is the emphasis given on possible problems relating to privacy rights. The specific mechanisms of the observed effect remain unclear and further research is, therefore, necessary.
Acknowledgements
We thank Christine Abele, Valentin Gold, Katharina Holzinger, and Elena Sewelies for their helpful suggestions, comments, and their (data gathering) support, and collaboration.
Appendix A
Description for figure Appendix A
Appendix B
Dear Mrs Example,
You are now in your first or second semester of your course of studies at the University of Konstanz. In these first few weeks you have gotten to know your department, the university, and the city. We would like to know about your experience with the university advisory services, the support received from the lecturers and your overall expectations. Therefore, we would like to invite you to participate in our First Year Students Survey. Filling in this survey will help us to improve the study conditions of the University of Konstanz.
Please click on the following link to the survey (English version):
http://personalizedlink
If you cannot enter the survey via the link, please copy and paste the address in your Web browser.
The participation in this survey is voluntary. The survey is subjected to the data protection regulations and the information you provide will only be used by the Quality Management Department of the University of Konstanz and for scientific purposes. More information on data protection can be found on the welcome screen of the survey.
For questions and comments, please e-mail our survey team: studieneingangsbefragung@uni-konstanz.de.
Thank you for your participation and we wish you all the best with your course of studies!
Kind regards
References
American Association for Public Opinion Research (AAPOR) (2011). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Retrieved February, 27, 2013, from http://aapor.org/Content/NavigationMenu/AboutAAPOR/StandardsampEthics/StandardDefinitions/StandardDefinitions2011.pdf.
Baker-Prewitt, J. (2003). All Web Surveys are not Created Equal: Your Design Choices Can Impact Results. Paper presented at the SumIT03 Global Market Research Symposium, Montreal.
Bandilla, W., Couper, M.P. and Kaczmirekt, L. (2012). The mode of invitation for web surveys. Survey Practice, 5.
Bosnjak, M., and Tute, T.L. (2001). Classifying response behavior in web surveys. Journal of Computer Medicated Communication, 6. Retrieved November, 21, 2012, from http://jcmc.indiana.edu/vol6/issue3/boznjak.html.
Bradburn, N.M. (1978). Respondent Burden. Proceedings of the Section of Survey Research Methods, American Statistical Association.
Brewer, P.B., Graf, J. and Willnat, L. (2003). Priming or Framing. Media Influence on Attitudes toward Foreign Countries. Gazette: The International Journal for Communication Studies, 65, 493-508.
Cook, C., Heath, F. and Thompson, R.L. (2000). A meta-analysis of response rates in web- or Internet-based surveys. Educational and Psychological Measurement, 60, 821-836.
Couper, M.P. (2000). Web surveys. A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.
Couper, M.P. (2008). Designing Effective Web Surveys. Cambridge: Cambridge University Press.
Couper, M.P., Traugott, M.W. and Lamais, M.J. (2001). Web survey design and administration. Public Opinion Quarterly, 65, 230-253.
Couper, M.P., and Miller, P.V. (2008). Web survey methods. Introduction. Public Opinion Quarterly, 72, 831-835.
Crawford, S.D., Couper, M.P. and Lamias, M.J. (2001). Web surveys. Percepsintions of Burden. Social Science Computer Review, 19, 146-62.
Dillman, D.A., Conradt, J. and Bowker, D. (1998). Influence of Plain vs. Fancy Design on Response Rates for Web Surveys. Paper presented at annual meeting of the American Statistical Association, Dallas, TX.
Dillman, D.A., Gertseva, A. and Mahon-Haft, T. (2005). Achieving usability in establishment survey through the application of visual design principles. Journal of Official Statistics, 21, 183-214.
Dillman, D.A. (2007). Mail and Internet Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley & Sons, Inc.
Deutskens, E., De Ruyter, K., Wetzels, M. and Oosterveld, P. (2004). Response rate and response quality of Internet-based surveys: An experimental study. Marketing Letters, 15, 21-36.
Edwards, P.J., Roberts, I., Clarke, M.J., Diguiseppi, C., Wentz, R., Kwan, I., Cooper, R., Felix, L.M. and Pratap, S. (2009). Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews, 8, 3. Retrieved November, 21, 2012, from http://www.ncbi.nlm.nih.gov/pubmed/19588449.
Etter, J., Cucherat, M. and Perneger, T.V. (2002). Questionnaire color and response rates to mailed surveys: A randomized trial and a meta-analysis. Evaluation & The Health Professions, 25, 185-99.
Faubert, J. (1994). Seeing depth in colour: More than just what meets the eyes. Vision Research, 34, 1165-1186.
Fay, R.E., Bates, N. and Moore, J. (1991). Lower mail response in the 1990 Census: A preliminary interpretation. In Proceedings of the Annual Research Conference of the U.S. Census Bureau, 3-32. Washington DC: Census Bureau. Retrieved November, 27, 2012 from https://www.census.gov/srd/papers/pdf/rsm2010-13.pdf.
Galesic, M. (2006). Dropouts on the web: Effects of interest and Burden experienced during an online survey. Journal of Official Statistics, 22, 313-328.
Galesic, M., and Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73, 349-360.
Ganassali, S. (2008). The influence of the design of web survey questionnaires on the quality of responses. Survey Research Methods, 2, 21-32.
Gorn, G.J., Chattopadhyay, A., Yi, T. and Dahl, D.W. (1997). Effects of color as an executional cue in advertising: They're in the shade. Management Science, 43, 1387-1400.
Groves, R.M., Fowler Jr., F.J., Couper, M.P., Lepkowski, J.M., Singer, E. and Tourangeau, R. (2004). Survey Methodology. Hoboken: John Wiley & Sons, Inc.
Hall, R.H., and Hanna, P. (2004). The impact of web page text-background color combinations on readability, retention, aesthetics, and behavioral intention. Behaviour & Information Technology, 23, 183-195.
Heerwegh, D. (2004). Using Progress Indicators in Web Surveys. Paper prepared for the 59th AAPOR conference (Phoenix, Arizona May 13-16 2004). Retrieved November, 27, 2012, from https://perswww.kuleuven.be/~u0034437/public/Files/Heerwegh%20Using%20Progress%20Indicators.pdf.
Hillygus, S., Nie, N., Prewitt, K., and Pals, G. (2006). Civic Mobilization and Privacy Concerns in the 2000 Census. New York: Russell Sage Foundation.
Hogg, A., and Miller, J. (2003). Watch out for Dropouts. Retrieved April, 18, 2011, from http://www.quirks.com.
Lozar Manfreda, K., Bosnjak, M., Berzelak, J., Haas, I. and Vehovar, V. (2008). Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research, 50, 79-104.
Mahon-Haft, T.A., and Dillman, D.A. (2010). Does visual appeal matter? Effects of web survey aesthetics on survey quality. Survey Research Methods, 4, 43-59.
Marcus, B, Bosnjak, M., Linder, S., Pilischenko, S. and Schütz, A. (2007). Compensating for low topic interest and long surveys. A field experiment on nonresponse in web surveys. Social Science Computer Review, 25, 372-383.
Peytchev, A. (2009). Survey breakoff. Public Opinion Quarterly, 73, 74-97.
Pope, D., and Baker, R.P. (2005). Experiments in Color for Web-Based Surveys. Paper presented at the FedCASIC Workshops, Washington, D.C.
Redline, C., and Dillman, D.A. (2002). The influence of alternative visual designs of respondent's performance with branching instructions in self-administered questionnaire. In Survey Response, (Eds., R. Groves, D.A. Dillman, E. Eltinge and R. Little), 179-196. New York: John Wiley & Sons, Inc.
Singer, E., Hippler, H.J. and Schwarz, N. (1992). Confidentiality assurances in surveys: Reassurance or threat? International Journal of Public Opinion Research, 4, 256-268.
Singer, E., Mathiowetz, N. and Couper, M.P. (1993). The role of privacy and confidentiality as factors in response to the 1990 census. Public Opinion Quarterly, 57, 465-82.
Singer, E., Von Thurn, D.R. and Miller, E.R. (1995). Confidentiality assurances and response: A quantitative review of the experimental literature. Public Opinion Quarterly, 59, 66-77.
Singer, E., Van Hoewyk, J. and Neugebauer, R.J. (2003). Attitudes and Behavior - the impact of privacy and confidentiality concerns on participation in the 2000 census. Public Opinion Quarterly, 67, 368-384.
Terhanian, G. (2005). How to Produce Credible, Trustworthy Information through Internet-Based Survey Research. Paper presented at the annual conference of the American Association for Public Opinion Research, Portland, OR.
Tourangeau, R., Couper, M.P. and Conrad, F. (2007). Color, labels, and interpretive heuristics for response scales. Public Opinion Quarterly, 71, 91-112.
Toepoel, V., Das, M. and Van Soest, A. (2008). Effects of design in web surveys. Comparing trained and fresh respondents. Public Opinion Quarterly, 72, 985-1007.
Vicente, P., and Reis, E. (2010). Using questionnaire design to fight nonresponse bias in web surveys. Social Science Computer Review, 28, 251-267.
Weller, L., and Livingston, R. (1988). Effect of color of questionnaire on emotional responses. The Journal of General Psychology, 115, 433-440.
Yan, T., Conrad, F.G., Tourangeau, R. and Couper, M.P. (2010). Should I stay or should I go: The effects of progress feedback, promised task duration, and length of questionnaire on completing web surveys. International Journal of Public Opinion Research 23, 131-147.
- Date modified: