3.3 Collecting
3.3.2 Questionnaire design

Questionnaires play a central role in the data collection process. A well-designed questionnaire efficiently collects the required data with a minimum number of errors. It facilitates the coding and capture of data and it leads to an overall reduction in the cost and time associated with data collection and processing. The biggest challenge in developing a questionnaire is to translate the objectives of the survey into a well conceptualized and methodologically sound study.

Before you can design the questionnaire, you must plan the survey as a whole, including the objectives, data needs and analysis. Once the questionnaire is designed, it must be tested before you can proceed with the data collection.

There is a lot to consider when developing a questionnaire. The following is a list of some key points to think about:

• Is the introduction informative? Does it stimulate respondent interest?
• Are the words simple, direct and familiar to all respondents?
• Do the questions read well? Does the overall questionnaire flow well?
• Are the questions clear and as specific as possible?
• Does the questionnaire begin with easy and interesting questions?
• Is there a specific time reference?
• Are any of the questions double-barrelled?
• Are any questions leading or loaded?
• Should the questions be open-ended or close-ended? If the questions are close-ended, are the response categories mutually exclusive and collectively exhaustive?
• Are the questions applicable to all respondents?

Introduction and conclusion of the questionnaire

The introduction of the questionnaire is very important because it outlines the pertinent information about the survey. The introduction should:

• provide the title or subject of the survey;
• identify the sponsor;
• explain the purpose of the survey;
• request the respondent’s co-operation;
• inform the respondent about confidentiality issues;
• specify whether the survey is (voluntary or mandatory), and
• state whether there are any data-sharing agreements with other organizations

Respondents frequently question the value of the gathered information to themselves and to others. Therefore, be sure to explain why it is important to complete the questionnaire, how the information will be used, and how respondents can access the results. Ensuring that respondents understand the value of their information is vital in undertaking a survey.

The following is an example of a good introduction to a questionnaire.

The opening questions of any survey should establish the respondents’ confidence in their ability to answer the remaining questions. If necessary, the opening questions should help determine whether the respondent is a member of the survey population.

A good questionnaire ends with a comments section that allows the respondent to record any other issues not covered by the questionnaire. This is one way of avoiding any frustration on the part of the respondent, as well as allowing them to express any thoughts, questions or concerns they might have. Lastly, there should be a message at the end thanking the respondents for their time and patience in completing the questionnaire.

Wording of questions

One of the most important factors in any survey is the design of the actual questionnaire. The questions and instructions should be easy to understand and respond to. The way a question is worded is very important as the same question worded in a different manner may achieve different results. Consider the following.

Abbreviations and acronyms

Always spell out the complete form of abbreviations and acronyms.

Complex words and terminology

Avoid specialized terminology and complicated words.

Frame of reference

Give all the details concerning the question’s frame of reference.

Specific questions

A question’s frame of reference is not the only specific detail required. In order to get a uniform response from the entire sample, the question sometimes needs to state the type of response needed.

Double-barreled questions

Each of the above examples asks two questions rather than one:

In the first example, the question asks respondents if they plan to leave their cars at home, and whether or not they are taking the bus for the next year.

The second example asks respondents if their company provides training for new employees as well as providing retraining for existing employees.

In some instances, the answer to each half of the question is the same. However, sometimes there could be two very separate answers, which would make interpreting this question difficult.

The best solution could be to split such questions in two.

Loaded questions

The following examples demonstrate how a loaded question can impact the respondent’s results.

The wording of the first question asks whether the respondents were in favour of Sunday shopping, while the second question was worded to ask respondents whether they were in favour of not working on Sundays. As a result, there was a significant change in the data.

A possible explanation for the difference in the results could be that some respondents did not quite understand the implications of the question. Some people may be opposed to working on Sundays, but are still in favour of shopping. However, if no one works on Sundays, then stores cannot stay open for shoppers!

Open or closed questions

Generally, there are two types of questions: open and closed. Open questions give respondents an opportunity to answer the question in their own words. Closed questions give respondents a choice of answers and the respondent is supposed to select one.

Open question

What is the most important issue facing today’s youth?

Closed question

Which of these is the most important problem facing today’s youth?

• Unemployment
• National unity
• Environment
• Youth violence
• Rising tuition fees
• Drugs in schools
• Need for more computers in schools
• Career counseling

There are advantages and disadvantages to using one type of question versus another. The open question allows the respondent to interpret the question and answer it anyway he or she chooses. The respondent writes the answer or the interviewer records verbatim what the respondent says in answer to the question.

The closed question restricts the respondent to select an answer from the specified response options. For the respondent, a closed question is easier and faster to answer and for the researcher, closed questions are easier and less expensive to code and analyse. Also, closed questions provide consistency, an element that is not necessarily going to occur with an open question.

Questionnaire testing

This is a fundamental step in developing a questionnaire.  Questionnaire testing allows:

• to discover poor wording or ordering of questions,
• to identify errors in the questionnaire layout and instructions,
• to determine problems caused by the respondent’s inability or unwillingness to answer the questions,
• to suggest additional response categories that can be pre-coded on the questionnaire, and
• to provide a preliminary indication of the length of the interview and any refusal problems.

Testing can include the complete questionnaire or only a particular portion of it. The complete questionnaire will at some point in time have to be fully tested.