Questionnaire

A paper form, yet to be filled in by the respondent.

A questionnaire is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents. Although they are often designed for statistical analysis of the responses, this is not always the case. The questionnaire was invented by the Statistical Society of London in 1838.^[1] A copy of the instrument is published in the Journal of the Statistical Society, Volume 1, Issue 1, 1838, pages 5–13.^[2]

Questionnaires have advantages over some other types of surveys in that they are cheap, do not require as much effort from the questioner as verbal or telephone surveys, and often have standardized answers that make it simple to compile data. However, such standardized answers may frustrate users. Questionnaires are also sharply limited by the fact that respondents must be able to read the questions and respond to them. Thus, for some demographic groups conducting a survey by questionnaire may not be concrete.

As a type of survey, questionnaires also have many of the same problems relating to question construction and wording that exist in other types of opinion polls.

Types

A distinction can be made between questionnaires with questions that measure separate variables, and questionnaires with questions that are aggregated into either a scale or index.^[3] Questionnaires within the former category are commonly part of surveys, whereas questionnaires in the latter category are commonly part of tests.

Questionnaires with questions that measure separate variables, could for instance include questions on:

• preferences (e.g. political party)
• behaviors (e.g. food consumption)
• facts (e.g. gender)

Questionnaires with questions that are aggregated into either a scale or index, include for instance questions that measure:

• latent traits (e.g. personality traits such as extroversion)
• attitudes (e.g. towards immigration)
• an index (e.g. Social Economic Status)

Examples

• A food frequency questionnaire (FFQ) is a questionnaire the type of diet consumed in people, and may be used as a research instrument. Examples of usages include assessment of intake of vitamins or toxins such as acrylamide.^[4][5]

Questionnaire construction

Question types

Usually, a questionnaire consists of a number of questions that the respondent has to answer in a set format. A distinction is made between open-ended and closed-ended questions. An open-ended question asks the respondent to formulate his own answer, whereas a closed-ended question has the respondent pick an answer from a given number of options. The response options for a closed-ended question should be exhaustive and mutually exclusive. Four types of response scales for closed-ended questions are distinguished:

• Dichotomous, where the respondent has two options
• Nominal-polytomous, where the respondent has more than two unordered options
• Ordinal-polytomous, where the respondent has more than two ordered options
• (Bounded)Continuous, where the respondent is presented with a continuous scale

A respondent's answer to an open-ended question is coded into a response scale afterwards. An example of an open-ended question is a question where the testie has to complete a sentence (sentence completion item).^[3]

Question sequence

In general, questions should flow logically from one to the next. To achieve the best response rates, questions should flow from the least sensitive to the most sensitive, from the factual and behavioural to the attitudinal, and from the more general to the more specific.

There typically is a flow that should be followed when constructing a questionnaire in regards to the order that the questions are asked. The order is as follows:

1. Screens
2. Warm-ups
3. Transitions
4. Skips
5. Difficult
6. Changing Formula

Screens are used as a screening method to find out early whether or not someone should complete the questionnaire. Warm-ups are simple to answer, help capture interest in the survey, and may not even pertain to research objectives. Transition questions are used to make different areas flow well together. Skips include questions similar to "If yes, then answer question 3. If no, then continue to question 5." Difficult questions are towards the end because the respondent is in "response mode." Also, when completing an online questionnaire, the progress bars lets the respondent know that they are almost done so they are more willing to answer more difficult questions. Classification, or demographic question should be at the end because typically they can feel like personal questions which will make respondents uncomfortable and not willing to finish survey.^[6]

Basic rules for questionnaire item construction

• Use statements which are interpreted in the same way by members of different subpopulations of the population of interest.
• Use statements where persons that have different opinions or traits will give different answers.
• Think of having an "open" answer category after a list of possible answers.
• Use only one aspect of the construct you are interested in per item.
• Use positive statements and avoid negatives or double negatives.
• Do not make assumptions about the respondent.
• Use clear and comprehensible wording, easily understandable for all educational levels
• Use correct spelling, grammar and punctuation.
• Avoid items that contain more than one question per item (e.g. Do you like strawberries and potatoes?).
• Question should not be biased or even leading the participant towards an answer.

Questionnaire administration modes

Main modes of questionnaire administration are:^[3]

• Face-to-face questionnaire administration, where an interviewer presents the items orally.
• Paper-and-pencil questionnaire administration, where the items are presented on paper.
• Computerized questionnaire administration, where the items are presented on the computer.
• Adaptive computerized questionnaire administration, where a selection of items is presented on the computer, and based on the answers on those items, the computer selects following items optimized for the testee's estimated ability or trait.

Concerns with questionnaires

While questionnaires are inexpensive, quick, and easy to analyze, often the questionnaire can have more problems than benefits. For example, unlike interviews, the people conducting the research may never know if the respondent understood the question that was being asked. Also, because the questions are so specific to what the researchers are asking, the information gained can be minimal.^[7] Often, questionnaires such as the Myers-Briggs Type Indicator, give too few options to answer; respondents can answer either option but must choose only one response. Questionnaires also produce very low return rates, whether they are mail or online questionnaires. The other problem associated with return rates is that often the people that do return the questionnaire are those that have a really positive or a really negative viewpoint and want their opinion heard. The people that are most likely unbiased either way typically don't respond because it is not worth their time.

Some questionnaires have questions addressing the participants gender. Seeing someone as male or female is something we all do unconsciously, we don’t give much important to one’s sex or gender as most people use the terms ‘sex’ and ‘gender’ interchangeably, unaware that they are not synonyms.^[8] Gender is a term to exemplify the attributes that a society or culture constitutes as masculine or feminine. Although your sex as male or female stands at a biological fact that is identical in any culture, what that specific sex means in reference to your gender role as a ‘woman’ or ‘man’ in society varies cross culturally according to what things are considered to be masculine or feminine. The survey question should really be what is your sex. Sex is traditionally split into two categories, which we typically don’t have control over, you were either born a girl or born a boy and that’s decided by nature.^[9] There's also the intersex population which is disregarded in the North American society as a sex. Not many questionnaires have a box for people that fall under Intersex.^[8] These are some small things that can be misinterpreted or ignored in questionnaires.

Further, if the questionnaires are not collected using sound sampling techniques, often the results can be non-representative of the population—as such a good sample is critical to getting representative results based on questionnaires.^[10]

See also

• Behavioral Risk Factor Surveillance System
• Computer-assisted personal interviewing
• Enterprise Feedback Management
• Quantitative marketing research
• Questionnaire construction
• Statistical survey
• Structured interviewing
• Web-based experiments
• Position analysis questionnaire

Further reading

• Foddy, W. H. (1994). Constructing questions for interviews and questionnaires: Theory and practice in social research (New ed.). Cambridge, UK: Cambridge University Press.
• Gillham, B. (2008). Developing a questionnaire (2nd ed.). London, UK: Continuum International Publishing Group Ltd.
• Leung, W. C. (2001). How to conduct a survey. StudentBMJ, 9, 143-5.
• Mellenbergh, G. J. (2008). Chapter 10: Tests and questionnaires: Construction and administration. In H. J. Adèr & G. J. Mellenbergh (Eds.) (with contributions by D. J. Hand), Advising on research methods: A consultant's companion (pp. 211–234). Huizen, The Netherlands: Johannes van Kessel Publishing.
• Mellenbergh, G. J. (2008). Chapter 11: Tests and questionnaires: Analysis. In H. J. Adèr & G. J. Mellenbergh (Eds.) (with contributions by D. J. Hand), Advising on research methods: A consultant's companion (pp. 235–268). Huizen, The Netherlands: Johannes van Kessel Publishing.
• Munn, P., & Drever, E. (2004). Using questionnaires in small-scale research: A beginner's guide. Glasgow, Scotland: Scottish Council for Research in Education.
• Oppenheim, A. N. (2000). Questionnaire design, interviewing and attitude measurement (New ed.). London, UK: Continuum International Publishing Group Ltd.

References

1. Gault, RH (1907). "A history of the questionnaire method of research in psychology". Research in Psychology 14 (3): 366–383. doi:10.1080/08919402.1907.10532551.
2. "Fourth Annual Report of the Council of the Statistical Society of London".
3. Mellenbergh, G.J. (2008). Chapter 10: Tests and Questionnaires: Construction and administration. In H.J. Adèr & G.J. Mellenbergh (Eds.) (with contributions by D.J. Hand), Advising on Research Methods: A consultant's companion (pp. 211--236). Huizen, The Netherlands: Johannes van Kessel Publishing.
4. Smedts HP, de Vries JH, Rakhshandehroo M, et al. (February 2009). "High maternal vitamin E intake by diet or supplements is associated with congenital heart defects in the offspring". BJOG 116 (3): 416–23. doi:10.1111/j.1471-0528.2008.01957.x. PMID 19187374.
5. "A Prospective Study of Dietary Acrylamide Intake and the Risk of Endometrial, Ovarian, and Breast Cancer". Cebp.aacrjournals.org. Retrieved 2013-02-18.
6. Burns, A. C., & Bush, R. F. (2010). Marketing Research. Upper Saddle River, NJ: Pearson Education.
7. Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological testing: Principles, applications, and issues. Belmont, CA: Wadsworth
8. Fausto-Sterling, Anne “Of Gender and Genitals” from Sexing the body: gender politics and the construction of sexuality New York, Basic Books, 2000, [Chapter 3, pp. 44-77]
9. Birke, Lynda. "Chapter 24, In Pursuit of Difference." The Gender and Science Reader. New York: Routledge, 2001. 309-22. Print.
10. Moser, Claus Adolf, and Graham Kalton. "Survey methods in social investigation." Survey methods in social investigation. 2nd Edition (1971).

External links

• Harmonised questions from the UK Office for National Statistics
• Hints for designing effective questionnaires - from the ERIC Clearinghouse on Assessment and Evaluation