Survey methodology

For the Statistics Canada publication, see Survey Methodology.

A field of applied statistics, survey methodology studies the sampling of individual units from a population and the associated survey data collection techniques, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys.

Statistical surveys are undertaken with a view towards making statistical inferences about the population being studied, and this depends strongly on the survey questions used. Polls about public opinion, public health surveys, market research surveys, government surveys and censuses are all examples of quantitative research that use contemporary survey methodology to answer questions about a population. Although censuses do not include a "sample", they do include other aspects of survey methodology, like questionnaires, interviewers, and nonresponse follow-up techniques. Surveys provide important information for all kinds of public information and research fields, e.g., marketing research, psychology, health professionals and sociology.[1]

Overview

A single survey is made of at least a sample (or full population in the case of a census), a method of data collection (e.g., a questionnaire) and individual questions or items that become data that can be analyzed statistically. A single survey may focus on different types of topics such as preferences (e.g., for a presidential candidate), opinions (e.g., should abortion be legal?), behavior (smoking and alcohol use), or factual information (e.g., income), depending on its purpose. Since survey research is almost always based on a sample of the population, the success of the research is dependent on the representativeness of the sample with respect to a target population of interest to the researcher. That target population can range from the general population of a given country to specific groups of people within that country, to a membership list of a professional organization, or list of students enrolled in a school system (see also sampling (statistics) and survey sampling).

Survey methodology as a scientific field seeks to identify principles about the sample design, data collection instruments, statistical adjustment of data, and data processing, and final data analysis that can create systematic and random survey errors. Survey errors are sometimes analyzed in connection with survey cost. Cost constraints are sometimes framed as improving quality within cost constraints, or alternatively, reducing costs for a fixed level of quality. Survey methodology is both a scientific field and a profession, meaning that some professionals in the field focus on survey errors empirically and others design surveys to reduce them. For survey designers, the task involves making a large set of decisions about thousands of individual features of a survey in order to improve it.[2]

Survey methodology topics

The most important methodological challenges of a survey methodologist include making decisions on how to:[2]

Selecting samples

Main article: Survey sampling

Survey samples can be broadly divided into two types: probability samples and non-probability samples. These are discussed in several sources including Salant and Dillman.[3] Stratified sampling is a method of probability sampling such that sub-populations within an overall population are identified and included in the sample selected in a balanced way.

Modes of data collection

There are several ways of administering a survey. The choice between administration modes is influenced by several factors, including

  1. costs,
  2. coverage of the target population,
  3. flexibility of asking questions,
  4. respondents' willingness to participate and
  5. response accuracy.

Different methods create mode effects that change how respondents answer, and different methods have different advantages. The most common modes of administration can be summarized as:[4]

Cross-sectional and longitudinal surveys

There is a distinction between one-time (cross-sectional) surveys, which involve a single questionnaire or interview administered to each sample member, and surveys which repeatedly collect information from the same people over time. The latter are known as longitudinal surveys. Longitudinal surveys have considerable analytical advantages but they are also challenging to implement successfully.

Consequently, specialist methods have been developed to select longitudinal samples, to collect data repeatedly, to keep track of sample members over time, to keep respondents motivated to participate, and to process and analyse longitudinal survey data. [5]

Response formats

Usually, a survey consists of a number of questions that the respondent has to answer in a set format. A distinction is made between open-ended and closed-ended questions. An open-ended question asks the respondent to formulate his or her own answer, whereas a closed-ended question has the respondent pick an answer from a given number of options. The response options for a closed-ended question should be exhaustive and mutually exclusive. Four types of response scales for closed-ended questions are distinguished:

A respondent's answer to an open-ended question can be coded into a response scale afterwards,[4] or analysed using more qualitative methods.

Nonresponse reduction

The following ways have been recommended for reducing nonresponse[6] in telephone and face-to-face surveys:[7]

Brevity is also often cited as increasing response rate. A 1996 literature review found mixed evidence to support this claim for both written and verbal surveys, concluding that other factors may often be more important.[9] A 2010 study looking at 100,000 online surveys found response rate dropped by about 3% at 10 questions and about 6% at 20 questions, with drop-off slowing (for example, only 10% reduction at 40 questions).[10] Other studies showed that quality of response degraded toward the end of long surveys.[11]

Interviewer effects

Survey methodologists have devoted much effort to determining the extent to which interviewee responses are affected by physical characteristics of the interviewer. Main interviewer traits that have been demonstrated to influence survey responses are race,[12] gender,[13] and relative body weight (BMI).[14] These interviewer effects are particularly operant when questions are related to the interviewer trait. Hence, race of interviewer has been shown to affect responses to measures regarding racial attitudes,[15] interviewer sex responses to questions involving gender issues,[16] and interviewer BMI answers to eating and dieting-related questions.[17] While interviewer effects have been investigated mainly for face-to-face surveys, they have also been shown to exist for interview modes with no visual contact, such as telephone surveys and in video-enhanced web surveys. The explanation typically provided for interviewer effects is social desirability bias: survey participants may attempt to project a positive self-image in an effort to conform to the norms they attribute to the interviewer asking questions.

See also

References

  1. "WhatIsASurvey.info". WhatIsASurvey.info. Retrieved 2013-10-03.
  2. 1 2 Groves, R.M.; Fowler, F. J.; Couper, M.P.; Lepkowski, J.M.; Singer, E.; Tourangeau, R. (2009). Survey Methodology. New Jersey: John Wiley & Sons. ISBN 978-1-118-21134-2.
  3. Salant, Priscilla, I. Dillman, and A. Don. How to conduct your own survey. No. 300.723 S3.. 1994.
  4. 1 2 Mellenbergh, G.J. (2008). Chapter 9: Surveys. In H.J. Adèr & G.J. Mellenbergh (Eds.) (with contributions by D.J. Hand), Advising on Research Methods: A consultant's companion (pp. 183–209). Huizen, The Netherlands: Johannes van Kessel Publishing.
  5. Lynn, P. (2009) (Ed.) Methodology of Longitudinal Surveys. Wiley. ISBN 0-470-01871-2
  6. Lynn, P. (2008) "The problem of non-response", chapter 3, 35-55, in International Handbook of Survey Methodology (ed.s Edith de Leeuw, Joop Hox & Don A. Dillman). Erlbaum. ISBN 0-8058-5753-2
  7. Dillman, D.A. (1978) Mail and telephone surveys: The total design method. Wiley. ISBN 0-471-21555-4
  8. De Leeuw, E.D. (2001). "I am not selling anything: Experiments in telephone introductions". Kwantitatieve Methoden, 22, 41–48.
  9. Bogen, Karen (1996). "THE EFFECT OF QUESTIONNAIRE LENGTH ON RESPONSE RATES -- A REVIEW OF THE LITERATURE" (PDF). Proceedings of the Section on Survey Research Methods (American Statistical Association): 1020–1025. Retrieved 2013-03-19.
  10. "Does Adding One More Question Impact Survey Completion Rate?". 2010-12-10. Retrieved 2013-03-19.
  11. "Respondent engagement and survey length: the long and the short of it". research. April 7, 2010. Retrieved 2013-10-03.
  12. Hill, M.E (2002). "Race of the interviewer and perception of skin color: Evidence from the multi-city study of urban inequality". American Sociological Review 67 (1): 99–108. doi:10.2307/3088935. JSTOR 3088935.
  13. Flores-Macias, F.; Lawson, C. (2008). "Effects of interviewer gender on survey responses: Findings from a household survey in Mexico". International Journal of Public Opinion Research 20 (1): 100–110. doi:10.1093/ijpor/edn007.
  14. Eisinga, R.; Te Grotenhuis, M.; Larsen, J.K.; Pelzer, B.; Van Strien, T. (2011). "BMI of interviewer effects". International Journal of Public Opinion Research 23 (4): 530–543. doi:10.1093/ijpor/edr026.
  15. Anderson, B.A.; Silver, B.D.; Abramson, P.R. (1988). "The effects of the race of the interviewer on race-related attitudes of black respondents in SRC/CPS national election studies". Public Opinion Quarterly 52 (3): 1–28. doi:10.1086/269108.
  16. Kane, E.W.; MacAulay, L.J. (1993). "Interviewer gender and gender attitudes". Public Opinion Quarterly 57 (1): 1–28. doi:10.1086/269352.
  17. Eisinga, R.; Te Grotenhuis, M.; Larsen, J.K.; Pelzer, B. (2011). "Interviewer BMI effects on under- and over-reporting of restrained eating. Evidence from a national Dutch face-to-face survey and a postal follow-up". International Journal of Public Health 57 (3): 643–647. doi:10.1007/s00038-011-0323-z. PMC 3359459. PMID 22116390.

Further reading

This article is issued from Wikipedia - version of the Wednesday, April 27, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.