Reporting bias

Not to be confused with Publication bias or Media bias.

In epidemiology, reporting bias is defined as "selective revealing or suppression of information" by subjects (for example about past medical history, smoking, sexual experiences).[1]

By extension, in empirical research in general, the term reporting bias may be used to refer to a tendency to under-report unexpected or undesirable experimental results, attributing the results to sampling or measurement error, while being more trusting of expected or desirable results, though these may be subject to the same sources of error. In this context, reporting bias can eventually lead to a status quo where multiple investigators discover and discard the same results, and later experimenters justify their own reporting bias by observing that previous experimenters reported different results. Thus, each incident of reporting bias can make future incidents more likely. [2][3] Sociologist Christopher B. Doob refers to this practice as selective reporting in explaining the Power of the Press and defines it as biased coverage of news issues that promotes corporate interests and downplays, denigrates, or ignores issues and groups challenging these issues.[4]

Reporting biases in research

Research can only contribute to knowledge if it is communicated from investigators to the community. The generally accepted primary means of communication is “full” publication of the study methods and results in an article published in a scientific journal. Sometimes, investigators choose to present their findings at a scientific meeting as well, either through an oral or poster presentation. These presentations are included as part of the scientific record as brief “abstracts” which may or may not be recorded in publicly accessible documents typically found in libraries or the World Wide Web.

Sometimes, investigators fail to publish the results of entire studies. The Declaration of Helsinki and other consensus documents have outlined the ethical obligation to make results from clinical research publicly available.

Reporting bias occurs when the dissemination of research findings is influenced by the nature and direction of the results.[5] Positive results is a commonly used term to describe a study finding that one intervention is better than another.

Various attempts have been made to overcome the effects of the reporting biases, including statistical adjustments to the results of published studies.[6] None of these approaches has proved satisfactory, however, and there is increasing acceptance that reporting biases must be tackled by establishing registers of controlled trials and by promoting good publication practice. Until these problems have been addressed, estimates of the effects of treatments based on published evidence may be biased.

Case study

Litigation brought upon by consumers and health insurers against Pfizer for the fraudulent sales practices in marketing of the drug gabapentin in 2004 revealed a comprehensive publication strategy that employed elements of reporting bias.[7] Spin was used to put emphasis on favorable findings that favored gabapentin, and also to explain away unfavorable findings towards the drug. In this case, favorable secondary outcomes became the focus over the original primary outcome, which was unfavorable. Other changes found in outcome reporting include the introduction of a new primary outcome, failure to distinguish between primary and secondary outcomes, and failure to report one or more protocol-defined primary outcomes.[8]

The decision to publish certain findings in certain journals is another strategy.[7] Trials with statistically significant findings were generally published in academic journals with higher circulation more often than trials with nonsignificant findings. Timing of publication results of trials was influenced, in that the company tried to optimize the timing between the release of two studies. Trials with nonsignificant findings were found to be published in a staggered fashion, as to not have two consecutive trials published without salient findings. Ghost authorship was also an issue, where professional medical writers who drafted the published reports were not properly acknowledged.

Fallout from this case is still being settled by Pfizer in 2014, 10 years after the initial litigation.[9]

Types of reporting bias

Publication bias

Main article: Publication bias

The publication or nonpublication of research findings, depending on the nature and direction of the results. Although medical writers have acknowledged the problem of reporting biases for over a century,[10] it was not until the second half of the 20th century that researchers began to investigate the sources and size of the problem of reporting biases.[11]

Over the past two decades, evidence has accumulated that failure to publish research studies, including clinical trials testing intervention effectiveness, is pervasive.[11] Almost all failure to publish is due to failure of the investigator to submit;[12] only a small proportion of studies are not published because of rejection by journals.[13]

The most direct evidence of publication bias in the medical field comes from follow-up studies of research projects identified at the time of funding or ethics approval.[14] These studies have shown that “positive findings” is the principal factor associated with subsequent publication: researchers say that the reason they don’t write up and submit reports of their research for publication is usually because they are “not interested” in the results (editorial rejection by journals is a rare cause of failure to publish).

Even those investigators who have initially published their results as conference abstracts are less likely to publish their findings in full unless the results are “significant”.[15] This is a problem because data presented in abstracts are frequently preliminary or interim results and thus may not be reliable representations of what was found once all data were collected and analyzed.[16] In addition, abstracts are often not accessible to the public through journals, MEDLINE, or easily accessed databases. Many are published in conference programs, conference proceedings, or on CD-ROM, and are made available only to meeting registrants.

The main factor associated with failure to publish is negative or null findings.[17] Controlled trials that are eventually reported in full are published more rapidly if their results are positive.[16] Publication bias leads to overestimates of treatment effect in meta-analyses, which in turn can lead doctors and decision makers to believe a treatment is more useful than it is.

It is now well-established that publication bias is associated with the source of funding for the study.[18]

Time lag bias

The rapid or delayed publication of research findings, depending on the nature and direction of the results. In a systematic review of the literature, Hopewell and her colleagues found that overall, trials with “positive results” (statistically significant in favor of the experimental arm) were published about a year sooner than trials with “null or negative results” (not statistically significant or statistically significant in favor of the control arm).[16]

Multiple (duplicate) publication bias

The multiple or singular publication of research findings, depending on the nature and direction of the results. Investigators may also publish the same findings multiple times using a variety of patterns of “duplicate” publication.[19] Many duplicates are published in journal supplements, potentially difficult to access literature. Positive results appear to be published more often in duplicate, which can lead to overestimates of a treatment effect.

Location bias

The publication of research findings in journals with different ease of access or levels of indexing in standard databases, depending on the nature and direction of results. There is also evidence that, compared to negative or null results, statistically significant results are on average published in journals with greater impact factors,[20] and that publication in the mainstream (non grey) literature is associated with an overall greater treatment effect compared to the grey literature.[21]

Citation bias

The citation or non-citation of research findings, depending on the nature and direction of the results. Authors tend to cite positive results over negative or null results, and this has been established over a broad cross section of topics.[22][23][24][25][26][27] Differential citation may lead to a perception in the community that an intervention is effective when it is not, and it may lead to over-representation of positive findings in systematic reviews if those left uncited are difficult to locate.

Selective pooling of results in a meta-analysis is a form of citation bias that is particularly insidious in its potential to influence knowledge. To minimize bias, pooling of results from similar but separate studies requires an exhaustive search for all relevant studies. That is, a meta-analysis (or pooling of data from multiple studies) must always have emerged from a systematic review (not a selective review of the literature), even though a systematic review does not always have an associated meta-analysis.

Language bias

The publication of research findings in a particular language, depending on the nature and direction of the results. There is longstanding question about whether there is a language bias such that investigators choose to publish their negative findings in non-English language journals and reserve their positive findings for English language journals. Some research has shown that language restrictions in systematic reviews can change the results of the review[28] and in other cases, authors have not found that such a bias exists.[29]

Outcome reporting bias

The selective reporting of some outcomes but not others, depending on the nature and direction of the results.[30] A study may be published in full, but pre-specified outcomes omitted or misrepresented.[8][31] Efficacy outcomes that are statistically significant have a higher chance of being fully published compared to those that are not statistically significant.

Selective reporting of suspected or confirmed adverse treatment effects is an area for particular concern because of the potential for patient harm. In a study of adverse drug events submitted to Scandinavian drug licensing authorities, reports for published studies were less likely than unpublished studies to record adverse events (for example, 56 vs 77% respectively for Finnish trials involving psychotropic drugs).[32] Recent attention in the lay and scientific media on failure to accurately report adverse events for drugs (e.g., selective serotonin uptake inhibitors, rosiglitazone, rofecoxib) has resulted in additional publications, too numerous to review, indicating substantial selective outcome reporting (mainly suppression) of known or suspected adverse events.

See also

References

  1. ↑ Porta, Miquel, ed. (5 June 2008). A Dictionary of Epidemiology. Oxford University Press. p. 275. ISBN 978-0-19-157844-1. Retrieved 27 March 2013.
  2. ↑ Green S, Higgins S, editors: Glossary. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5.
  3. ↑ McGauran, N; Wieseler, B; Kreis, J; Schüler, YB; Kölsch, H; Kaiser, T (2010). "Reporting bias in medical research - a narrative review" (PDF). Trials 11: 37. doi:10.1186/1745-6215-11-37.
  4. ↑ Doob, C. B. (2013). Social inequality and social stratification in US society. Upper Saddle River, NJ: Pearson.
  5. ↑ Higgins, JPT; Green, S (2008). "Cochrane Handbook for Systematic Review of Interventions". Retrieved 2 January 2015.
  6. ↑ Rosenthal, R (1979). "The file drawer problem and tolerance for null results". Psychological Bulletin 86 (3): 638–641. doi:10.1037/0033-2909.86.3.638.
  7. 1 2 Vedula, SS; Goldman, PS; Rona, IJ; Greene, TM; Dickersin, K (2012). "Implementation of a publication strategy in the context of reporting biases. A case study based on new documents from Neurontin litigation". Trials 13 (136). doi:10.1186/1745-6215-13-136. PMID 22888801.
  8. 1 2 Vedula, SS; Bero, L; Scherer, RW; Dickersin, K (2009). "Outcome reporting in industry-sponsored trials for gabapentin for off-label use". N Engl J Med 361 (120): 1963–1971. doi:10.1056/NEJMsa0906126. PMID 19907043.
  9. ↑ Stempel, Jonathan (2 June 2014). "Pfizer to pay $325 million in Neurontin settlement". Reuters. Retrieved 24 August 2014.
  10. ↑ Editorial (1909). "The reporting of unsuccessful cases". Boston Medical and Surgical Journal 161: 263–264. doi:10.1056/nejm190908191610809.
  11. 1 2 Dickersin, K. (2005). "Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm". In Rothstein, H.R.; Sutton, A.J.; Borenstein, M. Publication bias in meta-analysis: prevention, assessment, and adjustments. London: Wiley. pp. 11–13. ISBN 0470870141.
  12. ↑ Godlee, F.; Dickersin, K. (2003). "Bias, subjectivity, chance, and conflict of interest in editorial decisions". In Godlee, F.; Jefferson, T. Peer review in health sciences (2nd ed.). London: BMJ Books. ISBN 978-0727916853.
  13. ↑ Olson, CM; Rennie, D; Cook, D; Dickersin, K; Flanagin, A; Hogan, JW; Zhu, Q; Reiling, J; Pace, B (2002). "Publication bias in editorial decision making". JAMA 287 (21): 2825–2828. doi:10.1001/jama.287.21.2825. PMID 12038924.
  14. ↑ Song, F; Parekh, S; Hooper, L; Loke, YK; Ryder, J; Sutton, AJ; Hing, C; Kwok, CS; Pang, C; Harvey, I (2010). "Dissemination and publication of research findings: an updated review of related biases". Health Technol Assess 14 (8): iii, ix–xi. doi:10.3310/hta14080. PMID 20181324.
  15. ↑ Scherer, RW; Langenberg, P; von Elm, E (2007). "Full publication of results initially presented in abstracts". Cochrane Database Syst Rev 2: MR000005. doi:10.1002/14651858.MR000005.pub3. PMID 17443628.
  16. 1 2 3 Hopewell, S; Clarke, MJ; Stewart, L; Tierney, J (2007). "Time to publication for results of clinical trials". Cochrane Database Syst Rev 2: MR000011. doi:10.1002/14651858.MR000011.pub2. PMID 17443632.
  17. ↑ Hopewell, S; Loudon, K; Clarke, MJ; Oxman, AD; Dickersin, K (2009). "Publication bias in clinical trials due to statistical significance or direction of trial results". Cochrane Database Syst Rev 1: MR000006. doi:10.1002/14651858.MR000006.pub3. PMID 19160345.
  18. ↑ Lundh, A; Sismondo, S; Lexchin, J; Busuioc, OA; Bero, L (2012). "Industry sponsorship and research outcome". Cochrane Database Syst Rev 12: M R000033. doi:10.1002/14651858.MR000033.pub2. PMID 23235689.
  19. ↑ Von Elm, M; Poglia, G; Walder, B; Tramer, MR (2004). "Different patterns of duplicate publication. An analysis of articles used in systematic reviews". JAMA 291 (8): 974–980. doi:10.1001/jama.291.8.974. PMID 14982913.
  20. ↑ Easterbrook, PJ; Berlin, JA; Gopalan, R; Matthews, DR (1991). "Publication bias in clinical research". Lancet 337 (8746): 867–872. doi:10.1016/0140-6736(91)90201-y.
  21. ↑ Hopewell, S; McDonald, S; Clarke, MJ; Egger, M (2007). "Grey literature in meta-analyses of randomized trials of health care interventions". Cochrane Database Syst Rev 2: MR000010. doi:10.1002/14651858.MR000010.pub3. PMID 17443631.
  22. ↑ Gøtzsche, PC (1987). "Reference bias in reports of drug trials". BMJ 295 (6599): 654–656. doi:10.1136/bmj.295.6599.654. PMID 3117277.
  23. ↑ Ravnskov, U (1992). "Frequency of citation and outcome of cholesterol lowering trials". BMJ 305 (6855): 717. doi:10.1136/bmj.305.6855.717. PMC 1882943. PMID 1393143.
  24. ↑ Ravnskov, U (1995). "Quotation bias in reviews of the diet-heart idea". J Clin Epidemiol 48 (5): 713–719. doi:10.1016/0895-4356(94)00222-c. PMID 7730926.
  25. ↑ Kjaergard, LL; Gluud, C (2002). "Citation bias of hepato-biliary randomized clinical trials". J Clin Epidemiol 55 (4): 407–410. doi:10.1016/s0895-4356(01)00513-3. PMID 11927219.
  26. ↑ Schmidt, LM; Gøtzsche, PC (2005). "Of mites and men: reference bias in narrative review articles: a systematic review". J Fam Pract 54 (4): 334–338. PMID 15833223.
  27. ↑ Nieminen, P; Rucker, G; Miettunen, J; Carpenter, J; Schumacher, M (2007). "Statistically significant papers in psychiatry were cited more often than others". J Clin Epidemiol 60 (9): 939–946. doi:10.1016/j.jclinepi.2006.11.014. PMID 17689810.
  28. ↑ Pham, B; Klassen, TP; Lawson, ML; Moher, D (2005). "Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary". J Clin Epidemiol 58 (8): 769–776. doi:10.1016/j.jclinepi.2004.08.021. PMID 16086467.
  29. ↑ Juni, P; Holenstein, F; Sterne, J; Bartlett, C; Egger, M (2002). "Direction and impact of language bias of controlled trials: An empirical study". Int J Epidemiol 31 (1): 115–123. doi:10.1093/ije/31.1.115.
  30. ↑ Sterne, J.; Egger, M.; Moher, D. (2008). "Addressing reporting biases". In Higgins, J. P. T.; Green, S. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley. pp. 297–334. ISBN 978-0-470-69951-5.
  31. ↑ Chan, AW; Krleža-Jerić, K; Schmid, I; Altman, D (2004). "Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research". CMAJ 171 (7): 735–740. doi:10.1503/cmaj.1041086. PMID 15451835.
  32. ↑ Hemminki, E (1980). "Study of information submitted by drug companies to licensing authorities". BMJ 280 (6217): 833–836. doi:10.1136/bmj.280.6217.833. PMID 7370687.
This article is issued from Wikipedia - version of the Tuesday, April 05, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.