A typical biomedical researcher can recall at least one instance in a research career where he or she could not reproduce results that had been published in a peer–reviewed article. It is estimated that as high as 89% of published findings in major cancer journals, for example, are not reproducible. This alarming ‘reproducibility crisis’ emerged when the pharmaceutical company Amgen in the USA tried to confirm 53 ‘landmark’ studies in oncology. Astonishingly, 47 out of the 53 (89%) could not be reproduced (1).  This is in line with an earlier report by Bayer Healthcare in Germany that only 25% of the published results were reproducible (2). While not everyone will agree with reproducibility defined in these studies as ‘validation to the point at which projects could continue’, the sombre reality is that any scientist who has been involved in research for more than three years will acknowledge that a considerable number of findings in well-reputed biomedical journals are not reproducible. There are many reasons for the ‘reproducibility crisis’ (3, 4). Irrespective of the undeniable fact that a minority of researchers falsify their data, three additional reasons stand out: the reluctance of mainstream journals to publish contradictory or negative findings, the policies advocated by funding bodies and research / academic institutions, and the constraints that are placed on authors.

 

First, mainstream journals prefer positive results.  Anything that does not show a p<0.05 value is considered lacking in scientific merit and thus does not receive equal importance as that of a manuscript that reports positive data.  In the same vein, any studies that show a p<0.05 value that is contradictory to previously published data are looked at with incredulity and the authors have a difficult process to follow in order to proceed to publication. It is not uncommon to see researchers conducting research with the sole purpose of publishing their findings in particular journals. While it is acceptable to do so, negative findings observed during such a quest mostly never proceed to publication.  An impact factor-motivated publication system encourages authors to overlook negative or contradictory results. Moreover researchers at times are enticed to submit manuscripts to low impact factor rating journal, just so as to get the work published and overcome any publication frustrations encountered.  Moreover, it seems reasonable to assert that impact factors should have no place in the provision of research grants, tenure or appointment committees (5), however the reality is somewhat different.

 

Second, funding bodies require well-focused projects that show the proposed research will produce a positive outcome within the specified period of time. The ‘expected outcome’ section of the funding application should have something to state that the project will produce a positive outcome and that the results will potentially cure the disease of interest, or something to that effect. A considerable number of research project applications are based on preliminary data, generated with limited resources, sample numbers and repeats. When the project becomes successful in attracting funding and during the course of the actual experiments, if the researcher observes an effect either null or contradictory to the original hypothesis on which the funding became successful, there are no provisions for adequately addressing this issue.

Third, there are no incentives for scientists to explore or report negative findings. The culture of ‘publish or perish’ discourages authors taking negative findings seriously (6). Also there is an inherent fear of being viewed negatively by peers, adding to the potential detrimental effect it may have on future collaborations, publications and funding applications. An impression also exists that researchers who support contentious scientific research outcomes will be reviewed by experts in the field, whose research focus is in opposition and as such will fail the scholarly peer review process.

 

Whatever the reason, there is a requisite need that addresses the problem of irreproducibility so that contradictory, null and negative findings can be disclosed in a non-judgemental, unbiased, yet scientifically plausible manner.  Journal of Controversies in Biomedical Research (JCBMR) is an attempt to address these problems in biomedical research. JCBMR (www.jcbmr.com) is an online-only open access journal to address contentious research findings.  Basic science or clinical research articles that meet any of the following criteria are suitable for submission: a) original articles that show negative, neutral or contradictory findings; b) original articles that challenge previously published results in peer-reviewed journals; c) original articles that show effects of compounds on disease models (either in vitro or in vivo) are contradictory to the expected outcome; d) review articles that critically evaluate and challenge established norms and offer possible solutions to the problem; e) any manuscript that will assist the scientific community to re-think and value–add to the established norm.

 

Why should researchers care about publishing contradictory or negative results? There are at least five potential benefits. First, and most importantly, is the fact that publishing controversial or negative results could progress a beneficial scientific paradigm shift. Think of Galileo as he abandoned the popular geocentric (earth–at–the–center) view of the solar system of the time for the heliocentric (sun–at–the–center) view. Consequently his refusal to follow the popular teachings was to his own personal detriment.  Similarly, the case of the Nobel Laureate Peter D Mitchell (1978–awarded for Chemistry) who formulated the chemiosmotic hypothesis at a time when scientific research abounded on secondary / substrate level phosphorylation (7). Mitchell emphasized that all biological systems that carry out the synthesis of ATP were membrane systems and that they were all closed systems and that nobody had ever isolated a chemical intermediate before and so therefore maybe they just did not exist (7).  This posit was greeted with significantly less enthusiasm, given that researchers at the time were heavily engaged in elucidating the phosphorylated intermediate in ATP synthesis, which Mitchell disputed by proposing that it did not exist (7).

 

Second, the success rate of major competitive funding applications is less than 20%. If 75-89% of the peer reviewed published research is not reproducible and if this is then widely publicised, the end result is for the redirection of funding to other research areas of need.  Third, this could be the impetus for a paradigm shift that funding bodies and institutions require in order to implement a revolutionary change in scientific thinking and reporting for controversial and negative findings. This may well lead to the provision of support to researchers with novel biologically plausible scientific ideas.  Fourth, researchers have a responsibility to the community at large who indirectly fund scientific research and this forms an important part of value adding to the scientific understanding that is perceived by the community that then can be translated to the building of robust human capacity in the field. When transparency in research reporting is established, public awareness and confidence in researchers is established and maintained.

 

Fifth, and perhaps the most important, is the legacy that is left in place for future generation of scientists. When authenticity is established, time and resources are properly directed and utilised, and the pursuit of scientific knowledge abounds. Our hope is that JCBMR will play a significant role in achieving these goals.

Conflict of Interest

 

CM is the founder of JCBMR and LV is the Editor-in-Chief of JCBMR.

 

References

 

1. Begley C, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature 2012; 483: 531–533.
http://dx.doi.org/10.1038/48353

2. Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nature Rev. Drug Discov 2011; 10: 712. http://dx.doi.org/10.1038/nrd3439-c1

 

3. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005; 2: e124.
http://dx.doi.org/10.1371/journal.pmed.0020124 PMid:16060722 PMCid:PMC1182327.

 

4. Jager LR, Leek JT. An estimate of the science-wise false discovery rate and application to the top medical literature. Biostat. 2014; 15: 1-12.
http://dx.doi.org/10.1093/biostatistics/kxt007 PMid:2406824.

 

5. [No authors listed] Beware the impact factor. Nat Mater. 2013;12(2):89.
http://dx.doi.org/10.1038/nmat3566 PMid:23340463.

 

6. Fanelli D. Do pressures to publish increase scientists' bias? An empirical support from US States Data. 1. PLoS One. 2010;5(4):e10271.
http://dx.doi.org/10.1371/journal.pone.0010271

 

7. Allchin D. Cellular and theoretical chimeras: piecing together how cells process energy. Stud Hist Philos Sci. 1996;27(1):31-41.
http://dx.doi.org/10.1016/0039-3681(95)00027-5