Poor reporting of animal research hinders the quality of research and the potential for it to translate into the clinic. The ARRIVE (Animal Research: Reporting of In Vivo Research) guidelines, published in 2010 in PLOS Biology, were developed as part of an NC3Rs initiative to improve the reporting of animal research to ensure maximum benefit and to minimise unnecessary animal studies.
Research published this week in PLOS Biology shows experimental flaws and inadequacies in the reporting of animal research in the field of neuroimmunology. The research, led by Professor David Baker, Queen Mary University of London, was prompted from concerns in the lab about experimental design flaws in pre-clinical animal studies assessing treatment effects in experimental autoimmune encephalomyelitis (EAE), a mouse model of multiple sclerosis.
Initially we assessed the appropriate use of statistics to analyse treatment effects in EAE studies. We took the view that a non-parametric statistical approach, which does not make assumptions about population distribution, should be used when assessing clinical data from EAE studies. After analysing 180 papers reporting EAE studies, we found only 39% of papers used appropriate statistics. Even more concerning was the finding in an additional survey of high impact journals (Nature journals, Science and Cell) showing only 4% of high impact journals reported appropriate use of statistics. It was also found that reporting of statistical methods was absent in 13% of studies, which led us to consider inadequate reporting in addition to experimental design failures.
As a result of these findings, we carried out a survey to look at reporting of key elements described in the ARRIVE guidelines. We assessed publications two years before and after the endorsement of the guidelines in both PLOS and Nature journals. Following a literature search for relevant EAE papers in these journals, four key aspects of reporting were analysed including ethical approval, study design (methods to reduce bias), animal characteristics (species, sex, age and group size) and sample size estimation. Although ethical approval was commonly reported and there was an improvement in reporting of animal characteristics, other areas caused some concern. Fewer than 10% of journals reported methods to reduce bias (such as randomisation and blinding) and fewer than 7% reported sample size estimations (such as power calculations). The results show a lack of transparency and poor reporting suggesting that the ARRIVE guidelines are potentially being ignored and therefore having minimal impact.
So, is simply endorsing the ARRIVE guidelines sufficient to improve reporting? The vast majority of journals endorsing the ARRIVE guidelines do so by encouraging authors to refer to the guidelines when authoring and submitting manuscripts. However, this study demonstrates that animal research papers are still failing to meet minimum reporting standards. This suggests the current approach taken by journals is not working and that editors need to do more to ensure both authors and reviewers are aware of the minimal requirements for reporting animal research.
In response to this study and growing evidence highlighting lack of transparency in reporting research, PLOS Biology published an editorial to coincide with the original article (Open Science and Reporting Animal Studies: Who’s Accountable?). The editorial highlights the consequences of poor reporting and stresses the requirements of transparent reporting to ensure successful translation of pre-clinical research, reduce publication bias and allow study validation. In addition, the authors highlight the ethical obligation to ensure animals are not unnecessarily wasted as a result of poor reporting. PLOS journals are requesting authors to use the ARRIVE guidelines checklist when submitting relevant manuscripts to ensure reporting of animal research meets minimum standards. In April 2013, Nature journals adopted a similar approach and introduced editorial measures to improve transparency of research which includes a checklist incorporating elements of the ARRIVE guidelines. The true impact of the ARRIVE guidelines can only be accurately measured when journals take active steps towards implementation to eliminate poor reporting.
Journals play a central role in improving standards of reporting but a change in reporting practices requires input from the whole scientific community including funders, universities and learned societies. The New Year is traditionally seen as a time for making resolutions, to resolve bad habits and start the year with best intentions. The timely publication of this research could be used as a prompt to the scientific community to resolve to use the ARRIVE guidelines to improve reporting of animal research in 2014 and beyond. Let’s start the year as we mean to go on, request your copy of the ARRIVE guidelines here.
Baker D, Lidster K, Sottomayor A, Amor S (2014). Two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLOS Biol 11(12): e1001756.
Eisen JA, Ganley E, MacCallum CJ (2014). Open science and reporting animal studies: who’s accountable? PLOS Biol 12(1): e1001757.
*and now NC3Rs staff member.