Developing an AI tool to check ARRIVE compliance

The freely available compliance checker will use natural language processing to automatically assess scientific manuscripts for the information in the ARRIVE Essential 10, a checklist of the most important details to include in any publication describing animal research.

A major factor influencing the reliability and reproducibility of animal experiments is how transparently they are reported. The ARRIVE guidelines help researchers improve the reporting of animal studies by clearly laying out the information that should be included in a manuscript. Reporting animal experiments in line with the ARRIVE guidelines is a requirement of research funding organisations worldwide, including all major UK animal research funders, and the National Institutes of Health in the USA.

Assessing a manuscript’s compliance with ARRIVE is a valuable way for researchers and journals to identify when key information is missing from it. At present, ARRIVE compliance assessment can only be performed manually, using the ARRIVE compliance questionnaire. Automating this process will facilitate the inclusion of ARRIVE assessments in journals’ manuscript workflows, as well as the auditing of reporting practices by funding organisations or universities.

The compliance checker will be developed by the ScreenIT consortium, a group of developers led by SciCrunch, a California-based company focused on initiatives to improve research reproducibility. SciCrunch develop and operate platforms such as the Resource Identification Initiative, the Antibody Registry, and SciScore, a popular scientific manuscript review tool that uses the same technology that will be implemented in the ARRIVE compliance checker.

Dr. Nathalie Percie du Sert, head of experimental design and reporting at the NC3Rs, said:

“We are delighted to work with SciCrunch and colleagues to develop such an important tool for the research community. The automated ARRIVE checker will provide researchers, journals and reviewers with a freely-accessible tool to check that manuscripts are not missing critical information and it will allow us and other stakeholders to monitor improvements in the reporting of animal research.”

Dr. Anita Bandrowski, CEO of SciCrunch, said:

“The ARRIVE guidelines are a very important tool for improving the reporting of animal research, we are very pleased to be developing the compliance checker, which will help make their use easier for the community. At the same time, we’re glad to be able to take this opportunity to add ARRIVE compliance checks to SciScore, making ARRIVE assessment functionality available to the large number of journals who already use SciScore."

The compliance checker will be available via the ARRIVE guidelines website in 2025. Before that, ARRIVE compliance checking functionality will be added to SciScore in 2024.

- Ends -

About the group developing the compliance checker

The ScreenIT consortium is a group of developers led by SciCrunch. They are joined by:

• Dr Tracey Weissgerber, Berlin Institute of Health at Charité – Universitätsmedizin Berlin, a data visualisation and reproducibility expert who has created several automated tools that screen figures and images.
• Dr. Malcolm Macleod, University of Edinburgh, an expert in preclinical research reproducibility whose team have curated large, annotated, gold standard manuscript datasets that will be used to train the compliance checker
• Dr. Iain Marshall, King’s College London, co-leader of the RobotReviewer project, a machine learning system that automates evidence synthesis in systematic reviews.

About the NC3Rs

The National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs) is a leading independent scientific organisation dedicated to replacing, refining and reducing the use of animals in research and testing (the 3Rs). It supports the UK science base by driving and funding innovation and technological developments that minimise the need for animals in research and testing, and lead to improvements in welfare where animals continue to be used. It funds research, supports training and development, and stimulates changes in regulations and practice.

Further information can be found at: www.nc3rs.org.uk @nc3rs

About the ARRIVE guidelines

The ARRIVE guidelines are a checklist of recommendations to improve the reporting of experiments involving animals. Reporting in vivo experiments in line with ARRIVE helps to ensure that manuscripts contain enough information to add to the knowledge base. The guidelines are prioritised into two sets: the ARRIVE Essential 10, the basic minimum information to include in a manuscript, without which readers cannot assess its reliability; and the Recommended Set, which add important context to the study. They are accompanied by extensive explanation and elaboration information, which ensures that the items are easily understood, and provides useful information on rigorous experimental design in animal studies.

Reporting in compliance with ARRIVE is required or recommended by many important research funders, journals and universities worldwide. These include all major UK animal research funders, and the National Institutes of Health in the USA.

The guidelines, more information and resources for using them, can be found at: www.arriveguidelines.org