DRIVER recommendations – Experimental groups and exclusions

How to establish plans for handling experimental data and reporting any data exclusions.

Item 5: Experimental groups and exclusions

Data exclusions may sometimes be necessary, but how they are applied and reported can have a substantial impact on the analysis and interpretation of the results. Poorly justified or selectively applied exclusions can also introduce outcome reporting bias. When designing an experiment, potential reasons for excluding data should be identified in advance and used to define clear inclusion and exclusion criteria. When reporting the results, all exclusions should be described transparently and clearly justified.

You can find helpful tips and additional context in the grey boxes in each section below.

Design recommendation

Establish criteria for handling and reporting data from all experiments.

Identify potential reasons for excluding data

How to determine the circumstances that may require data exclusions.

Define inclusion and exclusion criteria prior to starting experiments

What to consider when establishing criteria for handling experimental data.

Identify potential reasons for excluding data

Data exclusion refers to the removal or disqualification of data points, experimental units or groups from an analysis. During the course of an experiment, a range of issues may arise that necessitate the exclusion of individual data points, experimental groups or even entire experimental runs.

Many legitimate reasons for excluding data points exist, however excluding data arbitrarily or according to researchers’ subjective opinions on their validity, has the potential to introduce bias to experimental results. When the inclusion of data in analyses is solely at researchers’ discretion, decisions on the inclusion or exclusion of particular data points can be influenced by expectations or preconceptions. This is particularly an issue in experiments where researchers are aware of the treatment each sample has received (this is explored in detail in Item 2: Risk of bias).

The most common reasons for data exclusion can be grouped into the following categories:

Technical failures: Exclusions resulting from errors introduced during an experiment or poor laboratory practice that compromise experimental outputs (e.g. pipetting errors).
Experimental criteria: Exclusions based on outcomes or events during the experiment that indicate it was unsuccessful (e.g. failed positive or negative controls).
Analysis criteria: Exclusions based on predefined criteria applied during data analysis to avoid introducing bias into the interpretation of experimental outcomes (e.g. threshold-based inclusion criteria).

For an expanded explanation and other potential reasons that exclusions may occur see the interactive content (i. What are exclusions and why might they occur?).

Some reasons for excluding data can be anticipated and addressed during the experimental design phase, while others occur unexpectedly during the course of the experiment. Where potential reasons can be identified in advance, they should be used to define explicit criteria to be applied consistently. For example, experiments may require a minimum number of cells or a specific level of confluency; any wells, plates or flasks that do not meet these requirements would therefore be excluded.

Define inclusion and exclusion criteria prior to starting experiments

Inclusion or exclusion criteria set out the specific limits or thresholds used to support and inform decisions on whether data should be included or excluded. Where possible these should be predefined. Setting criteria a priori – before the experiment starts – helps avoid the introduction of bias, by ensuring that decisions to exclude data are based on prespecified requirements rather than on the observed results.

To further minimise the introduction of bias, criteria should be as objective as possible, with clear thresholds. These criteria must also be applicable equally across all experimental groups or conditions to avoid any bias towards a particular group within an experiment.

For a worked example demonstrating how exclusions can impact the interpretation of experimental data see the interactive content (ii. Why do exclusions matter?).

Some criteria may emerge during the course of a study – especially when developing or testing new methods or models. Once identified, these criteria should be clearly defined and used in future studies. They can often serve as quality control measures; for example, an initial experiment using stem cells may identify that differentiation into a specific cell lineage is required for the system to function as intended. This can then be adopted as a quality control criterion, with any wells, plates or experimental runs that fail to meet it being excluded.

In general, all analysis-based criteria can be predefined, however it may not be possible to do this for every potential source of exclusion. The table below outlines some general categories to help identify potential sources of exclusions and how to manage them.

Category	Role of predefined criteria	Notes
Technical failures/errors	Not typically able to predefine	These are unintentional and unpredictable errors. While good lab practices and SOPs can reduce their occurrence, they cannot be fully anticipated or prespecified.
Experimental criteria	Ideally predefined, but may include emergent criteria	Some criteria (e.g. cell confluency thresholds, organoids forming the wrong tissue) can be prespecified. Others may emerge during method development or pilot studies.
Analysis criteria	Should be predefined	To ensure objectivity and avoid bias, analysis-related exclusions (e.g. outlier removal, thresholding) should be clearly defined in advance in the analysis plan.

Exclusion criteria should be defined before a study begins and documented in an experimental plan. This helps prevent biased data handling and analysis as the decisions about how data will be treated are made before experimental outcomes are known. While these plans do not need to be formally preregistered, documenting exclusion criteria in advance is considered good scientific practice. Where appropriate, the criteria can be published ahead of time as a preregistration (e.g. via a journal or registry). This approach helps build trust in the findings and provides a clear rationale for any data exclusions that occur.

One version of this process is known as a registered report, in which a study’s proposed methods and analyses are submitted to, and peer reviewed by, a journal in advance of the experiment taking place. Publication of the study is then dependent only on the quality of the methodology and adherence to the preregistered protocol, rather than the nature of the results obtained.

Reporting recommendation

Report all data obtained from all experimental groups (including controls) and justify any exclusions.

Report all data from experimental groups and controls

How to report experimental outcomes in full.

Transparently report all exclusions and use of criteria

How to explain and justify any exclusions of experimental data.

Report all data from experimental groups and controls

The selective (or incomplete) reporting of experimental results or analyses, in order to support a particular hypothesis (i.e. only reporting ‘positive’ results), has a considerable impact on the reliability and reproducibility of those results. Selectively reporting only those experiments (or parts of experiments) with statistically significant results that support a particular hypothesis misrepresents the true rate of confirmed hypotheses and can lead to over- or underestimation of the effect of interventions.

For example, if a cell culture experiment investigating the effect of a drug treatment on cancer cell growth is repeated three times, two of these experiments may show non statistically significant results and one may show a significant decrease in growth rate. Reporting only the experiment resulting in a statistically significant change can result in overestimation of the effect of the drug by readers. Future research building on these reported findings, or meta-analyses synthesising the results of published studies, will therefore be based on incomplete or misleading results.

For a reader to assess the reliability of the results reported, it is crucial that all the data obtained from each experimental group are reported, including results that are not statistically significant (or results that do not support the study’s hypothesis). This includes all control groups (negative or positive) used in the study and descriptions of whether positive controls had their intended effect.

This is explored further through a worked example in the interactive content (ii. Why do exclusions matter?).

Selective reporting and other similar practices like ‘cherry picking’ – the practice of excluding data to ensure that the most compelling data are presented, rather than reflect the true spectrum of experimental outcomes – are sources of outcome reporting bias.

For more on outcome reporting bias see the interactive content (iii. outcome reporting bias).

Transparently report all exclusions and use of criteria

Clearly reporting all data exclusions helps readers understand how the dataset was handled from start to finish. All exclusions should be reported transparently in the manuscript, including:

How many exclusions occurred.
Why exclusions occurred.
When the exclusions occurred.
If the decision to exclude was based on predefined criteria.
Differences in sample sizes between groups and specific n numbers.

Predefined inclusion and exclusion criteria should be reported in full, noting whether each criterion was applied or not. Being transparent about these decisions supports the interpretation of the results and helps build trust in the findings. When exclusions or criteria are not reported clearly, readers may question the robustness of the findings or suspect that data were selectively removed. Transparent reporting helps prevent this and strengthens the credibility of the research.

Reporting this information in full allows readers to understand any changes to the sample size, what led to any exclusions and at what point in the experimental process they occurred. Together, these provide the justification for the exclusion, enabling the reader or reviewer to assess the validity of exclusions in an informed manner.

If the criteria were predefined in the analysis plan this should be included alongside the article as this provides the justification for the exclusion with evidence that the criteria were defined a priori, before data were collected. Similarly where analysis plans were preregistered, the preregistration should be cited in the manuscript.

Do not report sample size ranges (e.g. n = 3-5), particularly in figures and figure legends, as this is ambiguous and does not clearly communicate where exclusions have occurred. Specific sample sizes for each experimental and control groups should be described either in the main body of text or in figure legends.

All exclusions can be clearly reported in the results; however, where and how they are explained may differ depending on the reason for exclusion. For example, exclusions based on pre‑defined criteria can be justified in the methods section, as the criteria are either met or not met. In contrast, exclusions arising from biological variation or other unexpected factors can be reported in the results and then explored further in the discussion. This may include consideration of potential sources of variability that led to the exclusions.