Experimental design/statistics

Well designed and correctly analysed experiments can lead to a reduction in animal use whilst increasing the scientific validity of the results. The number of animals used should be the minimum number that is consistent with the aims of the experiment.

Our experimental design hub contains a number of resources for use in this area.

A well designed experiment should be:

1. Unbiased

When two or more treatment groups are compared, the animals in the groups should be in identical environments and be similar in every way apart from the applied treatments. Bias can be minimised by:

  • Randomly allocating animals to the treatment groups (a physical process is needed e.g. tossing a coin, picking a number)
  • Ensuring that all subsequent treatments (including housing) are applied in a random order
  • Ensuring that researchers analysing experimental outcomes are unaware of the treatment received (blinded) until the final statistical analysis.

2. Adequately powered (i.e. use sufficient animals)

Powerful experiments are ones that have the maximum chance of detecting a true treatment effect. Power is achieved by:

  • Using appropriate numbers of animals (sample size)
  • Controlling inter-subject variation (e.g. using randomisation)

Sample size should be determined using a formal method such as power analysis. Although power is increased by increasing sample size, an unnecessarily large experiment will waste animals and scientific resources. Similarly, an underpowered experiment, with a sample size too small to detect a meaningful biological difference will waste animals and resources.

Variation is controlled by randomly allocating animals of similar genotypes, of a similar weight and age, which have had a similar environment throughout their lives. Variation due to circadian rhythms or fluctuations in the environment can often be reduced by appropriate experimental design, for example by using blocking factors.

Measurement error should be minimised by careful technique and good instrumentation, and blinding the researcher to treatment allocation.

Power analysis: A power analysis method for comparing two groups, for example, requires the following information:

  • The type of statistical test to be used (e.g. a t-test or a chi-squared test to compare two proportions)
  • The significance level to be used (often a 5% level)
  • The required statistical power (usually 80-90%),
  • The sidedness of the test (a 2-sided test is usual) 
  • The effect size of biological interest (i.e. how much of a difference in biological or clinical effect it is necessary to detect) 
  • An estimate of the standard deviation (when comparing means) this must come from a previous experiment

The StatPages.info website offers online calculations of sample size combining the above factors.

3. Have a wide range of applicability

It is often useful to find out whether similar results are obtained in males and females, in different strains, or as a result of different diets or environments. Similarly, the response to a drug may depend on prior treatment, the effects of other drugs, or the route of administration. These effects can be studied efficiently using factorial experimental designs.

Factorial experimental designs: These can be used to investigate the effect of a drug on both males and females without doing two separate experiments or using twice as many animals. Put simply, in each of the two experimental groups half the subjects are male and half female. An adequately powered factorial experiment will show whether or not the two sexes will respond in the same way, which is not possible if the two sexes are used in different experiments.

4. Be simple and efficient

Experiments should not be so complicated that mistakes are made in their execution, or the statistical analysis becomes unduly complicated.

Small pilot studies should be used before starting a major experiment to ensure that the experiment is logistically efficient and to give some preliminary indication of likely results.

All experiments should be pre-planned, and should not be changed while they are in progress.

5. Indicate the range of certainty

Each experiment should be statistically analysed so that the results can be used in planning future experiments. An appropriate statistical analysis should indicate the range of uncertainty in the results, or the measure of variation, usually indicated by significance levels or confidence intervals.