Key elements of a well-designed experiment

When two or more treatment groups are compared, the animals in each group should be in identical environments (or variations in environments should be the same across all groups) and be similar in every way apart from the applied treatments. Bias can be minimised by:

• Randomly allocating animals to treatment groups (the random sequence used must be generated by a valid process such as tossing a coin or using a computer generated random sequence).
• Ensuring that all subsequent treatments (including allocation to housing) are applied in a random order.
• Ensuring that when you measure experimental outcomes you are unaware of the treatment received (i.e. you are blinded) until the final statistical analysis.

An adequately powered experiment is one in which you have a high chance of detecting an effect if it is truly there. High power is achieved by:

• Using appropriate numbers of animals (e.g. formally calculating the sample size) [1].
• Controlling inter-subject variation (e.g. using randomisation and/or introducing systematic heterogeneity [2]).

Determine your sample size 

Determine your sample size using a formal method such as a power calculation [1]. Power is increased with increasing sample size, but an unnecessarily large experiment will waste animals and scientific resources. The number of animals you use should be the minimum number that is consistent with the aims of the experiment. Underpowered experiments, with sample sizes too small to detect a meaningful biological difference will waste animals and resources [3, 4].

If you are comparing two groups you will need the following information for your power calculation:

• The type of statistical test you are going to use (e.g. a t-test or a chi-squared test)
• The significance level (5% is often used)
• The power (usually 80-90%)
• The sidedness of the test (a 2-sided test is usually the most appropriate)
• The effect size of biological interest (i.e. what difference between the two groups would you consider interesting and worth taking forward into future research?)
• An estimate of the standard deviation (when comparing means), the best source for this is a previous experiment.

Control variation

Control variation by randomly allocating animals to treatment groups, while balancing differences in the animals (such as baseline weight or age) across groups. Introducing systematic variation into studies (e.g. testing the same intervention on several different strains, or performing tests across multiple sites) leads to results that are more generalisable [2].

Minimise measurement error

Minimise measurement error by using careful technique, good equipment, and by implementing blinding so you are not aware of the animal’s treatment allocation.

Your results will be more widely applicable if you have included both sexes, different strains, different environments, and other factors [2,5]. A response to a drug may depend on prior treatment of the animal, the effects of other drugs or the route of administration. These effects can be studied efficiently using factorial experimental designs.

Factorial experimental designs allow you to investigate the effect of a treatment on both males and females without doing two separate experiments, or using twice as many animals. To do this, make half of your subjects in each experimental group male and half female [6]. An adequately powered factorial experiment will show whether or not the two sexes respond in the same way, which is not possible if the sexes are studied individually in two separate experiments.

Overly complex experiments make mistakes more likely, and increase the chances of  statistical analyses becoming unduly complicated.

Before you start a major experiment use a small pilot study to ensure the experiment is feasible and enable you to make any changes to improve the experimental design or increase the efficiency of the main experiment.

Plan your experiments, including statistical analysis methods, before starting and do not change while they are in progress without good reason. If you do need to make changes make sure you document the changes and why you needed to deviate from your plans.

For each of your experiments generate descriptive statistics. These results can be used when planning future experiments. Descriptive statistics include mean and standard deviation or median and interquartile range. Indicate the range of uncertainty in your results, or a measure of variation such as standard deviations or confidence intervals.