The importance of experimental design

WARWICK DUNN: Experimental design, or the design of experiments, is a critical process in any scientific study. In metabolomics, getting your experimental design correct is critical. A simple definition of experimental design is the design of a data collection study where all sources of variation are assessed in the design process and contribute to the final project design. The important point to consider is that the different sources of variation may or may not be under the control of the researcher.

Ideally, we would like to measure the variation in one source while controlling all other sources of variation. For example, when identifying the metabolic changes resulting from a genetic change in yeast, all sources of variation will be controlled, such as the culture medium and temperature. The only difference in the experiment will be the genotype. By controlling all sources of variation except the one you are testing– in this example, the genetic difference in yeast– we are confident that our observations are due to the parameter that we are testing. So when we compare the metabolome with the two strains of yeast, our observations are due to the genetic difference and not another source of variation, or combination of sources of variation.

Whether we can control other sources of variability is dependent on the type of study that we wish to perform. In the laboratory, we can normally control the genotype and environment when studying microbes, plants, or mammalian cell lines, and so experimental design is relatively simple. When studying samples from the human population there are many sources of variation, related to the genotype, environment, and lifestyle. In many studies it is difficult to control these sources of variation, and so instead we need to ensure that the variability is equivalent between different groups of subjects.

For example, we want to ensure the age range and the ratio of males to females is equivalent in the sample groups, because we know that these will influence the phenotype and therefore the measured metabolome. Other factors, such as food intake, medications, and many more should also be considered in human studies. Where these sources of variation are not matched, then the study will be biassed, and the metabolic changes observed are a result of two or more sources of variation.

So if you are studying metabolic differences between a group of subjects who have a disease and a group of subjects who do not have a disease, and the control group are aged 30 to 45 years, and the disease group are aged 50 to 60 years, you have two sources of variation. Age and disease. So the observed changes in the metabolome are due to both age and disease. If the age range for both groups is similar, than the influence of age is removed, and the study now only has one source of variation, the disease. So you can be confident that your observed changes in the metabolome are due to the disease only.

Experimental design will assess many different aspects of the study workflow, from the biological study and sample collection through to data processing and analysis. One important point is to define the study question being asked. The type of study performed will influence the experimental design. For example, searching for biomarkers of a human disease will have a different experimental design to assessing the mechanistic changes in the early stages of a disease. The biological question will influence the experimental design. Sample numbers are another important consideration in experimental design, and should be defined at the start of the study to ensure an appropriate number of biological replicates are included in each biological group, to confidently answer the question in a statistically robust manner.

Where variability is highly controlled in the laboratory, fewer replicates are required. Typically, 6 to 10 biological replicates should be the minimum per biological group. However, in studies where variability is not well controlled when studying the human population, then hundreds or thousands of biological replicates are required to provide statistically significant and robust results. So as you can see, experimental design is an important aspect of any scientific study. Constructing a robust scientific study will provide confidence to the research community that the biological results reported are correct and should be validated in further hypothesis-testing studies.