An overview of different types of studies

As the examples we’ve looked at so far show, some of the confusion around what studies prove is down to a lack of understanding of the strength of evidence they convey. The strength of scientific evidence produced by different types of studies such as systematic reviews, meta-analyses, randomised controlled trials, observational research, animal studies, cell studies and expert opinions, can vary. All of them have their advantages and limitations.

Systematic reviews and meta-analysis

A meta-analysis is a method of combining results from separate studies to get broader overall conclusions about a hypothesis. Meta-analyses are a useful way to reconcile differences of statistical power or sample size between studies, or to aggregate relevant findings across studies.

The procedure is most appropriate when examining studies that look at the same question and use similar methods to measure relevant variables. For example, scientists used a meta-analysis of observational studies to examine the relationship between the consumption of red meat and the risk of bowel cancer. Although individual studies showed different results, pooling the data from 16 similar studies showed significant decreases in the risk of oral cancer with increased intakes of fruits and vegetables.

However, as with every study technique, the meta-analysis is not without its limitations. Data from flawed studies may be included, or the analysis may include data from studies that use different methods to measure variables resulting in a situation where direct comparisons are impossible (you’re ‘comparing apples with oranges’).

Systematic reviews gather together all the studies on a particular subject and present them in one paper, highlighting their strengths and weaknesses as well as any contradictory findings. The aim of a systematic review is not to come to a conclusion but to summarise the evidence to date. On the other hand, a meta-analysis aims to come to a conclusion based on the findings of a wide range of studies.

Randomised controlled trials

Randomised controlled trials (RCTs) can help prove causation. A selected group of people (a population sample) is split into two groups; one intervention group and one control group. The aim is to have different variables, such as gender, age, or lifestyle, equally distributed among the two groups. The intervention group is then given a selected treatment, while the control group does not get any treatment, or they get a placebo/fake treatment. If done correctly, observed differences between these two groups can then be attributed to the treatment investigated.

Uncertainty arises if the controlled trial becomes ‘uncontrolled’, either because of uneven splitting of groups, or because the people in the groups know which treatment they are getting as this can sometimes be hard to hide.

Observational research

These types of studies identify correlations (links) and develop hypotheses that can be further investigated. In these studies, researchers observe what happens without intervening. For example, they can observe whether there might be any link between eating eggs and cholesterol levels in a group of people. However, it’s important to keep in mind that correlation does not prove causation as other factors might be affecting the outcome.

There are various types of observational study:

• Cohort studies: the researchers follow a group of people who often share similar characteristics over a period of time (usually many years), to see if an outcome develops.

• Case-control studies: these help determine whether an exposure leads to an outcome, for example a disease. These studies are retrospective as they look at an outcome first and try to trace back to identify the exposure.

• Case series: these studies follow people with a known exposure or receiving the same treatment and examine their outcome.

Animal and cell studies

These studies are similar to the RCTs, but done on animals or on cells in a lab rather than in people. As animals and cells are different from whole human bodies, these types of studies are not considered as strong evidence but they provide indications that can help design further studies.

Expert opinions and anecdotes

Personal experiences and statements from experts are rated quite low on the evidence scale in scientific studies. Such statements are not objective. We are all very different, and what worked for one individual will not necessarily work for everyone.

Science is evolutionary, not revolutionary

We’ve looked at the strengths and limitations of different study designs but it’s also important to understand how to interpret the results they produce. The scientific process – how studies are designed, conducted, and reported – frequently generates a great deal of public debate. Tracking the debate is often key to putting new research into context. With that in mind, new research studies published in scientific journals should be viewed as discussions among scientists. In these discussions, almost no one gets to have the final word, as it is rare that a study provides a final, complete answer.

The bottom line is that dialogue characterised by cycles of revision, conjecture, assertion, and contradiction are key to investigating a subject. Although such cycles often frustrate non-scientists and contribute to increasing public scepticism about advice on food and health, it is important to understand that science is evolutionary, not revolutionary. Because scientific research explores the unknown, uncertainty is an unavoidable part of investigations. Only through repeated research and analyses can certainties emerge.