Methodology and results

Methodology

The key question the methodology addresses is ‘How?’ This section should enable critical readers to determine whether the research is valid: was the study adequately designed to achieve its purpose? It explains the detail of how the research was conducted and to whom or what the study results apply.

The methodology section also provides information about the sampling method and whether subjects were randomly assigned or not (in experimental studies). Pay specific attention to these points, because flaws that occur here can render the results invalid.

Methodological limitations

There are all kinds of limitations that make scientific research challenging. External limitations include access to finances and the regulations around ethics of human testing. Internal limitations include gaps in or limits to the current state of knowledge (particularly as it relates to data collection). Any limitations and constraints that affect the results should be discussed in the methodology or discussion sections of the study.

Here are some key questions to bear in mind when reading the methodology:

• Are there any major design flaws in this study?

• Are the data collection measures appropriate to answer the study questions?

• Were methodological limitations acknowledged and discussed?

• What influence might these limitations have had on the results?

Results

Few people would deny that reading the first three sections of a scientific study can be difficult and require focus. But then we finally get to the really interesting stuff: the answers. The results section provides the answers in the form of data and statistical analysis. Statistical measures can clearly and accurately describe the existence and strength of relationships observed in the study.

It is easy to get wrapped up in discussions of statistical significance when reading research, but it is important to remember that a statistically significant result does not necessarily mean that the outcomes are important or relevant to the public. And it doesn’t guarantee that the research is without bias or confounding factors that could make the statistical value irrelevant.

If the results are not statistically significant, the author may discuss the statistical power of the study. An in-depth discussion of statistical power is complex but such information does help the reader understand whether the study had a chance of finding the answer to the research questions in the first place.

Statistical significance is only part of the picture; to get the whole picture, the reader must consider the context of the study. Some key questions to bear in mind when reading the results are:

• What is the real as well as statistical significance of these results?

• To whom do these results apply?

In the next Step, you’ll explore the discussion and references before you have the opportunity to review a scientific paper yourself. Don’t forget to ‘mark as complete’ before you move on.