Stages of conducting a realist review

You may already be familiar with different types of literature reviews and how they are conducted. A realist review follows a similar process to a traditional systematic literature review. However, there are a few notable differences which we have outlined in table 1.

As highlighted in table 1, the key stages in a realist review are therefore non-linear, but can be separated as follows:

1. Clarify scope and locate existing theories
2. Search for evidence
3. Appraise and select evidence
4. Extract and organise evidence
5. Synthesise evidence and draw conclusions
6. Develop narrative and make recommendations

Some stages may be done in parallel e.g. appraising, selecting and extracting evidence (stages 3 and 4), and some can be repeated if necessary and not for the same number of times e.g. you may repeat stages 2-4 without moving to stage 5. The process is dynamic and the stages are repeated if deemed useful for further refining the programme theory.

Stage 1 – Clarify scope and locate existing theories

An important starting point is to identify the review question and define the scope. For example, the intervention should be clarified in terms of what it is, for whom the target recipient(s) are, in what specific settings (if any) the intervention is being considered, and what are the outcome(s) of interest. At this stage you may already have a set of potential explanations for how an intervention may work. Use these to devise an initial programme theory by also drawing on the experiences of the review team and any other stakeholder or subject matter experts you have links with in the field.

An exploratory literature search may also be helpful to help identify other potential explanations for how and why the intervention works. Does the intervention work differently in different settings? How is the intervention used in practice compared to what is recommended in local/national guidelines? Take some time to do this as the initial programme theory forms the starting point for which you will iteratively test and refine as you find more relevant and robust evidence. A helpful output from this stage could be a diagram or narrative of the programme theory.

Stage 2 – Search for evidence

A systematic literature search should be conducted to identify a more comprehensive range of research and non-research evidence that can be used to test your initial programme theory. At this stage, the priority is to identify potentially relevant papers, insights, policies etc. Assessment of rigour is important but excluding all except for a minority of experimentally rigorous studies would reduce the validity and generalisability of the review findings. As indicated in table 1, types and sources of evidence should not be restricted to empirical experimental studies. Furthermore, potentially relevant papers and evidence may also be identified through purposive approaches such as snowballing (reviewing the bibliography of relevant papers).

Stage 3 – Appraise and select the evidence

While the quality appraisal of studies in traditional systematic reviews is done to exclude flawed or weak evidence, this is not the case for a realist review. Instead, evidence is appraised to assess the weight of its contribution to refining the programme theory. Typically, quality appraisal is carried out by considering the relevance and rigour of a piece of evidence. Relevance is about whether the evidence addresses any of the aspects of the programme theory being tested. Selection is primarily based on relevance. Rigour is about the methodological robustness of the study and how credible a particular inference is drawn by the original researcher.

Stage 4 – Extract and organise evidence

The next stage after retrieving an initial core set of evidence is to systematically extract the data and organise them. This typically involves two or more reviewers to familiarise themselves with the papers, articles, feedback, policies etc starting with those that are the richest sources i.e. papers that have the most potential to inform the programme theory. An initial coding frame is produced and is then applied to the rest of the evidence. New codes may be added as more evidence is reviewed. Each code can then be analysed alongside associated texts to identify potential specific contexts and/or mechanisms related to the outcome(s) of interest.

Stage 5 – Synthesise evidence and draw conclusions

In a realist review, evidence synthesis is about refining the programme theory – that is, to determine how the intervention works, for whom, how, and in what respects. Here, the various contexts and mechanisms, identified across an array of evidence, are examined together to identify potential patterns of causality. These are important to explain how the intervention may change the context in such a way as to trigger a particular mechanism to result in a specific outcome. Like many of the previous stages, this process is iterative and requires careful consideration of the evidence in relation to the programme theory.

Stage 6 – Develop narrative and make recommendations

Review the findings and develop a narrative for the programme theory that can be shared with stakeholders. This stage can be done in parallel to evidence synthesis as part of refining the programme theory and is potentially a helpful way of improving face validity. Practical recommendations in a realist review can then be made by describing the relationships between the intervention and the contexts in which these are used.

Please find optional additional reading on this topic in the see also section below.

In the comments below please let us know:

• How could these stages be useful in your own healthcare setting?

We encourage you to interact with other learners by ‘liking’ and replying.