How to assess risk of bias?

Evaluating the risk of bias is a crucial step in systematic reviews to ensure the reliability and validity of evidence.

Bias can distort study results, leading to misleading conclusions. A structured approach helps identify and account for these potential flaws.

Why assess risk of bias?

• Bias affects the accuracy of research findings.

• A systematic assessment helps detect study design flaws.

• Transparent evaluation improves confidence in the evidence.

Cochrane’s domain-based approach

Cochrane recommends assessing bias across seven key domains, categorising each as low, high, or unclear risk:

1. Random sequence generation

• Ensures participants are assigned randomly to groups.

• Adequate methods: Computer-generated sequences, random number tables.

• Inadequate methods: Birth dates, alternating assignments (which introduce predictability).

2. Allocation concealment

• Prevents selection bias by ensuring assignment is hidden before allocation.

• Effective methods: Central randomisation, sealed opaque envelopes.

3. Blinding of participants and personnel

• Minimises performance bias by preventing treatment expectations from influencing outcomes.

• Double-blind studies ensure neither participants nor researchers know the group assignments.

4. Blinding of outcome assessors

• Prevents detection bias by ensuring those measuring outcomes are unaware of group assignments.

5. Incomplete outcome data (attrition bias)

• Occurs when participants drop out or data is missing.

• Assessing missing data ensures fair comparisons between groups.

6. Selective outcome reporting

• Happens when only certain (often favourable) results are reported.

• To detect this, compare study findings with the original protocol or trial registry.

7. Other sources of bias

• Includes funding bias, deviations from study protocols, or industry influence.

Using the Cochrane risk of bias tool

• Reviewers assess each domain and classify it as low, high, or unclear risk.

• Risk of bias tables and graphs visually summarise findings.

Green: Low risk

Red: High risk

Yellow: Unclear risk

Practical Implications

• Studies with a high risk of bias should be interpreted with caution.

• Transparency and consistency in bias assessment strengthen systematic reviews.

Assessing the risk of bias ensures systematic reviews provide accurate and reliable evidence. By carefully evaluating key domains, researchers can minimise misleading conclusions and enhance the quality of evidence-based decisions.