Why Are Randomised Controlled Trials Important?

Vector control interventions are the backbone of many disease control programmes, but there are repeated problems with vector control studies. In this article we summarise the findings of a recent article: Wilson, A.L., et al. (2015) Evidence-based vector control? Improving the quality of vector control trials. Trends in Parasitology. Vol 31 Iss 8.¹

The development of a new vector control tool follows a defined pathway from concept and testing, through laboratory assays (Phase I), semi-field tests (Phase II), field trials (Phase III), and pilot implementation (Phase IV). The WHO uses evidence from the Phase III and IV trials to make policy recommendations. The quality of evidence is rated using the Grading of Recommendations Assessment Development and Evaluation (GRADE) methodology. Unfortunately, vector control interventions are frequently lacking high quality research to support their recommendation in disease control programmes. It is therefore important that future studies on Zika and other vector borne diseases are well designed. Figure 1 shows the flow of activities that are involved in designing the trial through to outcome analysis and reporting.

Figure 1. Randomised controlled trial flow chart.

The importance of randomisation

Randomised Controlled Trials (RCT) are considered the gold standard in study design. Randomisation is important because if done properly it will minimise selection and other types of bias. Vector control tools will often work at the community level (for example, long lasting insecticidal nets can help protect everyone in the area, not just those sleeping under the nets), and in these cases it is important that randomisation takes place at the appropriate group or ‘cluster’ level, whether this is household, village, or district.

Studies involving interventions at the individual level should also be randomised, but they can have problems of contamination; if different individuals in a community are given different interventions, such as insect repellents or repellent clothing, any sharing of these products with other individuals would interfere with the study results. It is therefore often preferable to allocate the intervention to individuals in a cluster, then randomise the clusters.

It should be noted that while RCTs are rated as high-quality evidence by the GRADE methodology, studies can be up- or downgraded based on several factors: RCTs can be downgraded if there is risk of bias, inconsistency, or imprecision for example, and a non-RCT could be upgraded if a large effect size is observed. Nonetheless, non-randomised control trials are not recommended as selection bias is likely to be high and there are commonly no pre-intervention data to assess the comparability of test groups. Observational studies, such as case-control or cross-sectional studies, provide relatively weak evidence to support the effect of vector control interventions and rank below non-RCTs (Figure 2).

Figure 2. Hierarchy of study designs for assessing the efficacy of vector control interventions.¹

Weaknesses in vector control studies

A common problem in vector control studies is with implementation and adherence to the intervention. If an intervention, such as the use of repellents or nets within the home fails to have an effect, it is difficult to know whether this is because it did not work as expected or because it was not used as recommended. Therefore, it is important to ensure high coverage and monitor user compliance during the trial. Some studies use education and communication aimed at behaviour change to encourage user compliance² or may employ random spot checks, and supervision³.

Implementation and adherence to the intervention	Study designs should focus on quality control, high coverage and user compliance. Without these it will not be possible to determine the reasons behind any apparent lack of efficacy.
Choice and measurement of outcome measures	Entomological outcomes can demonstrate proof of concept but epidemiological outcomes are needed to demonstrate the efficacy of an intervention in protecting people.
Avoiding performance bias	Where possible, blinding of trial participants, health-care workers and others involved in a trial should be practiced to avoid introducing biases.
Selection of sites for entomological monitoring	Sites should be selected in a consistent way across intervention and control arms of the study; random selection avoids sampling bias.
Contamination or spillover effects	Clusters, such as villages, schools or households, should be well separated by a buffer zone to ensure there is no common boundary between the intervention and the control groups
Need for sample size calculations	Sample size calculations should be performed before conducting a study to confirm that the study has the power to show an effect.
Duration of the follow-up period	Follow-up periods need to be sufficiently long, with repeat measures, to gain an accurate assessment of the intervention effect. For RCTs this is likely to be at least one transmission season.

Table 1. Summary of vector control trial recommendations.¹

The outcome measure is another area where vector control trials can fail. Entomologists can focus on entomological outcomes such as reduction in mosquito numbers without adequately examining whether there has been an impact on disease transmission or human health. Entomological end points are not always good predictors of epidemiological outcomes. Therefore, it is extremely important that appropriate epidemiological outcomes should be used in Phase III trials, whether it is incidence of disease or disease-specific mortality, or the prevalence of infection in blood samples.

Entomological monitoring should, however, still be undertaken and sites should be chosen randomly. Historically, entomological monitoring takes place in sites already known or suspected to have high densities of vectors, and this can introduce sampling bias, so random selection is recommended. Further, detection bias can be a problem in vector control studies, particularly in trials where staff distributing the intervention are also collecting the outcome data. Ensuring all staff are blinded to the randomisation, as far as possible, can address this but for some interventions, such as using screens in homes, it is not always possible.

Contamination or spillover effects occur when there is movement of vectors or people between the study arms. This can dilute the intervention effect, leading to findings of lower effect or even no effect. However, some interventions lend themselves to the opposite effect, and can exaggerate the power of the intervention. For example topical repellents can divert mosquitoes to non-users thereby potentially increasing their risk of infection. Minimising this effect can rely on ensuring clusters are well separated or using a buffer zone, limiting enrolment within an area to avoid problems such as diversion caused by repellents.

Interventions that target outdoor biting mosquitoes such Aedes aegypti are particularly difficult, as people are bitten going about their daily activities. Larger cluster sizes and the use of travel histories can help in these cases to minimise this type of bias. Contamination is also an issue in crossover trials, in which individuals or clusters receive the intervention or control treatment for a set period of time then switch over to receive the control or intervention. If an intervention is persistent in the environment it will be difficult to escape the long-term effects it is having. This type of trial may not be suitable at all for some insecticides or for interventions that change habitats.

All trials should have a sample size calculation to ensure they are sufficiently powered to detect the intervention effect. Parameters such as disease incidence in the control group may be unknown and will therefore have to be estimated using baseline surveys before the start of a trial. Additionally, follow-up periods need to be carefully chosen based on the length of the transmission season and the ability to detect new infection within the follow up period.⁴ Follow-up periods should also take into consideration changes in user compliance: early on in a study individuals may show strong compliance but this is likely to fall as the trial continues.

In conclusion

Assessments of vector-borne interventions that are not rigorous and evidence-based can waste a considerable amount of money, time, and energy. Given the lack of vector control studies with epidemiological outcomes, and the development of new vector control tools for the control of Aedes mosquitoes, it is crucial that future studies are designed to provide high-quality evidence. Have you taken part in a randomised or non-randomised trial? Are you currently working on a trial? How do you think your study would rank in the GRADE methodology?