Skip main navigation

Research example: impact and process evaluation in the Maldives

TBC
11.3
This week, you’ve learned about impact evaluations and process evaluations. Let’s look at our study in the Maldives in more detail. Here, we used mixed methods to evaluate a social protection intervention designed to support people with disabilities. In the Maldives, people with disabilities can receive a monthly cash benefit called the disability allowance. It is a fixed amount of about $130 and it has not changed since the intervention was introduced in 2010. In order to see what changes, if any, were needed to improve the intervention, the Maldives National Social Protection Agency commissioned an impact evaluation. This was to detect, if, indeed, that the allowance was having an impact on the lives of people with disabilities.
59.3
To explore this, we conducted a process evaluation and then an impact evaluation. As you’ve heard in the last step, a process evaluation is a study designed to understand how an intervention functions. This looks at how the intervention is delivered, by whom, at what cost. It also looks at contextual factors that may be affecting the delivery and uptake of the intervention. In the Maldives study, we conducted a process evaluation to understand the disability allowance programme by collecting data through document reviews and in-depth interviews with implementers, stakeholders, and beneficiaries.
96
In collaboration with programme implementers, we also developed a theory of change, which is a map of how the intervention is expected to have an impact on beneficiaries– in this case, people with disabilities. Here, you can see a portion of the theory of change we developed for this study. The boxes in red indicate the intended outcomes of the programme– reduced poverty, improved health, participation, and quality of life. The other boxes indicate other processes that are expected to lead to this. We then used this information to design the impact evaluation. We used quantitative and qualitative methods in parallel, to explore whether the components depicted here worked as anticipated to produce an impact.
144.2
You can read more about the methods we used in the lesson on mixed methods in week 1. We knew from the process evaluation that the disability allowance was designed to improve beneficiaries’ functioning and participation by giving them access to assistive devices. However, the impact evaluation data showed that this did not always happen. That is, allowance beneficiaries were not always referred to the programme that provided assistive devices. This meant that the intervention did not improve beneficiaries’ participation as intended. We also knew from the process evaluation that the intervention was expected to improve people’s access to services.
184.8
The impact evaluation showed that, because disability-related services were scarce in the country, people were spending vast amounts on travel abroad or to the city, and the allowance was not enough to cover it. Lastly, the disability allowance was expected to protect beneficiaries from poverty by providing them with the monthly cash benefit with no conditions on how it was spent. However, the impact evaluation revealed that the monetary value of the allowance was insufficient to push people out of poverty and cover the high costs related to disability. Based on these findings, we were able to develop recommendations about how to improve the effectiveness of the intervention. We disseminated these findings in a number of ways.
235.9
We held in-depth discussions with the implementers to explain why their intervention had low impact. Because we had developed the theory of change with them, it was easy to communicate to them which processes need to be strengthened in order to improve the impact of the intervention. We conducted a series of small seminars throughout the country. We found that this was more effective in reaching various different audiences than having one major seminar in the city. We were able to reach study participants, community members, local government, DPOs, government agencies, international NGOs, academia, and policymakers. We also developed communication material like infographs, making versions in English and in the local language. This ensured that the important figures could be disseminated widely and easily.
287.9
In summary, with this case study, you should now be able to recognise how a social protection intervention can be evaluated, understand how a process evaluation can complement an impact evaluation, and appreciate a few ways of disseminating findings.

In the last two steps, you heard about impact and process evaluations. In this step, Dr Shaffa Hameed will share an example of a research project in the Maldives that assessed the impact of the disability allowance on the lives of people with disabilities, as well as examining the process of implementation.

Dr Hameed is a qualitative researcher at the International Centre for Evidence in Disability, and originally comes from the Maldives.

As ever, please share your thoughts on this step below. What did you think of the study findings? Was anything surprising or unexpected?

This article is from the free online

Global Disability: Research and Evidence

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education