Skip to 0 minutes and 7 seconds Was your programme a success or a failure? Did you achieve the outcomes that you set out to achieve? Did the programme deliver value and benefits to the stakeholders? Or did you have an impact on outcomes that you didn’t even set out to? And if you did do this, what was it about your programme that led to this impact? If your programme is ongoing, how can we improve it to be more effective? Or if it has a time limit, how can you show it was successful to advocate for the sustained funding to continue it?
Skip to 0 minutes and 37 seconds Or if you were effective at achieving outcomes, are there aspects of other programmes that are more effective, that you can learn from and incorporate into your own programmes? These are important questions, and without monitoring and evaluating your programmes are nearly impossible to answer. Although monitoring and evaluation are often spoken of as one phrase, they are two distinctly separate concepts. Monitoring generally refers to the routine and often ongoing collection of information of a programme, on how it is progressing towards achieving planned results. Specifically, monitoring is about measuring change, whether intended or unintended as a result of the programme. In most social programmes, it is about measuring changes in behaviour.
Skip to 1 minute and 18 seconds While evaluation on the other hand, is the periodic assessment of the outcomes and impacts of the programme, and it measures the success of the programme in achieving the overall goals that it set out to. Or more generally, evaluation is about working out the value of the programme to its stakeholders or donors. And to demonstrate the link between the two, you first need to understand what you’re going to evaluate before you decide what you’re going to monitor. There are three primary ways that monitoring and evaluation provides value to organisations. The first is learning what works on your programme, what doesn’t, and ultimately, improving your programme design.
Skip to 1 minute and 58 seconds The second is about demonstrating accountability for the resources you are using, for example, demonstrating accountability for the funding you might have received from a donor. And the third value of monitoring and evaluation is using results to advocate for continued funding for programmes sustainability, or even increased funding to expand programming. Monitoring and evaluation is often oversimplified by programme staff, and conducted as a simple tick box exercise to keep donors happy. Despite this, monitoring and evaluation needs to be carefully planned to ensure clarity in what you’re trying to achieve. You are collecting the most important information for your programme and that you have the financial and human resources to do so.
Skip to 2 minutes and 44 seconds There is a wide range of tools methods and monitoring and evaluation designs that can be used to capture impact. These range from the very simple, to the highly complex. Briefly, these can include interviewing, observation, written participant reflection, and other written and recorded data, which we referred to as qualitative methods. Or health assessments and surveys collecting number data that can be counted, which we refer to as quantitative methods. As an example of simple monitoring, a financial donor may want to know how many participants attended each session of your sport development programme, an a sign in sheet each week can capture this.
Skip to 3 minutes and 21 seconds Or more complex, you might want to know if your programme has led to a sustained behaviour change either health or social amongst the population in which it was delivered. This would require rigorous data collection using both qualitative and quantitative methods. And data collection will likely need to be from a group of participants, on multiple occasions, over an extended time period. Designing a plan on how you will monitor and evaluate your programme, commonly referred to as a monitoring and evaluation framework or plan is only the beginning. The reality of implementing it is very different and the challenges of conducting monitoring and evaluation are numerous.
Skip to 4 minutes and 2 seconds Aside from ensuring that your monitoring and evaluation plan will collect the information most useful or important for the programme, it’s also important to ensure that your monitoring and evaluation plan is feasible given your budget, human resources and expertise. The most common misconception of monitoring and evaluation is that it happens after a programme is started or even finished. The reality is that effective monitoring and evaluation needs to be designed and data collected, even before the first participant joins in your programme. Everyone involved in a programme is usually involved in monitoring and evaluation in some capacity. Imagine you are Sport for Development financial donor, and it was your money being spent on the programme.
Skip to 4 minutes and 50 seconds What would you want to know at the end of that programme about what you got for your money? Then ask what do we need to monitor to answer that question? Also, you need to remember that monitoring and evaluation is about working out what has changed as a result of the programme. So you first need to understand the situation at the beginning. We call this the baseline. Unless we understand the baseline, we really can’t work out what has changed and by how much.
What is monitoring and evaluation?
Although monitoring and evaluation (M&E) are often used as a single phrase, they refer to two distinctly separate concepts. What will be evaluated also needs to be identified before what will be monitored, with both concepts part of a continual review process.
Monitoring refers to the tracking of key programming elements and the routine collection of information. Evaluation refers to the assessment and interpretation of this information. Together monitoring and evaluation form the foundation of the ‘review, reflect, learn, adapt’ process, introduced as part of the theory of change approach.
Most importantly, monitoring and evaluation together help you to identify when things aren’t working, and identify them quickly! It enables you to learn from this information to improve your initiative and make changes to the design or implementation, which in turn strengthens your programme.
Stages of evaluation
Evaluation is the process of ongoing reflection and assessment against achievement of goals. Measuring change is a gradual process, necessary throughout your initiative, to make sure change is happening in the right direction to achieve your goals.
There are three stages to the evaluation cycle:
- Design evaluation: formative insights to help design your initiative (improve)
- Implementation evaluation: process insights to monitor and track progress
- Impact evaluation: summative assessment of impact and outcomes (prove)
More information about the formative (improve) and summative (prove) stages of evaluation can be found in the following Monitoring and Evaluation handout.
You might have already realised that you’ve been considering M&E aspects from the beginning of week 1, in particular during development of your Theory of Change. This is no coincidence. Working backwards from an identified goal and aligning activities to achieving this goal has M&E in-built as all these tools are themselves aligned to help you achieve your goal and identified value to stakeholders.
Capturing the impact of your intervention
Impact is a demonstrated contribution of the initiative to a community, creating change(s) to bring benefits to the economy, society, culture, public policy or services, health, the environment and/or quality of life.
Four types of impact to consider:
- Instrumental impact: influencing the development of policy, practice or service provision, shaping legislation, altering behaviour;
- Conceptual impact: contributing to the understanding of policy issues, reframing debates;
- Social impact: contributing to the change in attitudes, perceptions and behaviours;
- Capacity building impact: through technical and personal development.
What impact are you hoping to achieve with your initiative and what will you need to monitor to demonstrate this change? Share your current thoughts with others on the discussion board.