Skip to 0 minutes and 12 secondsWelcome. By the end of this presentation, you should be able to define monitoring, explain process, outcome, and impact indicators and relate them to the level they are collected at, and explore the role of managers in monitoring.
Skip to 0 minutes and 30 secondsIn this stage of the planning cycle, planners consider the implementation of the project. Step five is the monitoring phase. This is planned in detail at the beginning of the project and then managed throughout the project lifespan. Imagine that you have planned a journey from point A to point Z. In the plan, you know what car you will use, the routes you will travel, how many people will be travelling, and the amount of fuel needed for the journey. You also have an estimate of the time for travelling. Once the journey starts - the implementation of the plan - every few hours, you look at the fuel gauge, check the distance travelled, and perhaps even ask how the passengers are doing.
Skip to 1 minute and 12 secondsThis is known as monitoring. You may have planned to assess this information at several points along the way. Here, you decide if the car needs to be refuelled or if a break is necessary to rest the passengers. Information on fuel, distance, and how the passengers are doing are referred to as indicators. Indicators tell you how the journey is going.
Skip to 1 minute and 38 secondsMonitoring is the continuous surveillance of the implementation of a programme or project. Monitoring activities check if a project is proceeding according to the plan. Are you doing what you said you will do? Monitoring is important for a number of reasons. What gets monitored is more likely to get done. If you don't monitor performance, you can't tell success from failure. If you can't see success, you can't reward it. And if you can't recognise failure, you can't correct it. Finally, if you can't demonstrate results, you can't sustain support for your actions.
Skip to 2 minutes and 18 secondsIn any plan or programme, achievement is aligned with completion of objectives and there are usually several activities that need to be carried out for an objective to be achieved. Each activity requires inputs, such as finance, resources, et cetera, and for specific tasks to be completed. This is known as the process. As a result of process, activities and objectives are completed. And this, in turn, leads to an outcome and an impact.
Skip to 2 minutes and 48 secondsIndicators measure what or how much has been done. Process indicators provide information on tasks done and inputs consumed as part of activities to achieve objectives. Process indicators are collected regularly on a weekly, monthly, or quarterly basis. Outcome indicators are used during a project to assess if the path taken is working well and if changes need to be made to the plan's objectives. These are collected over longer intervals, once or twice a year. An impact indicator is an indication of change that has resulted from a plan. Impact indicators are collected and reported on at the end of a project, after objectives have been completed.
Skip to 3 minutes and 33 secondsIn any programme or plan, it is important to decide what information should be collected and when. Planners must decide on what could be monitored, what should be monitored, and what must be monitored. For example, a trachoma endemic district, Zrenya, develops a plan which aims to eliminate trachoma in the district using the SAFE strategy by 2020. One of the objectives is to eliminate the trichaisis backlog of 1900 cases in 3 years. The unit decides to carry out several activities to achieve this objective. Two of the main activities are to establish bi-monthly outreach services and to provide high quality TT surgery for the district.
Skip to 4 minutes and 16 secondsTo check that these two activities actually happen and to manage their progress, managers regularly collect data on several process indicators, such as the number of outreach operations done per month and the outcome of surgeries by surgeon on follow up. To see if activities actually make a difference, managers review the data and calculate an outcome indicator, in our example, the number of trichiasis surgeries done each year. This lets the manager assess how much of the backlog has been addressed. Impact indicators are more complicated and may require a new population-based survey to be carried out at the end of the implementation phase of the project.
Skip to 4 minutes and 57 secondsHere are some examples of process indicators: number of trichiasis patients operated per month, post-surgical trichiasis per surgeon per outreach, number of patients refusing and accepting surgery per month, outreach expenditure and staff salaries per month and number of outreach activities per month.
Skip to 5 minutes and 18 secondsHere are some examples of outcome indicators: the percentage of the trichiasis backlog addressed at the end of each year, the percentage of post surgical trichiasis in the district, and the trichiasis surgical acceptance rate in the district. The key impact indicator that the programme will aim for is reducing the TT backlog to less than 0.2% prevalence in adults aged 15 years and above.
Skip to 5 minutes and 46 secondsGolden rules for monitoring: - Do not collect too many monitoring indicators or collect indicators too often. - Use all the monitoring indicators you collect and discard indicators that aren't used. - Use the monitoring indicators at the level that they are collected at, as process, outcome, or impact. - Educate staff about the need to collect monitoring indicators. - And don't make things worse. Don't destroy a monitoring system that works.
Skip to 6 minutes and 18 secondsManaging monitoring: Planners need to decide who collects the indicators at each level. Once the data has been collected, where are the reports sent and who will review them? And how will the programme act on the feedback from the review? These are key details that must be managed by the programme or project manager. Appropriate selection and training of key people to carry out monitoring is essential. Let's return to our example, Zrenya District. In order to monitor activities towards the objective of eliminating the TT backlog in three years, the outreach trichiasis surgeon must send back a report on the number of outreach surgeries that are being done each month and the follow up outcome per surgeon per month.
Skip to 7 minutes and 2 secondsThese reports are collected and reflected on by the manager on a regular basis. It is important that the district manager gives feedback to staff about how the plan is doing. The district manager reports to the national coordinator. At the end of the programme period, it is the final responsibility of the national coordinator to identify resources and address the impact of the programme. Collecting the right data is important to guide the programme and its success. There is a saying that if you put rubbish in, you get rubbish out, so planners must remember to select indicators properly. In summary.
Skip to 7 minutes and 41 secondsMonitoring is important because it, improves accountability for the use of funds and resources, improves performance to achieve outcomes, and provides a system for recording lessons learned. These can then be shared to improve strategies in the future.
Principles of monitoring
Monitoring is the ongoing analysis of the progress of a programme using a pre-planned management system.
A monitoring framework is always set up when a programme is planned. The monitoring framework sets out the inputs, outputs, outcomes and impacts of the project.
Table. Monitoring framework elements: Examples for trachoma elimination programmes
|Inputs - what is invested||Staff, volunteers, time, money, materials, equipment, technology, partners|
|Outputs - activities and who we reach||Conduct outreach surgeries
Train health workers for MDA
Develop information, education and communication (IEC) materials
Reach school children
Reach trichiasis patients
|Outcomes - what happens during the programme||Reduce the trichiasis backlog
Quality of surgery provided
Coverage of MDA
Coverage of IEC activities
Increase in latrine use and reduction in open defecation
Improved access to water supply
|Impact - did we reach the intended goal?||Level of trichiasis backlog in the treated population
Level of trachomatous inflammation-follicular (TF) amongst children in the treated population
As you watch this video consider what are the possible internal and external factors that can affect the process indicators in trachoma elimination.
© London School of Hygiene and Tropical Medicine CC BY-NC-SA