Evaluation Approaches

I just want to reinforce again, though, that every objective you write has to be evaluated. What the people know? What changed? What attitudes have changed? What environmental conditions have changed? All of these relate to the objectives. So each objective has to have an evaluation strategy associated with it.

And it may be that you need several ways of evaluating something to be able to determine: has the objective actually been achieved? So for example, if you are a community worker trying to set up social support group for mothers to reduce the incident of loneliness and depression, we could have an impact evaluation that assess the changes made to social support. And you’ll see below that I now have another term, and this is outcome evaluation. And it may be that later on down the track, we do an assessment to see the changes in the prevalence of depression. Prior to the intervention, we may have assessed level of depression in the community.

We may have been done the intervention, made an assessment about changes to social support and then seen whether those changes to social support create a long-term change in depression. And that is one of the key elements around outcome evaluation because it often takes a longer time to get that evaluation measure. So sometimes, it can be difficult to do because you may have a budget that’s constrained or you may not have the resources to track the long term outcomes as a consequence of your intervention. There are also sometimes other factors, confounding factors, that may influence the effects or dilute the effects or perhaps of unintended consequences on the effects.

Perhaps in your community, there were other things that were occurring that you actually might not be able to control for. So just be aware that when you’re constructing your evaluation process, it’s absolutely critical to have impact evaluation. But also you need to understand, from the goal of your project, what are the outcomes you’re trying to achieve? And then consider what measures you might be able to take up? To actually show that your programs had an impact. Oh, sorry. I said shaped its outcome.

So these are just some of the things that you might want to consider, for example, the reduction and prevalence of a disease, the increase in physical activity, reducing smoking rates, or improvements of quality of life could all be some of the big outcome measures. Now when we’re selecting the approaches we’re going to use, we have to have real consideration to the literacy of the target group. Are the methods culturally appropriate? So, for example, I’ve worked with communities, and if I gave some of those communities a pen and a sheet of paper and asked them to fill in a questionnaire, they would automatically start to feel they were in an exam and they would feel threatened by that process.

Where there are other communities, where the use of yarning or storytelling is a way in which knowledge is transmitted. And so you would need to think about the approach that you would take in relation to the context and the community you’re working in. You know, also consider the skills and experience that the professionals that you are going to be doing the implementation. Are they able to collect focus group data? Are they able to do interviews? Are they’able to administer questionnaires effectively? And then the final bit on this might be what is that you’re required to do with that data?

For example, if you’re part of a randomized control trial, you need to have very definitive data and survey information, questionnaires at the end that allow you to distinguish between the pre, post, or the control and the intervention groups.