Introduction to evaluation frameworks

In this step, we move towards designing an evaluation by creating a framework which outlines what we are trying to evaluate and why. The Evaluation Principles provide important context to this activity.

In the previous activity, we explored the principles of good evaluation for cultural activities, to ensure that our work is:

• beneficial – contributing to positive change;
• robust –  accurate and as unbiased as possible;
• people-centred – considerate of all involved; and 
• connected – properly shared.

Why use a framework?

As discussed earlier in the course, cultural activities, projects and programmes have complex impacts – and capturing them can be a complicated business. Having a strong framework (some people call it a “model”) that is well-understood by all those involved can help you manage that complexity and stay focused on your end goal.

More than a method, a framework is a map showing how a project will make an impact or drive change. Usually expressed as a flowchart or diagram, it plots the aspects of the programme to be evaluated, the relationships between them, and their intended and actual value. You can then overlay data collection and research methods and check that the Principles have been considered at each stage.

An evaluation framework can help you to:

● visualise, summarise and organise the key elements of evaluation

● build consensus on what success looks like and why

● involve other colleagues and stakeholders in a coherent, people-centred and transparent way

● design relevant approaches to research, data collection and reflection

● identify key gaps in your method or data

● prioritise the resources available for evaluation

● track the progress of your evaluation and adapt the design if necessary

Developing a framework is the best starting point for an evaluation but also the map you can return to as a project progresses. 

Cause and Effect, Inputs to Impacts

Essentially, frameworks tend to follow a so-called “logic model” and seek to express and understand a cause-and-effect relationship. They can be used both to aid strategy – we will do X so that Y happens or changes – and/or evaluation – was X successful in making Y happen? Why? 

The logic model traces the chain reaction between resources/inputs, activities, outputs and outcomes/impacts. One of the most challenging aspects of crafting an evaluation is identifying, describing or agreeing the kinds of value you want to capture. It is worth taking some time to do so, working with as many stakeholders as you can manage to complete a logic model that reflects everyone’s understanding of intended outcomes and other critical aspects of a project.

The following diagram illustrates how a logic model works:

Click here to zoom the image

View accessible description

Example

Refer to the Example Model – A 5-step framework document attached in the downloads section below.

The logic model isn’t the only model available. In the following step we’ll explore various other types of models that have been refined for use in particular circumstances and adapted as useful frameworks for cultural evaluation.