Models of evaluation
In general terms, and similarly to what we do with the assessment of students, evaluations to improve teaching design and your practice are referred to as formative evaluation, while evaluations used in making decisions (for example, for purposes of promotion) are referred to as summative evaluations of teaching effectiveness.
In week 2 we will introduce a new three-step integrative framework to guide you through the process of evaluation. This framework integrates different approaches to academic development and professional development, models of reflective practice and recent applications related to educational data mining and learning analytics. The three-step framework will give you a structure and approach with which you can undertake your mini-evaluation.
As there are many models of evaluation available to help you plan and structure an evaluation, here we introduce you to three different models or resources. Each model or resource offers a different approach to understanding and undertaking evaluation.
The Evaluation Cookbook
A great place to start is the ‘cookbook’ developed by the Learning Technology Dissemination Initiative in Scotland. It is a practical guide aimed at lecturers and educational professionals interested in evaluating materials for their effectiveness in achieving specific learning objectives. The set of ‘recipes’ by evaluation experts, presents a comprehensive review of the methods that can be used as well as example of applications.
The cookbook is available for you to download.
The CIPP Evaluation Model
CIPP Evaluation Model (Stufflebeam, 2004) is a comprehensive framework developed over a decade of research into project evaluation by internationally recognised leaders of evaluation.
The acronym CIPP represents:
- Context - What needs to be done?
- Input - How should it be done?
- Process - Is it being done?
- Product evaluation - Did it succeed? For example: impact, effectiveness, sustainability, and transportability.
Originally the CIPP was used mainly to evaluate the impact of the introduction of learning technology but it is now used across the education sector.
To explain product evaluation in the context of evaluating learning technology:
- impact assesses whether the deployment of ICT facilities in teaching and learning has a direct effect on the lecturers and students, what the effects are and whether other aspects of the system change as a result of the development or implementation
- effectiveness checks whether the programme achieves the intended and unintended benefits, or whether it is effective for the purpose of improved teaching and learning for which it was designed.
- transportability measures whether the changes in teaching and learning and its improved effects can be directly attributed or associated to the deployment of ICT.
- sustainability considers the lasting effects of the ICT deployment on the students and lecturers and how well they utilise and maintain it for teaching and learning purposes.
ALTC Project Evaluation Resource
This evaluation resource developed by Chesterton and Cummings (2011) is specifically designed for:
- learning and teaching, and
- the Australian higher education sector.
This ten step resource provides clear questions to frame each step as well as details of each step. While the resource was originally developed for projects funded by a national body, it is relevant to any proposed evaluation to be undertaken in learning and teaching. The ten steps are:
- Project clarification: What is the nature of the project?
- What is the purpose and scope of the evaluation?
- Who are the stakeholders for the project and the audiences for the evaluation information?
- What are the key evaluation questions which the evaluation will address?
- How will the information be collected and analysed?
- What are the criteria for making judgments about the findings of the evaluation?
- What resources and skills are required to conduct the evaluation?
- How will the evaluation findings be disseminated?
- What is the timeline for the evaluation activities?
- Is the evaluation plan internally coherent and of high quality?
Which of the models for evaluation of learning and teaching, introduced in this step, resonates with you? Name the evaluation model which you prefer and support your choice with a sentence to explain why.
Want to know more?
If you would like to more about this topic on approaches to teaching there are additional resources listed in the Want to know more.pdf for this step.
Chesterton, P & Cummings, R. (2011). Evaluating your project. ALTC Grants Scheme - Evaluating projects, viewed 26 February 2015.
Harvey, J. (Ed). (1998). Evaluation cookbook. Edinburgh: Learning Technology Dissemination Initiative Institute for Computer Based Learning, Heriot-Watt University.
Kirkpatrick, D. L., Kirkpatrick, J. D. & Kirkpatrick, W.K. (2009). The Kirkpatrick model. Kirkpatrick partners.
Stufflebeam, D. L. (2007). CIPP evaluation model checklist. Second Edition. Western Michigan University Evaluation Center.
© UNSW Sydney 2017