Evaluation Process Applied to Training – Evaluation Planning
- Determining the need for evaluation and who the stakeholders are
- Formulating evaluation questions: What do we want to know about the course/initiative we are evaluating?
- Choosing indicators that are going to help us observe/measure specific aspect(s) we are looking at
- Collection of the information we need in an ethical manner
- Analysis of collected data in a valid and reliable way
- Sharing the results of the analysis and making decisions about further actions
Evaluation PlanningEvaluation of teaching and learning will be similar whether it is the evaluation of a face-to-face, hybrid or some form of digitally based learning and teaching. To make the evaluation equitable (i.e. just and fair to all involved), analysis of all the stakeholders’ needs and attributes should be undertaken and considered throughout the planning and implementation of the evaluation. Defining what we would like to know about a specific course, i.e. formulation of the evaluation question(s) is part of the planning. The scope and type of evaluation will depend on the questions we are asking. To help us define the questions, it is sometimes useful to represent the course/initiative in terms of inputs, activities, outputs and short-to-long term outcomes (this is called a logic model). The formative evaluation would then look at the input, activities and immediate output of the course, while summative evaluation would cover short, medium- and long-term outputs. Examples of formative evaluation questions asked can be related to how the course is implemented (competencies covered, use of certain resources, learners’ participation, trainers/facilitators roles, pedagogical approaches used, innovation applied, intended learning outcomes etc). Examples of summative evaluation questions are usually related to the course/initiative’s short, medium and long terms outcomes, including looking at the barriers that might have prevented the intended outcomes from happening. For example, when the educators designed the evaluation for the course you are currently following, our overarching evaluation question was about the ways and the extent to which this course influences learners’ practice and/or careers, propagates to the local communities and contributes to the organisational/institutional changes.
Want to keep
Wellcome Connecting Science online course,
Train the Trainer: Design Genomics and Bioinformatics Training
- Reaction of learners – their experience during the course
- Learning (including skills and changes in attitudes) – the increase in knowledge resulting from the course attendance
- Behaviour – the application of learning in employment and/or other context, such as study or career
- Results – the broader impact the learning has on the learners’ wider contexts, immediate community or organisation.
Data collectionSome of the typical qualitative methodologies used in evaluation of teaching and learning include case studies, ethnographic/cyber-ethnographic approaches, with data collection methods such as open question questionnaires and surveys, interviews, focus groups, observation, historical texts, reflective texts, visual artefacts and more. Quantitative methodology sometimes uses a quasi-experimental methodology, that tries to measure the effect of a ‘treatment’ in teaching. Usual data collection methods used in quantitative design include scales, such as a Likert scale rating how much people agree with statements, and surveys or questionnaires that use closed questions or open questions that can be quantified. Surveys are nowadays often conducted online via services such as SurveyMonkey, Google Forms, SmartSurvey. Institutional or openly available statistical data can also be source of quantitative data. In collecting data, blended and digital learning have the additional option of digital data collection through learning analytics, learners’ discussions forums or comments areas such as the one used in this course. Probably the most used ways of data collection are surveys or questionnaires and scales. Please find attached more tips on both at the end of this step. Data collection requires an ethical approach. For example, when collecting data about the learners, their rights to privacy, security and confidentiality, such as confidential and anonymous treatment of participants’ data, secure data storing, compliance with the relevant data protection laws, especially if you are working internationally, need to be respected. This includes observing learners’ right to withdraw their data at any point during the evaluation and this is an important principle of informed consent.
Data analysisThe method of analysis will depend on the scale and type of data. For qualitative data, thematic analysis is often used. Qualitative analysis requires triangulation in data sources, methodology and individuals who analyse it, so that credibility and confirmability of the results can be achieved. The analysis can be done manually on smaller scale data, but with larger sets, software can be used with the coding process. Quantitative analysis includes statistical approaches – often in the form of descriptive statistics, but more advanced statistical analysis can be provided using different software widely available. Risks and unintended consequences of every evaluation process should be considered or allowed for. Many contextual factors can affect the evaluation process and results, so the more factors are considered, the more valid and reliable the results will be. Finally, the results of the evaluation are normally shared through appropriate forms of dissemination with the majority if not all of the stakeholders involved. This course’s evaluation will use mixed methods to analyse data collected via FutureLearn analytics and metrics, end of course and follow-up surveys as well as the participant observation notes and feedback.
Train the Trainer: Design Genomics and Bioinformatics Training
Our purpose is to transform access to education.
We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.
We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.