Skip main navigation

Evaluate your training

Evaluate your training plan
© Wellcome Connecting Science

In Week 1 you were introduced to evaluation theory and processes and the need to develop a plan as part of course design. Kirkpatrick’s model, Step 1.21 was described as most widely used evaluation framework to evaluate the efficacy of training in short, medium and long term. An example of how this can be applied is shown in table below:

Evaluation plan:

The need (purpose) for evaluation Sample Instruments for data collection Methods, data analysis
Measuring impact (social utility), answer to stakeholder; future planning and allocation of resources; informing further decisions on course/programme Course participants, including trainers, sponsors Surveys, interviews, online forums, learning analytics, observation notes Mixed – Quantitative and Qualitative

Evaluation plan overview:

Evaluation questions Source Methods and format Frequency
What were the participant’s reactions to the course? Participants Mid-term and Post course survey; ad-hoc feedback requests during the course During the course and at the end of course
To what extent did participants acquire the intended knowledge and skills Participants Pre and Post course survey, Assessment or test Once pre and post course
To what extent did participants translate the knowledge and skills into practice Participants Follow-up survey, interviews; Stories or interviews with selected participants Once a year for 2 years
To what extent did the course impact on participants’ work, behaviours, practices, environment? Participants Follow-up survey, stories, interviews, case studies; Stories or interviews with selected participants Once a year for 2 years
Success of training in achieving the goal of the programme? Implementing at home institutions? Adoption in learners’ institutions/labs Participants, sponsors/ supervisors Follow-up survey, stories, interviews, case studies Once a year for 2 years

In the WCS Courses evaluation model for short, 1-week long genomics and bioinformatics courses, post-course feedback is the main form of evaluation of the learner’s satisfaction with the course, although feedback is solicited ad-hoc during the week through various check-ins and interactive platforms. The post-course feedback is collected using an online survey and asks participants about course organisation, course content, what participants most enjoyed in the course, and any suggestions for changes or improvements. While face-to-face courses may ask for feedback on venue, facilities and other logistical issues, feedback on use of virtual learning environments (VLE) and access to resources should be requested. You can find a PDF attached below with example questions for a short course feedback survey. After each question, a free text box invites respondents to explain their responses further.

In addition to post-course feedback from participants, feedback should also be obtained from the instructors or training team. This will be useful in further understanding the issues arising and also taking necessary action for future development.

To determine the impact the course has had on participants, a follow-up survey is conducted 6-12 months post-course. The focus is to understand whether and how participants had applied their knowledge in their research or professional practice. Example questions for the long term follow-up survey are listed below in the same pdf attached. Please note these surveys’ data are anonymised, data is treated safely and securely. The surveys are designed for collecting generalisable information primarily related to career progression, research outputs such as publications and contribution to building capacity and expanding reach as well as for the funding stakeholders.

Various steps during this week have demonstrated the implementation of the Train the Trainer course on Genomic pathogen surveillance. As discussed in the case study on Train the trainer courses Step 2.4, that Train the Trainer course was part of several integrated activities for a global capacity building programme supported by a funding organisation over a defined time frame. To meet the goals of the programme and expectations of the funders who are key stakeholders, a detailed report of outcomes and impact was important. Therefore in addition to the post-course feedback and the general follow-up survey, a more rigorous approach was applied to determine the impact especially in the context of all other factors in learners’ local environments, such as institutional support, funding, continued practice and collaboration. This provides a more specific follow-up survey than the generalised version described above. For this specific follow-up survey, participants responded anonymously. However, further consent was obtained for re-contact and in-depth case reports. Example questions for this type of survey are also included in the pdf attached.

© Wellcome Connecting Science
This article is from the free online

Train the Trainer: Design Genomics and Bioinformatics Training

Created by
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now