Evaluate your training
In Week 1 you were introduced to evaluation theory and processes and the need to develop a plan as part of course design. Kirkpatrick’s model, Step 1.21 was described as most widely used evaluation framework to evaluate the efficacy of training in short, medium and long term. An example of how this can be applied is shown in table below:
Evaluation plan:
| The need (purpose) for evaluation | Sample | Instruments for data collection | Methods, data analysis |
|---|---|---|---|
| Measuring impact (social utility), answer to stakeholder; future planning and allocation of resources; informing further decisions on course/programme | Course participants, including trainers, sponsors | Surveys, interviews, online forums, learning analytics, observation notes | Mixed – Quantitative and Qualitative |
Evaluation plan overview:
| Evaluation questions | Source | Methods and format | Frequency |
|---|---|---|---|
| What were the participant’s reactions to the course? | Participants | Mid-term and Post course survey; ad-hoc feedback requests during the course | During the course and at the end of course |
| To what extent did participants acquire the intended knowledge and skills | Participants | Pre and Post course survey, Assessment or test | Once pre and post course |
| To what extent did participants translate the knowledge and skills into practice | Participants | Follow-up survey, interviews; Stories or interviews with selected participants | Once a year for 2 years |
| To what extent did the course impact on participants’ work, behaviours, practices, environment? | Participants | Follow-up survey, stories, interviews, case studies; Stories or interviews with selected participants | Once a year for 2 years |
| Success of training in achieving the goal of the programme? Implementing at home institutions? Adoption in learners’ institutions/labs | Participants, sponsors/ supervisors | Follow-up survey, stories, interviews, case studies | Once a year for 2 years |
In the WCS Courses evaluation model for short, 1-week long genomics and bioinformatics courses, post-course feedback is the main form of evaluation of the learner’s satisfaction with the course, although feedback is solicited ad-hoc during the week through various check-ins and interactive platforms. The post-course feedback is collected using an online survey and asks participants about course organisation, course content, what participants most enjoyed in the course, and any suggestions for changes or improvements. While face-to-face courses may ask for feedback on venue, facilities and other logistical issues, feedback on use of virtual learning environments (VLE) and access to resources should be requested. You can find a PDF attached below with example questions for a short course feedback survey. After each question, a free text box invites respondents to explain their responses further.
In addition to post-course feedback from participants, feedback should also be obtained from the instructors or training team. This will be useful in further understanding the issues arising and also taking necessary action for future development.
To determine the impact the course has had on participants, a follow-up survey is conducted 6-12 months post-course. The focus is to understand whether and how participants had applied their knowledge in their research or professional practice. Example questions for the long term follow-up survey are listed below in the same pdf attached. Please note these surveys’ data are anonymised, data is treated safely and securely. The surveys are designed for collecting generalisable information primarily related to career progression, research outputs such as publications and contribution to building capacity and expanding reach as well as for the funding stakeholders.
Various steps during this week have demonstrated the implementation of the Train the Trainer course on Genomic pathogen surveillance. As discussed in the case study on Train the trainer courses Step 2.4, that Train the Trainer course was part of several integrated activities for a global capacity building programme supported by a funding organisation over a defined time frame. To meet the goals of the programme and expectations of the funders who are key stakeholders, a detailed report of outcomes and impact was important. Therefore in addition to the post-course feedback and the general follow-up survey, a more rigorous approach was applied to determine the impact especially in the context of all other factors in learners’ local environments, such as institutional support, funding, continued practice and collaboration. This provides a more specific follow-up survey than the generalised version described above. For this specific follow-up survey, participants responded anonymously. However, further consent was obtained for re-contact and in-depth case reports. Example questions for this type of survey are also included in the pdf attached.
Train the Trainer: Design Genomics and Bioinformatics Training
Train the Trainer: Design Genomics and Bioinformatics Training
