How Can Trainers Use Online Assessment?

As with all forms of assessment, online tests should measure the achievement of learning outcomes. If any outcomes cannot be measured by computer, some other form of assessment should be employed to ensure competence and patient safety.

Online assessments are relatively easy to create, and most modern eLearning platforms offer question designs in a variety of formats which can be marked by computer. For example,

• Short, free text answers
• True/false
• Multiple choice
• Select the missing words (from a list)
• Fill in the missing words (free text or first letter supplied).

Questions can also be created which require learners to provide answers in their own words. This type of question requires the learner to recall the answer from memory and is a reliable indicator of retention. Free text answers are marked by a tutor, so this design is necessarily limited to small group numbers.

Artificial Intelligence (AI) systems are in development that can mark free-text answers. This technology is likely to transform online assessment in the near future.

Many organisations rely on Multiple Choice Question (MCQ) tests for online assessments. It is important that they require thought and retrieval of knowledge to pass. The MCQ tests we have used on this course have been designed this way. For example, this question from Week 2:

How can we tell if we have learned something in a lasting way?

a) We recognise key sections of the material when we see them again
b) We can retrieve information in our own words after a period of time has passed
c) We have memorised key phrases and are able to repeat them later
d) We understand what we have read or been told.

Online Assessment with Simulation

What Should the Pass Mark Be?

If the pass mark is less than 100%, which bits of content are you happy for learners not to know?

If you are happy for lower pass mark, then why not remove the non-essential content, and ask for 100%?

Cheating; surely not?

Are all Online Assessments Adequate?

Describe how to use the organisations’ fire safety equipment

What are fire blankets for?
a) To keep people warm after they’ve been rescued from a fire
b) To cover food with to keep it warm
c) To smother a cooking fire and starve it of oxygen
d) All of the above

To what extent do you think this question assesses the outcome?

The incorrect answers were presumably intended to inject a sense of humour, or perhaps their author couldn’t think of any valid alternatives. Answer choices such as these don’t measure outcomes. At their worst, they can give an impression that the organisation does not take training seriously.

We would stress again that assessment should measure outcomes. There are no learning outcomes or real-life healthcare scenarios that require a professional to identify one correct phrase from three ridiculous alternatives. There are, however, occasions when we are required to make a difficult diagnosis from signs and symptoms which at first glance, could have more than one possible cause.

Talking Point

Consider sharing you experiences with online learning with other course members, and the answers to these questions:

• What has been your experience of MCQ tests? How well have they measured course outcomes?
• Are you involved with any training for which you’ve found it difficult to create a valid assessment?