Skip main navigation

How Can Trainers Use Online Assessment?

How can trainers use online assessment? Read to learn more.
Someone working at a laptop. The words 'online assessment' is beside the laptop.
© Health Education England Creative Commons 4.0 International

As with all forms of assessment, online tests should measure the achievement of learning outcomes. If any outcomes cannot be measured by computer, some other form of assessment should be employed to ensure competence and patient safety.

Online assessments are relatively easy to create, and most modern eLearning platforms offer question designs in a variety of formats which can be marked by computer. For example,

  • Short, free text answers
  • True/false
  • Multiple choice
  • Select the missing words (from a list)
  • Fill in the missing words (free text or first letter supplied).

Questions can also be created which require learners to provide answers in their own words. This type of question requires the learner to recall the answer from memory and is a reliable indicator of retention. Free text answers are marked by a tutor, so this design is necessarily limited to small group numbers.

Artificial Intelligence (AI) systems are in development that can mark free-text answers. This technology is likely to transform online assessment in the near future.

Many organisations rely on Multiple Choice Question (MCQ) tests for online assessments. It is important that they require thought and retrieval of knowledge to pass. The MCQ tests we have used on this course have been designed this way. For example, this question from Week 2:

How can we tell if we have learned something in a lasting way?

a) We recognise key sections of the material when we see them again
b) We can retrieve information in our own words after a period of time has passed
c) We have memorised key phrases and are able to repeat them later
d) We understand what we have read or been told.
This design is known as Single Best Answer (SBA). All four answers are plausible, but answer b) is the ‘best’. This format requires learners to concentrate and process the information. It strengthens mental model in a way that an easy question does not. In effect, it acts as a kind of revision.

Online Assessment with Simulation

Recent advances in digital media allow forms of online assessment which until now have not been widely available. The screenshot below comes from Laerdal Medical’s ‘Cognitive’ Online Basic life Support (BLS) assessment. Learners make decisions in real-time by selecting choices that are played out by avatars.
This system allows assessment of the outcome, ‘Make BLS decisions and apply them in the correct sequence, in real time. It is a viable alternative to training large numbers of hospital staff in resuscitation protocols. Ventilations and chest compressions are practiced separately on a manikin. These are also assessed by a computer.

What Should the Pass Mark Be?

There are differing opinions about what pass marks should be, and there is no common agreement.
One line of argument begs the question,
If the pass mark is less than 100%, which bits of content are you happy for learners not to know?
A similar line of debate asks,
If you are happy for lower pass mark, then why not remove the non-essential content, and ask for 100%?
It should be borne in mind that most healthcare professionals have come through an education system in which a ‘pass’ can be achieved with a mark of 60% or less in most subjects, right up to degree level and professional examinations.
In reality, many eLearning modules require a pass mark of 80%. Some require 100%, but allow learners additional attempts if needed.
The safest approach is to agree to a reasonable pass mark with managers and subject leads. Then pilot the assessment to ensure it is neither too easy nor too difficult.

Cheating; surely not?

Unfortunately, it is not unknown for notes to be shared that detail correct choices to eLearning multiple-choice questions. For this reason, many trainers use tests with rotating question banks. In this way, the sequence and content of questions change for each user.

Are all Online Assessments Adequate?

Not all online assessments are found to measure outcomes. Some measure very little at all. If the correct answers for an MCQ assessment are obvious at a glance, there is no way of knowing whether any knowledge has been acquired.
For example, the question below was taken from a Fire Safety assessment in 2020. The learning outcome is:
Describe how to use the organisations’ fire safety equipment
Below is one of its questions and its four possible answers:
What are fire blankets for?
a) To keep people warm after they’ve been rescued from a fire
b) To cover food with to keep it warm
c) To smother a cooking fire and starve it of oxygen
d) All of the above

To what extent do you think this question assesses the outcome?

The incorrect answers were presumably intended to inject a sense of humour, or perhaps their author couldn’t think of any valid alternatives. Answer choices such as these don’t measure outcomes. At their worst, they can give an impression that the organisation does not take training seriously.

We would stress again that assessment should measure outcomes. There are no learning outcomes or real-life healthcare scenarios that require a professional to identify one correct phrase from three ridiculous alternatives. There are, however, occasions when we are required to make a difficult diagnosis from signs and symptoms which at first glance, could have more than one possible cause.

Talking Point

Consider sharing you experiences with online learning with other course members, and the answers to these questions:

  • What has been your experience of MCQ tests? How well have they measured course outcomes?
  • Are you involved with any training for which you’ve found it difficult to create a valid assessment?
© Health Education England Creative Commons 4.0 International
This article is from the free online

Train the Healthcare Trainer

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education