Want to keep learning?

This content is taken from the Taipei Medical University's online course, AI and Big Data in Global Health Improvement. Join the course to learn more.

Skip to 0 minutes and 15 seconds So we’re now going on and looking at what happens when you introduce these systems in Pediatrics because Pediatrics is much more complicated in terms of medication safety. So how can we make these systems even more effective? And everyone is very excited, obviously, about electronic decision support as one way in which we can alert providers to errors before they actually make them, and we have some good evidence that when alerts such as these are well designed and targeted, they can be very effective.

Skip to 0 minutes and 50 seconds So this is something where we implemented very simple decision support in a pathology order entry system, so when clinicians go to order a pathology test, we implemented an alert which just told them, this is a duplicate order alert; someone has already ordered this test very recently, and this is how effective it was. We looked, in this study, at children under one who are intensive care units, and we looked at the repeat testing rate before and after the introduction of this very simple alert. The blue line represents the situation before the alert was implemented, and the red line afterwards. So it was highly effective at reducing the number of duplicate test results in this very vulnerable population.

Skip to 1 minute and 41 seconds So we know it can work very effectively, but we also are getting increasing evidence that many doctors override alerts, they get so many of them that they just ignore them and click through them, and this is what’s being termed “alert fatigue.”

Skip to 1 minute and 59 seconds So what we need to start to understand is: when is decision support effective? And how can we design it so that it is more effective? What are the situations and the context in which it can work? So we said about, first of all, during some observational work in hospitals, so we were interested to know what happened on ward rounds when clinicians received alerts in their electronic medication system. So we followed for about 60 hours ward rounds in hospitals, and we looked at them ordering medications, and we looked at how often alerts came up in the system and about 50% of the time they received an alert about an order that they had placed.

Skip to 2 minutes and 48 seconds They read 17% of those alerts, and we were very generous, if they just quickly looked at them, we said they’d read it, and we saw no orders change. NOT one. Does anybody know why? Some of you may be involved in, have been involved in ward rounds,

Skip to 3 minutes and 10 seconds and we know who makes the decisions in a ward round: it’s people like Professor Li,

Skip to 3 minutes and 15 seconds and who enters it and makes the orders in the computer system: it’s the junior doctor. And so we will watch the junior doctor would just click past all the alerts, never told anybody on the ward round that they had received an alert. And so in fact, what we were doing was training those junior doctors to ignore alerts.

Skip to 3 minutes and 38 seconds So then what we did was we followed junior doctors at night when there are no senior doctors available, and we looked at what they did when they were prescribing. And what we found was they read nearly all the alerts that they received, and they changed about 5% of their orders in response to those alerts. So context matters, and we need to consider that when we are designing alerts. We know that the design of what an alert looks like is also incredibly important. This is a recent example from some of the commercial systems, and we all look at this and thinking, “this is ridiculous,” And then you can look at the options, so if you manage to read all that informations,

Skip to 4 minutes and 25 seconds you have two options: “ok” or “cancel,” So. it’s not very clear. So human factors and getting involved with human factors experts is incredibly important.

The Results of Evidence that Targeted Decision Support

Is eMM the Solution? Dr. Westbrook first pointed out the benefits that electronic decision support as one way to alert providers to errors before error happened. And the results of the experiments show good evidence that when alerts are well designed and targeted, they can be very effective.

The educator also discusses several topics: when is decision support effective? How can the researcher design a more effective system? What are the situations and the context in which it can work?

Dr. Westbrook then shows an experiment result from observational work in hospitals. This observation is to observe the situation when clinicians received alerts in their electronic medication system. How do they respond to the decision support system? And How does the decision support system caused alert fatigue and fail its function? So the researchers need to consider when these alerts should be delivered.

Share this video:

This video is from the free online course:

AI and Big Data in Global Health Improvement

Taipei Medical University

Get a taste of this course

Find out what this course is like by previewing some of the course steps before you join: