So we’re now going on and looking at what happens when you introduce these systems in Pediatrics because Pediatrics is much more complicated in terms of medication safety. So how can we make these systems even more effective? And everyone is very excited, obviously, about electronic decision support as one way in which we can alert providers to errors before they actually make them, and we have some good evidence that when alerts such as these are well designed and targeted, they can be very effective.
So this is something where we implemented very simple decision support in a pathology order entry system, so when clinicians go to order a pathology test, we implemented an alert which just told them, this is a duplicate order alert; someone has already ordered this test very recently, and this is how effective it was. We looked, in this study, at children under one who are intensive care units, and we looked at the repeat testing rate before and after the introduction of this very simple alert. The blue line represents the situation before the alert was implemented, and the red line afterwards. So it was highly effective at reducing the number of duplicate test results in this very vulnerable population.
So we know it can work very effectively, but we also are getting increasing evidence that many doctors override alerts, they get so many of them that they just ignore them and click through them, and this is what’s being termed “alert fatigue.”
So what we need to start to understand is: when is decision support effective? And how can we design it so that it is more effective? What are the situations and the context in which it can work? So we said about, first of all, during some observational work in hospitals, so we were interested to know what happened on ward rounds when clinicians received alerts in their electronic medication system. So we followed for about 60 hours ward rounds in hospitals, and we looked at them ordering medications, and we looked at how often alerts came up in the system and about 50% of the time they received an alert about an order that they had placed.
They read 17% of those alerts, and we were very generous, if they just quickly looked at them, we said they’d read it, and we saw no orders change. NOT one. Does anybody know why? Some of you may be involved in, have been involved in ward rounds,
and we know who makes the decisions in a ward round: it’s people like Professor Li,
and who enters it and makes the orders in the computer system: it’s the junior doctor. And so we will watch the junior doctor would just click past all the alerts, never told anybody on the ward round that they had received an alert. And so in fact, what we were doing was training those junior doctors to ignore alerts.
So then what we did was we followed junior doctors at night when there are no senior doctors available, and we looked at what they did when they were prescribing. And what we found was they read nearly all the alerts that they received, and they changed about 5% of their orders in response to those alerts. So context matters, and we need to consider that when we are designing alerts. We know that the design of what an alert looks like is also incredibly important. This is a recent example from some of the commercial systems, and we all look at this and thinking, “this is ridiculous,” And then you can look at the options, so if you manage to read all that informations,
you have two options: “ok” or “cancel,” So. it’s not very clear. So human factors and getting involved with human factors experts is incredibly important.