Selecting measures to assess your AMS programme

We have just looked at some of the methods for measuring the success of an AMS programme. This step will examine how to choose the right measure for your programme.

We will look at the measures discussed previously: structure, process, outcome, and balancing.

Structure Measures

One way to assess your structural outcomes is using the core elements of the WHO AMS for healthcare facilities checklist discussed in step 2.2. This helpful tool allows you to check whether you have all the core elements in place.

Ensuring core structures are in place provides the building blocks of an AMS programme. Some of these structures are included in a set of European AMS indicators developed and validated via a consensus process. This 2015 study by Colligan et al. illustrates a high level of implementation of core structures with data collected via a survey reviewing stewardship activities across a national ASP in Scotland. The study provides an example of where national support and a framework for local ASPs work well to achieve reliable and sustainable AMS.

Click here to see a larger version of this image.

Process Measures

The implementation of AMS interventions aims to optimize antibiotic prescribing and use. This can be difficult to measure objectively; therefore, it is recommended to include process measures as a proxy for tracking improvement in prescribing. The WHO toolkit provides a list of indicators as process measures (chapter 6) which are commonly used to measure a range of AMS interventions (see image below).

You must define the numerator and denominator of the measure/indicator carefully when using a process measure to provide assurance of the quality of the data collected.

Click here to see a larger version of this image within the full WHO toolkit PDF.

This infographic is also available as a screen-reader compatible PDF.

Outcome Measures

An essential part of any AMS programme, national or facility, is studying antibiotic prescribing and use over time to demonstrate the impact of interventions on the outcome of ‘antibiotic use’. The most sustainable and least laborious way to measure antibiotic use over time is through the routine collection of antimicrobial consumption data, ideally via computer systems that can produce standard reports. It is recommended that antibiotic use indicators should be expressed in at least two metrics simultaneously to reflect the rate of prescribing rather than just bulk antibiotic use without relating it to clinical activity.

The WHO toolkit (chapter 6) provides a list of outcome measures relating to antimicrobial use. As you can see, Defined Daily Doses (DDDs) are a common measure used in outcome indicators. We will discuss DDDs more in a later section.

Click here to see a larger version of this image within the full WHO toolkit PDF.

This infographic is also available as a screen-reader compatible PDF.

Another method of studying antibiotic consumption is via a point prevalence survey which allows us to calculate the percentage of each antibiotic being used at a particular point in time. A good example of this approach is this 2019 study, which looked at the proportion of AWaRe antibiotics prescribed in children by country.

Another hallmark study, by Budd et al. from England, shows how the WHO AWaRe classification can be adapted to provide an index for a national AMS programme based on which antibiotics feature in national guidelines. The study provides a full list of all drugs prescribed in England, with some moved from the Access category to the Watch category to support national prescribing ambitions. This list has since been adopted across all UK nations. The authors encourage those in other countries to review their own antibiotic usage against the AWaRe classification to preserve Watch and Reserve antibiotics.

Other outcome measures/indicators:

As we discussed in our previous step, reviewing antimicrobial usage is not the only outcome that can be measured.

The WHO toolkit also provides a list of these outcome indicators (below).

Click here to see a larger version of this image within the full WHO toolkit PDF.

This infographic is also available as a screen-reader compatible PDF.

One example of an important outcome measure is mortality. Ensuring mortality is not increased is important to reassure clinicians about the safety of ASPs, which commonly aim to reduce overall antibiotic use. Studies often include mortality as an outcome measure to ensure that AMS interventions do not have unintended consequences.

Balancing Measures

When using balancing measures, the question we need to ask is, ‘are changes designed to improve one part of the system causing new problems in other parts of the system?’ For example:

• If you reduce the time patients spend on a ventilator after surgery, you need to be able to confirm that reintubation rates are not increasing.
• If patients are being changed to oral therapy as part of the IV to oral switch intervention, you need to confirm that patients are not deteriorating after the switch.
• If you reduce patients’ length of stay in the hospital, you need to confirm that readmission rates are not increasing

Choosing what method to use

Choosing the metrics and methods to evaluate the AMS interventions are important to ensure they are successful, sustainable and safe. Where possible, it is important to choose a realistic and achievable data set that reflects the reality of your ASP and local prescribing practice. Some of the factors that need to be considered are:

While focusing on measurement is essential, making the process of data collection sustainable is often challenging in many settings. Challenges may occur for various reasons such as measurement fatigue, change of focus or priority, and staff resource limitations. The eight principles of sustainable measurement outlined below provide a practical approach to ensure that data collection is realistic and can be maintained.

Now we have looked at what measurement is and why it’s important, in our next step, we will start to review in more detail the different types of measurement that can be used.