What do the numbers mean?

Hello, my name is Angus Campbell. I’m a livestock vet and epidemiologist from the Nossal Institute for Global Health. And I’m going to talk to you today about testing for COVID-19. We will discuss several common terms that help us understand how tests for COVID-19 are performing to identify people infected with SARS-CoV-2 or coronavirus and also whether the amount of testing being carried out is likely to be adequate to detect and monitor the spread of the disease. The first concept we’ll discuss is sensitivity which refers to the proportion of truly infected people that a test reports as positive If a test has poor sensitivity, it will incorrectly identify infected people as negative, producing a lot of false negative results.

The second concept we’ll discuss is the positive rate, the proportion of all tests conducted that are positive for SARS COVID-2 To think about how COVID-19 is affecting different parts of the community, we need to be able to interpret information about the number of tests countries report they are conducting and the number of new cases or the disease incidents that the tests are detecting. Do reports of low numbers of new cases mean that countries are managing the outbreak well? Or are they testing in the wrong places? Or not doing enough tests? This is especially important in the global context of COVID-19 because not all countries might be able to offer equitable access to testing.

Of course testing is important because it may help people access care if they are positive for COVID-19, it confirms who should isolate themselves until they are recovered and no longer risks spreading the virus to others. And it helps health authorities decide where to allocate resources. It’s important for governments to make decisions about what public health measures should be taken, such as the strength of social or work restrictions, which have direct consequences for the economy and our well-being. By the end of this step, you should be aware of how accurately test statistics might be reflecting the true COVID-19 situation in a country and some factors affecting this.

So, as I prepared this lecture, my home city of Melbourne has been returned to strict lockdown for a further six weeks after 700 odd cases were detected from 178 000 tests conducted across the state of Victoria in the first week of July. So, how do we understand these figures? Our knowledge of how many true COVID-19 cases are present in the community depend on how many tests are conducted, the kind of people being tested, in other words how more or less likely are they to be infected compared to the wider population, and the accuracy of the test; that technical term I referred to before, sensitivity. Let’s deal with that one first.

Sensitivity means how many true positive cases a test can detect Currently, the commonly used tests, the PCR tests, detect unique, small fragments of the SARS COVID-2 virus on a swab taken from your nose and throat. If there’s a minute quantity of virus in those locations, the test will detect it. But the problem is that the virus isn’t in those swab locations throughout the entire course of the illness. Now, let’s focus on the truly positive cases highlighted by the larger rectangle here, which has just turned to green. As we conduct our tests, a few truly positive cases aren’t detected by the test. They’re the black dots turning up in that lower left quadrant.

It’s likely that, for example, of 100 people who truly have COVID-19, current tests will declare about 80 of them to be positive at best. The test sensitivity in this situation is 80 percent. The take home message here is that even for a very good test like a PCR test, an important proportion of truly infected people can return a negative test result. Now if you’re testing the wider population and the presence of SARS-CoV-2 infection is low, this won’t be too significant because the number of false negatives will be small compared to all the true negatives that are correctly diagnosed. Compare those 20 false negative dots to the thousands of true negatives in the lower right box of the figure.

However, if you are testing a population that is more likely to be infected, the implication of those false negatives is much more important and you should consider not relying on a single test to confirm the patient is not infected with COVID-19. Next we need to understand who and how many people are being tested for COVID-19 so we can understand reported test statistics more insightfully.

If many of the tests being conducted return positive results, it’s likely they are being conducted in people already suspected of having the disease, suggesting that testing of the wider population is inadequate and our knowledge of what’s happening in the community is poor On the other hand, a very low proportion of tests returning positive results could suggest that there is little virus in the population or that the group tested is being tested is disproportionately low risk and that we should be looking elsewhere too. Now the proportion of positive results from total tests is called the positive rate.

The World Health Organization suggests that if you have a positive rate of between about 3 and 12 percent, it suggests that a country is testing widely enough to be monitoring the disease adequately. This graph shows the positive rate, the y-axis, for a selection of countries. Each a different color through the first half of 2020. You can see that many countries did in fact have results in that positive rate range target or even below it, but some are much higher suggesting testing should be widened there You can also see that countries positive rates have changed over time often decreasing towards this desirable range of about 3 to 12 percent as more comprehensive testing plans have been put in place.

Now what are the implications of having a high positive rate if that’s the target that we have, or if the target is 3 to 12 percent This graph shows the risk we run when positive rates become too high above that target range, which might suggest insufficient overall testing In this hypothetical situation that I’ve created, I’ve also factored in that the test is only about 80 percent sensitive Now, here the positive rate runs along the z-axis away from us and so different positive rates along the z-axis represent different scenarios that we’re discussing The axis across the front represents how much greater the likelihood of infection is in the population we’re testing compared to the wider population.

The right end where the numbers are higher is where there is a greater likelihood of infection in the target population compared to the average person on the street. For example, if the target population contained lots of travellers returning home, who were more likely to be infected or people who are likely to be occupationally exposed such as healthcare workers or meat processors, the left end of this front axis is for us, where the numbers are lower, is for a situation where we think the people being tested have a pretty similar likelihood of being infected as the rest of the community.

Now, realistically we don’t know what this number across the front axis really is, but estimates are, that commonly it’s in the 5 to 10 range B ut the thing to note is how much the likely true prevalence of infection varies when the positive rate is high Let’s move the graph around so that we can see that a little more clearly.

We can see it now in the widely varying heights of the different colored series at the back of the graph If your country’s rate positive rate is 15 or 20 percent your tests could be telling you the incidence of the infections, represented by the heights of the bars, could be one percent or they could be as high as 10 percent or even 14 16 or 18. You just don’t know.

And if the incidence is high then that suggests that infections are likely to be spreading very rapidly through the community and that health care systems could be at risk of being overwhelmed Remember in an earlier step we noted that if you had a population a total population of about 10 up to a fifth of those people or two percent of the population might require hospital care a sobering thing to contemplate if that’s building up without your knowledge Compare this to the coloured series at the front of the graph. The light blue, orange, grey and yellow.

In these situations, if your positive rate is down around 5 percent or maybe even lower the population prevalence is much is more likely to be in the range of maybe only 0.2 to 3 percent, which is a much more manageable situation. Lastly, let’s consider what might be driving these different testing scenarios. In this last slide, the map on the left shows the testing policies of different countries who was able to access tests, from no formal policy about who would be tested in red, through testing of different kinds of case definition, to an open access policy to tests in blue.

The map on the right runs over the same timeline and it shows what kind of contact tracing has been performed I put the two maps together because contact tracing usually closely follows a positive test result Now, note that many of the countries that had open testing policies those blue areas in the left map in the middle of the timeline were also ones that continued to have high positive rates, suggesting they weren’t performing enough tests. Was there enough information in those areas that successfully managing the outbreak at those times It’s also interesting to note as we run that timeline through and if we compare jurisdictions.

Often the ones that have open testing policies also had limited tracing practices However, don’t forget that testing might have been carried out for other reasons than just contact tracing. This step has described a few aspects of understanding high-level testing results for COVID-19. There are other factors associated with producing reliable test results, including for example, making it clear what kind of numbers are being reported. For example it’s important to describe whether number of tests means the number of people being tested or the number of tests being run because one person might be tested more than once and this can obscure the results Furthermore, confidence comes if tests are reported from internationally certified test procedures and laboratories.

To sum up, based on what we’ve discussed in this step confidence in testing is improved if the test results show lower positive rates below about 10 percent When testing across large sections of the community, and results are reported in a timely clear and consistent manner. Think about these features in relation to your own situation, and as we start to discuss some specific country outbreak examples in the next step