The Importance of Incentives: Data Quality

EME OWOAJE: So in this section, we’ll be looking at the importance of incentives and data quality. In this slide, we’re going to look at elements of data quality and these elements include validity, reliability, completeness, precision, timeliness, and integrity. The validity of the data is extremely important because it shows whether it’s been able to capture that which is supposed to capture and with the tools that– the indicators that we use were actually appropriate. Then reliability looks at the ability to go back and measure the same thing by other people and get the same results. Completeness. There are various aspects of the data that are looked at and it’s important to get those various aspects and not just part of it.

So it’s– data complete this is important. Precision looks at the ability to measure a specific aspect that we actually want to measure for that data. Then timeliness looks at the properties that which that data is being measured. And in a campaign like polio, it’s important to get the right data at the right time because people– children can move around and it would affect the dominators, it would affect the percentage of coverage. Then integrity– data integrity is very important because it reflects the truth and– of the particular data that is being used. So I’d like us to think about what happens if any of these elements of data quality are not met.

And I’d like you to relate this concept of implementation fidelity– that is the degree to which an intervention is delivered the way it’s supposed to be delivered like the polio program with the plan– the microplans and the activities of the health workers are supposed to actually carry out. So each of those activities is linked to data and the fidelity– the implementation fidelity is strongly linked to the initial microplan. So I’d like you to think about this and relate it.

So I’d like us to talk about what causes poor quality data. Before you look at the various options that we’ve given that you need to discuss amongst yourselves and come up with some of these things that, in your experience, result in poor quality data. So from my end, we’re looking at no standards for data collection or the standards are not enforced as they’re supposed to be enforced. This usually occurs when there’s no training or the training is inadequate. So it is essential that the health workers or the people involved in data collection are well-trained. Then there’s also the aspect of integration of data from systems with different data standards.

So we need to harmonize the data standards to ensure that the quality of the data that we collect is acceptable. Then, data quality issues are perceived as time consuming and expensive to fix. And that is something that training can take care of and then reiterating to the people that are involved in data collection that it is important, that we need to get that data right right from the beginning, also to explain to them that it’s more expensive to come back and try and fix it. So today’s work focus people may not be incentivized for high quality data or they may just have perverse incentives.

So if people are rewarded for showing high coverage, they may adjust the data to show high coverage and I’m sure most of you have experienced that. If people are told, oh, you’ll get extra money based on the number of children that you immunize– so they might just go out there and– in the case of polio– when they we’re using the oral polio vaccine, just keep on using the vaccine drops of the vaccine and write it up that they immunized children when they actually did not, just because they were being incentivized to do so.

So in terms of implementation fidelity, which I had already talked about before, the degree to which an intervention is delivered as it’s intended, polio data illustrates the root cause of poor fidelity, including pervasive incentives. We’ve talked about that with the oral polio virus that people could just go out and give two drops to what– even put them on the ground just because they were being incentivized to immunize as many children as possible. So we need to look at the strategies for tackling this, including technology-based strategies. And then this is also important in other immunization programs. It’s– there’s no use saying that people have immunized children when they haven’t.

For instance, in measles– people go out on measles campaigns and they say, oh, they have immunized this number of children and, at the end of the day, the truth of the matter is that they have not immunized these children, but there’s a false sense of security. So we need to address this and make sure that there’s fidelity in the implementation of our various programs. So we’re looking at the use of technology versus a manual way of estimating the number of children in Afghanistan.

So if you’re looking at this slide– in the 2017 GI– Geographical Information Systems, GIS, under five estimates of the population of children in Afghanistan was about 5.9 million– almost 6 million, whereas the under five polio targets in Afghanistan was 10.2 million. So what do you think, in your opinion, might have resulted in the divergence in the numbers and why might the geographical information system numbers be lower? What is GIS modeling missing that the administrative data has? What’s the importance of the knowledge that the administrative system has? And why do you think that the administrative numbers might be higher? Could it be as a result of the incentives that are being given?

This could potentially have to do with people inflating numbers so that they can get more resources. We’ve talked about this. Could you give examples in your old environment how this might work? So what would you do with this if you were a planner? For further discussion, here’s a comment from the Indian context. There are other reasons for polio planners to use higher figures than any demographic or GIS estimates. In India’s program, we always took the reported coverage as the base for planning the next campaign. Though we normally expect under five population to be 14% of the total population, polio program population used to be around 20%. This was a very practical approach– many reasons for that.

Vaccinators would like to err on the overage rather than the underage, really giving overestimates rather than underestimates for children five years below. So they would rather have more population on ground than predicted by the estimates. Some accounts unaccounted wastage– planning vaccine with higher figures for the fear of running out of vaccine and [INAUDIBLE].. How can public health programs better incentivize quality data? I’d like you to reflect on this question. The polio program has not fully cracked this particular nut. And I think it would be an excellent final project for one of you who might want to look in that area and who’s interested in data. One thing that has been tried is using technology to achieve this.

Let’s look at that next.