Skip main navigation

Bias in AI

Display of a situation in which bias in AI occurred.
ELENA SüGIS: A central motivation for using AI systems in health care is to advance patient care by reducing human error and to improve the accuracy of diagnoses and clinical decisions. There is, however, also rising awareness about bias in AI technology and the potentially harmful effects. It is important to mention at this point that strictly speaking, biases in AI are not only caused by technical problems, that can be solved by technological means. It has been also shown that human bias, such as gender and racial bias can be inherited by AI systems. For example, face recognition systems have been shown to misclassify darker skinned females exhibiting gender and ethnicity bias.
According to the ethical guidelines for trustworthy AI, technical robustness and safety, along with diversity, non-discrimination, and fairness, has to be ensured, while unfair bias has to be avoided. Wondering what can go wrong with an AI system when the bias is present? Let’s find out with the help of an example. There is a significant interest in applying convolutional neural networks to analyse radiology, pathology, and other types of clinical images for the purposes of computer-aided diagnostics. Before these tools can be used in real world clinical practice, we must verify their ability to generalise. In 2020 a study was conducted to investigate whether gender imbalance training data can affect the outcome of AI algorithm reading chest X-rays.
For this purpose, the authors of the study used a large data set of chest X-ray images. These images were collected for 14 diseases, including pneumonia, fibrosis, emphysema, and others. In this experiment, two AI models were trained, one using male only data and the other using female only training data. Additionally, intermediate scenarios were also analysed. In these scenarios female and male images were represented in the training data set at different proportions, such as 0 to 100, 25 to 75, and 50-50. The models then, were tested to correctly classify a disease correspondingly on female and male images.
The results of the study showed that algorithms trained with gender imbalanced data, performed slightly worse at reading chest X-rays for the underrepresented gender group, meaning that the male image based training model, where the proportion between male and female images in the training set was 75% to 25%, showed a worse ability to classify disease on female test images. Analogous results were shown for the female image trained model. In the given example you have observed, how the composition of patient data sets used for training can cause bias in AI models for computer-aided diagnostics? When it comes to training and AI algorithm, fewer data points can lead to less accurate predictions.
If not explicitly accounted for, bias in the training data will result in unequal impact from the majority and minority subpopulations when developing an AI system. There is, however, a different type of bias that can be encoded into an algorithm itself. This will be discussed in the next section.
One of the seven requirements for reaching trustworthy AI is the application of diversity, non-discrimination, and fairness. In order to meet this requirement, bias should be avoided.

In this video, lead educator Elena Sügis introduces the topic of bias and displays the results of a study in which the effects of bias were examined.

This article is from the free online

How Artificial Intelligence Can Support Healthcare

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education