Skip main navigation

Re-inforcement Learning in Healthcare: Opportunities and Challenges

Video
8.5
Hello everyone! And thank you very much for watching this video. In the next few minutes we’ll be talking about reinforcement learning in healthcare and I’ll be discussing some of the opportunities and challenges associated with this new field of research. If we want to take a step back and if we were to define what machine learning is really what this is about is about learning from the data. and there’s really three pillars in machine learning. The first one being supervised learning, and then you have unsupervised learning, and the third aspect is reinforcement learning which will which we’ll be talking about in a bit more detail. Supervised learning.
47.7
if we were to define this, it’s about learning the relationship between input data and an output data so what we took, what we call learning a function between between x and y and within supervised learning. Two really famous subtypes of supervised learning are regression and classification algorithms and all we give you; I will be giving you an example in a few seconds. The second aspect is unsupervised learning where we learn about the structure in the data so this is called unsupervised because we have no idea but the data is unlabeled.
91.3
We don’t need to know the specific categories of data before they are fed into the algorithm and in unsupervised learning - really well used types of algorithms are clustering and dimensionality reduction. We won’t be going into any detail about this and the third aspect is reinforcement learning where this is a bit different in the sense that what we want to do is learn an optimal strategy. Let’s talk about an example of supervised learning to start with. Of course, I chose an example that comes from the field of healthcare and this was a very groundbreaking paper that was published in 2017.
135.6
In the journal Nature showing that a computer system was able to diagnose skin lesions with at least as good accuracy as the the most experienced dermatologists. So this this is an example of supervised learning where the the the data used for training the model is labeled meaning that we know which lesions were cancerous which lesions were non cancerous rent when when training the model. But let’s jump straight into reinforcement learning the intuition behind reinforcement learning is that what we want to do is learn an optimal strategy. And it’s often I would argue… it’s often more complex than prediction tasks such as the one we we saw in a previous example, about skin lesions.
190.7
The idea behind the reinforcement learning and here is here is a very simplistic example if you think about a rat in a maze, this example we simply define some of the elements that are necessary in order to implement reinforce planning algorithms so in this example think about a rat in a maze the rat is the agent; the maze is the environment and the agent can be in in lots of different states that are represented by the cells in this maze. In every state of the system that the agent has the choice of making different actions can go up left, right or down.
229.9
and the objective of the agent is to maximize the reward to reach cells associated with the reward as often as possible and to avoid cells associated with a penalty as often as possible and the actions that lead to reward are reinforced hence the term reinforcement and if we take a very very simplistic approach this this could be put in parallel with with healthcare where we have a patient, the patient can be in various health states and various clinical conditions.
266.5
And the objective of physicians and the healthcare system in this example is to take the patient from state to state by intervening on our patient and with the objective of bringing this patient home rather than sending him to the cemetery, so again in this example we have different states and the the agent the physician has different options of actions and interventions in every state of the system and the objective of the doctor is to send the patient home to maximize the reward if survival is the reward. Taking a step back we can think about how do doctors achieve this.
315.5
So typically, nowadays when a patient comes in the hospital with a with a particular presentation what doctors do is that they will be gathering data about this new patient examining them doing all sorts of assessment and examinations. Then they will use their theoretical medical knowledge as well as their clinical experience meaning all the cases they have previously encountered. All this information will be integrated and will lead to a medical decision. The problem is that this is a very very suboptimal approach for various reasons. A very important aspect is that humans are are full of biases and cognitive biases in medical decisions is a very significant problem.
365.3
For many diseases, many clinical problems we don’t have a physiological model or sometimes we lack theoretical knowledge so… personally I’m an intensive care doctor I don’t pretend to have all the all the medical knowledge about all the medical specialties in the world for sure Also human doctors are not perfect they sometimes forget cases that they have encountered in the past sometimes the cases are so rare that very few people can even make a diagnosis sometimes we get the diagnosis wrong and practice variation is also a huge problem

An experienced anaesthetist and intensivist with 10 years in training and practice, Dr. Matthieu Komorowski holds a full specialist board certification in both France and the UK. He belongs to the MIT Critical Data group where he applied machine learning to large critical care databases, with the objective of developing decision support systems for sepsis, the number one killer in intensive care. In this video, he explains re-inforcement learning in healthcare.

This article is from the free online

Artificial Intelligence for Healthcare: Opportunities and Challenges

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education