Transfer learning

What is transfer learning?

Transfer learning refers to the idea that we can get an AI model to gain useful information from one domain, and transfer that knowledge across to a second domain.

Let’s take a simple example. Let’s suppose we have a trained model which is able to segment carrots on a conveyor belt system. We want to modify the model so that it will work in a different scenario, segmenting parsnips on a conveyer belt. Parsnips may looks a little different to carrots (different colour, size) but they also have a lot of similarities (texture, shape, etc.). Is it reasonable to assume we might be able to apply some of the knowledge we have learnt about labelling carrots, to the problem of labelling parsnips?

At its heart, this is the transfer learning task. We have a model trained for one task, and we want to apply it in a different area. This is advantageous, as if we can do this, it means we will need less images for the second task – the basic fundamentals of the problems are learned from the first application already, we just need to refine the model to the new task. We hope that a lot of what has already been learned is useful in the new domain.

It turns out transfer learning is very commonly used in AI and machine learning development. How successful it is depends on how close the domains are, but there is usually something which can be transferred to the new model. Many common AI models, such as convolutional neural networks, are hierarchical in nature. That is, they first extract low level features from images, such as lines and colours, and use this information to build up more complex concepts about what is in the image. Extracting lines and edges from an image is a really common early task in vision, including in our eyes, and so simply transferring this part of the system across is often useful event in very different domains.

It is quite common to pretrain convolution neural network models on large datasets of general images, and then fine tune – transfer – this model onto the new application domain. Often, we fix the parts of the model which are generally useful to transfer – such as what edges of objects look like – and allow it to learn the specifics of the new domain. These specifics might include what to do with those edges and colours – how to combine the colour and texture information to label a parsnip instead of a carrot, for example.

Successful transfer learning can mean we need a lot less training time (or compute power), and a lot less images, to build the new model.