Deep learning networks for regression

A short article that explains how the deep learning networks we’ve seen used for classification can also be used for regression-type problems.

We’ve already seen how deep learning can be used to make a classifier – outputting a predicted class for a given piece of data. But how do we use deep learning to solve regression type problems, where we require numerical output instead?

In our classifier example, we used a ResNet with a final, fully connected layer consisting of 512 nodes connected to 17 output nodes, each representing one of the 17 flower species in the dataset.

When we trained the model, it was trained so that for each image passed to network, the output node outputting the largest number corresponds to the most likely species depicted in that image. If trained well, for a clear decision, the correct output node will have a large positive value, and other nodes much smaller value, perhaps negative values.

Regression

For a regression problem though we need a different approach. Rather than picking a value out of a fixed set of values (e.g. our 17 flower species), we want just a single number that represents the result of the regression. This could be a count of objects within the image, or perhaps an estimate of the age of a plant. Anything that can be descripbed by a single number.

The obvious way to do this is to just use a single output node and train the network so that the number it outputs in response to a given item of data matches some known set of values.

In the practical example linked to in the following step we will do just that. We will use the same ResNet from PyTorch as before, but rather than setting it to output the probability for each of 17 classes for every image, we will set it to output a single value instead, which we will train to match a simple count of the number of flower heads in the images.

Images (c) The University of Nottingham