Tasks in machine learning

To expand on the previous video, these are the types of tasks that we often encounter when using machine learning for plant phenotyping.

1. Classification

Classification involves classifying, or labelling, an image with some sort of text label.
The meaning of the label – what the text represents – is irrelevant to the machine learning system.

The label, or class, for the above image could be wheat, or plant, or biomass – the semantics are something we apply ourselves. The machine learning system will simply learn to associate the features in different example images (the training set) with a particular label.

It is important to remember this – as it may be the system is learning something unintended in the image. For example, it may be that a set of wheat plants were imaged with a black background, and a set of rice plants were imaged with a blue background. Rather than learning features of the plants themselves to discriminate between the images, a machine learning system may well, in this simplistic example, just use the background colour to differentiate the classes!

2. Regression

A regression system aims to predict a number from an image. Once again, it is using image features, but using a model set up in a way that allows a number to be predicted.

In the above example, it may be we want to the model to predict the number of leaves – so, 2 in this case. We would need a training set that has examples with different leaf numbers, and the associated count of leaves. A regression system is likely to predict a real, continuous value, meaning that it may predict 2.1 for the above image, instead of exactly 2.

For small numbers of items, sometimes it can be more successful to frame the problem as a classification task. In other words, rather than predicting a continuous number for the above image, we predict the class “2” instead. The disadvantage of using classes is that the system cannot interpolate between values – a system trained to predict biomass for example, may be able to predict a value of 260g, even if it hasn’t seen an example image of exactly 260g in the training set.

3. Segmentation

In a segmentation task, we are wanting to label individual pixels. So, a segmentation system will predict where in an image whatever object or feature it has been trained on is. Note though that it is labelling individual pixels – it doesn’t understand topological details of an object, for example that it must be one enclosed boundary, or that it has a hole in the middle. Hence, often a segmentation system may produce some false positives – as you can see by the red spots in the bottom of the prediction above – or false negatives, where the model will incorrectly miss out pixels of images.

Segmentation systems are usually used to predict areas or volumes. If the segmentation prediction includes multiple regions, such as leaf, flower, fruit it is known as semantic segmentation; the segmentation includes information about the meaning of the pixels. If the segmentation system is trained to segment all items in the image – for example, all seeds scattered on a tray, it is known as instance segmentation. In an instance segmentation task, as well as having all the regions marked for the item of interest, we of course also get a count of the objects as well, by considering the number of regions.

Different tasks require different models or configurations as part of a machine learning system, and it is worth noting that some tasks will be easier or faster to train than others.

Images (c) The University of Nottingham