Advantages and disadvantages of deep learning

What are the postives and negatives of deep learning approaches when compared to more fundamental machine learning techniques?

We have seen that deep learning is a form of machine learning. So how do we characterise the differences and similarities between models we refer to with the different terms?

Training time versus manual feature selection

As mentioned in a previous article, one of the drawbacks of deep machine learning is the enormous number of parameters that need to be learnt. This in turn takes a long time, and requires a lot of examples of training data; with too little time, or not enough examples, it will be impossible to set the weights effectively, and the algorithm will perform poorly.

Machine learning normally requires the manual selection of features of interest from the dataset, whereas deep learning will learn its own feature set from the training data. Again this requires more data on the part of deep learning; but more time and domain expertise from a human expert to craft or select the features from a dataset for more traditional machine learning approaches.

There is a danger with deep learning that the most suitable general features for the data will not be learnt (e.g. due to insufficient data, or poor representation of the domain in the training set). Conversely when manually selecting features in machine learning, it is possible to select inappropriate or suboptimal features, if not careful.

Expertise in model selection

Machine learning comprises many different algorithms and approaches, and we have seen just a small introduction to some of the approaches here on this course. An appropriately-trained expert in machine learning could hand select an approach well suited to the problem, data and computational requirements. This might lead to improved results over the more generic deep learning approach. This is particularly true if a small dataset is available. Finding the simplest suitable algorithm usually has computational benefits too – it may train or run faster.

Complexity of datasets

Complex, non-intuitive datasets (such as hyperspectral image data) may prove challenging for both approaches to analyse. Although features exist for hyperspectral data from a biological standpoint (such as vegetative indices, for example), manually selecting other features from hyperspectral datasets to manually optimise the learning can prove problematic.

Conversely, the sheer amount of data captured in imaging modalities such as hyperspectral imaging, versus the limited supervised labelling available, can make it challenging for deep learning to learn appropriate features too – analogous to finding a needle in a haystack of data.

Summary

There is often not one approach which is completely correct, depending on the scenario, data available, computational requirements and skillset of the team. Deep and non-deep machine learning are both tools, and the selection of any tool needs an understanding of the assumptions of the tool itself, the specifics of the problem domain and an appreciation of data availability.