Skip main navigation

New offer! Get 30% off your first 2 months of Unlimited Monthly. Start your subscription for just £29.99 £19.99. New subscribers only. T&Cs apply

Find out more
Home / IT & Computer Science / AI & Robotics / Applications of AI Technology / Deep Learning Basics: Part2

Deep Learning Basics: Part2

v
View transcript
14
There are three major types of neural networks. The first is Convolutional Neural Networks The second is the Recurrent Neural Networks And the third is the Generative Adversarial Networks Let’s take a look of Convolutional Neural Networks first. The convolutional neural network is introduced by Yann LeCun in 1990s for recognizing handwritten digits. Convolution is a mathematical operation that has been widely used in many fields, such as digital signal processing, electrical engineering, and physics. In image processing, the convolution operations are 2D-filters, which can be applied to extract different image features. The training process is to adjust the parameters of filters to minimize the errors between predictions and labels. Historycally, CNN was used mainly for images.
76.3
But recently scientists have found that 1D-convolutional network with attention mechanism can achieve top performance in other research fields such as neural language processing or speech recognition. So, Convolutional Neural Networks becomes more and more important. For more details, you can refer to the famous Stanford’s free course cs231n. One drawback of CNN and other feedforward networks is that they don’t consider the interdependency of sequential data. The temporal relations are important for natural language understanding or speech recognition. For example, “Mary had a little lamb”, the owner of the lamb, Mary, is mentioned in the beginning of the sentence. The Recurrent Neural Network (RNN) solves this problem by adding a “loop” in the hidden layers.
137.1
The loop can keep the previous states of sequential data, which “remember” the temporal information. It may not be obvious how loops in hidden layers can be used to remember information. Actually, we can unroll the RNN loop to better understand this mechanism. Here is a good figure from Colah’s blog. The input data X0, X1, X2 are coming sequentially. The recurrent layer generates outputs h0, h1, h2, depending on the input X, and also send current output to next state. From this figure, we can see that the “Loop back” mechanism is equivalent to keep information for next state. There are many variants of RNN, the most important one is Long-short Term Memory (LSTM).
194.2
The main contribution of LSTM is to add the forget gate, which enables a neuron cell to reset its own state and “forget” out-of-date information. Modern RNN are all based on LSTM. Recently a new emerging architecture called Generative Adversarial Networks (GAN) was proposed by Ian Goodfellow.
219
GAN consists of two sub-networks: a generator and a discriminator. The generator tries to generate fake images based on random inputs, while the discriminator tries to classify if the generated images are fake or real. The trick is to let those two networks compete with each other. The generator learns to generate more realistic images, while the discriminator learns to identify more challenging fake images. GAN achieves the equilibrium state when the discriminator can no longer distinguish between the real images and fake images, and we will have a strong fake generator. After being invented in 2014, GAN has been widely adopted to generate articles, music, images, or videos. It creates many new applications but also makes some new problems, such as fake porn videos.

Next, Prof. Lai will introduce Major Types of Neural Networks:

  • Convolutional Neural Networks (CNN)
  • Recurrent Neural Networks (RNN)
  • Generative Adversarial Networks (GAN)

Convolutional Neural Network (CNN) was used mainly for images. Convolution is a mathematical operation that has been widely used in many fields, such as digital signal processing, electrical engineering, and physics. The Recurrent Neural Network (RNN) solves this problem by adding a “loop” in the hidden layers. The loop can keep the previous states of sequential data, which “remember” the temporal information.

Generative Adversarial Networks (GAN) was proposed by Ian Goodfellow. GAN consists of two sub-networks: a generator and a discriminator. The generator tries to generate fake images based on random inputs, while the discriminator tries to classify if the generated images are fake or real. The generator learns to generate more realistic images, while the discriminator learns to identify more challenging fake images.

Want to keep learning?

This content is taken from Taipei Medical University online course
Applications of AI Technology
View Course
See other articles from this course
This article is from the online course:

Applications of AI Technology

Created by
Join Now
This article is from the free online

Applications of AI Technology

Created by
Join Now
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now

Register to receive updates

  • Create an account to receive our newsletter, course recommendations and promotions.

    Register for free