GD-based algorithms for model parameter optimization

In Step 3.8, we introduced the gradient descent method, which is a general numerical method for optimization. In this video, Dr Alex Tse explains how to apply gradient-descent algorithms to estimate the optimal model parameters in supervised learning, and its two variants to cope with large-scale dataset.

This video covers three gradient-based algorithms for optimizing model parameters for supervised learning models as follows and the underlying mathematical intuitions:

• Batch Gradient Descent (BGD);
• Stochastic Gradient Descent (SGD);
• Mini-Batch gradient Descent (Mini-Batch GD).

Finally, Dr Alex Tse discusses the pros and cons of the above three algorithms in terms of various aspects, such as update frequency and update complexity. In the next step, we will continue to discuss the comparison of these algorithms.