Summary for Week Two
We are now half way through the course and have begun to look at applicable algorithms and techniques, building on the theory from week one.
The first half of this week focused on evaluating statistical models and data. Here is a reminder what you should have learnt from these components:
1. Model Evaluation
Know how to use hold-out and cross-validation techniques (if you did not know how to do so already), as well as understand the purpose of using a three-way rather than two-way split. You should also understand how to use statistical evaluation techniques such as the AIC and BIC, and be able to relate them to the bias-variance trade-off.
2. Learning Graphs
You should know how to create a learning graph to evaluate a suitable amount of training data for a model type for a given problem. You should know how to interpret learning graphs and work with them in real life scenarios so that they are interpretable.
3. Statistical Hypothesis Testing
You should know how to use basic statistical hypothesis testing to evaluate (i) whether you can confidently determine one model is better than another based on the results obtained from a specific data set; and (ii) the confidence we should have in the accuracy of the estimation of expected loss on new data for a final model. Relatedly, you should know how to place confidence intervals around estimates.
In the second half of the week, we looked at artificial neural networks. Here, you should have learnt:
4. Mathematics of Basic Neural Networks
You should understand that a basic neural network is a series of non-linear feature transformations followed by a linear model on the extracted features. In addition, you should know the form such feature transformation take and some common activation functions. You should also know about the vanishing gradient problem.
5. Gradient Descent for Neural Networks
You should understand the various versions of gradient descent used for training neural networks, including the trade-offs between them, as well as the common hyper-parameters that need to be specified in order to run such algorithms.
6. Regularization for Neural Networks
You should know the common forms of regularization used when training neural networks.
We hope you are still enjoying the course and are excited about next weeks topics, where we will be looking at kernel methods for supervised learning, as well as some unsupervised learning techniques!
© Dr Michael Ashcroft