Summary for Week Three
1. Kernel Regression, the Kernel Trick and the Representer TheoremYou should understand what kernels are and what the kernel trick is and, at a basic level, how it works and what sorts of models it can be used with. You should also understand how it can be applied to dual form linear regression (since this is a powerful and paradigmatic case) and the mathematical guarantee that comes with it in the form of the representer theorem. You most certainly do not need to understand the mathematics that lead to the kernel trick or representer theorem! You should also understand why kernel methods do not generally scale well to big data.
2. Support Vector MachinesYou should know what support vector machines do in terms of concepts like an optimal separating hyperplane, and know how to train them to data. You should know what sort of hyper-parameters are involved in training an SVM, and be aware of the peculiar case of using a ‘linear kernel’.
3. Gaussian ProcessesYou should know what Gaussian processes do, and how they do it. You should be able to train them to data, including having a knowledge of the decisions that need to be made, and the hyper-parameters that need to be specified (and what they control). You should be able to explain the outputs that a Gaussian process provides when applied to new data.
4. Basic ClusteringYou should understand the differences between the various basic types of clustering algorithms covered, such as sequential clustering, hierarchical clustering, graph clustering and optimization based clustering. You should also understand related concepts such as proximity measures, proximity graphs, dendrograms, etc.
5. Gaussian Mixture Models and the Expectation Maximization AlgorithmYou should understand what GMMs are, what they do, how to apply them to data, and what they produce. You should also know their draw-backs. Further you should understand the idea behind the EM algorithm – we will see it again!
6. Dirichlet Process Mixture ModelsYou should understand what Dirichlet distributions and Dirichlet processes are, and how they can be used in combination with Gaussian distributions to produce a Dirichlet Process Mixture Model. You should understand the relationship between such a model and GMMs.
7. LDA and Metropolis within Gibbs MCMC
Want to keep
The Open University online course,
Advanced Machine Learning
Advanced Machine Learning
Our purpose is to transform access to education.
We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.
We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.