SB2b: Statistical Machine Learning
Instructors: Prof. Mihaela van der Schaar and Dr. Seth Flaxman
||Hilary Term 2017, Jan 15 - Mar 11
||Statistics LG.01, Wed 12-13, Thu 16-17
||Statistics LG.01, Thu 26 January, 9 February, 16 February, 2 March, 15-16
New: Revision questions
Access past exam questions via OXAM which is on WebLearn [solutions here]
- Lecture 1 (18/01): Introduction to machine learning, brief tour of topics covered in course (lecturer: Yee Whye Teh)
- Lecture 2 (19/01): Empirical risk minimization, Bias/variance, Generalization, Overfitting, Crossvalidation (lecturer: Dino Sejdinovic)
- Lecture 3 (25/01): Regularization and Logistic regression (Seth)
slides, overfitting demo, regularization demo, logistic function demo [code]
- Lecture 4 (26/01): Neural Networks (Seth)
- Lecture 5 (1/02): Generative vs. Discriminative models, Maximum Likelihood Estimation, Mixture models (Mihaela)
- Lecture 6 (2/02): Expectation Maximization (Mihaela)
- Lecture 7 (8/02): Deep Learning (Seth)
slides, deep learning demo from Google
- Lecture 8 (9/02): Support Vector Machines (Seth)
slides, read more about Lagrangian duality and the Karush-Kuhn-Tucker conditions in Chapter 5 of Boyd and Vandenberghe and read a different take on SVMs on pages 129-132 and 417-438 in ESL.
- Lecture 9 (15/02): Clustering, K-means, Vector Quantization, Naive Bayes (Mihaela)
- Lecture 10 (16/02): Nearest Neighbour Classification (Mihaela)
- Lecture 11 (22/02): Support Vector Machines, LDA / QDA (Seth)
- Lecture 12 (23/02): Training ML methods, Optimisation (Seth)
Presentation was on whiteboard, based loosely on these notes.
- Lecture 13 (1/03): Decision trees, CART (Mihaela)
- Lecture 14 (2/03): Bagging decision trees, Random forests, and ROC curves (Dino)
- Lecture 15 (8/03): Ensemble learning methods, Adaboost (Mihaela)
- Lecture 16 (9/03): Reinforcement learning: a brief introduction (Mihaela)
Resources for R:
Recommended textbooks on statistical data mining and machine learning:
Hastie, Tibshirani and Friedman, The Elements of Statistical Learning, Springer.
Bishop, Pattern Recognition and Machine Learning, Springer.
Murphy, Machine Learning: a Probabilistic Perspective, MIT Press, 2012.
B. D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, 1996.
G. James, D. Witten, T. Hastie, R. Tibshirani, An Introduction to Statistical Learning, Springer, 2013. [ebook]