Teaching : Nonparametric Bayes

Review Articles

• Bayesian Nonparametric Models.
  P. Orbanz and Y.W. Teh. Encyclopedia of Machine Learning, 2010. Springer.
  [bibtex] [pdf] [djvu] [Encyclopedia of Machine Learning]

• Dirichlet Processes.
  Y.W. Teh. Encyclopedia of Machine Learning, 2010. Springer.
  [bibtex] [pdf] [djvu] [Encyclopedia of Machine Learning]

• Hierarchical Bayesian Nonparametric Models with Applications.
  Y.W. Teh and M.I. Jordan. Bayesian Nonparametrics, 2010. Cambridge University Press.
  [bibtex] [pdf] [djvu] [Cambridge University Press]

Tutorials

I have given a number of tutorials on Bayesian nonparametrics over the years. The most recent ones being at MLSS Bordeaux 2011 and NIPS 2011. Videos of some of the tutorials are available online:

• NIPS 2011: 2 hours, with Peter Orbanz. Motivation and high level concepts for Bayesian nonparametrics.
• MLSS 2011 Bordeaux: 4.5 hours, more motivation and high level concepts for Bayesian nonparametrics, and in depth development of Dirichlet processes, Chinese restaurant processes and other random partition models.
• MLSS 2009 Cambridge: 3 hours, tutorial introducing Dirichlet processes, beta processes, hierarchical Dirichlet processes and Pitman-Yor processes.
• MLSS 2007 Tuebingen: 1.5 hours tutorial focussing on Dirichlet processes.

Further details, including slides of the tutorials:

• Modern Bayesian Nonparametrics.
  P. Orbanz and Y.W. Teh. 2011.
  A nonparametric model is a model on an infinite dimensional parameter space. The parameter space represents the set of all possible solutions for a given learning problem -- for example, the set of smooth functions in nonlinear regression, or of all probability densities in a density estimation problem. A Bayesian nonparametric model defines a prior distribution on such an infinite dimensional space, where the traditional prior assumptions (e.g. "the parameter is likely to be small") are replaced by structural assumptions ("the function is likely to be smooth"), and learning then requires computation of the posterior distribution given data.
  The tutorial will provide a high-level introduction to modern Bayesian nonparametrics. Since first attracting attention at NIPS a decade ago, the field has evolved substantially in the machine learning, statistics and probability communities. We now have a much improved understanding of how novel models can be used effectively in applications, of their theoretical properties, of techniques for posterior computation, and of how they can be combined to fit the requirements of a given problem. In the tutorial, we will survey the current state of the art with a focus on recent developments of interest in machine learning.
  Slides: [NIPS 2011]
  Video: [NIPS 2011]
  

• Bayesian Nonparametrics.
  Y.W. Teh. 2011.
  Bayesian nonparametric models are a novel class of models for Bayesian statistics and machine learning. These are models over infinite dimensional spaces, e.g. of functions, densities or distributions. Bayesian nonparametric models allow for priors that have large coverage while at the same time allow for rich prior knowledge to be encoded. In this tutorial I will introduce Bayesian nonparametric models, focussing particularly on the Dirichlet process, a stochastic process that has been discovered and rediscovered many times in the last 50 years, and a cornerstone of the field. I will discuss its definitions, representations, applications, inference algorithms, as well as various generalizations and the rich classes of Bayesian nonparametric models that can be constructed using the Dirichlet process.
  Slides: [MLSS 2011 Bordeaux]
  Video: [MLSS 2011 Bordeaux]
  

• Bayesian Nonparametrics and Random Partitions.
  Y.W. Teh. 2011.
  Tutorial introducing Bayesian nonparametrics and going in depth into the role of random partitions in various Bayesian nonparametric models. Fragmentation and coagulation operators, models for hierarchical clustering, sequence memoizer, and fragmentation-coagulation processes.
  Slides: [MLSS 2011 Singapore]

• Bayesian Nonparametrics with Applications.
  Y.W. Teh. 2010.
  Tutorial covering Dirichlet processes, beta and Indian buffet processes, hierarchical and nested processes, time series models, and processes with power-law behaviours. The use of such models is motivated by successful applications across a range of domains.
  Slides: [CIMAT Mexico 2010] [KAIST South Korea 2010]
  [demonstrations]

• Bayesian Nonparametrics.
  Y.W. Teh. 2009.
  Tutorial covering Dirichlet processes, beta and Indian buffet processes, hierarchical Dirichlet processes and Pitman-Yor processes.
  Slides: [Gatsby ML2 2009] [MSR Cambridge 2009] [Toronto 2009] [MLSS 2009 Cambridge]
  Video: [MLSS 2009 Cambridge]
  [demonstrations]

• Dirichlet Processes.
  Y.W. Teh. Tutorial and practical course focussing on the Dirichlet process and its various representations and uses.
  Slides: [MLSS 2007 Tuebingen]
  Package: [tgz] [zip]
  Video: [MLSS 2007 Tuebingen]

Demonstrations

• Bayesian Nonparametrics: DP Mixtures.
  Y.W. Teh. 2009.
  Documented MATLAB code demonstrating DP mixtures. Also includes demonstrations that are part of my introductory lectures on Bayesian nonparametrics.
  Package: [tgz] [zip]
  [tutorial slides]