Part B Foundations of Statistical Inference (SB2.1) MT 2019 (first half)


Webpage for second half of the course


Lecturer

Julien Berestycki (lectures 1-8) and Dino Sejdinovic (Lectures 9-16)
  julien.berestycki@stats.ox.ac.uk     dino.sejdinovic@stats.ox.ac.uk

Course aims 

Understanding how data can be interpreted in the context of a statistical model. Working knowledge and understanding of key-elements of model-based statistical inference, including awareness of similarities, relationships and differences between Bayesian and frequentist approaches.

Synopsis

Exponential families: Curved and linear exponential families; canonical parametrization; likelihood equations.
Sufficiency: Factorization theorem; sufficiency in exponential families.

Frequentist estimation: unbiasedness; method of moments; the Cramer-Rao information inequality;

Rao-Blackwell theorem, Lehmann-Scheffe Theorem and Rao-Blackwellization.
Statement of complete sufficiency for Exponential families.

The Bayesian paradigm: likelihood principle; subjective probability; prior to posterior analysis; asymptotic normality;

conjugacy; examples from exponential families. Choice of prior distribution: proper and improper priors;
Jeffreys and maximum entropy priors. Hierarchical Bayes models.


Decision theory: risk function; Minimax rules, Bayes rules. Point estimators and admissibility of Bayes rules.

The James-Stein estimator, shrinkage estimators and Empirical Bayes. Hypothesis testing as decision problem.

Pre-course reading

A brief revision of the material covered in Part A Statistics course would be a good idea.
Course notes for Part A Statistics can be found here.

Lectures (all in Stats LG 0.1)

Tuesdays 3 pm and Thursdays 10 am weeks 1-4.
Mondays 11 am and Tuesdays 1 pm weeks 5-8.
( Part B timetable )


Lecture slides

Copies of lecture slides will usually appear here before each lecture.
It is recommended that you bring copies of the slides to each lecture.
The 4-up slides would be best for this purpose.

Chapters
Topic
1up slides
4up slides
1
Exponential families
Chapter1.pdf

2
Sufficiency, Factorization Theorem, Minimal sufficiency
Chapter2.pdf
3
Estimators, Minimum Variance Unbiased Estimators (MVUE)
The Cramér-Rao Lower Bound.
Chapter3.pdf
4
Consequences of the Cramér-Rao Lower Bound.
Searching for a MVUE. Rao-Blackwell Theorem, Lehmann-Scheffé Theorem.
Chapter4.pdf
5
The method of moments
Chapter5.pdf
6
Bayesian Inference.
Chapter6.pdf
7
Prior distributions. Hierarchical models. Predictive distributions.
Chapter7.pdf


UG Classes

There will be UG classes in weeks 3,5,7,8 (time and rooms on Minerva) .

Work should be handed in to the appropriate SB2.1 pigeon hole 48h before the class.

MSc Classes

There will be MSc classes in weeks 3,5,7,8 (Tuesday 10 am in LG.01) .

No work to be handed in.


Class
Sheet
Solutions
Week 3
ex1.pdf
Solutions 1
Week 5
ex2.pdf Solutions 2
Week 7
ex3.pdf
Week 8
ex4.pdf

Reading

  1. P. H. Garthwaite, I. T. Jolliffe and Byron Jones, Statistical Inference, Second ed. Oxford University Press, 2002
  2. G.A.Young and R.L. Smith,  Essentials of Statistical Inference, Cambridge University  Press, 2005. 
  3. T. Leonard and J.S.J. Hsu, Bayesian Methods, Cambridge University Press, 2005.

Further reading

  1. D. R. Cox, Principles of Statistical Inference, Cambridge University Press, 2006
  2. H. Liero and S Zwanzig, Introduction to the Theory of Statistical Inference, CRC Press,  2012
  3. D. Barber, Bayes Reasoning and Machine Learning, Cambridge University Press,
    2012