Part A Synopses 2019-2020
Aims and objectives
The first half of the course takes further the probability theory that was developed in the first year. The aim is to build up a range of techniques that will be useful in dealing with mathematical models involving uncertainty. The second half of the course is concerned with Markov chains in discrete time and Poisson processes in one dimension, both with developing the relevant theory and giving examples of applications.
Continuous random variables. Jointly continuous random variables, independence, conditioning, functions of one or more random variables, change of variables. Examples including some with later applications in statistics.
Moment generating functions and applications. Statements of the continuity and uniqueness theorems for moment generating functions. Characteristic functions (definition only). Convergence in distribution and convergence in probability. Weak law of large numbers and central limit theorem for independent identically distributed random variables. Strong law of large numbers (proof not examinable).
Discrete-time Markov chains: definition, transition matrix, n-step transition probabilities, communicating classes, absorption, irreducibility, periodicity, calculation of hitting probabilities and mean hitting times. Recurrence and transience. Invariant distributions, mean return time, positive recurrence, convergence to equilibrium (proof not examinable), ergodic theorem (proof not examinable). Random walks (including symmetric and asymmetric random walks on Z, and symmetric random walks on Zd).
Poisson processes in one dimension: exponential spacings, Poisson counts, thinning and superposition.
G. R. Grimmett and D. R. Stirzaker, Probability and Random Processes (3rd edition, OUP, (2001 ). Chapters 4, 6.1-6.5, 6.8.
G.R. Grimmett and D. R. Stirzaker, One Thousand Exercises in Probability (OUP, 2001).
G. R. Grimmett and D J A Welsh, Probability: An Introduction (OUP, 1986). Chapters 6, 7.4, 8, 11.1-11.3.
J. R. Norris, Markov Chains (CUP, 1997). Chapter 1.
D. R. Stirzaker, Elementary Probability (Second edition, CUP, 2003). Chapters 7-9 excluding 9.9.
Building on the first year course, this course develops statistics for mathematicians, emphasising both its underlying mathematical structure and its application to the logical interpretation of scientific data. Advances in theoretical statistics are generally driven by the need to analyse new and interesting data which come from all walks of life.
At the end of the course students should have an understanding of: the use of probability plots to investigate plausible probability models for a set of data; maximum likelihood estimation and large sample properties of maximum likelihood estimators; hypothesis tests and confidence intervals (and the relationship between them). They should have a corresponding understanding of similar concepts in Bayesian inference.
Order statistics, probability plots.
Estimation: observed and expected information, statement of large sample properties of maximum likelihood estimators in the regular case, methods for calculating maximum likelihood estimates, large sample distribution of sample estimators using the delta method.
Hypothesis testing: simple and composite hypotheses, size, power and p-values, Neyman-Pearson lemma, distribution theory for testing means and variances in the normal model, generalized likelihood ratio, statement of its large sample distribution under the null hypothesis, analysis of count data.
Confidence intervals: exact intervals, approximate intervals using large sample theory, relationship to hypothesis testing.
Probability and Bayesian Inference. Posterior and prior probability densities. Constructing priors including conjugate priors, subjective priors, Jeffreys priors. Bayes estimators and credible intervals. Statement of asymptotic normality of the posterior. Model choice via posterior probabilities and Bayes factors.
Examples: statistical techniques will be illustrated with relevant datasets in the lectures.
F Daly, D J Hand, M C Jones, A D Lunn and K J McConway, Elements of Statistics, Addison Wesley (1995) Chapters 7-10 (and Chapters 1-6 for background)
J A Rice, Mathematical Statistics and Data Analysis, 2nd edition, Wadsworth (1995), Sections 8.5, 8.6, 9.1-9.7, 9.9, 10.3-10.6, 11.2, 11.3, 12.2.1, 13.3, 13.4.
T Leonard and J S J Hsu Bayesian Methods, Cambridge, Chapters 2 and 3.
G Casella and R L Berger, Statistical Inference, 2nd edition, Wadsworth (2001)
A C Davison, Statistical Models, Chapter 11.
The workload of this course is equivalent to an 16-lecture course.
Aims and Objectives
Building on Part A probability and first year statistics, this course introduces Monte Carlo methods, collectively one of the most important toolkits for modern statistical inference. In parallel, students are taught programming in R, a programming language widely used in statistics. Lectures alternate between Monte Carlo methods and Statistical Programming so that students learn to programme by writing simulation algorithms.
Simulation: Transformation methods. Rejection sampling including proof for a scalar random variable, Importance Sampling. Unbiased and consistent IS estimators. MCMC including the Metropolis-Hastings algorithm.
Statistical Programming: Numbers, strings, vectors, matrices, data frames and lists, and Boolean variables in R. Calling functions. Input and Output. Writing functions and flow control. Scope. Recursion. Runtime as a function of input size. Solving systems of linear equations, Cholesky decomposition. Numerical stability. Regression and least squares, QR factorisation. Implementation of Monte Carlo methods for elementary Bayesian inference.
The course will consist of fourteen lectures. Six of these will be held in a computer laboratory and are followed by an associated practical session. There will be four classes on problem sheets.
W J Braun and D J Murdoch, A First Course in Statistical Programming with R, CUP 2007
S M Ross, Simulation, Elsevier, 4th edition, 2006
J R Norris, Markov Chains, CUP, 1997
C P Robert and G Casella, Monte Carlo Statistical Methods, Springer, 2004
B D Ripley, Stochastic Simulation, Wiley, 1987