George Deligiannidis

Associate Professor of Statistics
Department of Statistics, University of Oxford

Hugh Price Fellow in Statistics, Jesus College

Contact info

Email: deligian 'at' stats.ox.ac.uk
Telephone: 01865282855

Brief Bio

I studied mathematics (MMath) at Warwick university, then went on to study for a joint MSc in Financial Mathematics at Heriot-Watt University and the University of Edinburgh. I got my PhD from the University of Nottingham where I studied with Sergey Utev and Huiling Le. After my PhD I worked at the Department of Mathematics of the University of Leicester as Teaching Assistant and Teaching Fellow between 2009 and 2012. I moved to the Department of Statistics of the University of Oxford as a Departmental Lecturer, where I stayed until 2016 when I moved to King's College London as Lecturer in Statistics. I moved back to Oxford as Associate Professor of Statistics in late 2017.

Office hours

For Hilary term I will be holding office hours (for MSc related matters) on Tuesdays 14-15:00. Ideally send me an email in advance.

Research

Research Interests

I work in the intersection of probability and statistics to analyse random processes and objects, especially those arising from algorithms used in computational statistics and machine learning. I have worked extensively on the theory and methodology of sampling methods, especially Markov Chain Monte Carlo. I have also worked on random walks on lattices and groups.
At the moment I am particularly interested in the interplay between sampling, optimization and machine learning.

Publications

Benjamin Dupuis, George Deligiannidis, Umut Şimşekli (2023). Generalization Bounds with Data-dependent Fractal Dimensions.
arXiv

Providing generalization guarantees for modern neural networks has been a crucial task in statistical learning. Recently, several studies have attempted to analyze the generalization error in such settings by using tools from fractal geometry. While these works have successfully introduced new mathematical tools to apprehend generalization, they heavily rely on a Lipschitz continuity assumption, which in general does not hold for neural networks and might make the bounds vacuous. In this work, we address this issue and prove fractal geometry-based generalization bounds without requiring any Lipschitz assumption. To achieve this goal, we build up on a classical covering argument in learning theory and introduce a data-dependent fractal dimension. Despite introducing a significant amount of technical complications, this new notion lets us control the generalization error (over either fixed or random hypothesis spaces) along with certain mutual information (MI) terms. To provide a clearer interpretation to the newly introduced MI terms, as a next step, we introduce a notion of "geometric stability" and link our bounds to the prior art. Finally, we make a rigorous connection between the proposed data-dependent dimension and topological data analysis tools, which then enables us to compute the dimension in a numerically efficient way. We support our theory with experiments conducted on various settings.
```
@misc{https://doi.org/10.48550/arxiv.2302.02766,
                      doi = {10.48550/ARXIV.2302.02766},
                      
                      url = {https://arxiv.org/abs/2302.02766},
                      
                      author = {Dupuis, Benjamin and Deligiannidis, George and Şimşekli, Umut},
                      
                      keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
                      
                      title = {Generalization Bounds with Data-dependent Fractal Dimensions},
                      
                      publisher = {arXiv},
                      
                      year = {2023},
                      
                      copyright = {Creative Commons Attribution 4.0 International}
                    }
                    
                    
                    
```
A Multi-Resolution Framework for U-Nets with Applications to Hierarchical VAEs. Fabian Falck, Christopher Williams, Dominic Danks, George Deligiannidis, Christopher Yau, Chris C Holmes, Arnaud Doucet, Matthew Willetts. Accepted at NeurIPS 2022 (Oral).

Conference
Generalization Bounds with Data-dependent Fractal Dimensions. Benjamin Dupuis, George Deligiannidis, Umut Şimşekli.
arXiv

Providing generalization guarantees for modern neural networks has been a crucial task in statistical learning. Recently, several studies have attempted to analyze the generalization error in such settings by using tools from fractal geometry. While these works have successfully introduced new mathematical tools to apprehend generalization, they heavily rely on a Lipschitz continuity assumption, which in general does not hold for neural networks and might make the bounds vacuous. In this work, we address this issue and prove fractal geometry-based generalization bounds without requiring any Lipschitz assumption. To achieve this goal, we build up on a classical covering argument in learning theory and introduce a data-dependent fractal dimension. Despite introducing a significant amount of technical complications, this new notion lets us control the generalization error (over either fixed or random hypothesis spaces) along with certain mutual information (MI) terms. To provide a clearer interpretation to the newly introduced MI terms, as a next step, we introduce a notion of "geometric stability" and link our bounds to the prior art. Finally, we make a rigorous connection between the proposed data-dependent dimension and topological data analysis tools, which then enables us to compute the dimension in a numerically efficient way. We support our theory with experiments conducted on various settings.
```
@misc{https://doi.org/10.48550/arxiv.2302.02766,
                            doi = {10.48550/ARXIV.2302.02766},
                            
                            url = {https://arxiv.org/abs/2302.02766},
                            
                            author = {Dupuis, Benjamin and Deligiannidis, George and Şimşekli, Umut},
                            
                            keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
                            
                            title = {Generalization Bounds with Data-dependent Fractal Dimensions},
                            
                            publisher = {arXiv},
                            
                            year = {2023},
                            
                            copyright = {Creative Commons Attribution 4.0 International}
                          }
                          
                          
                          
```
A Continuous Time Framework for Discrete Denoising Models. Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, Arnaud Doucet. Accepted at NeurIPS 2022 (Oral).
arXiv Conference

We provide the first complete continuous time framework for denoising diffusion models of discrete data. This is achieved by formulating the forward noising process and corresponding reverse time generative process as Continuous Time Markov Chains (CTMCs). The model can be efficiently trained using a continuous time version of the ELBO. We simulate the high dimensional CTMC using techniques developed in chemical physics and exploit our continuous time framework to derive high performance samplers that we show can outperform discrete time methods for discrete data. The continuous time treatment also enables us to derive a novel theoretical result bounding the error between the generated sample distribution and the true data distribution.
```
@misc{https://doi.org/10.48550/arxiv.2205.14987,
                      doi = {10.48550/ARXIV.2205.14987},        
                      url = {https://arxiv.org/abs/2205.14987},                      
                      author = {Campbell, Andrew and Benton, Joe and De Bortoli, Valentin and Rainforth, Tom and Deligiannidis, George and Doucet, Arnaud},                      
                      keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},                      
                      title = {A Continuous Time Framework for Discrete Denoising Models},                      
                      publisher = {arXiv},                      
                      year = {2022}                  
                    }
                    
                    
```
Eugenio Clerico, Amitis Shidani, George Deligiannidis, Arnaud Doucet (2022). Chained Generalisation Bounds. Colt 2022.
arXiv Proceedings

This work discusses how to derive upper bounds for the expected generalisation error of supervised learning algorithms by means of the chaining technique. By developing a general theoretical framework, we establish a duality between generalisation bounds based on the regularity of the loss function, and their chained counterparts, which can be obtained by lifting the regularity assumption from the loss onto its gradient. This allows us to re-derive the chaining mutual information bound from the literature, and to obtain novel chained information-theoretic generalisation bounds, based on the Wasserstein distance and other probability metrics. We show on some toy examples that the chained generalisation bound can be significantly tighter than its standard counterpart, particularly when the distribution of the hypotheses selected by the algorithm is very concentrated.
```
@misc{clerico2022chained,
 								title={Chained Generalisation Bounds}, 
 								author={Eugenio Clerico and Amitis Shidani and George Deligiannidis and Arnaud Doucet},
 								year={2022},
 								eprint={2203.00977},
 								archivePrefix={arXiv},
 								primaryClass={stat.ML}
							}
                    
```
Yuyang Shi, Valentin De Bortoli, George Deligiannidis, Arnaud Doucet (2022). Conditional Simulation Using Diffusion Schr\" odinger Bridges. UAI 2022.
arXiv

Denoising diffusion models have recently emerged as a powerful class of generative models. They provide state-of-the-art results, not only for unconditional simulation, but also when used to solve conditional simulation problems arising in a wide range of inverse problems such as image inpainting or deblurring. A limitation of these models is that they are computationally intensive at generation time as they require simulating a diffusion process over a long time horizon. When performing unconditional simulation, a Schr\"odinger bridge formulation of generative modeling leads to a theoretically grounded algorithm shortening generation time which is complementary to other proposed acceleration techniques. We extend here the Schr\"odinger bridge framework to conditional simulation. We demonstrate this novel methodology on various applications including image super-resolution and optimal filtering for state-space models.
```
@misc{https://doi.org/10.48550/arxiv.2202.13460,
  									doi = {10.48550/ARXIV.2202.13460},
									url = {https://arxiv.org/abs/2202.13460},
    								author = {Shi, Yuyang and De Bortoli, Valentin and Deligiannidis, George and Doucet, Arnaud},
  									keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  									title = {Conditional Simulation Using Diffusion Schrödinger Bridges},
  									publisher = {arXiv},
  									year = {2022}
											}

                    
```
Oscar Clivio, Fabian Falck, Brieuc Lehmann, George Deligiannidis, Chris Holmes. Neural score matching for high-dimensional causal inference. AISTATS 2022.
arXiv

Traditional methods for matching in causal inference are impractical for high-dimensional datasets. They suffer from the curse of dimensionality: exact matching and coarsened exact matching find exponentially fewer matches as the input dimension grows, and propensity score matching may match highly unrelated units together. To overcome this problem, we develop theoretical results which motivate the use of neural networks to obtain non-trivial, multivariate balancing scores of a chosen level of coarseness, in contrast to the classical, scalar propensity score. We leverage these balancing scores to perform matching for high-dimensional causal inference and call this procedure neural score matching. We show that our method is competitive against other matching approaches on semi-synthetic high-dimensional datasets, both in terms of treatment effect estimation and reducing imbalance.
```
@misc{clivio2022neural,
						title={Neural Score Matching for High-Dimensional Causal Inference}, 
    					author={Oscar Clivio and Fabian Falck and Brieuc Lehmann and George Deligiannidis and Chris Holmes},
     					year={2022},
     					eprint={2203.00554},
     					archivePrefix={arXiv},
     					primaryClass={stat.ML}
					}
                    
```
Eugenio Clerico, George Deligiannidis, Arnaud Doucet (2021). Conditional Gaussian PAC-Bayes. AISTATS 2022.
Conference arXiv

Recent studies have empirically investigated different methods to train a stochastic classifier by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that the bounds obtained with this approach are tighter than those found in the literature.
```
@misc{clerico2021conditional,
      						title={Conditional Gaussian PAC-Bayes}, 
      						author={Eugenio Clerico and George Deligiannidis and Arnaud Doucet},
      						year={2021},
      						eprint={2110.11886},
      						archivePrefix={arXiv},
      						primaryClass={cs.LG}
										}
                    
```
Eugenio Clerico, George Deligiannidis, Arnaud Doucet (2021). Wide stochastic networks: Gaussian limit and PAC-Bayesian training arXiv:2106.09798.
arXiv

The limit of infinite width allows for substantial simplifications in the analytical study of overparameterized neural networks. With a suitable random initialization, an extremely large network is well approximated by a Gaussian process, both before and during training. In the present work, we establish a similar result for a simple stochastic architecture whose parameters are random variables. The explicit evaluation of the output distribution allows for a PAC-Bayesian training procedure that directly optimizes the generalization bound. For a large but finite-width network, we show empirically on MNIST that this training approach can outperform standard PAC-Bayesian methods.
```
@misc{clerico2021wide,
      title={Wide stochastic networks: Gaussian limit and PAC-Bayesian training}, 
      author={Eugenio Clerico and George Deligiannidis and Arnaud Doucet},
      year={2021},
      eprint={2106.09798},
      archivePrefix={arXiv},
      primaryClass={stat.ML}
}
                    
```
Alexander Camuto, George Deligiannidis, Murat A. Erdogdu, Mert Gürbüzbalaban, Umut Şimşekli, Lingjiong Zhu (2021). Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms. Accepted at NeurIPS ’21 (Spotlight).
arXiv

Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account in order to obtain meaningful generalization bounds, it is still theoretically not clear which properties of the data and the algorithm determine the generalization performance. In this study, we approach this problem from a dynamical systems theory perspective and represent stochastic optimization algorithms as random iterated function systems (IFS). Well studied in the dynamical systems literature, under mild assumptions, such IFSs can be shown to be ergodic with an invariant measure that is often supported on sets with a fractal structure. As our main contribution, we prove that the generalization error of a stochastic optimization algorithm can be bounded based on the `complexity' of the fractal structure that underlies its invariant measure. Leveraging results from dynamical systems theory, we show that the generalization error can be explicitly linked to the choice of the algorithm (e.g., stochastic gradient descent -- SGD), algorithm hyperparameters (e.g., step-size, batch-size), and the geometry of the problem (e.g., Hessian of the loss). We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden-layered neural networks) and algorithms (e.g., SGD and preconditioned variants), and obtain analytical estimates for our bound.For modern neural networks, we develop an efficient algorithm to compute the developed bound and support our theory with various experiments on neural networks.
```
@misc{camuto2021fractal,
                    title={Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms}, 
                        author={Alexander Camuto and George Deligiannidis and Murat A. Erdogdu and Mert Gürbüzbalaban and Umut Şimşekli and     Lingjiong Zhu},
                    year={2021},
                    eprint={2106.04881},
                    archivePrefix={arXiv},
                    primaryClass={stat.ML}
                        }
                    
```
Adrien Corenflos, James Thornton, Arnaud Doucet, George Deligiannidis (2020). Differentiable Particle Filtering via Entropy- Regularized Optimal Transport. Accepted for long presentation at ICML 2021. arXiv:2102.07850.
arXiv

Particle Filtering (PF) methods are an established class of procedures for performing inference in non-linear state-space models. Resampling is a key ingredient of PF, necessary to obtain low variance likelihood and states estimates. However, traditional resampling methods result in PF-based loss functions being non-differentiable with respect to model and PF parameters. In a variational inference context, resampling also yields high variance gradient estimates of the PF-based evidence lower bound. By leveraging optimal transport ideas, we introduce a principled differentiable particle filter and provide convergence results. We demonstrate this novel method on a variety of applications.
```
@misc{corenflos2021differentiable,
                        title={Differentiable Particle Filtering via Entropy-Regularized Optimal Transport}, 
                            author={Adrien Corenflos and James Thornton and Arnaud Doucet and George Deligiannidis},
                            year={2021},
                                eprint={2102.07850},
                            archivePrefix={arXiv},
                            primaryClass={stat.ML}
                            }
                    
```
Simsekli, U., Sener, O., Deligiannidis, G., & Erdogdu, M. A. (2020). Hausdorff Dimension, Stochastic Differential Equations, and Generalization in Neural Networks. NeurIPS ’20 (Spotlight Presentation).
arXiv

Despite its success in a wide range of applications, characterizing the generalization properties of stochastic gradient descent (SGD) in non-convex deep learning problems is still an important challenge. While modeling the trajectories of SGD via stochastic differential equations (SDE) under heavy-tailed gradient noise has recently shed light over several peculiar characteristics of SGD, a rigorous treatment of the generalization properties of such SDEs in a learning theoretical framework is still missing. Aiming to bridge this gap, in this paper, we prove generalization bounds for SGD under the assumption that its trajectories can be well-approximated by a Feller process, which defines a rich class of Markov processes that include several recent SDE representations (both Brownian or heavy-tailed) as its special case. We show that the generalization error can be controlled by the Hausdorff dimension of the trajectories, which is intimately linked to the tail behavior of the driving process. Our results imply that heavier-tailed processes should achieve better generalization; hence, the tail-index of the process can be used as a notion of “capacity metric”. We support our theory with experiments on deep neural networks illustrating that the proposed capacity metric accurately estimates the generalization error, and it does not necessarily grow with the number of parameters unlike the existing capacity metrics in the literature.
```
@misc{simekli2020hausdorff,
                        title = {Hausdorff Dimension, Stochastic Differential Equations, and Generalization in Neural Networks},
                                author = {Simsekli, Umut and Sener, Ozan and Deligiannidis, George and Erdogdu, Murat A.},
                    year = {2020},
                arxiv = {https://arxiv.org/abs/2006.09313},
                journal = {NeurIPS '20}
                        }
                    
```

Hayou, S., Clerico, E., He, B., Deligiannidis, G., Doucet, A., & Rousseau, J. (2020). Stable ResNet. AISTATS 2021.

arXiv Conference

                    @InProceedings{pmlr-v130-hayou21a,
                    title = 	 { Stable ResNet },
                    author =       {Hayou, Soufiane and Clerico, Eugenio and He, Bobby and Deligiannidis, George and Doucet, Arnaud and Rousseau, Judith},
                    booktitle = 	 {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
                    pages = 	 {1324--1332},
                    year = 	 {2021},
                    editor = 	 {Banerjee, Arindam and Fukumizu, Kenji},
                    volume = 	 {130},
                    series = 	 {Proceedings of Machine Learning Research},
                    month = 	 {13--15 Apr},
                    publisher =    {PMLR},
                    pdf = 	 {http://proceedings.mlr.press/v130/hayou21a/hayou21a.pdf},
                    url = 	 {http://proceedings.mlr.press/v130/hayou21a.html},
                    }

Cornish, R., Caterini, A. L., Deligiannidis, G., & Doucet, A. (2019). Localised Generative Flows. ICML ’20.
arXiv

We show that normalising flows become pathological when used to model targets whose supports have complicated topologies. In this scenario, we prove that a flow must become arbitrarily numerically noninvertible in order to approximate the target closely. This result has implications for all flow-based models, and especially Residual Flows (ResFlows), which explicitly control the Lipschitz constant of the bijection used. To address this, we propose Continuously Indexed Flows (CIFs), which replace the single bijection used by normalising flows with a continuously indexed family of bijections, and which can intuitively "clean up" mass that would otherwise be misplaced by a single bijection. We show theoretically that CIFs are not subject to the same topological limitations as normalising flows, and obtain better empirical performance on a variety of models and benchmarks.
```
@article{cornish2019localised,
                        title = {Localised Generative Flows},
                        author = {Cornish, Rob and Caterini, Anthony L and Deligiannidis, George and Doucet, Arnaud},
                        arxiv = {https://arxiv.org/pdf/1909.13833},
                        journal = {ICML '20},
                        year = {2019}
                        }
                        
```

Cornish, R., Caterini, A. L., Deligiannidis, G., & Doucet, A. (2019). Relaxing bijectivity constraints with continuously indexed normalising flows. ArXiv Preprint ArXiv:1909.13833.

@article{cornish2019relaxing,
              title = {Relaxing bijectivity constraints with continuously indexed normalising flows},
              author = {Cornish, Rob and Caterini, Anthony L and Deligiannidis, George and Doucet, Arnaud},
              journal = {arXiv preprint arXiv:1909.13833},
              year = {2019}
            }

M El Khribch, G Deligiannidis, D Paulin (2021). On Mixing Times of Metropolized Algorithm With Optimization Step (MAO): A New Framework
arXiv

In this paper, we consider sampling from a class of distributions with thin tails supported on Rd and make two primary contributions. First, we propose a new Metropolized Algorithm With Optimization Step (MAO), which is well suited for such targets. Our algorithm is capable of sampling from distributions where the Metropolis-adjusted Langevin algorithm (MALA) is not converging or lacking in theoretical guarantees. Second, we derive upper bounds on the mixing time of MAO. Our results are supported by simulations on multiple target distributions.
```
@misc{khribch2021mixing,
      	title={On Mixing Times of Metropolized Algorithm With Optimization Step (MAO) : A New Framework}, 
      	author={EL Mahdi Khribch and George Deligiannidis and Daniel Paulin},
      	year={2021},
      	eprint={2112.00565},
      	archivePrefix={arXiv},
      	primaryClass={stat.ML}
}
                        
```
George Deligiannidis, Valentin De Bortoli, Arnaud Doucet (2021). Quantitative Uniform Stability of the Iterative Proportional Fitting Procedure. Accepted at Annals of Applied Probability.
arXiv URL

We establish the uniform in time stability, w.r.t. the marginals, of the Iterative Proportional Fitting Procedure, also known as Sinkhorn algorithm, used to solve entropy-regularised Optimal Transport problems. Our result is quantitative and stated in terms of the 1-Wasserstein metric. As a corollary we establish a quantitative stability result for Schrödinger bridges.
```
@article{deligiannidis2021quantitative,    title={Quantitative Uniform Stability of the Iterative Proportional Fitting Procedure}, 
                                author={George Deligiannidis and Valentin De Bortoli and Arnaud Doucet},
                                year={2021},
                                eprint={2108.08129},
                                archivePrefix={arXiv},
                                primaryClass={stat.ML}
                                }
                        
```
Syed, S., Bouchard-Côté, A., Deligiannidis, G., & Doucet, A. (2021). Non-Reversible Parallel Tempering: an Embarassingly Parallel MCMC Scheme. Journal of the Royal Statistical Society, Series B,
A cool application can be found here where it was used to generate the first high-res images of the M87 black hole.
arXiv URL

Parallel tempering (PT) methods are a popular class of Markov chain Monte Carlo schemes used to explore complex high-dimensional probability distributions. These algorithms can be highly effective but their performance is contingent on the selection of a suitable annealing schedule. In this work, we provide a new perspective on PT algorithms and their tuning, based on two main insights. First, we identify and formalize a sharp divide in the behaviour and performance of reversible versus non-reversible PT methods. Second, we analyze the behaviour of PT algorithms using a novel asymptotic regime in which the number of parallel compute cores goes to infinity. Based on this approach we show that a class of non-reversible PT methods dominates its reversible counterpart and identify distinct scaling limits for the non-reversible and reversible schemes, the former being a piecewise-deterministic Markov process (PDMP) and the latter a diffusion. In particular, we identify a class of non-reversible PT algorithms which is provably scalable to massive parallel implementation, in contrast to reversible PT algorithms, which are known to collapse in the massive parallel regime. We then bring these theoretical tools to bear on the development of novel methodologies. We develop an adaptive non-reversible PT scheme which estimates the event rate of the limiting PDMP and uses this estimated rate to approximate the optimal annealing schedule. We provide a wide range of numerical examples supporting and extending our theoretical and methodological contributions. Our adaptive non-reversible PT method outperforms experimentally state-of-the-art PT methods in terms of taking less time to adapt, as well as providing better target approximations. Our scheme has no tuning parameters and appears in our simulations robust to violations of the theoretical assumption used to carry out our analysis.
```
@article{syed2019non,
                      title = {Non-Reversible Parallel Tempering: an Embarassingly Parallel MCMC Scheme},
                      author = {Syed, Saifuddin and Bouchard-C{\^o}t{\'e}, Alexandre and Deligiannidis, George and Doucet, Arnaud},
                      journal = {Journal of the Royal Statistical Society, Series B},
                      volume = {84},
                      issue = {2},
                      pages= {321--350},
                      arxiv = {https://arxiv.org/pdf/1905.02939},
                      year = {2021}
                    }
                    
```
Deligiannidis, G., Paulin, D., & Doucet, A. (2018). Randomized Hamiltonian Monte Carlo as Scaling Limit of the Bouncy Particle Sampler and Dimension-Free Convergence Rates. Annals of Applied Probability.
arXiv URL

The Bouncy Particle Sampler is a Markov chain Monte Carlo method based on a nonreversible piecewise deterministic Markov process. In this scheme, a particle explores the state space of interest by evolving according to a linear dynamics which is altered by bouncing on the hyperplane tangent to the gradient of the negative log-target density at the arrival times of an inhomogeneous Poisson Process (PP) and by randomly perturbing its velocity at the arrival times of an homogeneous PP. Under regularity conditions, we show here that the process corresponding to the first component of the particle and its corresponding velocity converges weakly towards a Randomized Hamiltonian Monte Carlo (RHMC) process as the dimension of the ambient space goes to infinity. RHMC is another piecewise deterministic non-reversible Markov process where a Hamiltonian dynamics is altered at the arrival times of a homogeneous PP by randomly perturbing the momentum component. We then establish dimension-free convergence rates for RHMC for strongly log-concave targets with bounded Hessians using coupling ideas and hypocoercivity techniques.
```
@article{deligiannidis2018randomized,
                  title = {Randomized Hamiltonian Monte Carlo as Scaling Limit of the Bouncy Particle Sampler and Dimension-Free Convergence Rates},
                  author = {Deligiannidis, George and Paulin, Daniel and Doucet, Arnaud},
                  journal = {Annals of Applied Probability},
                  volume = {31},
                  issue = {6},
                  pages = {2612--2662},
                  arxiv = {https://arxiv.org/pdf/1808.04299},
                  year = {2021}
                }
                
```
Deligiannidis, G., Doucet, A., & Rubenthaler, S. (2020). Ensemble Rejection Sampling.
arXiv

We introduce Ensemble Rejection Sampling, a scheme for exact simulation from the posterior distribution of the latent states of a class of non-linear non-Gaussian state-space models. Ensemble Rejection Sampling relies on a proposal for the high-dimensional state sequence built using ensembles of state samples. Although this algorithm can be interpreted as a rejection sampling scheme acting on an extended space, we show under regularity conditions that the expected computational cost to obtain an exact sample increases cubically with the length of the state sequence instead of exponentially for standard rejection sampling. We demonstrate this methodology by sampling exactly state sequences according to the posterior distribution of a stochastic volatility model and a non-linear autoregressive process. We also present an application to rare event simulation.
```
@article{deligiannidis2020ensemble,
                        title = {Ensemble Rejection Sampling},
                        author = {Deligiannidis, George and Doucet, Arnaud and Rubenthaler, Sylvain},
                        arxiv = {https://arxiv.org/pdf/2001.09188},
                        year = {2020}
                        }
                    
```
Deligiannidis, G., Bouchard-Côté, A., & Doucet, A. (2019). Exponential Ergodicity of the Bouncy Particle Sampler. Annals of Statistics, 47(3), 1268–1287.
arXiv URL

Non-reversible Markov chain Monte Carlo schemes based on piecewise deterministic Markov processes have been recently introduced in applied probability, automatic control, physics and statistics. Although these algorithms demonstrate experimentally good performance and are accordingly increasingly used in a wide range of applications, geometric ergodicity results for such schemes have only been established so far under very restrictive assumptions. We give here verifiable conditions on the target distribution under which the Bouncy Particle Sampler algorithm introduced in \citeP_dW_12 is geometrically ergodic. This holds whenever the target satisfies a curvature condition and has tails decaying at least as fast as an exponential and at most as fast as a Gaussian distribution. This allows us to provide a central limit theorem for the associated ergodic averages. When the target has tails thinner than a Gaussian distribution, we propose an original modification of this scheme that is geometrically ergodic. For thick-tailed target distributions, such as t-distributions, we extend the idea pioneered in \citeJ_G_12 in a random walk Metropolis context. We apply a change of variable to obtain a transformed target satisfying the tail conditions for geometric ergodicity. By sampling the transformed target using the Bouncy Particle Sampler and mapping back the Markov process to the original parameterisation, we obtain a geometrically ergodic algorithm.
```
@article{deligiannidis2017exponential,
                  title = {Exponential Ergodicity of the Bouncy Particle Sampler},
                  author = {Deligiannidis, George and Bouchard-C{\^o}t{\'e}, Alexandre and Doucet, Arnaud},
                  journal = {Annals of Statistics},
                  url = {https://projecteuclid.org/euclid.aos/1550026836},
                  volume = {47},
                  number = {3},
                  page = {1268--1287},
                  arxiv = {arxiv:1705.04579.pdf},
                  year = {2019}
                }
                
```
Heng, J., Bishop, A. N., Deligiannidis, G., & Doucet, A. (2019). Controlled sequential monte carlo. Annals of Statistics, 48(5), 2904–2929.
arXiv

Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques to approximate high-dimensional probability distributions and their normalizing constants. They have found numerous applications in statistics and related fields as they can be applied to perform state estimation for non-linear non-Gaussian state space models and Bayesian inference for complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which have a crucial impact on their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. We provide theoretical analysis of our proposed methodology and demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications.
```
@article{heng2017controlled,
                  title = {Controlled sequential monte carlo},
                  author = {Heng, Jeremy and Bishop, Adrian N and Deligiannidis, George and Doucet, Arnaud},
                  journal = {Annals of Statistics},
                  volume = {48},
                  number = {5},
                  page = {2904-2929},
                  arxiv = {https://arxiv.org/pdf/1708.08396},
                  year = {2019}
                }
                
```

Doucet, A., Deligiannidis, G., Middleton, L., & Jacob, P. (2019). Unbiased smoothing using Particle Independent Metropolis-Hastings. AISTATS 2019.

@article{doucet2019unbiased,
                  title = {Unbiased smoothing using Particle Independent Metropolis-Hastings},
                  author = {Doucet, A and Deligiannidis, G and Middleton, L and Jacob, P},
                  year = {2019},
                  journal = {AISTATS 2019},
                  publisher = {MLR Press}
                }

Schmon, S. M., Doucet, A., & Deligiannidis, G. (2019). Bernoulli Race Particle Filters. AISTATS 2019.

arXiv

@article{schmon2019bernoulli,
                      title = {Bernoulli Race Particle Filters},
                      author = {Schmon, Sebastian M and Doucet, Arnaud and Deligiannidis, George},
                      arxiv = {https://arxiv.org/pdf/1903.00939},
                      journal = {AISTATS 2019},
                      year = {2019}
                    }

Cornish, R., Vanetti, P., Bouchard-Côté, A., Deligiannidis, G., & Doucet, A. (2019). Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets. ICML 2019.
arXiv

Bayesian inference via standard Markov Chain Monte Carlo (MCMC) methods such as Metropolis–Hastings is too computationally intensive to handle large datasets, since the cost per step usually scales like O(n) in the number of data points \(n\). We propose the Scalable Metropolis–Hastings (SMH) kernel that exploits Gaussian concentration of the posterior to require processing on average only \(O(1)\)or even \(O(1/\sqrtn)\)based on a combination of factorized acceptance probabilities, procedures for fast simulation of Bernoulli processes, and control variate ideas. Contrary to many MCMC subsampling schemes such as fixed step-size Stochastic Gradient Langevin Dynamics, our approach is exact insofar as the invariant distribution is the true posterior and not an approximation to it. We characterise the performance of our algorithm theoretically, and give realistic and verifiable conditions under which it is geometrically ergodic. This theory is borne out by empirical results that demonstrate overall performance benefits over standard Metropolis-Hastings and various subsampling algorithms.
```
@article{cornish2019scalable,
                  title = {Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets},
                  author = {Cornish, Robert and Vanetti, Paul and Bouchard-C{\^o}t{\'e}, Alexandre and Deligiannidis, George and Doucet, Arnaud},
                  arxiv = {https://arxiv.org/pdf/1901.09881},
                  journal = {ICML 2019},
                  year = {2019}
                }
                
```

Schmon, S. M., Deligiannidis, G., Doucet, A., & Pitt, M. K. (2020). Large Sample Asymptotics of the Pseudo-Marginal Method. Biometrika.

arXiv URL

@article{10.1093/biomet/asaa044,
                                author = {Schmon, S M and Deligiannidis, G and Doucet, A and Pitt, M K},
                                title = "{Large-sample asymptotics of the pseudo-marginal method}",
                                journal = {Biometrika},
                                year = {2020},
                                month = {07},
                                abstract = "{The pseudo-marginal algorithm is a variant of the MetropolisâHastings algorithm which samples asymptotically from a probability distribution when it is only possible to estimate unbiasedly an unnormalized version of its density. Practically, one has to trade off the computational resources used to obtain this estimator against the asymptotic variances of the ergodic averages obtained by the pseudo-marginal algorithm. Recent works on optimizing this trade-off rely on some strong assumptions, which can cast doubts over their practical relevance. In particular, they all assume that the distribution of the difference between the log-density, and its estimate is independent of the parameter value at which it is evaluated. Under regularity conditions we show that as the number of data points tends to infinity, a space-rescaled version of the pseudo-marginal chain converges weakly to another pseudo-marginal chain for which this assumption indeed holds. A study of this limiting chain allows us to provide parameter dimension-dependent guidelines on how to optimally scale a normal random walk proposal, and the number of Monte Carlo samples for the pseudo-marginal method in the large-sample regime. These findings complement and validate currently available results.}",
                                issn = {0006-3444},
                                doi = {10.1093/biomet/asaa044},
                                url = {https://doi.org/10.1093/biomet/asaa044},
                                note = {asaa044},
                                eprint = {https://academic.oup.com/biomet/advance-article-pdf/doi/10.1093/biomet/asaa044/33487358/asaa044.pdf},
                            }

Deligiannidis, G., Doucet, A., & Pitt, M. K. (2018). The Correlated Pseudo-Marginal Method. JRSSB, 80(5), 839–870.
URL

The pseudo-marginal algorithm is a popular variant of the Metropolis–Hastings scheme which allows us to sample asymptotically from a target probability density π, when we are only able to estimate an unnormalized version of π pointwise unbiasedly. It has found numerous applications in Bayesian statistics as there are many scenarios where the likelihood function is intractable but can be estimated unbiasedly using Monte Carlo samples. Using many samples will typically result in averages computed under this chain with lower asymptotic variances than the corresponding averages that use fewer samples. For a fixed computing time, it has been shown in several recent contributions that an efficient implementation of the pseudo-marginal method requires the variance of the log-likelihood ratio estimator appearing in the acceptance probability of the algorithm to be of order 1, which in turn usually requires scaling the number N of Monte Carlo samples linearly with the number \(T\)of data points. We propose a modification of the pseudo-marginal algorithm, termed the correlated pseudo-marginal algorithm, which is based on a novel log-likelihood ratio estimator computed using the difference of two positively correlated log-likelihood estimators. We show that the parameters of this scheme can be selected such that the variance of this estimator is order 1 as \(N,T\to ∞\)whenever \(N/T\to 0\). By combining these results with the Bernstein-von Mises theorem, we provide an analysis of the performance of the correlated pseudo-marginal algorithm in the large T regime. In our numerical examples, the efficiency of computations is increased relative to the standard pseudo-marginal algorithm by more than 20 fold for values of \(T\)of a few hundreds to more than 100 fold for values of T of around 10,000-20,000.
```
@article{deligiannidis2015correlated,
                  title = {The Correlated Pseudo-Marginal Method},
                  author = {Deligiannidis, George and Doucet, Arnaud and Pitt, Michael K},
                  journal = {JRSSB},
                  volume = {80},
                  number = {5},
                  pages = {839--870},
                  year = {2018},
                  url = {https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/rssb.12280}
                }
                
```
Deligiannidis, G., & Lee, A. (2018). Which ergodic averages have finite asymptotic variance? Annals of Applied Probability, 28(4), 2309–2334.
arXiv URL

We show that the class of \(L^ 2\)functions for which ergodic averages of a reversible Markov chain have finite asymptotic variance is determined by the class of \(L^ 2\)functions for which ergodic averages of its associated jump chain have finite asymptotic variance. This allows us to characterise completely which ergodic averages have finite asymptotic variance when the Markov chain is an independence sampler. In addition, we obtain a simple sufficient condition for all ergodic averages of \(L^ 2\)functions of the primary variable in a pseudo-marginal Markov chain to have finite asymptotic variance.
```
@article{deligiannidis2018ergodic,
                  title = {Which ergodic averages have finite asymptotic variance?},
                  author = {Deligiannidis, George and Lee, Anthony},
                  journal = {Annals of Applied Probability},
                  volume = {28},
                  number = {4},
                  pages = {2309--2334},
                  year = {2018},
                  url = {https://projecteuclid.org/euclid.aoap/1533780274},
                  arxiv = {https://arxiv.org/pdf/1606.08373},
                  publisher = {IMS}
                }
                
```
Middleton, L., Deligiannidis, G., Doucet, A., & Jacob, P. E. (2018). Unbiased Markov chain Monte Carlo for intractable target distributions. To Appear in Electronic Journal of Statistics.
arXiv

Performing numerical integration when the integrand itself cannot be evaluated point-wise is a challenging task that arises in statistical analysis, notably in Bayesian inference for models with intractable likelihood functions. Markov chain Monte Carlo (MCMC) algorithms have been proposed for this setting, such as the pseudo-marginal method for latent variable models and the exchange algorithm for a class of undirected graphical models. As with any MCMC algorithm, the resulting estimators are justified asymptotically in the limit of the number of iterations, but exhibit a bias for any fixed number of iterations due to the Markov chains starting outside of stationarity. This" burn-in" bias is known to complicate the use of parallel processors for MCMC computations. We show how to use coupling techniques to generate unbiased estimators in finite time, building on recent advances for generic MCMC algorithms. We establish the theoretical validity of some of these procedures by extending existing results to cover the case of polynomially ergodic Markov chains. The efficiency of the proposed estimators is compared with that of standard MCMC estimators, with theoretical arguments and numerical experiments including state space models and Ising models.
```
@article{middleton2018unbiased,
              title = {Unbiased Markov chain Monte Carlo for intractable target distributions},
              author = {Middleton, Lawrence and Deligiannidis, George and Doucet, Arnaud and Jacob, Pierre E},
              arxiv = {https://arxiv.org/pdf/1807.08691},
              journal = {to appear in Electronic Journal of Statistics},
              year = {2018}
            }
            
```
Vanetti, P., Bouchard-Côté, A., Deligiannidis, G., & Doucet, A. (2017). Piecewise-Deterministic Markov Chain Monte Carlo. Https://Arxiv.org/Pdf/1707.05296.
A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a deterministic dynamics which is modified using a Markov transition kernel at random event times. These methods enjoy remarkable features including the ability to update only a subset of the state components while other components implicitly keep evolving and the ability to use an unbiased estimate of the gradient of the log-target while preserving the target as invariant distribution. However, they also suffer from important limitations. The deterministic dynamics used so far do not exploit the structure of the target. Moreover, exact simulation of the event times is feasible for an important yet restricted class of problems and, even when it is, it is application specific. This limits the applicability of these techniques and prevents the development of a generic software implementation of them. We introduce novel MCMC methods addressing these shortcomings. In particular, we introduce novel continuous-time algorithms relying on exact Hamiltonian flows and novel non-reversible discrete-time algorithms which can exploit complex dynamics such as approximate Hamiltonian dynamics arising from symplectic integrators while preserving the attractive features of continuous-time algorithms. We demonstrate the performance of these schemes on a variety of applications.
```
@article{vanetti2017piecewise,
  title = {Piecewise-Deterministic Markov Chain Monte Carlo},
  author = {Vanetti, Paul and Bouchard-C{\^o}t{\'e}, Alexandre and Deligiannidis, George and Doucet, Arnaud},
  journal = {https://arxiv.org/pdf/1707.05296},
  year = {2017}
}
```
Doucet, A., Pitt, M. K., Deligiannidis, G., & Kohn, R. (2015). Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika, 102(2), 295–313.
arXiv URL

When an unbiased estimator of the likelihood is used within a Metropolis–Hastings chain, it is necessary to trade off the number of Monte Carlo samples used to construct this estimator against the asymptotic variances of the averages computed under this chain. Using many Monte Carlo samples will typically result in Metropolis–Hastings averages with lower asymptotic variances than the corresponding averages that use fewer samples; however, the computing time required to construct the likelihood estimator increases with the number of samples. Under the assumption that the distribution of the additive noise introduced by the loglikelihood estimator is Gaussian with variance inversely proportional to the number of samples and independent of the parameter value at which it is evaluated, we provide guidelines on the number of samples to select. We illustrate our results by considering a stochastic volatility model applied to stock index returns.
```
@article{doucet2015efficient,
              title = {Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator},
              author = {Doucet, Arnaud and Pitt, Michael K and Deligiannidis, George and Kohn, Robert},
              journal = {Biometrika},
              volume = {102},
              number = {2},
              pages = {295--313},
              year = {2015},
              publisher = {Oxford University Press},
              url = {https://academic.oup.com/biomet/article-abstract/102/2/295/246727},
              arxiv = {https://arxiv.org/pdf/1210.1871}
            }
            
```

Fahim Faizi, Pedro J Buigues, George Deligiannidis & Edina Rosta (2020). Simulated tempering with irreversible Gibbs sampling techniques.
arXiv

We present here two novel algorithms for simulated tempering simulations, which break detailed balance condition (DBC) but satisfy the skewed detailed balance to ensure invariance of the target distribution. The irreversible methods we present here are based on Gibbs sampling and concern breaking DBC at the update scheme of the temperature swaps. We utilise three systems as a test bed for our methods: an MCMC simulation on a simple system described by a 1D double well potential, the Ising model and MD simulations on Alanine pentapeptide (ALA5). The relaxation times of inverse temperature, magnetic susceptibility and energy density for the Ising model indicate clear gains in sampling efficiency over conventional Gibbs sampling techniques with DBC and also over the conventionally used simulated tempering with Metropolis-Hastings (MH) scheme. Simulations on ALA5 with large number of temperatures indicate distinct gains in mixing times for inverse temperature and consequently the energy of the system compared to conventional MH. With no additional computational overhead, our methods were found to be more efficient alternatives to conventionally used simulated tempering methods with DBC. Our algorithms should be particularly advantageous in simulations of large systems with many temperature ladders, as our algorithms showed a more favorable constant scaling in Ising spin systems as compared with both reversible and irreversible MH algorithms. In future applications, our irreversible methods can also be easily tailored to utilize a given dynamical variable other than temperature to flatten rugged free energy landscapes.
```
@article{faizi2020simulated,
                              title={Simulated tempering with irreversible Gibbs sampling techniques},
                              author={Faizi, Fahim and Buigues, Pedro J and Deligiannidis, George and Rosta, Edina},
                              journal={arXiv preprint arXiv:2008.05976},
                              year={2020}
                            }
                            
```
Deligiannidis, G., Maurer, S., & Tretyakov, M. V. (2020). Random walk algorithm for the Dirichlet problem for parabolic integro-differential equation. BIT, Numerical Mathematics.
Journal arXiv

We consider stochastic differential equations driven by a general Lévy processes (SDEs) with infinite activity and the related, via the Feynman-Kac formula, Dirichlet problem for parabolic integro-differential equation (PIDE). We approximate the solution of PIDE using a numerical method for the SDEs. The method is based on three ingredients:(i) we approximate small jumps by a diffusion;(ii) we use restricted jump-adaptive time-stepping; and (iii) between the jumps we exploit a weak Euler approximation. We prove weak convergence of the considered algorithm and present an in-depth analysis of how its error and computational cost depend on the jump activity level. Results of some numerical experiments, including pricing of barrier basket currency options, are presented.
```
@article{deligiannidis2020random,
                        title = {Random walk algorithm for the Dirichlet problem for parabolic integro-differential equation},
                        author = {Deligiannidis, G and Maurer, S and Tretyakov, MV},
                        arxiv = {https://arxiv.org/pdf/2001.05531},
                        year = {2020}
                            }
                            
```
Faizi, F., Deligiannidis, G., & Rosta, E. (2020). Efficient Irreversible Monte Carlo samplers. J. Chem. Theory Comput., 16:4, 2124–2138.
arXiv Journal

We present here two irreversible Markov chain Monte Carlo algorithms for general discrete state systems, one of the algorithms is based on the random-scan Gibbs sampler for discrete states and the other on its improved version, the Metropolized-Gibbs sampler. The algorithms we present incorporate the lifting framework with skewed detailed balance condition and construct irreversible Markov chains that satisfy the balance condition. We have applied our algorithms to 1D 4-state Potts model. The integrated autocorrelation times for magnetisation and energy density indicate a reduction of the dynamical scaling exponent from z≈1 to z≈1/2. In addition, we have generalized an irreversible Metropolis-Hastings algorithm with skewed detailed balance, initially introduced by Turitsyn et al.(2011) for the mean field Ising model, to be now readily applicable to classical spin systems in general; application to 1D 4-state Potts model indicate a square root reduction of the mixing time at high temperatures.
```
@article{faizi2019efficient,
                  title = {Efficient Irreversible Monte Carlo samplers},
                  author = {Faizi, Fahim and Deligiannidis, George and Rosta, Edina},
                  journal = {J. Chem. Theory Comput.},
                  volume={16},
                  number={4},
                  pages={2124–-2138},
                  arxiv = {https://arxiv.org/pdf/1911.00883},
                  year = {2019}
                }
                
```

Deligiannidis, G., Gouëzel, S., & Kosloff, Z. (2021). Boundary of the Range of a random walk and the Fölner property. Electronic Journal of Probability. 26, 1-39.
arXiv URL

The range process \(R_n\)of a random walk is the collection of sites visited by the random walk up to time \(n\). In this work we deal with the question of whether the range process of a random walk or the range process of cocycle over an ergodic transformation is almost surely a Fölner sequence and show the following results &#58 (a) The size of the inner boundary \( |∂R_n | \)of the range of the recurrent aperiodic random walks on \(\mathbbZ^2\)with finite variance and aperiodic random walks on \(\mathbbZ\)in the standard domain of attraction of the Cauchy distribution, divided by \(n /\!\log^2(n)\)converges to a constant almost surely. (b)We establish a formula for the Fölner \(log^2 (n)\)asymptotic of transient cocycles over an ergodic probability preserving transformation and use it to show that for transient random walk on groups which are not virtually cyclic, for almost every path, the range is not a Fölner sequence. (c) For aperiodic random walks in the domain of attraction of symmetric \(α\)-stable distributions with \(1<α≤2\), we prove a sharp polynomial upper bound for the decay at infinity of \(|∂R_n|/|R_n|\). This last result shows that the range process of these random walks is almost surely a Fölner sequence.
```
@article{10.1214/21-EJP667,
                      author = {George Deligiannidis and S{\'e}bastien Gou{\"e}zel and Zemer Kosloff},
                      title = {{The boundary of the range of a random walk and the Følner property}},
                      volume = {26},
                      journal = {Electronic Journal of Probability},
                      number = {none},
                      publisher = {Institute of Mathematical Statistics and Bernoulli Society},
                      pages = {1 -- 39},
                      year = {2021},
                      doi = {10.1214/21-EJP667},
                      URL = {https://doi.org/10.1214/21-EJP667}
                      }
                      
                
```
Deligiannidis, G., & Kosloff, Z. (2017). Relative Complexity of Random Walks in Random Scenery in the absence of a weak invariance principle for the local times. Annals of Probability, 45(4), 2505–2532. Retrieved from https://projecteuclid.org/euclid.aop/1502438433
arXiv URL

We answer the question of Aaronson about the relative complexity of Random Walks in Random Sceneries driven by either aperiodic two dimensional random walks, two- dimensional Simple Random walk, or by aperiodic random walks in the domain of attraction of the Cauchy distribution. A key step is proving that the range of the random walk satisfies the F ̈olner property almost surely.
```
@article{deligiannidis2017relative,
              title = {Relative Complexity of Random Walks in Random Scenery in the absence of a weak invariance principle for the local times},
              author = {Deligiannidis, George and Kosloff, Zemer},
              journal = {Annals of Probability},
              volume = {45},
              number = {4},
              pages = {2505--2532},
              year = {2017},
              url = {https://projecteuclid.org/euclid.aop/1502438433},
              arxiv = {https://arxiv.org/pdf/1506.00422}
            }
            
```
Deligiannidis, G., & Utev, S. (2016). Optimal bounds for the variance of self-intersection local times. International Journal of Stochastic Analysis, 2016. Retrieved from https://www.hindawi.com/journals/ijsa/2016/5370627/abs/
URL

For a \(\mathbbZ^d\)- valued random walk \( (S_n)_n∈\mathbbN_0 \)in \( \mathbbZ^d \), let \(l(n,x) \)be its local time at the site \(x ∈\mathbbZ^d \). For \( α∈\mathbbN \)define the \( α\)-fold self intersection local time as \(L_n (α)=\sum_x l(n,x)^α\). Also let \(L_n^\mathsfSRW(α)\)be the corresponding quantities for the simple random walk in \(\mathbbZ^d\). Without imposing any moment conditions, we show that the variance of the self-intersection local time of any genuinely \(d\)-dimensional random walk is bounded above by the corresponding quantity for the simple symmetric random walk, that is \( \mathrmvar (L_n(α)) = O\big(\mathrmvar\big(L_n^\mathsfSRW(α)\big)\big)\). In particular, for any genuinely d-dimensional random walk, with \(d≥4\), we have \(\mathrmvar(L_n(α)) = O(n)\). On the other hand, in dimensions \(d≤3\)we show that if the behaviour resembles that of simple random walk, in the sense that \(\liminf_n\to ∞ \mathrmvar(L_n(α))/\mathrmvar(L_n^\mathsfSRW(α))>0\), then the increments of the random walk must have zero mean and finite second moment.
```
@article{deligiannidis2016optimal,
                          title = {Optimal bounds for the variance of self-intersection local times},
                          author = {Deligiannidis, George and Utev, Sergey},
                          journal = {International Journal of Stochastic Analysis},
                          volume = {2016},
                          url = {https://www.hindawi.com/journals/ijsa/2016/5370627/abs/},
                          year = {2016},
                          publisher = {Hindawi}
                        }
                        
```
Deligiannidis, G., Peligrad, M., & Utev, S. (2014). Asymptotic variance of stationary reversible and normal Markov processes. Electronic Journal of Probability, 20, 1–26. Retrieved from https://projecteuclid.org/euclid.ejp/1465067126
URL

We obtain necessary and sufficient conditions for the regular variation of the variance of partial sums of functionals of discrete and continuous-time stationary Markov processes with normal transition operators. We also construct a class of Metropolis-Hastings algorithms which satisfy a central limit theorem and invariance principle when the variance is not linear in \(n\).
```
@article{deligiannidis2014asymptotic,
                      title = {Asymptotic variance of stationary reversible and normal Markov processes},
                      author = {Deligiannidis, George and Peligrad, Magda and Utev, Sergey},
                      journal = {Electronic Journal of Probability},
                      volume = {20},
                      pages = {1--26},
                      year = {2014},
                      url = {https://projecteuclid.org/euclid.ejp/1465067126}
                    }
                    
```

Deligiannidis, G., & Utev, S. (2013). Variance of partial sums of stationary sequences. Annals of Probability, 41(5), 3606–3616. Retrieved from https://projecteuclid.org/euclid.aop/1378991850

URL

@article{deligiannidis2013variance,
                          title = {Variance of partial sums of stationary sequences},
                          author = {Deligiannidis, George and Utev, Sergey},
                          journal = {Annals of Probability},
                          volume = {41},
                          number = {5},
                          pages = {3606--3616},
                          year = {2013},
                          url = {https://projecteuclid.org/euclid.aop/1378991850}
                        }

Deligiannidis, G., & Utev, S. A. (2011). Asymptotic variance of the self-intersections of stable random walks using Darboux-Wiener theory. Siberian Mathematical Journal, 52(4), 639–650. Retrieved from https://link.springer.com/article/10.1134/S0037446611040082

URL

@article{deligiannidis2011asymptotic,
                          title = {Asymptotic variance of the self-intersections of stable random walks using Darboux-Wiener theory},
                          author = {Deligiannidis, George and Utev, Sergei Aleksandrovich},
                          journal = {Siberian mathematical journal},
                          volume = {52},
                          number = {4},
                          pages = {639--650},
                          year = {2011},
                          url = {https://link.springer.com/article/10.1134/S0037446611040082},
                          publisher = {Springer New York}
                        }

Deligiannidis, G., & Utev, S. (2010). An asymptotic variance of the self-intersections of random walks.

arXiv

@article{deligiannidis2010asymptotic,
              title = {An asymptotic variance of the self-intersections of random walks},
              author = {Deligiannidis, George and Utev, Sergey},
              arxiv = {https://arxiv.org/pdf/1004.4845},
              year = {2010}
            }

Deligiannidis, G. (2009). Some results associated with random walks (PhD thesis). PhD thesis, University of Nottingham 2010. Retrieved from http://eprints.nottingham.ac.uk/13104/1/537670.pdf

URL

@phdthesis{deligiannidis2009some,
                      title = {Some results associated with random walks},
                      author = {Deligiannidis, George},
                      year = {2009},
                      url = {http://eprints.nottingham.ac.uk/13104/1/537670.pdf},
                      school = {PhD thesis, University of Nottingham 2010}
                    }

Deligiannidis, G., Le, H., & Utev, S. (2009). Optimal Stopping for processes with independent increments, and applications. Journal of Applied Probability, 46(4), 1130–1145. Retrieved from https://www.cambridge.org/core/journals/journal-of-applied-probability/article/optimal-stopping-for-processes-with-independent-increments-and-applications/B7E6499862E770195C01A7710E0E243F
URL

In this paper we present an explicit solution to the infinite-horizon optimal stopping problem for processes with stationary independent increments, where reward functions admit a certain representation in terms of the process at a random time. It is shown that it is optimal to stop at the first time the process crosses a level defined as the root of an equation obtained from the representation of the reward function. We obtain an explicit formula for the value function in terms of the infimum and supremum of the process, by making use of the Wiener–Hopf factorization. The main results are applied to several problems considered in the literature, to give a unified approach, and to new optimization problems from the finance industry.
```
@article{deligiannidis2009optimal,
                  title = {Optimal Stopping for processes with independent increments, and applications},
                  author = {Deligiannidis, G and Le, H and Utev, S},
                  journal = {Journal of Applied Probability},
                  volume = {46},
                  number = {4},
                  pages = {1130--1145},
                  year = {2009},
                  publisher = {Applied Probability Trust},
                  url = {https://www.cambridge.org/core/journals/journal-of-applied-probability/article/optimal-stopping-for-processes-with-independent-increments-and-applications/B7E6499862E770195C01A7710E0E243F}
                }
                
```

Selected Talks

Session on PDMPs and hypo-coercivity, MCQMC, Oxford, 2020
Mathematical Conversations, Institute for Advanced Study, Princeton, 2020
Bayescomp, Florida 2020
European Meeting of Statisticians, Palermo 2019
Probability Seminar, Warwick, February 2019.
SIAM Conference on High Dimensional Inference and Monte Carlo Techniques, Warwick 2019
Statistics Seminar, Bristol, October 2018.
Bayesian Computation for High-Dimensional Statistical Models, IMS, Singapore, September 2018.
Opening talk of Applied Mathematics Session, 3rd UK India Frontiers of Science Meeting, Royal Society, Chicheley Hall, May 2018.
BayesComp, Session on PDMPs, Barcelona, March 2018.
Machine Learning Seminar, Department of Information Technology, Uppsala, January 2018.
Department of Statistics, Warwick, June 2017.
Department of Statistics, Harvard University, May 2017.
Stochastic Analysis Seminar, Imperial College, December 2016.
Midlands Probability Seminar, University of Warwick, May 2016.
Mathematical Finance Seminar, Mathematical Institute, Oxford, March 2016.
Christmas Workshop on Sequential Monte Carlo, Imperial College, December 2015.
Probability Colloquium, Maxwell Institute, Edinburgh, September 2015.
Probability and Statistics Seminar, University of Bristol, March 2015.
Limit theorems in probability and dynamics, {CIRM, Marseilles}, July 2014.

Selected Talks

Athens Probability Colloquium , March 2023
PRAIRIE Colloquium , December 2022
ESSEC Statistics Seminar , May 2022
AUEB (ΟΠΑ) Statistics Seminar , October 2021
Imperial Statistics Seminar, October 2021
Session on PDMPs and hypo-coercivity, MCQMC, Oxford, 2020
Mathematical Conversations, Institute for Advanced Study, Princeton, 2020
Bayescomp, Florida 2020
European Meeting of Statisticians, Palermo 2019
Probability Seminar, Warwick, February 2019.
SIAM Conference on High Dimensional Inference and Monte Carlo Techniques, Warwick 2019
Statistics Seminar, Bristol, October 2018.
Bayesian Computation for High-Dimensional Statistical Models, IMS, Singapore, September 2018.
Opening talk of Applied Mathematics Session, 3rd UK India Frontiers of Science Meeting, Royal Society, Chicheley Hall, May 2018.
BayesComp, Session on PDMPs, Barcelona, March 2018.
Machine Learning Seminar, Department of Information Technology, Uppsala, January 2018.
Department of Statistics, Warwick, June 2017.
Department of Statistics, Harvard University, May 2017.
Stochastic Analysis Seminar, Imperial College, December 2016.
Midlands Probability Seminar, University of Warwick, May 2016.
Mathematical Finance Seminar, Mathematical Institute, Oxford, March 2016.
Christmas Workshop on Sequential Monte Carlo, Imperial College, December 2015.
Probability Colloquium, Maxwell Institute, Edinburgh, September 2015.
Probability and Statistics Seminar, University of Bristol, March 2015.
Limit theorems in probability and dynamics, {CIRM, Marseilles}, July 2014.

Teaching

Advanced Simulation SC5 (2019-2020)

an image alt text

Lectures

Monday, 16:00-17:00, LG01 Department of Statistics
Tuesday, 11-12:00, LG01 Department of Statistics.

Classes

Undergraduates:

Tuesday, weeks 2,5,7, TT1 13:30-15:00
Thursday, weeks 2,5,7, TT1 10-11:30

MSc

Tuesday, weeks 3, 5, 7, 8 16-17:00

Problem Sheets

Problem Sheet 0, this is a preparatory problem sheet covering some prerequisites.
Problem Sheet 1, Due Monday, wk 2 @ 9 a.m. (UG only).
Problem Sheet 2, Due Monday, wk 5 @ 9 a.m. (UG only).

Solutions to R exercises

Notes

These may be updated as we go.

Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7 (old reversible jump MCMC no longer in the syllabus)
Chapter 7
Chapter 8
Chapter 9

Slides

These will be updated as we go, however only minor changes will be made. Feel free to use these to prepare ahead of the lecture.

Disclaimer

Disclaimer This course is based on the material developed by previous instructors, including Arnaud Doucet, Pierre E. Jacob, Rémi Bardenet, George Deligiannidis, Lawrence M. Murray, Tigran Nagapetyan, Patrick Rebeschini and Paul Vanetti.

Modern Statistical Theory (StatML CDT)

Notes

Notes
Slides

SB21 Foundations of Statistical Inference

I will be updating these from time to time, but not very regularly. For the most up-to-date notes and problem sheets please visit the Canvas website.

Notes

MSc in Statistical Science

Offering external projects for the MSc in Statistical Science

If you are reading this page then you may be an industrial partner, or an academic or postdoc from another department of the university interested in co-supervising an MSc project for students on the MSc in Statistical Science offered by the Department of Statistics.

We are very happy to have projects proposed by members of other departments or industrial partners and such projects have been very successful in the past.

Here are a few key facts to keep in mind:

Projects run for 3 months from June to early September. The deadline for submitting the project is usually the second Monday of September.
The topics can vary greatly, but the project must demonstrate a good command of advanced statistical techniques acquired in the duration of the degree.
Projects proposed externally, ie by industry or other departments, need to be signed-off and co-supervised by an internal supervisor.
The deadline for submitting external projects is Friday wk 4 of Hilary term, ie for 2022 the 11th of February.
The project will then be advertised to potential internal supervisors who will be in contact with you to discuss and prepare the formal proposal.
Unfortunately not all projects will be picked up, but these may be resubmitted for consideration the following year.
The templates can be found here in pdf and in docx. There you can see the information that is usually required.
For projects with industry partners please also indicate whether a confidentiality agreement is needed. These are handled by Research Services and they often take some time so the sooner we know about this the better.

The timeline is roughly as follows:

Projects are approved and released by wk 10-11 of HT, roughly mid-end of March.
Students submit their top 10 choices in order of preference in wk 0.
Projects are allocated by wk 2-3 in TT.
Students start work on the project after the end of their exams around wk 6-7 of TT.

If you are interested in submitting an external MSc project please send me an email by the 15th of January 2022 with a single document (.pdf .doc .txt) including your contact details and a brief description that I can advertise to academics in the Department of Statistics. I will then bring you in touch with anyone interested and you can compile and submit the proposal in collaboration with them. As it often takes a bit of time for this to happen it is crucial that you send me the proposal by the 15th of January.