Prof Patrick Rebeschini has been awarded a €2m ERC Consolidator Grant in Computer Science and Informatics.
The project aims to develop novel theoretical foundations for machine learning based on the paradigm of implicit regularisation. The goal is to design a new algorithmic framework that can structurally combine notions of statistical optimality with requirements of computational efficiency. The project will create a world-leading centre in statistical learning theory, recruiting a group of PDRAs working at the intersection of statistics, probability, and optimisation.
More information
In the era of Big Data—characterized by large, high-dimensional and distributed datasets—we are increasingly faced with the challenge of establishing scalable methodologies that can achieve optimal statistical guarantees under computational constraints. To fundamentally address this challenge, new paradigms need to be established.
Over the past 50 years, statistical learning theory has relied on the framework of explicit regularization to control the model complexity of estimators. By design, this approach decouples notions of statistical optimality and computational efficiency and, in applications, often leads to expensive model selection procedures. This framework faces fundamental limitations to explain the practical success of modern machine learning paradigms, which are based on running simple gradient descent methodologies without any explicit effort to control model complexity.
Overcoming these limitations prompts the investigation of the implicit regularization properties of iterative algorithms, namely the bias enforced as a by-product of the very choice of optimization routine and tuning parameters. Implicit regularization structurally combines statistics with optimization and it has the potential to promote the design of new algorithmic paradigms built around the notion of statistical and computational optimality. However, to fully realize its potential, several challenges need to be overcome.
This project aims to develop a general theory of implicit regularization that can optimally address fundamental primitives in modern applications—e.g. involving sparse and low-rank noisy models, decentralized multi-agent learning, and adaptive and robust procedures—and establish novel cross-disciplinary connections with far-reaching consequences.
This goal will be achieved by combining non-asymptotic tools for the study of random structures in high-dimensional probability with the general framework of mirror descent from optimization and online learning.
What does a statistician do for the England football team?
From squad selection to modelling how footballs behave at altitude, statistician Matt Penn explains how data is helping shape the modern game, and why coaches will always matter more than the numbers.
New evidence suggests vast hidden magma systems inside Mars
Researchers from the Departments of Earth Science and Statistics have found evidence that Mars once hosted enormous, Earth-like magmatic systems deep below its surface – even though the planet lacks the plate tectonics long considered essential for this kind of geological complexity. The findings open up new possibilities for how rocky planets become habitable.
Finding a needle in the genomic haystack: Targeting rare genes using statistical outliers
In statistical modelling, extreme outliers are often written off as 'noise'. But a new study by researchers from Oxford's Department of Statistics and Big Data Institute published this week in The American Journal of Human Genetics reverses that principle, using these outliers as the basis of a targeting system for locating rare, high-impact genetic mutations.