As the scale and complexity of available data increase, developing rigorous understanding of the computational properties of statistical procedures has become a key scientific priority of our century. In line with such priority, this project is developing a mathematical theory of computational scalability for Bayesian learning methods, with a focus on extremely popular high-dimensional and hierarchical models. Unlike most research on these topics, the project integrates computational and statistical aspects in the analysis of Bayesian learning algorithms, providing novel insight into the interaction between commonly used model structures and fitting algorithms. In particular, the project derives a broad collection of results for popular Bayesian computation algorithms, especially Markov chain Monte Carlo ones, in a variety of modeling frameworks, such as random-effect, shrinkage, hierarchical and nonparametric ones. These are routinely used for various statistical tasks, such as multilevel regression, factor analysis and variable selection in various disciplines ranging from political science to genomics.
Throughout the project, the research team involved in the project is tackling several open problems in Bayesian computation. In the first scientific theme, we develop a rigorous mathematical theory of the computational behavior of Bayesian hierarchical models, analyzing algorithmic performance in sparse, structured settings and advancing convergence diagnostics via dimensionality reduction techniques for MCMC algorithms. In the second theme, we focus on the computational cost of high-dimensional Bayesian inference, linking Bayesian asymptotics with Markov chain theory to design provably scalable algorithms. Finally, we study robustness of commonly used algorithms to data heterogeneity, model misspecification, and tail behavior, leading to the development of more stable and generalizable computational techniques.
The results have direct implications on the design of novel and more scalable computational schemes, as well as on the optimization of existing ones. Focus is given to develop algorithms with provably linear overall cost both in the number of datapoints and unknown parameters. The project contributes to significantly reduce the gap between theory and practice in Bayesian computation and allow practitioners to fully benefit of the huge potential of the Bayesian and probabilistic modeling in popular data science pipelines.
