Periodic Reporting for period 4 - MAJORIS (Majoration-Minimization algorithms for Image Processing)
Reporting period: 2024-07-01 to 2025-06-30
Constantly improving signal acquisition devices, from spectrometry to medical imaging machines, impose to work with increasingly large and complex data. From the mathematical perspective, this implies the need to solve large-scale optimization problems involving many variables. The MAJORIS project aims to propose a new generation of optimization algorithms, relying on the key concept of majorization-minimization (MM), that remain efficient in the context of “big data” processing, by tackling several challenging questions regarding algorithm design (acceleration strategies), convergence analysis (non-convex costs, inexact schemes), and practical implementation (usage of massively parallel and/or distributed architectures). The methodological outcomes of MAJORIS are expected to yield a set of novel methods for the efficient processing of large amounts of data and variables.
Work-Package 1: Accelerated MM approaches
Modern signal and image recovery problems require powerful optimization frameworks to handle high dimensionality while providing reliable numerical solutions in a reasonable time. In this perspective, asynchronous parallel optimization algorithms have received an increasing attention by overcoming memory limitation issues and communication bottlenecks.
Our main results around these aspects are (1) proposition of block alternating, and block distributed MM algorithms, with convergence guarantees, to tackle efficiently large scale non-convex differentiable optimization problems, (2) design of deep unrolled MM algorithms with GPU implementation, and supervised data-driven strategy to learn optimal hyper-parameters, (3) application of the methodological developments to the resolution of inverse problems of image and signal processing, and to sparse graph inference problems.
Work-Package 2: Robust MM approaches
In WP2, we analyzed the impact of numerical errors arising at the practical level on the convergence of the method with the aim to propose MM algorithms robust to inaccurately implemented steps. Our main results are (1) new fundamental results of the stability properties of MM and proximal algorithms under the situation of adjoint mismatch, frequently encountered in tomographic image reconstruction, (2) new convergence results of MM algorithms facing stochastic errors in the gradient computation, beyond the convex case, (3) application to the resolution of inverse problems of imaging, and to machine learning scenarios.
Work-Package 3 – Flexible MM approaches
MM algorithms are powerful tools for optimization at a large scale. Nonetheless, for several classes of problems, open questions regarding their convergence guarantees and even their applicability remained open, that we proposed to investigate in this work-package. Our main results are (1) proposition of new MM method for solving large scale constrained differentiable optimization problems based on local search and trust-region mechanism, (2) implementation of MM strategies to boost sampling exploration in the context of Monte-Carlo algorithms, (3) development of MM algorithms going beyond Euclidian metrics, to deal with Bregman divergence minimization problems, arising from various situations in variational inference, proposition law adaptation, Poisson image reconstruction.
The results have been disseminated in multiple journal and conference publications. An updated list can be found in the project's website https://opis-inria.eu/majoris/(opens in new window).
Modern signal and image recovery problems require powerful optimization frameworks to handle high dimensionality while providing reliable numerical solutions in a reasonable time. In this perspective, asynchronous parallel optimization algorithms have received an increasing attention by overcoming memory limitation issues and communication bottlenecks.
Our main results around these aspects are (1) proposition of block alternating, and block distributed MM algorithms, with convergence guarantees, to tackle efficiently large scale non-convex differentiable optimization problems, (2) design of deep unrolled MM algorithms with GPU implementation, and supervised data-driven strategy to learn optimal hyper-parameters, (3) application of the methodological developments to the resolution of inverse problems of image and signal processing, and to sparse graph inference problems.
Work-Package 2: Robust MM approaches
In WP2, we analyzed the impact of numerical errors arising at the practical level on the convergence of the method with the aim to propose MM algorithms robust to inaccurately implemented steps. Our main results are (1) new fundamental results of the stability properties of MM and proximal algorithms under the situation of adjoint mismatch, frequently encountered in tomographic image reconstruction, (2) new convergence results of MM algorithms facing stochastic errors in the gradient computation, beyond the convex case, (3) application to the resolution of inverse problems of imaging, and to machine learning scenarios.
Work-Package 3 – Flexible MM approaches
MM algorithms are powerful tools for optimization at a large scale. Nonetheless, for several classes of problems, open questions regarding their convergence guarantees and even their applicability remained open, that we proposed to investigate in this work-package. Our main results are (1) proposition of new MM method for solving large scale constrained differentiable optimization problems based on local search and trust-region mechanism, (2) implementation of MM strategies to boost sampling exploration in the context of Monte-Carlo algorithms, (3) development of MM algorithms going beyond Euclidian metrics, to deal with Bregman divergence minimization problems, arising from various situations in variational inference, proposition law adaptation, Poisson image reconstruction.
The results have been disseminated in multiple journal and conference publications. An updated list can be found in the project's website https://opis-inria.eu/majoris/(opens in new window).
Work-Package 1: Accelerated MM approaches
First derivation of convergence guarantees for a distributed version of the MM memory gradient method in the challenging non-convex setting.
First deep unrolled architecture in the context of variational-Bayes image processing.
Work-Package 2: Robust MM approaches
First study of the effect of an adjoint mismatch on the proximal gradient convergence (that we further extend to primal-dual proximal algorithms), in the case of generic priors associated with a nonlinear proximal operator, in infinite dimension.
Work-Package 3: Flexible MM approaches
First work that combines MM strategy within the exterior penalty framework, with original convergence proof for local surrogates.
First work proposing fast importance sampling algorithm for effective training of Bayesian neural networks.
First derivation of convergence guarantees for a distributed version of the MM memory gradient method in the challenging non-convex setting.
First deep unrolled architecture in the context of variational-Bayes image processing.
Work-Package 2: Robust MM approaches
First study of the effect of an adjoint mismatch on the proximal gradient convergence (that we further extend to primal-dual proximal algorithms), in the case of generic priors associated with a nonlinear proximal operator, in infinite dimension.
Work-Package 3: Flexible MM approaches
First work that combines MM strategy within the exterior penalty framework, with original convergence proof for local surrogates.
First work proposing fast importance sampling algorithm for effective training of Bayesian neural networks.