Nonlocal Methods for Arbitrary Data Sources

Informations projet

NoMADS

N° de convention de subvention: 777826

DOI

10.3030/777826

Projet clôturé

Date de signature de la CE 8 Novembre 2017

Date de début 1 Mars 2018

Date de fin 31 Août 2023

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 1 111 500,00

Contribution de l’UE

€ 1 111 500,00

Coordonné par

FRIEDRICH-ALEXANDER-UNIVERSITAET ERLANGEN-NUERNBERG
Germany

Periodic Reporting for period 2 - NoMADS (Nonlocal Methods for Arbitrary Data Sources)

Période du rapport: 2020-03-01 au 2023-08-31

Résumé du contexte et des objectifs généraux du projet

This research network addresses problems with data that can be interpreted and/or processed via nonlocal relationships. Typical examples include 3D point-cloud data used in several industrial applications, data derived from imaging techniques in biomedicine, remote sensing for earth observation and conservation, or scans of historical documents from cultural heritage. The key issues to be addressed in this project are basic processing of data (e.g. reconstruction, filtering, denoising, or inpainting) as well as methods that extract advanced information from the data, like clustering or decomposition. These goals are to be achieved using a merely data-driven approach trying to exploit ephemeral patterns and self-similarity at larger distances hidden in the data, which are called ‘nonlocal methods’ and are often realized on weighted graphs. The latter are an active field of research in mathematics and computer science with a strong overlap to modern machine learning, in particular several current deep learning architectures.
For this sake, NoMADS builds on a large multidisciplinary network of universities and companies, bringing together a strong international group of leading researchers from mathematics (applied and computational analysis, statistics, and optimisation), computer vision, and data mining. A major objective is to significantly increase understanding and applicability of nonlocal methods in a wide range of applications that covers a variety of different data sources. The overall objectives of this project are the theoretical understanding of nonlocal relationships of data and associated mathematical operators, the efficient implementation of numerical algorithms for nonlocal problems, and their application to real-world problems in industry and society.

During the implementation of this project our research network has discovered fundamental mathematical principles for the characterization of nonlocal operators both in a discrete setting as well as in continuous mathematics and has established important relationships between these two worlds. Furthermore, we were able to develop new efficient algorithms, that exploit the geometry of given data and consequently boost the performance of software implementations using these methods. Finally, many interesting approaches and ideas have been investigated on real-world problems that allowed to bridge the gap between theory of nonlocal operators and their future use in industrial applications.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

Strong progress has been made in all work packages. The first concerns the theory of nonlocal operators and spectral decompositions, which was further developed in a variety of publications within the network. Moreover, different flows and iterative schemes to compute eigenfunctions or spectral decompositions were presented applied to novel setups, e.g. large graph clustering problems.

In a row of works we have been developing graph-based classification approaches that are characterised by their ability to capture guiding rules for clustering high-dimensional data into pre-specified classes being guided by only a small fraction of the data being labeled (minimal supervision) and their amenity to deeply learned features (turning them into geometric deep learning algorithms). In these works, we demonstrate that these graph-based methods can achieve state-of-the-art classification performance in a range of applications (including classification of chest x-rays, landcover classification, etc.) while requiring a minimal amount of labeled data.

Phase-field and Mumford-Shah type models on graphs as well as their continuum limits have been further developed. Examples are an MBO-scheme allowing for the efficient implementation of nonlocal Ginzburg-Landau theory and semi-supervised clustering, the continuum limit of Mumford-Shah models on large graph, as well as nonlocal limits of p-Laplacian variational and evolution problems.

Efficient computational and optimization methods for nonlocal and learning problems have been pushed further, in particular by randomized methods. We have added important results to the understanding as well as application and efficient implementation of such approaches. Examples of the first are the convergence analysis of a stochastic proximal gradient method and the bias-variance analysis of a stochastic gradient method, leading to accelerated versions. An example application is positron emission tomography (PET) reconstruction, which promises to revolutionise how PET reconstruction is done in the clinic.

Various real world problems have been tackled during the project. Examples are novel ways to analyze intracellular flow data, methods for visibility estimation in point clouds, automated restoration and interpretation of illuminated ancient manuscripts, as well as artefact and shadow removal in photographs.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

Exciting progress has been made concerning the connection of variational problems of graphs and machine learning, in particular deep learning, which is an area where we expect further results far beyond the state of the art and with high impact. In particular limits of many deep layers in residual neural networks could be derived in the spirit of variational limits on graphs, which will further stimulate research between the core approaches in NoMADS and the highly timely topic of deep learning. Deep neural networks for solving inverse problems have been investigated and task-induced neural network reconstruction as well as deeply learned (adversarial) regularisers in variational reconstruction approaches have been proposed. Due to the increasing presence of deep learning techniques in daily life we expect a strong societal impact of this direction.
We were able to derive and analyze a new class of graph-based operators, which is not bound to values in Euclidean vector spaces but maps to more general Riemannian manifolds. For this we had to adapt the fundamental metrics used in the traditional graph operators, e.g. graph p-Laplacians, in order to measure distances on manifolds. This new development allows for novel applications with manifold-valued data, e.g. in diffuse tensor imaging of the human brain. After the investigation of graph p-Laplace operators within the scope of NoMADS, we realized that this family of operators can be generalized to the more abstract structure of so-called hypergraphs. These mathematical objects are not restricted to model relationships between entities (i.e. the modeled data in our applications) via pairwise connections as in traditional graphs, but allows to group any number of vertices in hyperedges. Our recent investigation of hypergraphs and related nonlocal operators has shown some first promising results for modeling network interactions such as opinion formation as well as in image processing. We expect valuable insights from our research for the scientific community.
The COVID-19 pandemic created real-world applications of high societal impact and we were able to apply novel and efficient methods investigated in the scope of NoMADS that could directly be applied to data related to COVID-19, e.g. for the diagnosis and prognosis for COVID-19 on the basis of chest X-ray images and additional clinical data

Simultaneous motion correction and image reconstruction for clinical MRI data

Simultaneous image registration and superresolution model for hyperspectral data

Hierarchical graph representations for efficient point cloud simplification

Simultaneous motion correction and image reconstruction for clinical PET data

Nonlinear eigenfunctions of the graph 1-Laplacian for data clustering in machine learning

A sketch of a neural network for shadow segmentation in videos

Nonlinear gradient flows for computation of distance maps on weighted graphs for arbitrary data

Nonlinear gradient flows for decomposition of images in different scales of detail