Network Inpainting via Optimal Transport

Project Information

NIOT

Grant agreement ID: 101103631

DOI

10.3030/101103631

Project closed

EC signature date 8 March 2023

Start date 1 April 2023

End date 31 March 2025

Funded under

• Marie Skłodowska-Curie Actions (MSCA)

Total cost

No data

EU contribution

€ 210 911,04

Coordinated by

UNIVERSITETET I BERGEN
Norway

Periodic Reporting for period 1 - NIOT (Network Inpainting via Optimal Transport)

Reporting period: 2023-04-01 to 2025-03-31

Summary of the context and overall objectives of the project

Current State and Challenges in Vascular Imaging

Medical imaging techniques like Magnetic Resonance Imaging (MRI) allow for non-invasive visualization of blood vessel networks. While valuable for diagnostics and research, MRI data can present challenges, such as noise and artifacts, sometimes leading to incomplete or seemingly disconnected representations of the vascular system, especially for smaller vessels. These fragmented vascular network models can complicate accurate medical diagnosis and treatment planning. They also pose difficulties for researchers using computational models to simulate blood flow, as incomplete data can reduce the reliability of simulation outcomes.

Addressing the Needs of Clinicians and Mathematicians

Clinicians require reliable methods to obtain accurate vascular models from scans, while mathematicians work on developing algorithms to process imaging data and reconstruct these networks. The NIOT (Numerical challenges in Optimal Transport) project aims to contribute to this area by applying Optimal Transport (OT) theory, a branch of Mathematics studying the optimal strategy for moving resources at the least cost possible.

Project Goals and Approach

The project's goal is to develop computational tools for improved reconstruction of blood vessel networks from medical imaging. Leveraging OT principles, which are well-suited for analyzing branched structures and pathways in graph-based systems, the project will adapt existing expertise in OT solvers to address the specific challenges of vascular network reconstruction. This work is supported by the host institution's capabilities in processing vascular data.

Long-Term Contributions and Collaborative Benefits

In the longer term, the tools developed aim to streamline the process of converting raw MRI scan data into formats suitable for numerical simulations. This could enable more routine use of patient-specific models in computational studies.
By providing improved data for medical assessment and computational modeling, this research seeks to foster closer interaction between medical practitioners and mathematical scientists. More reliable data can lead to better predictions, which in turn can support the clinical use of mathematical tools, creating a positive feedback loop for continued development.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Foundational Development and 2D Framework

The initial phase of the project focused on integrating theoretical tools from branched optimal transport into the practical framework of inverse problems and inpainting. This foundational work aimed to leverage the strengths of optimal transport for tasks involving data completion and reconstruction. To support this, a dedicated software tool was developed to facilitate the testing of various scenarios in 2D networks. This software enabled the validation of new ideas and provided a platform for comparison with existing software and alternative methods. The methodologies developed during these initial steps, along with the capabilities of the 2D software, have been detailed in a scientific manuscript. This paper is currently under review for publication. In conjunction with the preprint of this manuscript, the 2D testing software was publicly released to ensure transparency and allow for broader academic use.

We showed how the proposed strategy was able to consistently "rewire" corrupted network, where other classical in-painting methods fail in capturing the whole structure of the network.

Transition to 3D MRI Data and Solver Enhancement

Following the advancements in 2D, the project's focus shifted towards the more complex challenge of processing 3D MRI data, specifically for vascular network reconstruction. A significant effort was dedicated to enhancing the computational efficiency of the underlying solver to handle the increased data dimensionality and complexity, while keeping it open to further developments. Key achievements in this area include the successful parallelization of the software, allowing it to leverage multi-core architectures. Furthermore, the software was designed for ease of setup with various parameter combinations and is capable of running on high-performance computing servers equipped with adequate computational resources.

Achievements in 3D Vascular Network Reconstruction

By late 2024 and early 2025, the project achieved a significant milestone: the first successful reconstructions of 3D vascular networks from MRA (Magnetic Resonance Angiography) data at full resolution scale, handling datasets with approximately 27 million degrees of freedom. These initial reconstructions demonstrated the capability of the developed tools to process complex, real-world medical imaging data.

However, analysis of these first 3D reconstructions indicated the need for further refinement of the algorithm. Specifically, enhancements are required to better incorporate physiological information inherent in the data. This includes identifying and utilizing regions where blood vessels are unlikely to be present and developing methods for the suppression of imaging artifacts, a process being guided by consultation with medical professionals.

Current Activities and Outlook

Currently, the project is actively working on integrating these physiological constraints and artifact suppression strategies into the 3D solver. Different approaches and parameter combinations are being systematically tested to improve the accuracy and clinical relevance of the reconstructed vascular networks. The ongoing work aims to produce more robust and physiologically faithful 3D reconstructions, ultimately enhancing the utility of these tools for medical applications.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The project has yielded several significant results, establishing a novel approach to network reconstruction, particularly from complex medical imaging data.

A key achievement is the establishment, for the first time, of a theoretical and numerical framework capable of coherently reconstructing networks from corrupted or incomplete data. This framework, rooted in branched optimal transport, distinguishes itself from other in-painting approaches that often fail to capture the essential global structure of the network. Our method prioritizes structural integrity, leading to more realistic and complete network representations. This results are summarized in the attached image.

Furthermore, the method and the accompanying software developed during this project are the first of their kind specifically demonstrated to operate effectively on complex 3D data derived from Magnetic Resonance Angiography (MRA). This capability to handle real-world, high-resolution medical imaging datasets represents a significant step forward in applying advanced mathematical reconstruction techniques directly to clinical data.

At the point of writing this report, while successful 3D reconstructions have been achieved, further tuning of the algorithms is underway. This ongoing work is focused on incorporating a wider range of physiological information into the reconstruction process to enhance the anatomical accuracy and clinical relevance of the generated vascular networks.

Looking ahead, to transform the algorithm developed during this project into a standard tool utilized by both medical doctors and mathematicians, further research is identified as crucial. A primary research path involves enhancing the reliability of the algorithm and reducing the need for extensive human oversight in its operation. This will necessitate scaling its capabilities to robustly handle diverse datasets from various MRA acquisition protocols and automatically integrating information from multiple MRI sources. For instance, expanding beyond the current MRI Time-of-flight techniques to include complementary information from other sequences like T1- or T2-weighted images could significantly improve segmentation and reconstruction accuracy in complex anatomical regions.

A second promising research path involves leveraging the developed software as a preprocessing and validation tool for datasets intended for machine learning applications. As machine learning methods become increasingly prevalent in medical image analysis, the ability of our software to generate high-fidelity network reconstructions or to complete sparse datasets can provide valuable ground truth or augmented data, thereby improving the training and performance of such AI-driven approaches.

Compasion between the NIOT and other inpainting methods for the reconstruction of a simple network