Periodic Reporting for period 4 - C-SENSE (Exploiting low dimensional models in sensing, computation and signal processing)
Période du rapport: 2021-03-01 au 2022-08-31
The aim of this project was to develop the next generation of compressive and computational sensing and processing techniques.
The ability to identify and exploit good signal representations is pivotal in many signal and data processing tasks. During the last decade sparse representations have provided stunning performance gains for applications such as: imaging coding, computer vision, super-resolution microscopy and most recently in MRI, achieving many-fold acceleration through compressed sensing (CS). However, iterative reconstruction techniques are often not adopted in commercial imaging/sensing systems as they typically incur at least an order of magnitude more computation than traditional techniques.
Today imaging and sensing are becoming increasingly important as a core enabler for our data centric society, whether this be for informing advanced technology such as self driving cars, or improved characterization and diagnosis in medicine, or even for fundamental data driven scientific discovery.
There is therefore a need for a new framework for generalized computationally accelerated sensing and processing techniques capable of dealing with the increasingly complex sensing challenges through science and technology. This project aims to enable us to tackle a new generation of data-driven, physics-aware and task-orientated sensing systems in application domains such as advanced radar, CT and MR imaging and emerging sensing modalities such as time-of-flight cameras.
The overall objectives of the project were to set out a fundamental theoretical framework, develop and analyse new algorithms, signal models and data processing tools, accommodating everything from physical laws to data-driven models and neural networks, exploiting underlying low dimensional structure to reduce computation and sensing costs, as well as enhance performance.
To this end we have made substantial advances to generalize compressed sensing theory to infinite dimensional systems and data driven signal models. We have advanced the field in performance and understanding of compressive algorithms for machine learning and imaging, developing a range of algorithms that significantly reduce computation and data storage.
Most significantly we have developed new theoretical and algorithmic frameworks for learning to image without ground truth data - something that we believe offers the breakthrough capability of true data-driven knowledge discovery in sensing and imaging.
The ability to identify and exploit good signal representations is pivotal in many signal and data processing tasks. During the last decade sparse representations have provided stunning performance gains for applications such as: imaging coding, computer vision, super-resolution microscopy and most recently in MRI, achieving many-fold acceleration through compressed sensing (CS). However, iterative reconstruction techniques are often not adopted in commercial imaging/sensing systems as they typically incur at least an order of magnitude more computation than traditional techniques.
Today imaging and sensing are becoming increasingly important as a core enabler for our data centric society, whether this be for informing advanced technology such as self driving cars, or improved characterization and diagnosis in medicine, or even for fundamental data driven scientific discovery.
There is therefore a need for a new framework for generalized computationally accelerated sensing and processing techniques capable of dealing with the increasingly complex sensing challenges through science and technology. This project aims to enable us to tackle a new generation of data-driven, physics-aware and task-orientated sensing systems in application domains such as advanced radar, CT and MR imaging and emerging sensing modalities such as time-of-flight cameras.
The overall objectives of the project were to set out a fundamental theoretical framework, develop and analyse new algorithms, signal models and data processing tools, accommodating everything from physical laws to data-driven models and neural networks, exploiting underlying low dimensional structure to reduce computation and sensing costs, as well as enhance performance.
To this end we have made substantial advances to generalize compressed sensing theory to infinite dimensional systems and data driven signal models. We have advanced the field in performance and understanding of compressive algorithms for machine learning and imaging, developing a range of algorithms that significantly reduce computation and data storage.
Most significantly we have developed new theoretical and algorithmic frameworks for learning to image without ground truth data - something that we believe offers the breakthrough capability of true data-driven knowledge discovery in sensing and imaging.
This project has developed new theory and algorithms for signal processing, optimisation and advanced image reconstruction. In each case the focus has been the role of underlying low-dimensional structure within the data that can potentially be exploited for computation or performance gains.
In advancing the fundamental theory of sensing and imaging we have:
- developed a new foundational theory for learning to image, giving conditions for the identifiability of low dimensional signal models from incomplete measurements. This theory exploits a combination of low dimensional signal models with either assumed symmetry from the problem physics or access to a sufficient diversity of measurement operators. This theoretical framework has further driven a new class of unsupervised learning techniques (see below) that offers the potential for genuine data-driven knowledge discovery through imaging.
- generalized the classical compressed sensing theory to account for infinite dimensional systems and data driven signal models
- extended compressive learning theory to include the task of compressive independent component analysis (ICA) and proposed various associated algorithms. This enables ICA to be performed on big data problems where it is impossible to store the vast quantities of data.
In our work on machine learning, optimization and imaging algorithms we have:
- defined a new class of self supervised learning algorithms for learning to image from incomplete measurements without ground truth data, exploiting our new theory on learning to image. These algorithms have shown a breakthrough capability of learning state of the art image reconstruction without access to any ground truth data.
- developed theoretically rigorous accelerated extensions of the approximate message passing/ expectation propagation algorithms
- new accelerated compressed and stochastic gradient algorithms for imaging and machine learning, including identifying conditions and limitations for the use of stochastic optimization in imaging tasks
- developed a mean field theory analysis of a class of unsupervised overparameterized neural networks (deep image priors), highlighting links to traditional non-local filters in image processing
In our applied work we have:
- new protocols and algorithms for accelerated quantitative MRI (magnetic resonance fingerprinting)
- new algorithms for data driven quantitative CT imaging, estimating electron density form a single polyenergetic source in CT imaging
- developed a new compressed data transfer technique for photon counting lidar systems, called sketched Lidar, that, inspired by the ideas from compressive learning, enables a massive reduction of on-chip data to be stored and transferred off. This work is subject of a patent application and has been licenced for evaluation with a leading lidar manufacturer.
- applied the ideas from our underpinning work to develop improved algorithms for a range of other applications from compressed optical coherent tomography to radiation portal imaging.
In advancing the fundamental theory of sensing and imaging we have:
- developed a new foundational theory for learning to image, giving conditions for the identifiability of low dimensional signal models from incomplete measurements. This theory exploits a combination of low dimensional signal models with either assumed symmetry from the problem physics or access to a sufficient diversity of measurement operators. This theoretical framework has further driven a new class of unsupervised learning techniques (see below) that offers the potential for genuine data-driven knowledge discovery through imaging.
- generalized the classical compressed sensing theory to account for infinite dimensional systems and data driven signal models
- extended compressive learning theory to include the task of compressive independent component analysis (ICA) and proposed various associated algorithms. This enables ICA to be performed on big data problems where it is impossible to store the vast quantities of data.
In our work on machine learning, optimization and imaging algorithms we have:
- defined a new class of self supervised learning algorithms for learning to image from incomplete measurements without ground truth data, exploiting our new theory on learning to image. These algorithms have shown a breakthrough capability of learning state of the art image reconstruction without access to any ground truth data.
- developed theoretically rigorous accelerated extensions of the approximate message passing/ expectation propagation algorithms
- new accelerated compressed and stochastic gradient algorithms for imaging and machine learning, including identifying conditions and limitations for the use of stochastic optimization in imaging tasks
- developed a mean field theory analysis of a class of unsupervised overparameterized neural networks (deep image priors), highlighting links to traditional non-local filters in image processing
In our applied work we have:
- new protocols and algorithms for accelerated quantitative MRI (magnetic resonance fingerprinting)
- new algorithms for data driven quantitative CT imaging, estimating electron density form a single polyenergetic source in CT imaging
- developed a new compressed data transfer technique for photon counting lidar systems, called sketched Lidar, that, inspired by the ideas from compressive learning, enables a massive reduction of on-chip data to be stored and transferred off. This work is subject of a patent application and has been licenced for evaluation with a leading lidar manufacturer.
- applied the ideas from our underpinning work to develop improved algorithms for a range of other applications from compressed optical coherent tomography to radiation portal imaging.
The project has made the following advancements beyond the state of the art the in theory, algorithm and application solutions:
- We have developed a new foundational theory for learning to image, giving conditions for the identifiability of low dimensional signal models from incomplete measurements.
- We have generalized the classical compressed sensing theory to account for infinite dimensional systems and data driven signal models
- We have extended compressive learning theory to include the task of compressive independent component analysis (ICA) for big data scenarios and proposed various associated algorithms.
- We have defined a new class of self supervised learning algorithms for learning to image from incomplete measurements without ground truth data, showing a breakthrough capability of learning state of the art image reconstruction without access to any ground truth data.
- We have developed theoretically rigorous accelerated extensions of the approximate message passing/ expectation propagation algorithms
- We have advanced compressed and stochastic gradient algorithms for imaging and machine learning, including identifying conditions and limitations for the use of stochastic optimization in imaging tasks
- developed a mean field theory analysis of a class of unsupervised overparameterized neural networks (deep image priors), highlighting links to traditional non-local filters in image processing
In our applied work we have:
- new protocols and algorithms for accelerated quantitative MRI (magnetic resonance fingerprinting)
- new algorithms for data driven quantitative CT imaging, estimating electron density form a single polyenergetic source in CT imaging
- developed a new compressed data transfer technique for photon counting lidar systems, called sketched Lidar, that enables a massive reduction of on-chip data to be stored or transferred.
- We have developed a new foundational theory for learning to image, giving conditions for the identifiability of low dimensional signal models from incomplete measurements.
- We have generalized the classical compressed sensing theory to account for infinite dimensional systems and data driven signal models
- We have extended compressive learning theory to include the task of compressive independent component analysis (ICA) for big data scenarios and proposed various associated algorithms.
- We have defined a new class of self supervised learning algorithms for learning to image from incomplete measurements without ground truth data, showing a breakthrough capability of learning state of the art image reconstruction without access to any ground truth data.
- We have developed theoretically rigorous accelerated extensions of the approximate message passing/ expectation propagation algorithms
- We have advanced compressed and stochastic gradient algorithms for imaging and machine learning, including identifying conditions and limitations for the use of stochastic optimization in imaging tasks
- developed a mean field theory analysis of a class of unsupervised overparameterized neural networks (deep image priors), highlighting links to traditional non-local filters in image processing
In our applied work we have:
- new protocols and algorithms for accelerated quantitative MRI (magnetic resonance fingerprinting)
- new algorithms for data driven quantitative CT imaging, estimating electron density form a single polyenergetic source in CT imaging
- developed a new compressed data transfer technique for photon counting lidar systems, called sketched Lidar, that enables a massive reduction of on-chip data to be stored or transferred.