Community Research and Development Information Service - CORDIS

FP7

RAPID Report Summary

Project reference: 335491
Funded under: FP7-IDEAS-ERC

Mid-Term Report Summary - RAPID (Rapid parsimonious modelling)

Parsimony, preferring a simple explanation to a more complex one, is probably one of the most intuitive heuristic principles widely adopted in the modeling of nature. The past two decades of research have shown the power of parsimonious representation in a vast variety of applications from diverse domains of science, especially in signal and image processing, computer vision, and machine learning. A common form of parsimony is sparsity, postulating that data can be represented by a small number of non-zero coefficients in an appropriate dictionary. This model is satisfied by many classes of natural signals and can be efficiently pursued through convex, greedy, and other optimization methods. Other manifestations of parsimony successfully capturing more intricate data structures are various forms of structured sparsity and low rank matrices and tensors.

Existing parsimonious modeling approaches are model-centric, following the same pattern: First, an objective comprising a fitting term and parsimony-promoting penalty terms is constructed; next, an iterative optimization algorithm is invoked to minimize the objective, pursuing either the parsimonious representation of the data in a given dictionary, or the dictionary itself. Despite the steady improvement of iterative optimization tools, their inherently sequential structure and data-dependent complexity and latency constitute a major limitation in many applications requiring real-time performance or involving large-scale data. Also, representations obtained via optimization are hard to incorporate into a higher-level optimization problem, practically restricting existing parsimonious models to unsupervised regimes. Consequently, these models are typically generative rather than discriminative, rendering difficult several important applications such as similarity learning for large-scale information retrieval.

The goal of the RAPID project is to introduce a paradigm shift in the construction and application of parsimonious models. We proposed to move the emphasis from the model to the pursuit algorithm, and develop a process-centric view of parsimonious modeling, in which the learning of the pursuit process replaces dictionary learning. Very recently, we were able to establish a theoretical framework analysing inexact versions of projected gradient descent-type of algorithms with a fixed number of iterations, showing how the iteration has to look like in order to achieve best signal recovery quality. We believe that this is the first step toward a systematic study of the trade-off between computational complexity of a pursuit process and the reconstruction error it achieves, in the spirit of the rate-distortion trade-off studied in information theory.

The computational embodiment of these theoretical results has lead to non-iterative parsimonious modeling and pursuit algorithms with guaranteed optimal performance within a given complexity budget. We also introduced several theoretical results shedding light on the connections between parsimonious modelling, and compressed sensing in particular, and deep learning. We showed several applications, mainly in computational imaging, in which the proposed departure from the classical iterative techniques brings a dramatic improvement in performance. We demonstrated a real-time FPGA implementation of several image reconstruction tasks, including extended depth of field imaging, that are beyond the reach of the standard iterative techniques. We are confident that these applications are just the tip of the iceberg of what can be done with the proposed methodology, and we are currently embarking on more ambitious applications related to coded exposure and aperture imaging and time-of-flight depth sensing.

The proposed departure from the classical iterative techniques also allows novel uses of parsimony in discriminative tasks. Thus far, this has been demonstrated on tasks like large-scale content-based retrieval and shape analysis, where we were able to produce state-of-the-art results. Furthermore, our work on the use of sparsity in shape analysis led to a fundamental theoretical result related to graph isomorphism problems.
Finally, the development of medical applications of the proposed methodology have been consolidated into a standalone ERC Proof-of-Concept project NETEEG, dedicated to high-resolution EEG neuroimaging using an acquisition technique resembling in principle the idea of compressed sensing.

Contact

Davison, Mark (Director Excellent Science Programme)
Tel.: +972 4 829 3097
E-mail
Record Number: 187766 / Last updated on: 2016-08-23