Final Report Summary - RAPID (Rapid parsimonious modelling) Parsimony, preferring a simple explanation to a more complex one, is probably one of the most intuitive heuristic principles widely adopted in the modeling of nature. The past two decades of research have shown the power of parsimonious representation in a vast variety of applications from diverse domains of science, especially in signal and image processing, computer vision, and machine learning. The simplest form of parsimony is sparsity, postulating that data can be represented by a small number of non-zero coefficients in an appropriate dictionary. More complex models capturing intricate data structures rely on various forms of structured sparsity and low rank assumptions. These approaches are very much “model-centric”, following the same pattern: First, an objective comprising a data fitting term and parsimony-promoting penalty terms is constructed; next, an iterative optimization algorithm is invoked to minimize the objective. In 2012, when the RAPID project proposal was written, deep learning methods only started to show their spectacular successes; today, seven years since, they have established themselves as a de facto standard in practically every imaginable machine learning task. These methods largely lack the theoretical grounding of the parsimonious data models, and approach the modeling in a very process-centric way: the sequence of operations applied to the input data (e.g. in a convolutional neural network) is what creates the model. The purpose of the RAPID project was, essentially, to bridge between these very different world views. On the theoretical side, we have developed several interesting insights into the working on deep neural networks based, for example, on tools from non-linear compressed sensing. We also studied the family of inexact projected gradient algorithms and showed how it explains the success of LISTA-type neural networks. We studied the tradeoffs between the model accuracy and its computational complexity, and proposed several practical computational techniques based on neural network quantization that achieved state-of-the-art performance in low-complexity and low-power neural network inference. The computational tools developed within the project were applied to numerous tasks in various domains of signal and image processing, imaging, machine vision and learning; the common denominator of these tasks was the fact that all relied on an efficient solution to an inverse problem. Among the most successful applications was a computational camera we developed based on the idea of a phase-coded aperture that allowed extended depth-of-field imaging and passive depth imaging. We showed that learning the inverse operator directly together with the forward model (the camera hardware) greatly improves the accuracy of a specific end task. We successfully applied a similar methodology to ultrasound and MRI medical imaging. Another unexpected inverse problem that we tackled with was the reconstruction of shape from the lower part of its Laplacian spectrum. The problem, known as “hearing the shape of the drum” is known to be generally unsolvable in theory; however, we showed a computational framework allowing to solve it in practice for many classes of naturally occurring 2D and 3D shapes. In the domain of shape analysis, we also developed state-of-the-art tools for computing intrinsic correspondence between non-Euclidean domains. We introduced a novel point of view, according to which correspondence is considered as a denoising/deblurring problem on the product space of the two shapes being matched.
