Final Report Summary - SPARSE (Next Generation Sparsity-Based Signal Modeling)
Michael Elad’s ERC project, SPARSE, has led to substantial advancements in the field of sparse modeling and its implication on image processing. His group at the Technion has been adopting applied and fundamental theoretic research points of view in exploring novel model structures for visual data, aiming for ways that would lead to better solution of inverse problems in imaging. During the 5 years duration of this research project, a series of significant achievements have been made by this research group. These include:
• Harnessing patch-ordering for better image processing, a new and out-of-the-box idea that was shown to lead to state-of-the-art results in various applications, and in several contexts. This idea has been used for constructing spatially adaptive frames, plainly to permute the pixels in an image to a maximally smooth curve, and as a way to form a highly effective TV-like regularization on permuted pixels.
• Extensive study of the co-sparse analysis model, starting from its very conception and definition, through the development of adequate pursuit algorithms along with their theoretical analysis, and concluding with ways to practically learn the dictionary for this model. This work also included a theoretical study of the thresholding algorithm for the co-sparse analysis model to identify key features that make an analysis dictionary successful, and on another front, the derivation of an algorithm for fusion of analysis and synthesis dictionary learning for image deblurring.
• As deep learning ideas are making their way into this field, this project investigated some relations between Neural Networks (NN) and sparsity models. Within this activity, a NN method for compressed-sensing was developed, which learns both the sensing matrix and the recovery algorithm together, targeting a desired compression factor. In a different line of work, this project led to the construction of an image super-resolution algorithm as a chain of enhancement steps that have a deep learning architecture, where each can be trained separately, again showing high-quality results.
• One of the main questions this project has addressed is how local models can be trained/used for solving global problems effectively. This has been a major challenge of this project, and a significant progress has been made on this front. Elad's team managed to substantially improve existing image denoising algorithms by adopting multi-scale processing of patches, using game-theoretic (consensus) ideas, and boosting-related ideas that operate on both the method-noise and the resulting-image. In addition, and towards the end of the project period, this activity led to a breakthrough in identifying a theoretical connection between the convolutional sparse coding model and deep-learning architectures.
Other significant steps taken in this project include the development of novel MRI reconstruction algorithms that deploy self-calibration, and are capable of handling dynamics, in order to visualize the heart or blood perfusion; the derivation of general purpose kernelization for dictionary learning algorithms, handling of the Poisson image denoising problem, and more. Future challenges dealt with in this project include a thorough theoretical study of the local-global gap in the context of sparse approximation, understanding of model errors via image synthesis, structured dictionary learning algorithms for the synthesis model that overcome dimensionality problems, handling of graph-structured signals by sparse modeling while exploiting their inner topology, and more. See PI's website for publications that emanated from this project at www.cs.technion.ac.il/~elad .
• Harnessing patch-ordering for better image processing, a new and out-of-the-box idea that was shown to lead to state-of-the-art results in various applications, and in several contexts. This idea has been used for constructing spatially adaptive frames, plainly to permute the pixels in an image to a maximally smooth curve, and as a way to form a highly effective TV-like regularization on permuted pixels.
• Extensive study of the co-sparse analysis model, starting from its very conception and definition, through the development of adequate pursuit algorithms along with their theoretical analysis, and concluding with ways to practically learn the dictionary for this model. This work also included a theoretical study of the thresholding algorithm for the co-sparse analysis model to identify key features that make an analysis dictionary successful, and on another front, the derivation of an algorithm for fusion of analysis and synthesis dictionary learning for image deblurring.
• As deep learning ideas are making their way into this field, this project investigated some relations between Neural Networks (NN) and sparsity models. Within this activity, a NN method for compressed-sensing was developed, which learns both the sensing matrix and the recovery algorithm together, targeting a desired compression factor. In a different line of work, this project led to the construction of an image super-resolution algorithm as a chain of enhancement steps that have a deep learning architecture, where each can be trained separately, again showing high-quality results.
• One of the main questions this project has addressed is how local models can be trained/used for solving global problems effectively. This has been a major challenge of this project, and a significant progress has been made on this front. Elad's team managed to substantially improve existing image denoising algorithms by adopting multi-scale processing of patches, using game-theoretic (consensus) ideas, and boosting-related ideas that operate on both the method-noise and the resulting-image. In addition, and towards the end of the project period, this activity led to a breakthrough in identifying a theoretical connection between the convolutional sparse coding model and deep-learning architectures.
Other significant steps taken in this project include the development of novel MRI reconstruction algorithms that deploy self-calibration, and are capable of handling dynamics, in order to visualize the heart or blood perfusion; the derivation of general purpose kernelization for dictionary learning algorithms, handling of the Poisson image denoising problem, and more. Future challenges dealt with in this project include a thorough theoretical study of the local-global gap in the context of sparse approximation, understanding of model errors via image synthesis, structured dictionary learning algorithms for the synthesis model that overcome dimensionality problems, handling of graph-structured signals by sparse modeling while exploiting their inner topology, and more. See PI's website for publications that emanated from this project at www.cs.technion.ac.il/~elad .