Periodic Reporting for period 4 - SPADE (from SPArsity to DEep learning)
Reporting period: 2022-04-01 to 2023-09-30
Deep learning is a sub-field of machine learning that is already used today in everyday life. Deep neural networks, which are the tool used in deep learning, achieve state-of-the-art results in many applications in various fields ranging from medicine to autonomous driving. Yet, one of the main challenges in neural networks is that they require a lot of data for training them. Another problem is that they are usually designed for data that has a regular structure (for example, an image that has pixels organized in a rectangular structure).
The goal of this project is to extend the usability of neural networks beyond the case of large labeled data or regular data.
In the project we provided new theoretical analysis of deep neural networks that led to novel techniques for self-supervised learning, learning with small data and learning with non-conventional data types.
The goal of this project is to extend the usability of neural networks beyond the case of large labeled data or regular data.
In the project we provided new theoretical analysis of deep neural networks that led to novel techniques for self-supervised learning, learning with small data and learning with non-conventional data types.
In the project, we have developed various techniques that allow training a neural network using even just a single example and still getting remarkable results.
We also have shown that one can train a neural network for one simple task (e.g. image denoising) and still use it for various other tasks such as increasing the resolution of the image, filling missing parts in it, or removing blur from it.
Our work also extended beyond the standard data that is used with neural networks. This includes:
1. Develop novel neural network strategies for processing, generation and editing of 3D shapes. The following video explains one of our approaches https://youtu.be/3cKGSV-VUVI(opens in new window)
2. Detecting exoplanets in the outer space
3. Processing of radar data.
4. We have shown how one can automatically design optical systems using neural networks. This can bring the ability to new areas such as achieving all in focus imaging and depth reconstruction from only a single lens.
5. We have developed novel image processing techniques.
6. We proposed novel strategies for self-supervised and few-shot learning
7. We improved the understanding of adversarial examples and proposed new techniques to mitigate them
8. We proposed novel theoretical understanding of neural networks based on signal processing based tools, where sparsity is a notable example.
All these results brought deep learning to areas in which it was less used before.
We also have shown that one can train a neural network for one simple task (e.g. image denoising) and still use it for various other tasks such as increasing the resolution of the image, filling missing parts in it, or removing blur from it.
Our work also extended beyond the standard data that is used with neural networks. This includes:
1. Develop novel neural network strategies for processing, generation and editing of 3D shapes. The following video explains one of our approaches https://youtu.be/3cKGSV-VUVI(opens in new window)
2. Detecting exoplanets in the outer space
3. Processing of radar data.
4. We have shown how one can automatically design optical systems using neural networks. This can bring the ability to new areas such as achieving all in focus imaging and depth reconstruction from only a single lens.
5. We have developed novel image processing techniques.
6. We proposed novel strategies for self-supervised and few-shot learning
7. We improved the understanding of adversarial examples and proposed new techniques to mitigate them
8. We proposed novel theoretical understanding of neural networks based on signal processing based tools, where sparsity is a notable example.
All these results brought deep learning to areas in which it was less used before.
Our project achieved beyond state-of-the-art results in various problems. In particular, for the problems of
1. Image classification when training only with few labeled examples both for the case of a single object in an image and multiple objects in an image.
2. Object detection in images when training only with few labeled examples.
3. Detection of exoplanets from simulated data
4. Unsupervised Alignment of 2D and 3D data.
5. 3D object classification and segmentation (see a video in https://youtu.be/3cKGSV-VUVI(opens in new window))
6. Depth reconstruction accuracy from a single lens (see attached image)
7. Image enhancement quality (see attached image)
8. Object detection in radar data
9. State-of-art performance in self-supervised and few shot learning
10. Novel reconstruction, generation and editing techniques for 3D data
11. Improved theory for deep neural networks
1. Image classification when training only with few labeled examples both for the case of a single object in an image and multiple objects in an image.
2. Object detection in images when training only with few labeled examples.
3. Detection of exoplanets from simulated data
4. Unsupervised Alignment of 2D and 3D data.
5. 3D object classification and segmentation (see a video in https://youtu.be/3cKGSV-VUVI(opens in new window))
6. Depth reconstruction accuracy from a single lens (see attached image)
7. Image enhancement quality (see attached image)
8. Object detection in radar data
9. State-of-art performance in self-supervised and few shot learning
10. Novel reconstruction, generation and editing techniques for 3D data
11. Improved theory for deep neural networks