In the expanding realm of artificial intelligence, deep learning has emerged as the key technology behind revolutionary applications across various domains, from interpreting complex images to decoding human language.
Yet, despite its vast achievements, the deep learning we know today stands on the cusp of an evolutionary leap—a transition from hand-engineered problem-solving to sophisticated learned solutions that outsmart hand-engineered ones. The success of deep learning is based on replacing domain-specific handcrafted feature engineering with features that are learned for the particular task at hand. This is an example of the fact that, throughout the history of AI, there is a clear trend in machine learning that hand-crafted solutions are eventually replaced by more effective, learned solutions. The vision behind our ambitious project DEEP LEARNING 2.0 is to repeat the success of deep learning at the meta-level, replacing handcrafted algorithmic components of deep learning by improved, learned ones.
At the heart of our proposed next-gen AI evolution is the concept of meta-learning customized components of deep learning pipelines; a concept where a meta-level AI (meta-)learns improved components of its learning process, molding the neural networks, optimizers, hyperparameters, and training data to fit specific needs and tasks dynamically. Among these transformative possibilities is the potential for AI to recognize and actively integrate multiple user-desired objectives next to predictive accuracy, like robustness, energy usage, latency, interpretability, and fairness. Embracing these user-defined objectives and directly optimizing for them, DEEP LEARNING 2.0 aspires to set the stage for truly Trustworthy AI by design.
The transformative potential of DEEP LEARNING 2.0 will be demonstrated in various applications, such as tabular data, EEG decoding, RNA folding, or reinforcement learning.
The significance of DEEP LEARNING 2.0 pivots not only on advancing the science of AI but also on its promise to democratize deep learning—making it a tool accessible for broader social, academic, and commercial utility. Reflecting its potential to advance the foundation of modern deep learning, the project's impact could ripple through the entire ecosystem of AI, where each breakthrough elevates industry capabilities, advances academic frontiers, and touches lives profoundly with smarter, fairer, and more sustainable AI technology.
