Periodic Reporting for period 5 - LEMAN (Deep LEarning on MANifolds and graphs)
Reporting period: 2022-01-01 to 2023-09-30
For nearly two millennia, the word "geometry" was synonymous with Euclidean geometry. Euclid's monopoly came to an end in the 19th century, when multiple examples of non-Euclidean geometries were shown. However, these studies quickly diverged into disparate fields, with mathematicians debating the relations between different geometries and what defines one. A way out of this pickle was shown by Felix Klein in his Erlangen Programme, which proposed approaching geometry as the study of invariants or symmetries using the language of group theory. In the 20th century, these ideas have been fundamental in developing modern physics.
The state of deep learning resembles the situation in the field of geometry in the 19h century: On the one hand, in the past decade deep learning has brought a revolution in data science and made possible many tasks previously thought to be beyond reach. At the same time, we have a zoo of neural network architectures for various kinds of data, but few unifying principles.
Geometric Deep Learning aims to bring geometric unification to deep learning in the spirit of the Erlangen Programme. Such an endeavour serves a dual purpose: provide a common mathematical framework to study neural network architectures, and give a constructive procedure to build future architectures in a principled way.
The project aims to establish the mathematical principles underlying Geometric Deep Learning on grids, graphs, and manifolds, and show some of the exciting and groundbreaking applications of these methods in the domains of computer vision, social science, and biomedicine.
The state of deep learning resembles the situation in the field of geometry in the 19h century: On the one hand, in the past decade deep learning has brought a revolution in data science and made possible many tasks previously thought to be beyond reach. At the same time, we have a zoo of neural network architectures for various kinds of data, but few unifying principles.
Geometric Deep Learning aims to bring geometric unification to deep learning in the spirit of the Erlangen Programme. Such an endeavour serves a dual purpose: provide a common mathematical framework to study neural network architectures, and give a constructive procedure to build future architectures in a principled way.
The project aims to establish the mathematical principles underlying Geometric Deep Learning on grids, graphs, and manifolds, and show some of the exciting and groundbreaking applications of these methods in the domains of computer vision, social science, and biomedicine.
1. Science
Our team has pioneered the field of Geometric ML (including coining the term) and helped shape it into one of the most exciting current topics in ML research from a niche community in a matter of a few years. We have proposed some first and already popular architectures for deep learning on manifolds and graphs achieving state-of-the-art results in classically challenging problems in computer graphics and vision such as deformable dense correspondence. We also explored the use of geometric deep learning for the defence against adversarial attacks, recommender systems, astrophysics (our paper on neutrino detection with Geometric DL in collaboration with Berkeley, NYU, and IceCube won the best paper award at ICMLA), fake news detection (our technology was commercialized and acquired by Twitter), protein science (our paper on geometric DL-based protein analysis appeared on the cover of Nature Methods), and even molecular gastronomy (our paper on using graph methods for finding drug-like molecules in food appeared in Nature Scientific Report and attracted significant media attention; the same algorithmic pipeline is now used for drug repositioning against COVID-19).
2. Technology
We patented our geometric deep learning technology (two granted patents and several additional patents pending) and spun it off into a startup company Fabula AI funded by ERC Proof of Concept grant “GoodNews” (2018) as well as additional grants from the industry (Google, Facebook, and Amazon) and private investment. Our PhD student Federico Monti took the role of CTO in the company. The company was acquired by Twitter in 2019.
3. Dissemination
The increasing popularity of Geometric ML has seen the PI invited to speak at multiple high-level events such as TEDx (2019, 2023). The PI gave keynote talks at MICCAI Workshops GRAIL, ShapeMI, Graph Signal Processing Workshop, DiffCVML, 3DV, MIDL and LOD, ICLR, IJCNN.
Invited talks and seminars included top-notch institutions such as Institute for Advanced Study in Princeton MIT, Harvard, Princeton, Yale, Oxford, and Cambridge, and top tech companies such as Google, DeepMind, Facebook, Twitter, Samsung, and Intel.
The PI also gave invited talks at summer schools at SGP MLSS, MISS, GeoCow. The PI was a visitor at the IPAM (UCLA) and IAS (Princeton). The PI organized tutorials on geometric deep learning at NIPS 2017 (extremely popular, with nearly 3000 participants), CVPR, SIGGRAPH, and EUROGRAPHICS.
The PI has organized several conference in the domain of Geometric ML, including the series of conferences GMDL, NeurIPS, IPAM, and held various positions as senior area chair nd program committee member in numerous events.
4. Organized conferences
NeurIPS Workshop on Graph Representation Learning, 019.
GMDL, 2019.
Workshop on New Deep Learning Techniques, 2018.
LoG 2023--
5. Selected Publications
M. M. Bronstein et al, Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine 34(4):18-42, 2017.
Y. Wang et al., Dynamic graph CNN for learning on point clouds, TOG 38(5):1-12, 2019.
K. Veselkov et al. HyperFoods: Machine intelligent mapping of cancer-beating molecules in foods, Nature Scientific Reports 9:9237, 2019.
P. Gainza et al., Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning, Nature Methods 17:184–192, 2020.
B. Chamberlain et al., GRAND: Graph Neural Diffusion, ICML 2021.
M. M. Bronstein et al., Geometric Deep Learning, preprint 2022 (book to appear with MIT Press)
Gainza et al., De novo design of protein interactions with learned surface fingerprints, Nature 617 (7959), 176-184, 2022.
6. Major Awards
Turing World-Leading AI Research Fellowship, 2023
Royal Academy of Engineering Silver Medal, 2020
Royal Society Wolfson Research Merit Award, 2018
Dalle Molle Foundation Prize, 2018
Fellow, IEEE, 2018
Fellow, IAPR 2018
Radcliffe fellowship, Harvard University, 2017
Our team has pioneered the field of Geometric ML (including coining the term) and helped shape it into one of the most exciting current topics in ML research from a niche community in a matter of a few years. We have proposed some first and already popular architectures for deep learning on manifolds and graphs achieving state-of-the-art results in classically challenging problems in computer graphics and vision such as deformable dense correspondence. We also explored the use of geometric deep learning for the defence against adversarial attacks, recommender systems, astrophysics (our paper on neutrino detection with Geometric DL in collaboration with Berkeley, NYU, and IceCube won the best paper award at ICMLA), fake news detection (our technology was commercialized and acquired by Twitter), protein science (our paper on geometric DL-based protein analysis appeared on the cover of Nature Methods), and even molecular gastronomy (our paper on using graph methods for finding drug-like molecules in food appeared in Nature Scientific Report and attracted significant media attention; the same algorithmic pipeline is now used for drug repositioning against COVID-19).
2. Technology
We patented our geometric deep learning technology (two granted patents and several additional patents pending) and spun it off into a startup company Fabula AI funded by ERC Proof of Concept grant “GoodNews” (2018) as well as additional grants from the industry (Google, Facebook, and Amazon) and private investment. Our PhD student Federico Monti took the role of CTO in the company. The company was acquired by Twitter in 2019.
3. Dissemination
The increasing popularity of Geometric ML has seen the PI invited to speak at multiple high-level events such as TEDx (2019, 2023). The PI gave keynote talks at MICCAI Workshops GRAIL, ShapeMI, Graph Signal Processing Workshop, DiffCVML, 3DV, MIDL and LOD, ICLR, IJCNN.
Invited talks and seminars included top-notch institutions such as Institute for Advanced Study in Princeton MIT, Harvard, Princeton, Yale, Oxford, and Cambridge, and top tech companies such as Google, DeepMind, Facebook, Twitter, Samsung, and Intel.
The PI also gave invited talks at summer schools at SGP MLSS, MISS, GeoCow. The PI was a visitor at the IPAM (UCLA) and IAS (Princeton). The PI organized tutorials on geometric deep learning at NIPS 2017 (extremely popular, with nearly 3000 participants), CVPR, SIGGRAPH, and EUROGRAPHICS.
The PI has organized several conference in the domain of Geometric ML, including the series of conferences GMDL, NeurIPS, IPAM, and held various positions as senior area chair nd program committee member in numerous events.
4. Organized conferences
NeurIPS Workshop on Graph Representation Learning, 019.
GMDL, 2019.
Workshop on New Deep Learning Techniques, 2018.
LoG 2023--
5. Selected Publications
M. M. Bronstein et al, Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine 34(4):18-42, 2017.
Y. Wang et al., Dynamic graph CNN for learning on point clouds, TOG 38(5):1-12, 2019.
K. Veselkov et al. HyperFoods: Machine intelligent mapping of cancer-beating molecules in foods, Nature Scientific Reports 9:9237, 2019.
P. Gainza et al., Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning, Nature Methods 17:184–192, 2020.
B. Chamberlain et al., GRAND: Graph Neural Diffusion, ICML 2021.
M. M. Bronstein et al., Geometric Deep Learning, preprint 2022 (book to appear with MIT Press)
Gainza et al., De novo design of protein interactions with learned surface fingerprints, Nature 617 (7959), 176-184, 2022.
6. Major Awards
Turing World-Leading AI Research Fellowship, 2023
Royal Academy of Engineering Silver Medal, 2020
Royal Society Wolfson Research Merit Award, 2018
Dalle Molle Foundation Prize, 2018
Fellow, IEEE, 2018
Fellow, IAPR 2018
Radcliffe fellowship, Harvard University, 2017
Our group was among the first to bridge geometric and machine learning models, pioneering and spearheading the nascent field of "Geometric Machine Learning" (a term coined by our group). Our geometric deep learning algorithms have been recently used for neutrino interaction classification (a collaboration with IceCube, NYU and UC Berkeley) and fake news detection on social media (our startup Fabula AI acquired by Twitter in 2019). Our group collaborates with partners from world’s leading institutions such as Stanford, MIT, UC Berkeley, NYU, Oxford, Cambridge, EPFL, Ecole Polytechnique, TUM, and Technion, as well as top industrial labs at Google, Facebook, Intel, and Twitter. Graph neural networks are already used in the industry by companies such as Pinterest, Twitter, Google, and Alibaba.
Geometric deep learning is already proving a groundbreaking tool in numerous applications, including drug and material design and discovery, drug repositioning, physical sciences, and computational social sciences. Our group has produced groundbreaking results in the detection of fake news on social media from their spreading patterns using geometric deep learning.
An extremely promising direction is computational biology, where our joint work with Prof. Bruno Correia (EPFL) shows the possibility to solve notoriously hard problems in protein science, such as data-driven de novo design protein binders. This research could potentially lead to significantly and faster development of new generations of protein-based drugs (biologics) and therapies against oncological diseases. Research by an MIT group used graph deep learning as virtual screening tool for drug candidates to discover a new class of antibiotics. We believe that in the next few years Geometric ML methods could potentially revolutionize the field.
Geometric deep learning is already proving a groundbreaking tool in numerous applications, including drug and material design and discovery, drug repositioning, physical sciences, and computational social sciences. Our group has produced groundbreaking results in the detection of fake news on social media from their spreading patterns using geometric deep learning.
An extremely promising direction is computational biology, where our joint work with Prof. Bruno Correia (EPFL) shows the possibility to solve notoriously hard problems in protein science, such as data-driven de novo design protein binders. This research could potentially lead to significantly and faster development of new generations of protein-based drugs (biologics) and therapies against oncological diseases. Research by an MIT group used graph deep learning as virtual screening tool for drug candidates to discover a new class of antibiotics. We believe that in the next few years Geometric ML methods could potentially revolutionize the field.