Periodic Reporting for period 4 - SWING (Signals, Waves, and Learning: A Data-Driven Paradigm for Wave-Based Inverse Problems)
Periodo di rendicontazione: 2024-07-01 al 2024-12-31
Wave scattering underlies key scientific and technological challenges, from visualizing biological structures to probing Earth’s interior. With the rise of deep learning, a central question is whether neural networks can reliably learn and invert wave-based phenomena beyond the scope of traditional signal-theoretic methods. SWING addresses this by merging signal processing and machine learning to devise theoretically grounded algorithms for forward and inverse scattering, and related problems on graphs. Its three main thrusts were to: (1) design minimal deep architectures, rooted in Fourier integral operators, with provable approximation guarantees for learning scattering operators; (2) understand operator approximation and sample complexity results for the new architectures, including an analysis of graph neural networks which are the backbone of several advances in SWING; and (3) apply these methods to emerging problems in bioimaging and Earth science. Throughout, we adhere to the principle that “the best designs combine data-based models with an understanding of the underlying physics.”
Conclusions
Over the project’s duration, we developed new deep networks based on a mathematical understanding of waves that surpass generic architectures in accuracy, robustness, and out-of-distribution performance, while dramatically reducing sample requirements. We established rigorous injectivity guarantees and universal approximation properties for structured networks, paving the way for stable, high-fidelity reconstructions even on strongly nonlinear problems. In cryo-electron tomography, our new end-to-end algorithms match or exceed state-of-the-art reconstruction quality while cutting processing times from hours to minutes and not needing parameter tuning—an outcome of combining a careful understanding of the signal processing structure of tomomgraphy with a development of new deep learning architectures. We extended our theoretical framework to inverse problems on graphs and developed new graph neural networks (e.g. WalkPool) that we also applied to single-particle cryo-EM. Along the way we spearheaded new approaches to analyze generalization and sample complexity of graph neural nets, and new practical algorithms for graph denoising. We also developed new algorithms for the seismic signal processing pipeline, including phase association---the problem of associating wave arrivals at multiple seismic stations to their originating events. An inverse problems perspective enables us to perform phase association even without knowing the background wave speed, and with extremely dense arrivals. This is exciting as it may allow us to leverage microseismicity which was thus far untapped. These advances stand to impact a broad swath of fields—biology, geophysics, and beyond—by uniting "deep learning engineering" with the mathematical understanding of wave physics, which was SWING’s central objective.
Conclusions
Over the project’s duration, we developed new deep networks based on a mathematical understanding of waves that surpass generic architectures in accuracy, robustness, and out-of-distribution performance, while dramatically reducing sample requirements. We established rigorous injectivity guarantees and universal approximation properties for structured networks, paving the way for stable, high-fidelity reconstructions even on strongly nonlinear problems. In cryo-electron tomography, our new end-to-end algorithms match or exceed state-of-the-art reconstruction quality while cutting processing times from hours to minutes and not needing parameter tuning—an outcome of combining a careful understanding of the signal processing structure of tomomgraphy with a development of new deep learning architectures. We extended our theoretical framework to inverse problems on graphs and developed new graph neural networks (e.g. WalkPool) that we also applied to single-particle cryo-EM. Along the way we spearheaded new approaches to analyze generalization and sample complexity of graph neural nets, and new practical algorithms for graph denoising. We also developed new algorithms for the seismic signal processing pipeline, including phase association---the problem of associating wave arrivals at multiple seismic stations to their originating events. An inverse problems perspective enables us to perform phase association even without knowing the background wave speed, and with extremely dense arrivals. This is exciting as it may allow us to leverage microseismicity which was thus far untapped. These advances stand to impact a broad swath of fields—biology, geophysics, and beyond—by uniting "deep learning engineering" with the mathematical understanding of wave physics, which was SWING’s central objective.
We systematically advanced signal processing and deep learning methods and theory for wave-based inverse problems. Early on we proposed FIONet which showed that respecting the mathematical structure of scattering operators (in this case Fourier integral operators) dramatically improves generalization while resulting in smaller, much less data-hungry networks. We also developed a multiscale sparse coding scheme that addresses why U-Net may “memorize” seen patterns instead of learning underlying physics.
In order to address well-posedness and stability aspects that critical in scientific inverse problems, we proposed “Trumpets,” a class of injective deep flows that can be inverted fast, which enabled stable reconstructions even in strongly nonlinear problems like inverse scattering. We proved that these flows can universally approximate distributions on low-dimensional manifolds and identified relevant topological obstructions. Going beyond, we showed probabilistic transformers can be universal approximators under arbitrary constraints—the first result of its kind—critical for enforcing physically meaningful outputs and developed a unified "randomized" universal approximation framework which coveres everything from functions between graphs to operators between function spaces arising in inverse problems and simulation.
Challenges in unknown-view tomography and other "unlabeled" problems prompted us to work on graph neural networks. We developed WalkPool---an algorithm for link prediction in networks---which achieved state of the art performance on graphs with very different structural properties and is now used by labs around the world (somewhat unexpectedly) for drug repurposing and gene targeting studies. We adapted WalkPool to denoise graphs arising in signgle-particle cryo-EM and showed that this pushes feasible SNR for reconstruction lower than previously possible. We also developed new techniques to study generalization in graph neural networks which are now being adopted by a number of theory groups.
Beyond single-particle reconstruction we developed new reconstruction methods for cryo-electron tomography, including ICE-TIDE (joint deformation correction and reconstruction) and CryoLithe (end-to-end localized deep learning), achieving state-of-the-art 3D reconstructions at greatly reduced computational times, without parameter tuning. On the Earth science front, we introduced new tools for seismic inverse problems, including Harpa for seismic phase association, which operates without prior background models and handles very dense data. This is exciting as it may enable real-time monitoring of seismicity and volcanoes. Finally, our seismic waveform foundation model, SeisLM, promises to improve earthquake early warning systems.
Dissemination
Throughout the project, results were disseminated through publications in high-impact journals (e.g. PNAS, IEEE Transactions, Applied and Computational Harmonic Analysis) and conferences and international conference presentations (NeurIPS, ICLR, ICML, AAAI, ICASSP, Asilomar). We have presented the results of SWING in numerous invited talks at leading institutions in Switzerland and abroad.
In order to address well-posedness and stability aspects that critical in scientific inverse problems, we proposed “Trumpets,” a class of injective deep flows that can be inverted fast, which enabled stable reconstructions even in strongly nonlinear problems like inverse scattering. We proved that these flows can universally approximate distributions on low-dimensional manifolds and identified relevant topological obstructions. Going beyond, we showed probabilistic transformers can be universal approximators under arbitrary constraints—the first result of its kind—critical for enforcing physically meaningful outputs and developed a unified "randomized" universal approximation framework which coveres everything from functions between graphs to operators between function spaces arising in inverse problems and simulation.
Challenges in unknown-view tomography and other "unlabeled" problems prompted us to work on graph neural networks. We developed WalkPool---an algorithm for link prediction in networks---which achieved state of the art performance on graphs with very different structural properties and is now used by labs around the world (somewhat unexpectedly) for drug repurposing and gene targeting studies. We adapted WalkPool to denoise graphs arising in signgle-particle cryo-EM and showed that this pushes feasible SNR for reconstruction lower than previously possible. We also developed new techniques to study generalization in graph neural networks which are now being adopted by a number of theory groups.
Beyond single-particle reconstruction we developed new reconstruction methods for cryo-electron tomography, including ICE-TIDE (joint deformation correction and reconstruction) and CryoLithe (end-to-end localized deep learning), achieving state-of-the-art 3D reconstructions at greatly reduced computational times, without parameter tuning. On the Earth science front, we introduced new tools for seismic inverse problems, including Harpa for seismic phase association, which operates without prior background models and handles very dense data. This is exciting as it may enable real-time monitoring of seismicity and volcanoes. Finally, our seismic waveform foundation model, SeisLM, promises to improve earthquake early warning systems.
Dissemination
Throughout the project, results were disseminated through publications in high-impact journals (e.g. PNAS, IEEE Transactions, Applied and Computational Harmonic Analysis) and conferences and international conference presentations (NeurIPS, ICLR, ICML, AAAI, ICASSP, Asilomar). We have presented the results of SWING in numerous invited talks at leading institutions in Switzerland and abroad.
SWING has significantly advanced wave-based imaging on both theoretical and algorithmic fronts. On the theory side, our development of injective deep networks with rigorous approximation guarantees goes well beyond the previous state of the art, enabling stable, physically constrained learning. Similarly, our “randomized” universal approximation framework extends all prior universal approximation results. Our precise analysis of generalization curves in graph neural networks connects statistical physics and block models with graph learning for the first time, sparking a new line of investigation now pursued by multiple groups. On the applications front, our deep-learning-based 3D cryo-electron tomography reconstructions—exploiting the geometry of CT—match or exceed existing pipelines while cutting computation times by orders of magnitude and eliminating the need for manual tuning. This result culminates a line of work on a new class of localized deep estimators for imaging. We also developed new seismic processing algorithms, including Harpa for dense phase association, pushing performance into regimes previously out of reach, and SeisLM, a foundation model for seismic waveforms that achieves state-of-the-art results on a range of seismic signal processing tasks, including, excitingly, foreshock–aftershock classification.