We systematically advanced signal processing and deep learning methods and theory for wave-based inverse problems. Early on we proposed FIONet which showed that respecting the mathematical structure of scattering operators (in this case Fourier integral operators) dramatically improves generalization while resulting in smaller, much less data-hungry networks. We also developed a multiscale sparse coding scheme that addresses why U-Net may “memorize” seen patterns instead of learning underlying physics.
In order to address well-posedness and stability aspects that critical in scientific inverse problems, we proposed “Trumpets,” a class of injective deep flows that can be inverted fast, which enabled stable reconstructions even in strongly nonlinear problems like inverse scattering. We proved that these flows can universally approximate distributions on low-dimensional manifolds and identified relevant topological obstructions. Going beyond, we showed probabilistic transformers can be universal approximators under arbitrary constraints—the first result of its kind—critical for enforcing physically meaningful outputs and developed a unified "randomized" universal approximation framework which coveres everything from functions between graphs to operators between function spaces arising in inverse problems and simulation.
Challenges in unknown-view tomography and other "unlabeled" problems prompted us to work on graph neural networks. We developed WalkPool---an algorithm for link prediction in networks---which achieved state of the art performance on graphs with very different structural properties and is now used by labs around the world (somewhat unexpectedly) for drug repurposing and gene targeting studies. We adapted WalkPool to denoise graphs arising in signgle-particle cryo-EM and showed that this pushes feasible SNR for reconstruction lower than previously possible. We also developed new techniques to study generalization in graph neural networks which are now being adopted by a number of theory groups.
Beyond single-particle reconstruction we developed new reconstruction methods for cryo-electron tomography, including ICE-TIDE (joint deformation correction and reconstruction) and CryoLithe (end-to-end localized deep learning), achieving state-of-the-art 3D reconstructions at greatly reduced computational times, without parameter tuning. On the Earth science front, we introduced new tools for seismic inverse problems, including Harpa for seismic phase association, which operates without prior background models and handles very dense data. This is exciting as it may enable real-time monitoring of seismicity and volcanoes. Finally, our seismic waveform foundation model, SeisLM, promises to improve earthquake early warning systems.
Dissemination
Throughout the project, results were disseminated through publications in high-impact journals (e.g. PNAS, IEEE Transactions, Applied and Computational Harmonic Analysis) and conferences and international conference presentations (NeurIPS, ICLR, ICML, AAAI, ICASSP, Asilomar). We have presented the results of SWING in numerous invited talks at leading institutions in Switzerland and abroad.
