Analyzing and processing 3D geometric data is a critical task in a wide range of disciplines, from biomedical imaging to cultural preservation. However, the currently dominant paradigm in geometric deep learning relies heavily on training neural networks entirely from scratch for each specific data modality and task. Because 3D geometric data lacks a canonical representation (often represented as point clouds, meshes, voxel grids, Gaussian splats, etc.) and deals with a multitude of distinct, small-scale datasets—creating a very long-tailed distribution—this approach is not only highly inefficient but also ignores the wider information overlap across different object categories and tasks. More fundamentally, most existing learning-based approaches are severely limited to problem settings where a massive amount of annotated training data is required, rendering them ill-adapted for realistic applications with limited supervision.
To address these fundamental limitations, our project VEGA aims to depart from the standard design of class-specific or data-rich architectures for 3D deep learning. Instead, our overall objective is to lay the theoretical foundations and design efficient computational methods for universal geometric _transfer learning_. We propose to develop universally-applicable methods by combining robust, pre-trainable multi-scale geometric feature extractors with effective fine-tuning modules. By establishing rigorous formal and computational relations across different geometric data representations, and by injecting prior physical and structural reasoning, our goal is to create a unified framework that allows solutions to be seamlessly transferred and adapted across problems, regardless of the underlying geometric data type.
The expected impact of this project is to open the door to fundamentally new problem settings and modeling tools that are currently entirely outside the scope of existing 3D deep learning methods. By creating practical, data-efficient tools for 3D transfer learning, we aim to enable accurate geometric data analysis with minimal or no training data. This will allow practitioners to tackle highly complex real-world applications—such as tracking the dynamic evolution of biological systems, analyzing variability in archaeological collections, or performing complex shape optimization under physical constraints—even when only a handful of domain-specific examples are present. Ultimately, this project expects to usher in a new era for geometric data analysis, coupling the generality and robustness of classical axiomatic methods with the power and semantic awareness of modern data-driven learning approaches.
