Skip to main content

Exploring Relations in Structured Data with Functional Maps

Periodic Reporting for period 2 - EXPROTEA (Exploring Relations in Structured Data with Functional Maps)

Reporting period: 2019-07-01 to 2020-12-31

The main goal of this project is to develop theoretical tools and design efficient computational methods for analyzing, quantifying and exploring relations and variability in structured data sets such as geometric shapes, graphs or 3D point clouds.

This means, in particular, being able to find detailed correspondences across geometric 3D shapes, and also identify specific regions in which the objects are similar and different.

Our ultimate goal is to design a unified framework in which variability can be processed in a way that would be largely agnostic to the underlying data modality. Such a unified computational framework of variability will enable entirely novel applications including accurate shape matching, efficiently tracking and highlighting most relevant changes in evolving systems, such as dynamic graphs, and analysis of shape collections. Thus, it will permit not only to compare or cluster objects, but also to reveal where and how they are different and what makes instances unique, which can be especially useful in medical imaging applications.

The task of shape correspondence and variability identification is a fundamental problem that arises in many areas of science and engineering from identifying defects in manufactured objects, providing accurate models for statistical shape analysis, creation of virtual avatars, medical imaging (for instance for detecting anomalies, tracking recovery and performing follow-up analysis in a reliable and accurate way) but also in other areas such as archeology and paleontology (e.g. comparing artefacts, identifying parts of existing models or even artistic styles across diverse) among myriad others.

As a result, efficient algorithms for quantifying shape similarity problem, have immediate impact in those areas. Currently, the bulk of these tasks is performed using manual intervention and expert (human) knowledge, which is both time-consuming, expensive and error prone.

Our overall objective is to develop algorithms that can alleviate the need for manual intervention and provide robust, reliable measurements that can help across all tasks of geometric data analysis. Our ultimate goal is to enable applications such as 3D search and comparison and make them accessible to the non-expert users in as wide as possible range of scientific and professional disciplines. At the same time, we aim to lay the foundations for future study in geometric data analysis through rigorous and theoretically well-justified approaches.
During the first period (30 months), the project has led to 24 publications in top venues including SIGGRAPH, SIGGRAPH Asia, CVPR, and ICCV among others.

We have primarily worked on creating a robust and reliable framework for measuring and quantifying similarity in collections of 3D shapes.

For this, we have built upon a novel paradigm in geometric data analysis, which considers objects as functional spaces rather than collections of points or triangles. This point of view has proved extremely productive and we have worked on both laying the theoretical foundations for data analysis using this perspective and also designing efficient algorithms that can process, manipulate, analyze and compare 3D shapes, while treating them as functional spaces.

Our key contributions include novel effective algorithms that can automatically compute dense correspondences between non-rigid 3D shapes, theoretical analysis of existing methods in terms of their robustness and novel robust and accurate approaches for shape similarity and 3D shape design (synthesis).

As a result of this research we have been able to obtain both new theoretical insights and also design a practical set of methods for analyzing complex geometric data. Our work has been recognized through extensive publication in peer-reviewed journals and conference proceedings. Furthermore, several papers have been selected for prestigious awards during this period (4 best paper awards or nominations). Finally, the members of the project, including the PI Maks Ovsjanikov have been active in the computer vision, computer graphics and geometry processing communities to disseminate their works and as active community service members.
During the first period (30 months), the project has led to significant progress beyond the state-of-the-art in a wide variety of tasks, including:

1. Automatic algorithms for shape matching of non-rigid 3D shapes
2. Methods for quantifying and extracting variability (differences) in non-rigid collections
3. Techniques for synthesizing novel 3D models, in particular, through shape interpolation and extrapolation.
4. Methods for shape repair, especially for complex shapes represented as collections of 3D points (point clouds).
5. Novel theoretical analysis of shape matching techniques and new insights into the structure of shape collections.

among others. All of these results have appeared in selective peer-reviewed venues and have been referred to and built upon by other members of the scientific community. In several well-established applications, such as dense non-rigid 3D shape correspondence, the methods developed within this project have led to state-of-the-art accuracy on challenging benchmarks in both 2019 and 2020.