Periodic Reporting for period 4 - EXPROTEA (Exploring Relations in Structured Data with Functional Maps)
Reporting period: 2022-07-01 to 2023-12-31
The main goal of this project is to develop theoretical tools and design efficient computational methods for analyzing, quantifying and exploring relations and variability in structured data sets, with focus on geometric shapes in 3D.
This means, in particular, being able to find detailed correspondences across geometric non-rigid 3D shapes, and also identify specific regions in which the objects are similar and different.
Our ultimate goal is to design a unified framework in which variability can be processed in a way that would be largely agnostic to the underlying data modality. Such a unified computational framework of variability will enable entirely novel applications including accurate shape matching, efficiently tracking and highlighting most relevant changes in evolving systems in 3D, and analysis of shape collections. Thus, it will permit not only to compare or cluster objects, but also to reveal where and how they are different and what makes instances unique, which can be especially useful in medical imaging applications.
The task of shape correspondence and variability identification is a fundamental problem that arises in many areas of science and engineering from identifying defects in manufactured objects, providing accurate models for statistical shape analysis, creation of virtual avatars, medical imaging (for instance for detecting anomalies, tracking recovery and performing follow-up analysis in a reliable and accurate way) but also in other areas such as archeology and paleontology (e.g. comparing artefacts, identifying parts of existing models or even artistic styles across diverse) among myriad others.
As a result, efficient algorithms for quantifying shape similarity problem, have immediate impact in those areas. Prior to this project, the bulk of such tasks was performed using manual intervention and expert (human) knowledge, making the resulting analysis pipelines time-consuming, expensive and error prone.
Our overall objective is to develop algorithms that can alleviate the need for manual intervention and provide robust, reliable measurements that can help across all tasks of geometric data analysis. Our ultimate goal is to enable applications such as 3D search and comparison and make them accessible to the non-expert users in as wide as possible range of scientific and professional disciplines. At the same time, the project aims to lay the foundations for future study in geometric data analysis through rigorous and theoretically well-justified approaches.
This means, in particular, being able to find detailed correspondences across geometric non-rigid 3D shapes, and also identify specific regions in which the objects are similar and different.
Our ultimate goal is to design a unified framework in which variability can be processed in a way that would be largely agnostic to the underlying data modality. Such a unified computational framework of variability will enable entirely novel applications including accurate shape matching, efficiently tracking and highlighting most relevant changes in evolving systems in 3D, and analysis of shape collections. Thus, it will permit not only to compare or cluster objects, but also to reveal where and how they are different and what makes instances unique, which can be especially useful in medical imaging applications.
The task of shape correspondence and variability identification is a fundamental problem that arises in many areas of science and engineering from identifying defects in manufactured objects, providing accurate models for statistical shape analysis, creation of virtual avatars, medical imaging (for instance for detecting anomalies, tracking recovery and performing follow-up analysis in a reliable and accurate way) but also in other areas such as archeology and paleontology (e.g. comparing artefacts, identifying parts of existing models or even artistic styles across diverse) among myriad others.
As a result, efficient algorithms for quantifying shape similarity problem, have immediate impact in those areas. Prior to this project, the bulk of such tasks was performed using manual intervention and expert (human) knowledge, making the resulting analysis pipelines time-consuming, expensive and error prone.
Our overall objective is to develop algorithms that can alleviate the need for manual intervention and provide robust, reliable measurements that can help across all tasks of geometric data analysis. Our ultimate goal is to enable applications such as 3D search and comparison and make them accessible to the non-expert users in as wide as possible range of scientific and professional disciplines. At the same time, the project aims to lay the foundations for future study in geometric data analysis through rigorous and theoretically well-justified approaches.
Overall, the project has successfully achieved its goals and in several aspects has even exceeded them. In particular, the project has led to 64 scientific publications in top venues including NeurIPS, SIGGRAPH, CVPR, ICCV, ICML, SIGGRAPH Asia and others. These papers have received 7 best paper awards or nominations. The majority of these publications are of core relevance to the project, and have helped to lay the foundations, both theoretical and computational for analysing, processing and even synthesizing geometric data using the general methodology outlined in the project proposal, which is based on the functional maps framework.
Within this project we have built upon a novel paradigm in geometric data analysis, which considers objects as functional spaces rather than collections of points or triangles. This point of view has proved extremely productive and has enabled efficient algorithms that can process, manipulate, analyze and compare 3D shapes, while treating them as functional spaces.
This project has led to several breakthrough results, and has significantly pushed the state-of-the-art in terms of accuracy and robustness of the best available techniques for shape comparison. A major contribution of this project is a set of novel theoretical insights and practical methods for 3D shape analysis based on the functional maps framework. The new results obtained within this project have significantly improved the robustness, accuracy and speed of the best available methods for computing correspondences and quantifying similarity across different 3D shapes. This includes both axiomatic as well as learning-based approaches.
In addition to its foundational nature, the project has also led to several practical applications in diverse scientific and industrial settings. This includes collaborations by the PI with the Necker Children's Hospital, The Museum of Mankind, as well as in bio-medical context such as for analysis of protein or evolving biological cell data. In all of these disciplines, the key problem is to reliably establish correspondences and quantify similarity and differences across diverse shapes. The methods that we have developed within this project have thus allowed researchers and practitioners in these domains to significantly improve the accuracy of their shape analysis pipelines.
Our work has been recognized through extensive publication record in peer-reviewed journals and conference proceedings. Furthermore, several papers have been selected for prestigious awards. Finally, the members of the project, including the PI Maks Ovsjanikov have been active in the computer vision, computer graphics and geometry processing communities to disseminate their works and as active community service members.
Within this project we have built upon a novel paradigm in geometric data analysis, which considers objects as functional spaces rather than collections of points or triangles. This point of view has proved extremely productive and has enabled efficient algorithms that can process, manipulate, analyze and compare 3D shapes, while treating them as functional spaces.
This project has led to several breakthrough results, and has significantly pushed the state-of-the-art in terms of accuracy and robustness of the best available techniques for shape comparison. A major contribution of this project is a set of novel theoretical insights and practical methods for 3D shape analysis based on the functional maps framework. The new results obtained within this project have significantly improved the robustness, accuracy and speed of the best available methods for computing correspondences and quantifying similarity across different 3D shapes. This includes both axiomatic as well as learning-based approaches.
In addition to its foundational nature, the project has also led to several practical applications in diverse scientific and industrial settings. This includes collaborations by the PI with the Necker Children's Hospital, The Museum of Mankind, as well as in bio-medical context such as for analysis of protein or evolving biological cell data. In all of these disciplines, the key problem is to reliably establish correspondences and quantify similarity and differences across diverse shapes. The methods that we have developed within this project have thus allowed researchers and practitioners in these domains to significantly improve the accuracy of their shape analysis pipelines.
Our work has been recognized through extensive publication record in peer-reviewed journals and conference proceedings. Furthermore, several papers have been selected for prestigious awards. Finally, the members of the project, including the PI Maks Ovsjanikov have been active in the computer vision, computer graphics and geometry processing communities to disseminate their works and as active community service members.
The project has led to significant progress extending the state-of-the-art in a wide variety of tasks, including:
1. Automatic algorithms for shape matching of non-rigid 3D shapes
2. Methods for quantifying and extracting variability (differences) in non-rigid collections
3. Novel theoretical analysis of shape matching techniques and new insights into the structure of shape collections.
4. Novel methods for deep learning on 3D data.
5. Techniques for shape interpolation and extrapolation.
6. Applications of shape correspondence and similarity in a range of problems, including those in morphology, archaeology and paleontology, as well as many others.
All of these results have appeared in selective peer-reviewed venues and have been referred to and built upon by other members of the scientific community. In several well-established applications, such as dense non-rigid 3D shape correspondence, the methods developed within this project have led to state-of-the-art accuracy on challenging benchmarks up to 2023.
1. Automatic algorithms for shape matching of non-rigid 3D shapes
2. Methods for quantifying and extracting variability (differences) in non-rigid collections
3. Novel theoretical analysis of shape matching techniques and new insights into the structure of shape collections.
4. Novel methods for deep learning on 3D data.
5. Techniques for shape interpolation and extrapolation.
6. Applications of shape correspondence and similarity in a range of problems, including those in morphology, archaeology and paleontology, as well as many others.
All of these results have appeared in selective peer-reviewed venues and have been referred to and built upon by other members of the scientific community. In several well-established applications, such as dense non-rigid 3D shape correspondence, the methods developed within this project have led to state-of-the-art accuracy on challenging benchmarks up to 2023.