Geometry Processing as Inference

Project Information

EMERGE

Grant agreement ID: 101055448

DOI

10.3030/101055448

EC signature date 8 July 2022

Start date 1 September 2022

End date 31 August 2027

Funded under

• European Research Council (ERC)

Total cost

€ 2 496 559,00

EU contribution

€ 2 496 559,00

Coordinated by

TECHNISCHE UNIVERSITAT BERLIN
Germany

Periodic Reporting for period 1 - EMERGE (Geometry Processing as Inference)

Reporting period: 2022-09-01 to 2025-02-28

Summary of the context and overall objectives of the project

*Geometry processing* deals specifically with the processing and analyzing of three-dimensional data, i.e. digital models of things surrounding us. It is also a success story because of its efficiency, and nowadays there is virtually no field in which three-dimensional objects are not digitally modeled and analyzed: from cars to teeth, from architecture to zoology. Accurate three-dimensional models consist of large amounts of data that must be processed in many applications in fractions of a second.

The project EMERGE (Geometry Processing as Inference) aims to use the methods of geometry processing also for the processing of higher-dimensional structures. The hope and central thesis of the project is that the refinement of methods in geometry processing over the last decades will still be successful when extended to higher dimensions and exploit potentials that are complementary to developments in machine intelligence and classical digital signal processing. Thus, the methods for three-dimensional data will become more general analysis tools that can also be applied to problems in medicine, for example. The necessary extensions to algorithms and data structures are fundamental and are the central research topic of EMERGE. In particular, the question is in which dimension of the data which methods can best be implemented on today's common computer infrastructure.

Concrete application scenarios arise from the modeling of so-called shape spaces: here, three-dimensional objects are represented as points in a high-dimensional space. The change in shape is then a curve in this space. For example, human motion sequences or the degeneration of organs can be represented from sensor data (ultrasound, MRI). "EMERGE" is expected to produce methods that will enable better analysis of these data, ultimately enabling better prediction of disease progression and therapeutic success.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have studied the fundamental building blocks as well as concrete algorithms and data structures of geometry processing from the perspective of extending them to higher dimension. Several directions that appeared immediately fruitful have been followed. Among several achievements, the following three may be considered the highlichts:
- A standard problem in engineering and science is fitting a line or plane (or a higher dimensional hyper-plane) to data. It is well established how to do that if the data are points. But what if the data themselves are lines, planes, or hyper-planes? While this problem is understood from the perspective of mathematical theory, the current solutions are complicated, slow in practice, and lack important and natural properties. We have worked on a new solution to the problem of fitting lines or planes to lines or planes. It is simple, fast, and, importantly, the solutions it provides are invariant to rigid transformations, a property so far missing.
- The most successful technique for reconstructing surfaces in three-dimensional space from point samples is Poisson Surface Reconstruction. We have extended this method to work for other, more general cases, such as also reconstructing curves in space or surfaces in higher-dimensional surfaces. This required introducing a new concept in the approach, the so-called exterior calculus, and lead to a more complicated optimization problem. We have worked on a multi-level approach for this problem that is both efficient and finds plausible solutions.
- An important tool in engineering are parameterizations, allowing to map data from one domain to another, or establish correspondence between different data sets. This problem is well-understood for two-dimensional data. A basic building block for almost all such methods are so-called Tutte embeddings, which can be computed by solving linear system and guarantee one-to-one mappings. This seemingly natural approach fails to extend to higher dimension in general. We have worked on an analysis and provide the first characterization of the situation in 3D.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

All of the results achieved so far go beyond the state of the art. The three highlights mentioned as main achievements are milestones, and will surely foster follow-up work and be used in other projects. We also continue their development. The fitting approach opens up new methods in reconstructing the world from videos. The publication describing the extension of Poisson Surface Reconstruction has directly garnered a best paper award. And the analysis of Tutte embeddings in 3D has been praised as "Great progress" by the scientific community.

There are more than enough interesting avenues with great potential, yet not enough scientists to work on them. The key issue is finding talented researchers, given there are so many options for well-educated computer scientists.