Harmonising Observations and Underlying Principles for Visual Data Association

Descripción del proyecto

Los principios optimizadores de la asociación visual de datos

Las aplicaciones de la asociación de datos visuales incluyen la asignación de modelos físicos a escenarios complejos en la conducción autónoma o la correspondencia de colecciones de formas tridimensionales para análisis médicos. Sin embargo, los métodos actuales de asociación de datos visuales están mal equipados para afrontar los recientes avances en el aprendizaje profundo. En consecuencia, no pueden garantizar la optimalidad global, la escalabilidad o la coherencia geométrica en aplicaciones fundamentales, porque son incapaces de interpretar interconexiones complejas entre diferentes entidades observables. Para resolver este problema, el proyecto Harmony, financiado con fondos europeos, propone armonizar conceptual y algorítmicamente las complejas interconexiones entre entidades observables y principios fundamentales subyacentes como la geometría y la física. Los resultados beneficiarán a investigadores y profesionales, ofreciéndoles soluciones optimizadas a tareas complejas en aplicaciones relevantes.

Objetivo

Visual data association aims to find task-specific mappings involving visual data. Two significant examples are the mapping of physics models to complex scenes for planning overtaking manoeuvrers in autonomous driving, or matching collections of 3D shapes for medical analysis. Despite the high relevance of visual data association, its progress has not kept pace with the revolutionary developments fuelled by recent deep learning advances: existing data association machinery lacks theoretical guarantees (e.g. global optimality, or structure such as geometric consistency in 3D shape matching) that are critical for high-stakes settings, or suffers from poor scalability. Moreover, current procedures fall short of understanding complex interconnections across different observable entities (collections of e.g. objects or scenes). The vision of Harmony is to tackle these shortcomings by harmonising the complex interconnections between observable entities and underlying fundamental principles (e.g. geometry, or physics). This research direction is challenging, largely unexplored and will require to break substantially new ground at conceptual, algorithmic and practical levels simultaneously. Harmony is organised into four complementary challenges:
Challenge A addresses global optimality and scalability for 3D shape matching;
Challenge B addresses structure and dynamics inference from static images;
Challenge C addresses non-linear synchronisation in data collections defined over graphs;
Challenge D will exploit synergies and cross-fertilise insights across Harmony.
Overall, Harmony will benefit both researchers and practitioners by providing solutions to more complex tasks in practically relevant settings (e.g. geometrically consistent medical shape analysis, or physics-based scene understanding).