Skip to main content

LEarning from our collective visual memory to Analyze its trends and Predict future events

Objective

People constantly draw on past visual experiences to anticipate future events and better understand, navigate, and interact with their environment, for example, when seeing an angry dog or a quickly approaching car. Currently there is no artificial system with a similar level of visual analysis and prediction capabilities. LEAP is a first step in that direction, leveraging the emerging collective visual memory formed by the unprecedented amount of visual data available in public archives, on the Internet and from surveillance or personal cameras - a complex evolving net of dynamic scenes, distributed across many different data sources, and equipped with plentiful but noisy and incomplete metadata. The goal of this project is to analyze dynamic patterns in this shared visual experience in order (i) to find and quantify their trends; and (ii) learn to predict future events in dynamic scenes.
With ever expanding computational resources and this extraordinary data, the main scientific challenge is now to invent new and powerful models adapted to its scale and its spatio-temporal, distributed and dynamic nature. To address this challenge, we will first design new models that generalize across different data sources, where scenes are captured under vastly different imaging conditions. Next, we will develop a framework for finding, describing and quantifying trends that involve measuring long-term changes in many related scenes. Finally, we will develop a methodology and tools for synthesizing complex future predictions from aligned past visual experiences.
Breakthrough progress on these problems would have profound implications on our everyday lives as well as science and commerce, with safer cars that anticipate the behavior of pedestrians on streets; tools that help doctors monitor, diagnose and predict patients’ health; and smart glasses that help people react in unfamiliar situations enabled by the advances from this project.

Call for proposal

ERC-2013-StG
See other projects for this call

Host institution

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
Address
Domaine De Voluceau Rocquencourt
78153 Le Chesnay Cedex
France
Activity type
Research Organisations
EU contribution
€ 1 496 736
Principal investigator
Josef Sivic (Dr.)
Administrative Contact
Jacqueline Maestracci (Ms.)

Beneficiaries (1)

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
France
EU contribution
€ 1 496 736
Address
Domaine De Voluceau Rocquencourt
78153 Le Chesnay Cedex
Activity type
Research Organisations
Principal investigator
Josef Sivic (Dr.)
Administrative Contact
Jacqueline Maestracci (Ms.)