Adaptive functions of visual systems

Project Information

AdaptiveVision

Grant agreement ID: 101045003

DOI

10.3030/101045003

EC signature date 15 September 2022

Start date 1 April 2023

End date 31 March 2028

Funded under

• European Research Council (ERC)

Total cost

€ 1 999 613,00

EU contribution

€ 1 999 613,00

Coordinated by

JOHANNES GUTENBERG-UNIVERSITAT MAINZ
Germany

Periodic Reporting for period 1 - AdaptiveVision (Adaptive functions of visual systems)

Reporting period: 2023-04-01 to 2025-09-30

Summary of the context and overall objectives of the project

The processing of visual information allows humans, animals, and computer-vision based machines to navigate the world. All visual systems face common challenges when the world rapidly changes. Such changes are often generated by an animal’s own movement. Self-motion for example causes fast changes in illumination and generates global motion patterns on the eye, due to the movement of the world relative to the observer. Diverse visual systems face these common challenges but must also deal with important differences. First, animals experience different environments. Second, animals show different types of behavior, such as walking or flying, and behavior will alter the visual cues that the animal encounters. The goal of AdaptiveVision is to first understand common principles of visual system function, and to then work out how diverse visual systems adapt to specific environmental and behavioral constraints. To achieve this, AdaptiveVision will study two essential visual computations, the robust estimation of contrast in dynamically changing environments, and the encoding of global motion cues generated by self-motion.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have indentified how flies stably process contrast n visual circuitry downstream of photoreceptors such that it remains unaffected by rapid changes in luminance. Using in vivo 2-photon imaging of neural responses in a living fly who was exposed to visual stimuli, we showed that neurons Tm9 and Tm1 in the visual OFF pathway are the first to exhibit luminance-invariant responses, unlike Tm2 and Tm4. We worked out the biophysical mechanisms and demonstrated that an inhibitory, glutamatergic mechanism is necessary this rapid luminance gain control in Tm9, but not Tm1. We combined experimental and theoretical approaches to show that local spatial pooling of luminance is required for luminance-invariant visual processing. Thus, the fly visual system has implemented a mechanisms by which contrast signals are corrected by a luminance signal that explores locally around the point at which contrast must be computed. This work is now published in Gür et al. 2024, Nature Communications.
We have also done extensive work on the recently released full adult fly brain connectome, and are currently exploring connectome datasets to understand the basis for global motion processing. The basis for our current work is published in Cornean et al. 2024, Nature Communications, and we have contributed to the release of the FlyWire connectome, as part of the FlyWire consortium. This is published in Schlegel et al. 2024 Nature, and Dorkenwald et al. 2024 Nature.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

This project deals with unsolved problems computing the most fundamental feature in vision - contrast and motion cues – in challenging conditions. These can be rapid changes in background luminance or the presence of global motion patterns caused by the object's own movement. These are challenges that no only flies face, but all visually guided animals including humans, but also all devices relying on machine vision, or artifical neural networks that compute information from dynamic visual scenes.
In an infamous example in the development of autonomously-driving cars, the human safety driver of the vehicle died because the car’s camera-based vision system could not identify a suddenly appearing bright truck against a bright background.

Although the self-driving car industry has found a solution to this problem by including additional radar- or lidar-based technology, it is striking that our eyes are suited to accommodate these apparently challenging conditions that are common to natural scenes. Our eyes (and the eyes of other animals) can do what computer-vision-based devices fail to do and perform a luminance-invariant computation of contrast at rapid time scales, allowing us to identify both clouds in the sky and trees in the dark forest, when facing a scenic view. Here we aim to learn from the eyes of animals about how contrast and global motion cues can be computed stably, especially when we are facing challenging, rapidly changing conditions.

summary-figure-sr1-01.png