Periodic Reporting for period 2 - RIDING (Physics informed algorithms for sensing and navigating turbulent environments)
Reporting period: 2023-02-01 to 2024-07-31
Imagine you are surrounded by water and there is a target you need to reach. Even if you cannot see it, the target still sends you plenty information through water. First: you smell its odor through chemical sensors in your nose. Second: as the target moves it generates currents that you measure through mechanical sensors on your skin. Many living organisms solve this problem routinely to find food and mates and escape predators, but how they accomplish this incredible feat is not known. RIDING asks the following question: what algorithms are needed to reach a target combining chemical & mechanical information?
Turbulence is both the problem and the solution: it carries signals a long way, but at the same time it breaks them and distorts them which poses a major challenge. To address this challenge RIDING blends theory, numerical simulations of fluid turbulence and behavioral experiments with aquatic animals.
Understanding these strategies is key to many real world applications required to function in the presence of turbulence: from search and rescue to demining and patrolling. Robots need to be programmed to track chemicals emitted by e.g. drugs, explosives or mines and must navigate autonomously in uncontrolled environments. While much is known on navigation in more predictable environments, these approaches fail in the presence of turbulence. RIDING dissects the interplay between different sensory signals and will inspire novel algorithms to clarify how organisms and machines can cope with uncertainty.
The overall ojectives are the following:
O1. Assemble a massive dataset of chemical and mechanical signals emitted by a target using state of the art numerical simulations of fluid turbulence and mathematical approximations.
O2. Develop algorithmic approaches for sensing and navigation using tools from machine learning trained on multiple sensory signals assembled within RIDING (see O1).
O3. Examine how sensory signals (O1) and algorithms (O2) adapt to different environments.
O4. Test predictions by recording prey capture in the laboratory using marine organisms. Analysis includes a fascinating species of fish which evolved unique sensory “legs” to catch prey in different environments.
Turbulence is both the problem and the solution: it carries signals a long way, but at the same time it breaks them and distorts them which poses a major challenge. To address this challenge RIDING blends theory, numerical simulations of fluid turbulence and behavioral experiments with aquatic animals.
Understanding these strategies is key to many real world applications required to function in the presence of turbulence: from search and rescue to demining and patrolling. Robots need to be programmed to track chemicals emitted by e.g. drugs, explosives or mines and must navigate autonomously in uncontrolled environments. While much is known on navigation in more predictable environments, these approaches fail in the presence of turbulence. RIDING dissects the interplay between different sensory signals and will inspire novel algorithms to clarify how organisms and machines can cope with uncertainty.
The overall ojectives are the following:
O1. Assemble a massive dataset of chemical and mechanical signals emitted by a target using state of the art numerical simulations of fluid turbulence and mathematical approximations.
O2. Develop algorithmic approaches for sensing and navigation using tools from machine learning trained on multiple sensory signals assembled within RIDING (see O1).
O3. Examine how sensory signals (O1) and algorithms (O2) adapt to different environments.
O4. Test predictions by recording prey capture in the laboratory using marine organisms. Analysis includes a fascinating species of fish which evolved unique sensory “legs” to catch prey in different environments.
- We conducted high resolution direct numerical simulations of fluid turbulence with state-of-the-art performance using a GPU parallel implementation. The simulations demonstrate that currents created by a target in motion affect the odor emitted by the target itself. The implication is that small preys like plankton can optimize their locomotion pattern to minimize chances of being caught
- A major challenge of turbulent navigation consists in the fact that odor and fluid motion are intermittent, continuously switching on and off. As a consequence, simple measures that are informative in smooth environments become meaningless. We developed machine learning algorithms to elucidate what are the most salient features of a turbulent time series: coupling measures of intensity and sparsity achieves most accurate predictions. Importantly, both classes can actually be measured by the brain
- We used these informative features to design a navigation algorithm (Q-learning) that learns to navigate by trial and error responding solely to odor with no spatial perception. Our algorithm manipulates signals explicitly, with no black box or deep architecture that are popular in the literature. Agents have memory, as they measure turbulence for a finite amount time. We show that memory must be finely tuned as dictated by turbulence
- We developed another completely different navigation algorithm (POMDP), where an agent forms a guess of target location and uses odor and prior information to refine its guess. Similar to insects, the POMDP alternates casting crosswind and surging upwind, and we show mathematically what is the decision boundary between the two. We show that when odors are sparse, it is beneficial to acquire information even when this is costly, demonstrating that alternating different sensory modalities is beneficial
- We conducted experiments with sea robins, a fish that finds prey buried in sand with leg-like appendages. We demonstrated that the fish uses the chemical substance emanating from the buried target. We conducted a comparative analysis with closely related species unable to dig prey from sand
- We conducted experiments with sea anemones and show that their stinging behavior – also a problem of decision making using olfactory and chemical signals – is consistent with predictions from optimal control theory
- A major challenge of turbulent navigation consists in the fact that odor and fluid motion are intermittent, continuously switching on and off. As a consequence, simple measures that are informative in smooth environments become meaningless. We developed machine learning algorithms to elucidate what are the most salient features of a turbulent time series: coupling measures of intensity and sparsity achieves most accurate predictions. Importantly, both classes can actually be measured by the brain
- We used these informative features to design a navigation algorithm (Q-learning) that learns to navigate by trial and error responding solely to odor with no spatial perception. Our algorithm manipulates signals explicitly, with no black box or deep architecture that are popular in the literature. Agents have memory, as they measure turbulence for a finite amount time. We show that memory must be finely tuned as dictated by turbulence
- We developed another completely different navigation algorithm (POMDP), where an agent forms a guess of target location and uses odor and prior information to refine its guess. Similar to insects, the POMDP alternates casting crosswind and surging upwind, and we show mathematically what is the decision boundary between the two. We show that when odors are sparse, it is beneficial to acquire information even when this is costly, demonstrating that alternating different sensory modalities is beneficial
- We conducted experiments with sea robins, a fish that finds prey buried in sand with leg-like appendages. We demonstrated that the fish uses the chemical substance emanating from the buried target. We conducted a comparative analysis with closely related species unable to dig prey from sand
- We conducted experiments with sea anemones and show that their stinging behavior – also a problem of decision making using olfactory and chemical signals – is consistent with predictions from optimal control theory
- Current research lacks an algorithm that learns to navigate turbulence by trial and error from turbulent data. Our model-free Q-learning algorithm successfully learns from interacting with our simulated turbulent environment. Moreover, the algorithm is interpretable as it manipulates odor time series explicitly: it defines memory as a time scale; measures specific properties of the odor time series and chooses actions from a look-up table. These are testable hypothesis for neuroscience, behavior and sensory biology.
- The state of the art lacks a normative algorithm to combine multiple senses in turbulence. Our model-based POMDP algorithm for navigation examines this problem and is a proof of concept to be further extended. POMDPs are known to be computationally cumbersome and to require approximations. We optimized a well-known approximation scheme and obtained a dramatic speed up, which makes it now possible to compare model free and model based paradigms for navigation.
- The state of the art has overlooked the effects of motion of the target itself on its own odor trail. Our work clarifies that for groups of targets these effects cannot be neglected and we expect to quantify and test the ensuing evolutionary pressure.
- Turbulent navigation is most studied in model systems. Sea robins evolved leg-like appendages that appear to be evolutionary novelties. We characterized the legs and the behavior that digs olfactory targets from sand, thus connecting this evolutionary novelty to its potential function.
- Normative theories for behavior are rare, as ranking the “best” behaviors is generally hard. Our normative model of stinging in the sea anemone is an example of normative theory that successfully explains behavior and offers the exciting opportunity to connect it to neural activity, due to the sea anemone’s primitive nervous system.
Expected results by the end of the project:
1. Current literature is fragmented: radically different algorithms have been propose to address turbulent navigation, but a systematic comparison is missing. We will develop an open-access platform to compare algorithms for turbulent navigation. Teams will commit algorithms and the platform will assess performance, robustness and computational requirements.
2. We will publish an open-access comprehensive dataset of 4-D turbulent fields
3. We will conduct a systematic analysis of animal behavior during navigation and test the predictions from distinct algorithmic paradigms
- The state of the art lacks a normative algorithm to combine multiple senses in turbulence. Our model-based POMDP algorithm for navigation examines this problem and is a proof of concept to be further extended. POMDPs are known to be computationally cumbersome and to require approximations. We optimized a well-known approximation scheme and obtained a dramatic speed up, which makes it now possible to compare model free and model based paradigms for navigation.
- The state of the art has overlooked the effects of motion of the target itself on its own odor trail. Our work clarifies that for groups of targets these effects cannot be neglected and we expect to quantify and test the ensuing evolutionary pressure.
- Turbulent navigation is most studied in model systems. Sea robins evolved leg-like appendages that appear to be evolutionary novelties. We characterized the legs and the behavior that digs olfactory targets from sand, thus connecting this evolutionary novelty to its potential function.
- Normative theories for behavior are rare, as ranking the “best” behaviors is generally hard. Our normative model of stinging in the sea anemone is an example of normative theory that successfully explains behavior and offers the exciting opportunity to connect it to neural activity, due to the sea anemone’s primitive nervous system.
Expected results by the end of the project:
1. Current literature is fragmented: radically different algorithms have been propose to address turbulent navigation, but a systematic comparison is missing. We will develop an open-access platform to compare algorithms for turbulent navigation. Teams will commit algorithms and the platform will assess performance, robustness and computational requirements.
2. We will publish an open-access comprehensive dataset of 4-D turbulent fields
3. We will conduct a systematic analysis of animal behavior during navigation and test the predictions from distinct algorithmic paradigms