Final Report Summary - CODDE (Co-ordination for optimal decisions in dynamic environments)
How does the brain use information from the senses to guide decisions and actions?
Everyday interactions depend on making prompt decisions in a dynamic world. Despite the seeming ease with which we detect, recognise, and categorise the targets of our attention and actions, extracting the key signals from the complex and ambiguous input of the natural world is a computationally demanding task that is far from understood. The challenge is to understand the link between sensory input, neural activity and adaptive behaviour. CODDE addresses this challenge through a multi-disciplinary approach to characterise the processes mediating our ability to make rapid decisions that translate into successful actions. We study the three key stages involved:
- The brain's efficient use of sensory data - how is information extracted, combined and integrated in time and space?
- The coordination of movements based on sensory input - what are the dynamics linking incoming sensory signals with movement outputs?
- The importance of lifelong learning and adaptation in shaping sensory processing and movement coordination - how does the brain adapt and change through experience and age?
Our aim is to provide fundamental new insight into the healthy brain, and to inspire new interactive technologies and facilitate methods that promote learning and rehabilitation following brain injury.
What does the CODDE network do?
The CODDE network consists of nine partners from across Europe that train 17 early career researchers (PhD) and early post-doc level. We conduct cutting-edge experiments that integrate the study of perception, motor behaviour and neural activity.
Mixing these different approaches (psychophysics, movement recording, functional brain imaging, virtual reality) depends centrally on good theoretical models.
Projects conducted by the network include:
- probing and modelling the visual information that drives eye movements,
- testing the integration of signals from the eyes and hands when judging a surface's compliance,
- determining how constraints from the environment influence arm and finger movements,
- developing new techniques to help prevent falls in old age,
- quantifying how the brain uses errors when moving to adapt and optimise its future responses,
- determining how the brain uses multi-sensory signals to detect timed events and how this information is used to control synchronisation,
- examining the neural basis of cross-modal judgments of surface texture,
- modelling the neural dynamics that encode both a decision and confidence in that decision,
- assessing the brain networks that support our perception of 3D and demonstrating how they change with experience,
- modelling the role of adaptation in helping the brain resolve perceptual ambiguities,
- probing learning and adaptation induced changes in the control of body posture,
- determining whether learning shapes attentional responses in early brain areas.
What have we found?
Our work has resulted in 59 presentations at international conferences and 18 published papers in leading peer-reviewed journals.
Examples of our findings are:
Taking the rough with the smooth - how hands and eyes work together.
Judith Eck and the team at Brain Innovation have shown how the hands and eyes work together when judging the roughness of nearby objects. Roughness signals are critical in everyday life-helping the brain plan movements and grasp objects. By combining behavioural tasks with advanced brain imaging, Judith demonstrated the importance of signals from the hands in guiding perception and brain responses.
Surprisingly, parts of the brain associated with sight show striking changes produced by information from the hands. The results demonstrate the importance of touch in visual appearance with implications for virtual reality applications that aim for a naturalistic experience of our textured environment.
Correcting the idea that 'one plus one equals one'. Using two senses helps us make parallel decisions.
Tom Otto and Pascal Mamassian have overturned the widely held view that the senses are automatically merged before decisions are made. They used a task that involved participants responding to changes that could come from a visual display, sounds they were listening to, or both. Through careful measurements of the time it takes to respond to such changes, they were able to show that people are better with two senses because they have access to two, parallel decision making mechanisms.
Importantly, they also showed that there is a cost associated with multi-sensory displays - people's judgments are more susceptible to random errors.
Make me sway to help me stay on my feet
D. J. Eikema and the team from Aristotle University have been developing new methods to help older adults stay on their feet. Falls pose a big health risk as people age, having effects ranging from mobility to psychological well-being and reduced immune system response. The team developed new methods to help train individuals to remain upright. They demonstrated a new method for optimising visual displays to improve balance and assist rehabilitation, and are using virtual reality systems to help train older adults in more life-like situations. The results could be used to help screen older adults, identifying individuals most at risk from falls. Moreover, balance retraining following a fall may be enhanced based on the team's findings.
How might our results be useful?
Work conducted by the CODDE network has implications for:
- tailoring technology to the user - from three-dimensional (3D) movies to advanced driver training,
- enhancing virtual reality methods based on the 'touch and feel',
- helping older adults stay on their feet,
- finding better ways of promoting learning in healthy adults and rehabilitation in brain-injured individuals.
To promote the application of our findings, we work with a number of industrial partners including Brain Innovation, VRSense, Bic and Volvo Technology.
Everyday interactions depend on making prompt decisions in a dynamic world. Despite the seeming ease with which we detect, recognise, and categorise the targets of our attention and actions, extracting the key signals from the complex and ambiguous input of the natural world is a computationally demanding task that is far from understood. The challenge is to understand the link between sensory input, neural activity and adaptive behaviour. CODDE addresses this challenge through a multi-disciplinary approach to characterise the processes mediating our ability to make rapid decisions that translate into successful actions. We study the three key stages involved:
- The brain's efficient use of sensory data - how is information extracted, combined and integrated in time and space?
- The coordination of movements based on sensory input - what are the dynamics linking incoming sensory signals with movement outputs?
- The importance of lifelong learning and adaptation in shaping sensory processing and movement coordination - how does the brain adapt and change through experience and age?
Our aim is to provide fundamental new insight into the healthy brain, and to inspire new interactive technologies and facilitate methods that promote learning and rehabilitation following brain injury.
What does the CODDE network do?
The CODDE network consists of nine partners from across Europe that train 17 early career researchers (PhD) and early post-doc level. We conduct cutting-edge experiments that integrate the study of perception, motor behaviour and neural activity.
Mixing these different approaches (psychophysics, movement recording, functional brain imaging, virtual reality) depends centrally on good theoretical models.
Projects conducted by the network include:
- probing and modelling the visual information that drives eye movements,
- testing the integration of signals from the eyes and hands when judging a surface's compliance,
- determining how constraints from the environment influence arm and finger movements,
- developing new techniques to help prevent falls in old age,
- quantifying how the brain uses errors when moving to adapt and optimise its future responses,
- determining how the brain uses multi-sensory signals to detect timed events and how this information is used to control synchronisation,
- examining the neural basis of cross-modal judgments of surface texture,
- modelling the neural dynamics that encode both a decision and confidence in that decision,
- assessing the brain networks that support our perception of 3D and demonstrating how they change with experience,
- modelling the role of adaptation in helping the brain resolve perceptual ambiguities,
- probing learning and adaptation induced changes in the control of body posture,
- determining whether learning shapes attentional responses in early brain areas.
What have we found?
Our work has resulted in 59 presentations at international conferences and 18 published papers in leading peer-reviewed journals.
Examples of our findings are:
Taking the rough with the smooth - how hands and eyes work together.
Judith Eck and the team at Brain Innovation have shown how the hands and eyes work together when judging the roughness of nearby objects. Roughness signals are critical in everyday life-helping the brain plan movements and grasp objects. By combining behavioural tasks with advanced brain imaging, Judith demonstrated the importance of signals from the hands in guiding perception and brain responses.
Surprisingly, parts of the brain associated with sight show striking changes produced by information from the hands. The results demonstrate the importance of touch in visual appearance with implications for virtual reality applications that aim for a naturalistic experience of our textured environment.
Correcting the idea that 'one plus one equals one'. Using two senses helps us make parallel decisions.
Tom Otto and Pascal Mamassian have overturned the widely held view that the senses are automatically merged before decisions are made. They used a task that involved participants responding to changes that could come from a visual display, sounds they were listening to, or both. Through careful measurements of the time it takes to respond to such changes, they were able to show that people are better with two senses because they have access to two, parallel decision making mechanisms.
Importantly, they also showed that there is a cost associated with multi-sensory displays - people's judgments are more susceptible to random errors.
Make me sway to help me stay on my feet
D. J. Eikema and the team from Aristotle University have been developing new methods to help older adults stay on their feet. Falls pose a big health risk as people age, having effects ranging from mobility to psychological well-being and reduced immune system response. The team developed new methods to help train individuals to remain upright. They demonstrated a new method for optimising visual displays to improve balance and assist rehabilitation, and are using virtual reality systems to help train older adults in more life-like situations. The results could be used to help screen older adults, identifying individuals most at risk from falls. Moreover, balance retraining following a fall may be enhanced based on the team's findings.
How might our results be useful?
Work conducted by the CODDE network has implications for:
- tailoring technology to the user - from three-dimensional (3D) movies to advanced driver training,
- enhancing virtual reality methods based on the 'touch and feel',
- helping older adults stay on their feet,
- finding better ways of promoting learning in healthy adults and rehabilitation in brain-injured individuals.
To promote the application of our findings, we work with a number of industrial partners including Brain Innovation, VRSense, Bic and Volvo Technology.