Final Report Summary - TEMPUS_G (Temporal Enhancement of Motor Performance Using Sensory Guides)
Controlling the timing of our movements to successfully perform an action (e.g. directing our eyes to read this text, drinking from a cup, crossing a road or catching a ball) is something we do thousands of times a day without thinking twice. How does the brain control the timing of these types of movement so that we do not knock over the cup we want to drink from or drop the ball we want to catch? How does the information we pick up through our senses (e.g. the location of the cup or flight path of the ball) influence our decisions about when and how to act? What happens when we want to learn the timing of a movement to perform a new skill or if the action timing system in the brain breaks down (as in the case of Parkinson’s disease)? Can specially engineered patterns of sensory information be used to improve the timing of our movement in a health related (e.g. walking better) or a sports related context (e.g. hitting a ball)?
To answer these questions the first part of the project involved carrying out a number of experiments that help us understand how the brain tunes into and uses sensory information (visual and auditory) to guide action. To start with we looked at how our ability to synchronise our movements to a beat changes when we have a continuous sound. We found that the way in which a continuous sound changes over time can provide a robust temporal framework that allows participants to move in a more consistent manner. We have also shown that for a given event, such as a ball rolling down a ramp covered in sandpaper, we can time our actions so we can catch the ball at the bottom of the ramp using only the sound of the approaching ball. We have shown how the relationship between the changing patterns of sound intensity as the ball accelerates down the ramp provides important information for the brain about when the ball will arrive at the catching point. This information can then be used to accurately time the catch and is exploited more successfully by those who are visually impaired compared to those who are sighted. Interestingly we also showed that this type of dynamic event involving a moving ball can also speed movement in people suffering from Parkinson’s disease, with them being able to accurately time and adapt their movement to the speed of the ball. This provides important insights into perception action networks in the brain and how sensory information can improve the temporal control of movement by coupling onto an extrinsic temporal framework.
We have also looked at how the spatio-temporal dynamics of the movement of a person or a ball, with respect to an observer, carries important information that guides our actions. More specifically, we were interested in understanding how events that are more unpredictable in nature are perceived (e.g. a ball with added spin flying through the air or a player that uses their body movement to deceive an opponent) and how these events influence decisions about when and how to act. By comparing different groups (experts vs. novices) we were able to see how different action capabilities also impact on performance. Our newly developed state of the art immersive, interactive virtual reality system allowed us to control the perceptual information presented to an observer (e.g. ball flight or player movement) and precisely measure how the observer responds. By ensuring that the type of movement one makes in a real sporting situation directly maps onto the movements made in the virtual reality setting (e.g. goal-keeping in soccer), more ecologically valid experiments were implemented. Furthermore, by using hand trackers to animate in real time virtual depictions of the participant’s hands, the level of behavioural presence in the virtual environments was greatly enhanced. We have shown how it is more difficult to anticipate where a ball that bends is going to go (e.g. a curved free kick in soccer) but that experts can overcome this problem by waiting longer before deciding in which direction to move. We have been able to model how the visual information influences the movement with our model strongly predicting observed behaviour, including the effect of the position of the goal posts. We also now understand how deception can be signalled through movement and which parts of the body signal deception and which cannot when side-stepping a player. We have used tau-coupling theory to show how experts tune into the temporal dynamics of the honest signals while the novices tend to tune into the temporal dynamics of the deceptive body based signals, leading to significantly higher rates of error.
The findings from the first part of the project have helped us understand what is important in the way patterns of sensory information change over time so they can be used to improve the timing of our actions. In collaboration with engineers, we have developed a visual guide using a programmable LED device that has been used in the context of learning a new skill (e.g. golf putting) and speeding movements in patients with Parkinson’s disease. The device contains a sequence of embedded leds that can each be individually programmed so as to determine when they light up and for how long for. This allows us to reproduce the temporal dynamics of biological motion or a moving object. With a group of patients suffering from Parkinson’s disease we were able to show that they could very successfully pick up and use this type of visual information to speed their movements. We also showed that by reproducing the temporal dynamics of the biological motion of an expert putting action, novice golfers could couple their movement onto this visual guide to consistently control the distance of a putt. We also explored movement guides in the auditory domain. By both recording real sounds and synthesising new sounds we explored what the essential properties of sound are that can help improve movement performance. By using ecologically relevant sounds, such as walking on a gravel path, we have shown how people with Parkinson’s disease can use these sounds to move better. By coupling their walking to the gravel footstep sounds they significantly reduced the incidence of shuffling, walking with a longer stride length and at a faster cadence. It would appear that these sounds, which convey action relevant properties, allow the brain to switch the control of movement away from the damaged part of the brain to another part of the brain that works better.
Finally one of the most exciting developments in the project has been harnessing the power of movement based game controllers, such as the Nintendo Wii balance board, as a means of interacting with virtual environments. Using virtual reality software, we have successfully created our own tailor-made balance games for older adults, which invite players to move in a specific way to both train balance and explore limits of balance stability. In these games we control the timing of the movement of virtual objects (visual and acoustic) and get the user to control the timing of their movements to intercept them. Increased levels of difficulty can be adapted to meet the needs of the player. A large group of older adults who played the games over a 4 week period significantly improved both their functional balance and balance confidence. Furthermore, the older adults who played the games reported that they were both fun and engaging, and motivated them to seek new ways to further improve their balance. We also found similar results in people with Parkinson’s disease, who found that playing the games helped reduced the level of tonicity in the muscles around the trunk area. Plans are afoot to try and commercialise the games and make them more widely available to both older adults and people with Parkinson’s.
To answer these questions the first part of the project involved carrying out a number of experiments that help us understand how the brain tunes into and uses sensory information (visual and auditory) to guide action. To start with we looked at how our ability to synchronise our movements to a beat changes when we have a continuous sound. We found that the way in which a continuous sound changes over time can provide a robust temporal framework that allows participants to move in a more consistent manner. We have also shown that for a given event, such as a ball rolling down a ramp covered in sandpaper, we can time our actions so we can catch the ball at the bottom of the ramp using only the sound of the approaching ball. We have shown how the relationship between the changing patterns of sound intensity as the ball accelerates down the ramp provides important information for the brain about when the ball will arrive at the catching point. This information can then be used to accurately time the catch and is exploited more successfully by those who are visually impaired compared to those who are sighted. Interestingly we also showed that this type of dynamic event involving a moving ball can also speed movement in people suffering from Parkinson’s disease, with them being able to accurately time and adapt their movement to the speed of the ball. This provides important insights into perception action networks in the brain and how sensory information can improve the temporal control of movement by coupling onto an extrinsic temporal framework.
We have also looked at how the spatio-temporal dynamics of the movement of a person or a ball, with respect to an observer, carries important information that guides our actions. More specifically, we were interested in understanding how events that are more unpredictable in nature are perceived (e.g. a ball with added spin flying through the air or a player that uses their body movement to deceive an opponent) and how these events influence decisions about when and how to act. By comparing different groups (experts vs. novices) we were able to see how different action capabilities also impact on performance. Our newly developed state of the art immersive, interactive virtual reality system allowed us to control the perceptual information presented to an observer (e.g. ball flight or player movement) and precisely measure how the observer responds. By ensuring that the type of movement one makes in a real sporting situation directly maps onto the movements made in the virtual reality setting (e.g. goal-keeping in soccer), more ecologically valid experiments were implemented. Furthermore, by using hand trackers to animate in real time virtual depictions of the participant’s hands, the level of behavioural presence in the virtual environments was greatly enhanced. We have shown how it is more difficult to anticipate where a ball that bends is going to go (e.g. a curved free kick in soccer) but that experts can overcome this problem by waiting longer before deciding in which direction to move. We have been able to model how the visual information influences the movement with our model strongly predicting observed behaviour, including the effect of the position of the goal posts. We also now understand how deception can be signalled through movement and which parts of the body signal deception and which cannot when side-stepping a player. We have used tau-coupling theory to show how experts tune into the temporal dynamics of the honest signals while the novices tend to tune into the temporal dynamics of the deceptive body based signals, leading to significantly higher rates of error.
The findings from the first part of the project have helped us understand what is important in the way patterns of sensory information change over time so they can be used to improve the timing of our actions. In collaboration with engineers, we have developed a visual guide using a programmable LED device that has been used in the context of learning a new skill (e.g. golf putting) and speeding movements in patients with Parkinson’s disease. The device contains a sequence of embedded leds that can each be individually programmed so as to determine when they light up and for how long for. This allows us to reproduce the temporal dynamics of biological motion or a moving object. With a group of patients suffering from Parkinson’s disease we were able to show that they could very successfully pick up and use this type of visual information to speed their movements. We also showed that by reproducing the temporal dynamics of the biological motion of an expert putting action, novice golfers could couple their movement onto this visual guide to consistently control the distance of a putt. We also explored movement guides in the auditory domain. By both recording real sounds and synthesising new sounds we explored what the essential properties of sound are that can help improve movement performance. By using ecologically relevant sounds, such as walking on a gravel path, we have shown how people with Parkinson’s disease can use these sounds to move better. By coupling their walking to the gravel footstep sounds they significantly reduced the incidence of shuffling, walking with a longer stride length and at a faster cadence. It would appear that these sounds, which convey action relevant properties, allow the brain to switch the control of movement away from the damaged part of the brain to another part of the brain that works better.
Finally one of the most exciting developments in the project has been harnessing the power of movement based game controllers, such as the Nintendo Wii balance board, as a means of interacting with virtual environments. Using virtual reality software, we have successfully created our own tailor-made balance games for older adults, which invite players to move in a specific way to both train balance and explore limits of balance stability. In these games we control the timing of the movement of virtual objects (visual and acoustic) and get the user to control the timing of their movements to intercept them. Increased levels of difficulty can be adapted to meet the needs of the player. A large group of older adults who played the games over a 4 week period significantly improved both their functional balance and balance confidence. Furthermore, the older adults who played the games reported that they were both fun and engaging, and motivated them to seek new ways to further improve their balance. We also found similar results in people with Parkinson’s disease, who found that playing the games helped reduced the level of tonicity in the muscles around the trunk area. Plans are afoot to try and commercialise the games and make them more widely available to both older adults and people with Parkinson’s.