Periodic Reporting for period 4 - NovelExperiSENSE (How experience shapes brain specializations)
Reporting period: 2022-10-01 to 2024-03-31
Hubel & Wiesel’s Nobel prize winning experiments showed that early visual loss (during critical periods of development) limits brain plasticity, leading to irreversible deficits in brain function. In other research works, the same has been suggested for hearing and cognitively impaired populations. Our research challenges this notion. In line with the assumed objectives, we investigated whether brain organization can be modified with the use of technology and optimally designed perceptual learning programs, facilitating novel sensory experiences (NSEs). Using behavioral measures and neuroimaging we show that massive and relatively rapid adult brain plasticity is possible both in healthy and clinical populations, including individuals with sensory and cognitive deficits. Specifically (and as discussed below), we show that with the use of multisensory interventions, brain areas dedicated to one sensory modality can adapt to new sensory inputs later in life, and new sensory maps can form in the adult brain even in the absence of exposure to sensory inputs at an early age. We furthermore demonstrate the possibility of complementing and enhancing the existing senses in healthy subjects. Our research has societal implications, as it opens the way for innovative training and therapeutic programs by improving daily functioning and wellbeing, while also illustrating the potential to broaden the boundaries of human sensory abilities.
To investigate our theories, we first embarked on a series of studies focused on sensory deprivations. In an early blind individual using our sensory substitution device converting visual images to sounds and training we show utterly new sensory maps in high-level brain regions, organized according to pitch and temporal changes, which have not existed beforehand and normally develop only in early sensory regions in childhood. We further demonstrate in blind individuals that the deprived visual brain areas preserve specialization for processing visual objects such as faces or shapes, albeit presented through sounds, as well as their unique connectivity patterns across the whole brain. Finally, with our sound-based sensory-motor device for audio-guided navigation, we show that people can rapidly learn to navigate complex mazes in real and virtual environments. Interestingly the engaged brain area for this task is the navigation area ‘V6’ (which typically integrates signals from the visual receptors), even in people who have never used it for vision. Our these findings indicate that the visual brain functional development is not limited by critical periods, and can be reprogrammed through training to process other sensory information. This also led us to frame the ‘reversible plasticity gradient’ theory in our 2020 review paper concerning sensory-deprived populations where we suggest that while brain plasticity indeed decreases with age, it can be reignited at any point in life. Next, we tested the potential for auditory rehabilitation through NSEs in hearing aid and cochlear implant (CIs) users. The motivation came from the fact that an estimated 2.5 billion people worldwide are predicted to suffer from hearing loss by 2050 (WHO), and untreated hearing impairment is linked to a higher risk of neurodegenerative diseases and dementia. Partnering with leading European and Israeli centers we created tactile-based training methods that enable perception of 3D surroundings in both simple and complex distracting environments, as well as improve speech in noise comprehension. The spatial localization abilities of the hearing impaired improved significantly, with 50% experiencing the 3D moving tactile sources on the fingertips as originating from beyond their physical reach; a phenomenon known as “distal attribution” which typically develops only for the visual and auditory sense during early childhood. We also developed an audio-tactile speech rehabilitation program which was revealed through neuroimaging studies to engage multiple brain regions, some of which are typical for the life-long trained audio-visual speech perception (e.g. during lip-reading). These findings have immense potential for auditory rehabilitation, harnessing the brain's remarkable ability for sensory adaptation in adulthood. In order to examine the possibility of enhancing existing senses in healthy subjects through NSEs, we also developed a system enabling participants to reconstruct 360º images using combined 3D auditory and visual inputs after minimal training, thus “expanding” the frontal visual field to the back. Similarly, by converting auditory signals into tactile feedback, we created a proof-of-concept NSE that allowed participants to merge normal hearing with info-sounds, resulting in enhanced auditory perception (thus developing superior sensing abilities). We also explored cross-modal perception, showing that people can distinguish water temperatures from the sounds of pouring, which was also confirmed by a deep machine learning model. We have continuously focused on disseminating our research to academic and general public audiences; publishing more than 30 peer-reviewed papers in high impact journals (e.g. Current Biology, PloS One, J. Neurosci., Neurosci. Biobehav. Rev., PNAS, Sci Rep) and presenting at numerous international symposiums and conferences (e.g. International Multisensory Research Forum, International Society for Neuroscience, Organisation for Human Brain Mapping, and World Congress of Audiology). We have also been featured on various media platforms including podcasts and news outlets (e.g. Forbes, Haaretz, etc.) and national television, and even created an NSE exhibition at a UNESCO world heritage site (Bet Govrin).
Beyond the assumed goals of the research program, we explored the interaction between multisensory integration and brain plasticity, and health of the brain and body. Specifically, in our studies with early blind subjects we showed that brief training with a NSE device for navigation can induce beneficial changes in brain regions first affected by Alzheimer’s disease and Parkinson’s Disease, suggesting that our training protocols may help alter the course of neurodegenerative disease. Furthermore, we developed a unique NSE which combines training to improve spatial awareness and memory, environmental enrichment elements, and stress-regulation and mindfulness exercises based on attentional-training and Cognitive Behavioral Therapy principles. In a pilot study in older adults with subjective cognitive decline and a follow up study of adults under elevated stress, we demonstrated that daily use of this intervention led to improvements in resilience, reductions in depression, anxiety, and inflammatory markers, and increased connectivity in memory and navigation-related brain regions, those first impacted by neurodegenerative diseases. We continued with this multi-system framework by applying technologies to improve wellbeing and interoception through external sensory feedback, including through tactile vibrations corresponding to heartbeats and breathing combined with synchronized visual inputs; and a hand-held tactile device which enhances music appraisal and decreases anxiety and rumination. Based on these initial findings, we believe that using NSE interventions and dedicated optimized training regimes has the potential to halt age-related declines in brain plasticity and improve health outcomes and well-being.