Final Report Summary - BRAINVISIONREHAB (‘Seeing’ with the ears, hands and bionic eyes: from theories about brain organization to visual rehabilitation)
What is the main driving force shaping the emergence of category selectivity in the human brain? It is well established that higher-order sensory cortices are divided into regions that are highly anatomically consistent across individuals, and that respond to specific stimuli categories (e.g. in vision: broad division of labor between ventral (“what”) and dorsal (“where”) streams; within these streams category selectivity to specific visual objects such as letters, numbers or body parts has been repeatedly observed). During this project, we investigated the extent to which such specializations can arise when visual experience is entirely lacking across the life span, by testing visual cortex organization in people born blind. We did it by using a quite unique approach, namely, combining fMRI with Sensory Substitution Devices (SSDs), which are devices that translate the visual information into audition using a predetermined algorithm that maintains the exact shape, location and color of objects in a visual scene. We developed specific training programs for our blind users to learn to interpret via SSD specific “visual” categories and then we tested the results of such SSD training with fMRI. Our key result is that higher-order ‘visual’ cortices do develop anatomically consistent category-selectivity (e.g. for dorsal/ventral division of labor as well as for specific stimuli categories such as body shapes, letters, numbers) without any visual experience even if the input is provided by an atypical sensory modality learned in few hours of dedicated training during adulthood (e.g. learning to interpret body shapes via SSD). Our work promotes a paradigm shift in how we conceptualize our sensory brain by suggesting that 1- the brain might be organized as a task-machine rather than as a sensory-machine as classically conceived and that 2- visual experience during critical periods early in development is not essential to develop anatomically consistent specializations in higher-order ‘visual’ regions. This work was published in over 15 papers in leading neuroscience and neuroimaging journals, and yielded a full bloomed theoretical framework which includes also two proposed mechanisms driving the emergence of such brain organization that was presented in several recent reviews. In parallel, this line of work also resulted in the creation of two novel Sensory Substitution Devices (SSDs) to offer visual information to blind users and proved their efficacy for rehabilitation. This resulted in >20 translational related papers. Specifically, we developed the EyeCane, a minimal-SSD to aid mobility in the blind. We showed that blind EyeCane users could successfully navigate through simple environments after less than 5 minutes of training and that the EyeCane was more successful than the classic white-cane to allow users to avoid waist-up obstacles. We also showed that with slightly longer training, EyeCane users with different levels of visual experience – congenitally blind, late blind and blindfolded-sighted – could successfully navigate through complex mazes. On a practical rehab level, we are ready to bring this device to the wider blind population, as a consumer version of the device is ready for mass production. The EyeMusic SSD utilizes pleasant musical notes to convey visual information about colors, shapes and location of objects in the world (http://amedilab.com/). We showed that EyeMusic users (blind; blindfolded sighted) were able to correctly perceive and interact with objects, even in real-life environments (A live demonstration can be seen at http://youtu.be/r6bz1pOEJWg) and in virtual ones, and they were also able to transfer the information from the senses to motor actions. We also showed that visual abilities such as visual acuity and visual parsing can be successfully accomplished through SSD-focused training, further strengthening the feasibility of SSDs as efficient tools for visual rehabilitation and recovery. Overall our research in rehab suggests (contrary to the current prevalent approach), that multisensory rather than unisensory training might be more effective in cases of sensory restoration, ultimately providing crucial guidelines for maximizing the outcomes of sensory recovery.