Periodic Reporting for period 1 - LEAS (Looking for Emotions in the Absence of Sight)
Reporting period: 2021-12-01 to 2023-11-30
Affective capabilities play a pivotal role in human social interactions and are indispensable for optimal daily functioning. Nonverbal visual cues, such as facial expressions and body postures, frequently convey emotional signals, relying on the sense of sight for detection. This prompts intriguing questions about how emotions are experienced and perceived in the absence of vision, particularly in congenitally blind individuals. How do they articulate emotions through language? Is their representation of affective states in the body distinct from that of sighted individuals? Do they retain the same neural mechanisms for emotional coding?
Notably, there are currently around 2 billion visually impaired individuals globally, according to the World Health Organization. Given the pivotal role of vision in social interactions, it prompts reflection on whether the absence of sight leads to enduring changes in the psychological well-being of blind individuals. Understanding how sensory deprivation impacts their social existence becomes imperative to address this pertinent societal issue and tailor effective rehabilitative strategies. The LEAS project's innovative framework successfully addressed these questions, advancing knowledge in the realm of sensory and affective neuroscience by mapping emotions across language, body, and brain.
The LEAS project was designed with the explicit goal of unraveling how visual experience shapes the development of emotional representation. The project provided a comprehensive, multifaceted, and interdisciplinary description of the role of sensory experience in emotion processing, which had not been performed before. Indeed, the study of sensory deprived people represents a challenge for researchers, as it requires the ideation of ad hoc devices and experimental paradigms. Yet, it offers a compelling opportunity to address fundamental questions about human functioning. In LEAS, we developed novel strategies to capture the inner emotional experience of blind individuals that can be applied to a gamut of conditions that have not been accessible before.
Notably, there are currently around 2 billion visually impaired individuals globally, according to the World Health Organization. Given the pivotal role of vision in social interactions, it prompts reflection on whether the absence of sight leads to enduring changes in the psychological well-being of blind individuals. Understanding how sensory deprivation impacts their social existence becomes imperative to address this pertinent societal issue and tailor effective rehabilitative strategies. The LEAS project's innovative framework successfully addressed these questions, advancing knowledge in the realm of sensory and affective neuroscience by mapping emotions across language, body, and brain.
The LEAS project was designed with the explicit goal of unraveling how visual experience shapes the development of emotional representation. The project provided a comprehensive, multifaceted, and interdisciplinary description of the role of sensory experience in emotion processing, which had not been performed before. Indeed, the study of sensory deprived people represents a challenge for researchers, as it requires the ideation of ad hoc devices and experimental paradigms. Yet, it offers a compelling opportunity to address fundamental questions about human functioning. In LEAS, we developed novel strategies to capture the inner emotional experience of blind individuals that can be applied to a gamut of conditions that have not been accessible before.
Across three main work packages, we used behavioral and functional magnetic resonance imaging (fMRI) to address how emotions are represented in the absence of vision.
The first group of research work packages (WP4-6) was based on behavioral evaluations (Study 1). First, using semi-structured interviews and computational linguistics, we tested whether congenitally and late blind and sighted subjects share the same semantic representation of emotion. Specifically, we used a natural language approach, where blind and sighted individuals were asked to describe in three minutes a specific affective word (e.g. sadness) in terms of bodily sensations, events and related concepts. The procedure has been repeated for 30 emotion categories, providing a total of about 2 hours of descriptions for each subject. Our preliminary findings provide the first emotion taxonomy of sensory deprivation, showing how blind individuals conceive emotions and how differently they process affective states in comparison with sighted people.
The second group of work packages (WP7-9) also relied on a behavioral task (Study 2). We translated a previously validated paradigm for the investigation of bodily maps of emotion to the haptic modality. Congenitally and late blind and blindfolded sighted subjects interacted with a 50cm mannequin representing a human body mounted on a wooden platform. For the same emotion categories employed in Study 1, participants were asked where in the body they perceived changes associated to each state. Results show that for emotional states significantly related to visual cues (e.g. redness of the face for aggressiveness), both congenital and late blind individuals focus on other bodily sensations and identify those as more crucial in the experience (e.g. highlight of the mouth for aggressiveness). At the same time, there are also emotional states in which bodily sensations are similar between sensory deprived and typically developed individuals. The work of Study 2 has been presented at the IMRF (International Multisensory Research forum) 2023 meeting and the AIP (Italian Association of Psychology) 2023 conference.
The third group of work packages (WP10-12) refers to an fMRI investigation (Study 3). We employed a naturalistic stimulation to explore how emotions are represented in sighted, blind and deaf brains. In this regard, ecological stimuli as movies, constitute an ideal proxy of our social daily life and condense a variety of emotional events in a short time. We built different versions of the same movie (i.e. only audio, only video or audio and video) and asked sighted subjects to report their affective experience during the watching or listening of the stimulus. Then, we collected brain data from sensory deprived and typically developed individuals to unveil the role of sensory experience in the cortical representation of affect. Our findings highlight how the ventromedial prefrontal cortex stores a categorical representation of emotion independent of stimulus modality and previous sensory experience. The work of Study 3 has been presented to the BAPS (Belgian Association for Psychological Sciences) 2022 meeting, the CERE (Consortium of European Research on Emotion) 2022 conference, the OHBM (Organization for Human Brain Mapping) 2022 meeting, and the SIPF (Italian Society of Psychophysiology) 2022 meeting.
The first group of research work packages (WP4-6) was based on behavioral evaluations (Study 1). First, using semi-structured interviews and computational linguistics, we tested whether congenitally and late blind and sighted subjects share the same semantic representation of emotion. Specifically, we used a natural language approach, where blind and sighted individuals were asked to describe in three minutes a specific affective word (e.g. sadness) in terms of bodily sensations, events and related concepts. The procedure has been repeated for 30 emotion categories, providing a total of about 2 hours of descriptions for each subject. Our preliminary findings provide the first emotion taxonomy of sensory deprivation, showing how blind individuals conceive emotions and how differently they process affective states in comparison with sighted people.
The second group of work packages (WP7-9) also relied on a behavioral task (Study 2). We translated a previously validated paradigm for the investigation of bodily maps of emotion to the haptic modality. Congenitally and late blind and blindfolded sighted subjects interacted with a 50cm mannequin representing a human body mounted on a wooden platform. For the same emotion categories employed in Study 1, participants were asked where in the body they perceived changes associated to each state. Results show that for emotional states significantly related to visual cues (e.g. redness of the face for aggressiveness), both congenital and late blind individuals focus on other bodily sensations and identify those as more crucial in the experience (e.g. highlight of the mouth for aggressiveness). At the same time, there are also emotional states in which bodily sensations are similar between sensory deprived and typically developed individuals. The work of Study 2 has been presented at the IMRF (International Multisensory Research forum) 2023 meeting and the AIP (Italian Association of Psychology) 2023 conference.
The third group of work packages (WP10-12) refers to an fMRI investigation (Study 3). We employed a naturalistic stimulation to explore how emotions are represented in sighted, blind and deaf brains. In this regard, ecological stimuli as movies, constitute an ideal proxy of our social daily life and condense a variety of emotional events in a short time. We built different versions of the same movie (i.e. only audio, only video or audio and video) and asked sighted subjects to report their affective experience during the watching or listening of the stimulus. Then, we collected brain data from sensory deprived and typically developed individuals to unveil the role of sensory experience in the cortical representation of affect. Our findings highlight how the ventromedial prefrontal cortex stores a categorical representation of emotion independent of stimulus modality and previous sensory experience. The work of Study 3 has been presented to the BAPS (Belgian Association for Psychological Sciences) 2022 meeting, the CERE (Consortium of European Research on Emotion) 2022 conference, the OHBM (Organization for Human Brain Mapping) 2022 meeting, and the SIPF (Italian Society of Psychophysiology) 2022 meeting.
The LEAS project has propelled the study of emotional experience beyond the current state of the art, marking a significant milestone in the exploration of sensory deprivation and affective neuroscience. A fundamental shift in our understanding of the intricate relationship between emotions and the sense of sight has emerged through this innovative endeavor.
An impactful outcome of the project is its potential to catalyze the development of a pioneering algorithm designed to extract emotional information from text. This signifies not only a technological advancement but also the promise of refining existing methods in sentiment analysis. The creation of the LEAS emotion taxonomy introduces a novel conceptualization system in emotion semantics, providing a valuable resource for future studies in related fields. Moreover, the haptic tool developed in the course of our project holds significant promise and applicability beyond the specific context of blind individuals, making it a valuable asset for future research across various experimental groups.
Foreseeing a profound impact, these findings are poised to reshape the landscape of affective science and neuroscience, pushing the boundaries of our comprehension in emotional processing, sensory deprivation, and the broader workings of the human brain. Beyond academic realms, the project illuminates the critical role of social interactions for individuals with sensory deprivation, challenging societal perceptions. LEAS urges a departure from viewing blindness solely as a disability, emphasizing the importance of fostering inclusive environments that recognize and appreciate the rich social lives of those with visual impairments. In this way, LEAS not only contributes to scientific progress but also holds the potential to instigate positive societal change by promoting a more inclusive and understanding society. A specific website has been developed for the dissemination of results (http://giadalettieri.eu).
An impactful outcome of the project is its potential to catalyze the development of a pioneering algorithm designed to extract emotional information from text. This signifies not only a technological advancement but also the promise of refining existing methods in sentiment analysis. The creation of the LEAS emotion taxonomy introduces a novel conceptualization system in emotion semantics, providing a valuable resource for future studies in related fields. Moreover, the haptic tool developed in the course of our project holds significant promise and applicability beyond the specific context of blind individuals, making it a valuable asset for future research across various experimental groups.
Foreseeing a profound impact, these findings are poised to reshape the landscape of affective science and neuroscience, pushing the boundaries of our comprehension in emotional processing, sensory deprivation, and the broader workings of the human brain. Beyond academic realms, the project illuminates the critical role of social interactions for individuals with sensory deprivation, challenging societal perceptions. LEAS urges a departure from viewing blindness solely as a disability, emphasizing the importance of fostering inclusive environments that recognize and appreciate the rich social lives of those with visual impairments. In this way, LEAS not only contributes to scientific progress but also holds the potential to instigate positive societal change by promoting a more inclusive and understanding society. A specific website has been developed for the dissemination of results (http://giadalettieri.eu).