Periodic Reporting for period 1 - EMBRACE (tEchnology for Multimodal inter-BRain dynAmiCs invEstigation)
Reporting period: 2021-02-01 to 2023-10-31
Most social interactions involve interpersonal coordinated movements in real-time and real space, but research on the complex processes determining social behavior is limited by inadequate technology and analytical tools.
EMBRACE merges complementary expertise and resources in biomedical engineering, material science, signal processing, neuroscience and social psychology at 3 academic and 3 industrial partners from 3 EU countries (Italy, Germany and Spain) to develop:
1) a new mobile and wireless dry electrode EEG system suitable for monitoring brain activity during full body movements
2) novel body-network sensors and a multimodal alignment system for the simultaneous recording of neural, physiological and kinematic signals from two interacting subjects
3) novel analytical solutions for motion artefact removal and multi-level analysis of multimodal data
4) a new research dyadic paradigm to exploit the technological advances.
The novel technologies address requirements of mobility, high signal quality, high temporal and spatial resolution, stable alignment of multiple signals, wearing comfort and long-term use.
The novel analytical methods will permit the investigation of joint action at the neural, cognitive-behavioral, and social levels.
Both technologies and analytical methods will be validated in real time and real space, face-to-face studies to produce prototypes and software models optimized for broad application.
To achieve these objectives, the EMBRACE consortium realizes extensive intersectoral transfer of knowledge and experience through shared research, secondments, and summer schools, promoting the research capacity and competitiveness of its partners and becoming a lasting EU network promoting basic and applied biomedical research.
EMBRACE merges complementary expertise and resources in biomedical engineering, material science, signal processing, neuroscience and social psychology at 3 academic and 3 industrial partners from 3 EU countries (Italy, Germany and Spain) to develop:
1) a new mobile and wireless dry electrode EEG system suitable for monitoring brain activity during full body movements
2) novel body-network sensors and a multimodal alignment system for the simultaneous recording of neural, physiological and kinematic signals from two interacting subjects
3) novel analytical solutions for motion artefact removal and multi-level analysis of multimodal data
4) a new research dyadic paradigm to exploit the technological advances.
The novel technologies address requirements of mobility, high signal quality, high temporal and spatial resolution, stable alignment of multiple signals, wearing comfort and long-term use.
The novel analytical methods will permit the investigation of joint action at the neural, cognitive-behavioral, and social levels.
Both technologies and analytical methods will be validated in real time and real space, face-to-face studies to produce prototypes and software models optimized for broad application.
To achieve these objectives, the EMBRACE consortium realizes extensive intersectoral transfer of knowledge and experience through shared research, secondments, and summer schools, promoting the research capacity and competitiveness of its partners and becoming a lasting EU network promoting basic and applied biomedical research.
During the first 24 months of the project 76 months of international and intersectoral secondments were implemented, several dissemination and communication activities were performed by the EMBRACE partners, and the first and second EMBRACE Summer Schools were organized for teaching technical/applicative/social topics and transferable skills. External invited speakers contributed to the Summer Schools.
Concerning the implementation of the scientific objectives, the following results have been achieved:
1. A prototype of the novel mobile EEG system with integrated electronics for the novel dry sports caps mounting the novel dry sports electrodes (the flower electrodes).
2. A prototype of the novel multimodal body-network sensors and electronics for the synchronous registration of EEG, ECG and respiration effort signals.
3. A prototype of the inter-device synchronization via wired and wireless communication media for the multimodal body-network sensors and motion capture system with integrated EMG sensors.
4. A dataset of EEG signals affected by stereotyped motion-related artefacts to develop new algorithms to correct motion-related artefacts.
5. A set of analytical methods to analyze the multimodal data at the neural, cognitive-behavioral, and social levels.
6. Definition of the study protocol to perform dyadic studies on joint actions performed through full body free movements occurring face to face, in cooperative and competitive modes, and at two difficulty levels.
Concerning the implementation of the scientific objectives, the following results have been achieved:
1. A prototype of the novel mobile EEG system with integrated electronics for the novel dry sports caps mounting the novel dry sports electrodes (the flower electrodes).
2. A prototype of the novel multimodal body-network sensors and electronics for the synchronous registration of EEG, ECG and respiration effort signals.
3. A prototype of the inter-device synchronization via wired and wireless communication media for the multimodal body-network sensors and motion capture system with integrated EMG sensors.
4. A dataset of EEG signals affected by stereotyped motion-related artefacts to develop new algorithms to correct motion-related artefacts.
5. A set of analytical methods to analyze the multimodal data at the neural, cognitive-behavioral, and social levels.
6. Definition of the study protocol to perform dyadic studies on joint actions performed through full body free movements occurring face to face, in cooperative and competitive modes, and at two difficulty levels.
EMBRACE draws together innovations in biomedical engineering, neuroscience, signal processing, and social psychology, with the potential to go beyond the state-of-the-art and knowledge base in all these fields.
Our approach combines innovations in technology (a new mobile and wireless dry electrode EEG system and a novel multimodal alignment system), analytical methods (new solutions for motion-related artefact removal and new methods for the multi-level analysis of multimodal data), and study paradigms (the novel table tennis paradigm) that will permit to situate the findings on the functional mechanisms at the basis of face-to-face cooperative and competitive joint action within a psycho-social framework to gain unprecedented understanding of the human social brain during interpersonal interactions in real time and real space.
Innovation in engineering: The new EMBRACE dry sport EEG system will permit the simultaneous neuroimaging of two interacting subjects while they freely move with no time or space restrictions, featuring high signal quality, stability and subject comfort, short mounting times, long recording times, high space and time resolutions, robust and light-weight electronics, optimized skin-sensor contact, robust modular high-density caps with personalized shape and size. The novel multimodal alignment system will outclass available solutions by offering wireless simultaneous acquisitions of EEG, EMG, ECG, respiration effort and kinematic signals from two subjects during full body movements. The production of this multi-device and multi-subject alignment system has the potential to become a real commercial breakthrough in the multilevel investigation of human interactions.
Innovation in analytical solutions: The innovative methods for the automated correction of motion-related artefacts in the neurophysiological data acquired during joint action, and a new analytical approach combining hyperbrain functional connectivity, microstate analysis, Graph Theory and Network Physiology will fully exploit the richness of information contained in the collected multimodal data, and will permit to investigate the mechanisms at the basis of human social behavior during joint action at the three recommended levels of analysis: neural, cognitive-behavioral, and social.
Innovation in study design: Our new study protocol fully exploits the innovative technology developed within EMBRACE and permits to investigate the complex processes underpinning face-to-face joint action during the performance of full body cooperative and competitive tasks in real time and real space and how these processes adjust to variable external conditions.
The exploitation of the EMBRACE results will bring benefits to European industries in nano-electronics, medical electronics, and even entertainment electronics, with the potential extension of EEG technology markets to products like intelligent prostheses or brain computer interfaces. Consequently, employment opportunities will increase, especially for young engineers, physicists, neuroscientists and behavioral psychologists.
The exploitation of the EMBRACE results in organizations and work settings where social behavior is a main feature, may improve teamwork and the effectiveness of the many task-related goals that are achieved through human coordination and interaction.
Finally, the international mobility and dissemination of the goals and achievements of the EMBRACE project will contribute to sharing cultures and knowledge with the scientific community and to promote communication on the importance of research in biomedical engineering to the society.
Our approach combines innovations in technology (a new mobile and wireless dry electrode EEG system and a novel multimodal alignment system), analytical methods (new solutions for motion-related artefact removal and new methods for the multi-level analysis of multimodal data), and study paradigms (the novel table tennis paradigm) that will permit to situate the findings on the functional mechanisms at the basis of face-to-face cooperative and competitive joint action within a psycho-social framework to gain unprecedented understanding of the human social brain during interpersonal interactions in real time and real space.
Innovation in engineering: The new EMBRACE dry sport EEG system will permit the simultaneous neuroimaging of two interacting subjects while they freely move with no time or space restrictions, featuring high signal quality, stability and subject comfort, short mounting times, long recording times, high space and time resolutions, robust and light-weight electronics, optimized skin-sensor contact, robust modular high-density caps with personalized shape and size. The novel multimodal alignment system will outclass available solutions by offering wireless simultaneous acquisitions of EEG, EMG, ECG, respiration effort and kinematic signals from two subjects during full body movements. The production of this multi-device and multi-subject alignment system has the potential to become a real commercial breakthrough in the multilevel investigation of human interactions.
Innovation in analytical solutions: The innovative methods for the automated correction of motion-related artefacts in the neurophysiological data acquired during joint action, and a new analytical approach combining hyperbrain functional connectivity, microstate analysis, Graph Theory and Network Physiology will fully exploit the richness of information contained in the collected multimodal data, and will permit to investigate the mechanisms at the basis of human social behavior during joint action at the three recommended levels of analysis: neural, cognitive-behavioral, and social.
Innovation in study design: Our new study protocol fully exploits the innovative technology developed within EMBRACE and permits to investigate the complex processes underpinning face-to-face joint action during the performance of full body cooperative and competitive tasks in real time and real space and how these processes adjust to variable external conditions.
The exploitation of the EMBRACE results will bring benefits to European industries in nano-electronics, medical electronics, and even entertainment electronics, with the potential extension of EEG technology markets to products like intelligent prostheses or brain computer interfaces. Consequently, employment opportunities will increase, especially for young engineers, physicists, neuroscientists and behavioral psychologists.
The exploitation of the EMBRACE results in organizations and work settings where social behavior is a main feature, may improve teamwork and the effectiveness of the many task-related goals that are achieved through human coordination and interaction.
Finally, the international mobility and dissemination of the goals and achievements of the EMBRACE project will contribute to sharing cultures and knowledge with the scientific community and to promote communication on the importance of research in biomedical engineering to the society.