Periodic Reporting for period 2 - EMBRACE (tEchnology for Multimodal inter-BRain dynAmiCs invEstigation)
Reporting period: 2023-11-01 to 2025-10-31
Most social interactions involve interpersonal coordinated movements in real-time and real space, but research on complex processes determining social behavior has been limited by inadequate technology and analytical tools.
EMBRACE merged complementary expertise and resources in biomedical engineering, material science, signal processing, neuroscience and social psychology at 3 academic and 3 industrial partners from 3 EU countries (Italy, Germany and Spain) to develop:
1) a new mobile dry electrode EEG system suitable for monitoring brain activity during full body movements
2) novel body-network sensors and a multimodal alignment system for the synchronous recording of neural, cardiac, electromyographic, respiration and kinematic signals from two interacting subjects during free full body interpersonal interaction
3) a new dyadic protocol employing table tennis to exploit the implemented technological advances
4) novel analytical solutions for motion artefact removal and multi-level analysis of multimodal data.
The novel technologies address requirements of mobility, high signal quality, high temporal and spatial resolution, stable alignment of multiple signals, wearing comfort and long-term use.
The novel analytical methods permit the investigation of joint action at the neural, cognitive-behavioral, and social levels.
A dataset of 36 dyadic multimodal recordings was collected, which includes EEG, ECG, EMG, respiration and kinematic data synchronously recorded during cooperative and competitive table tennis at 2 difficulty levels (DOI:10.5281/zenodo.17471747).
To achieve its objectives, the EMBRACE consortium realized extensive intersectoral transfer of knowledge through shared research, secondments, and summer schools, promoting the research capacity and competitiveness of its partners and becoming a long-lasting EU network promoting basic and applied biomedical research.
We can conclude that the EMBRACE multimodal approach to joint action demonstrated to be effective, and that the results that will be obtained from the multilevel analysis of the collected multimodal data will contribute to a deeper understanding of the mechanisms underpinning and differentiating cooperation and competition in real world contexts.
EMBRACE merged complementary expertise and resources in biomedical engineering, material science, signal processing, neuroscience and social psychology at 3 academic and 3 industrial partners from 3 EU countries (Italy, Germany and Spain) to develop:
1) a new mobile dry electrode EEG system suitable for monitoring brain activity during full body movements
2) novel body-network sensors and a multimodal alignment system for the synchronous recording of neural, cardiac, electromyographic, respiration and kinematic signals from two interacting subjects during free full body interpersonal interaction
3) a new dyadic protocol employing table tennis to exploit the implemented technological advances
4) novel analytical solutions for motion artefact removal and multi-level analysis of multimodal data.
The novel technologies address requirements of mobility, high signal quality, high temporal and spatial resolution, stable alignment of multiple signals, wearing comfort and long-term use.
The novel analytical methods permit the investigation of joint action at the neural, cognitive-behavioral, and social levels.
A dataset of 36 dyadic multimodal recordings was collected, which includes EEG, ECG, EMG, respiration and kinematic data synchronously recorded during cooperative and competitive table tennis at 2 difficulty levels (DOI:10.5281/zenodo.17471747).
To achieve its objectives, the EMBRACE consortium realized extensive intersectoral transfer of knowledge through shared research, secondments, and summer schools, promoting the research capacity and competitiveness of its partners and becoming a long-lasting EU network promoting basic and applied biomedical research.
We can conclude that the EMBRACE multimodal approach to joint action demonstrated to be effective, and that the results that will be obtained from the multilevel analysis of the collected multimodal data will contribute to a deeper understanding of the mechanisms underpinning and differentiating cooperation and competition in real world contexts.
We implemented 178 months of international and intersectoral secondments and several dissemination and communication activities. The three EMBRACE Summer Schools succeeded in teaching technical/applicative/social topics and transferable skills not only to the EMBRACE staff members but also to researchers and professionals from all over the world. External invited speakers contributed to the Summer Schools.
The following scientific results were achieved:
1. A novel mobile EEG system with integrated electronics for the novel dry sports caps mounting novel dry sports electrodes (the Flower electrodes).
2. A novel multimodal body-network sensors and electronics for the synchronous registration of EEG, ECG and respiration effort signals.
3. An inter-device synchronization system via wired and wireless communication media for the multimodal body-network sensors and motion capture system with integrated EMG sensors.
4. Definition of a new study protocol to perform dyadic studies on non-verbal joint actions performed through free full body movements occurring face to face, in cooperative and competitive modes, and at two difficulty levels.
5. A dataset of 36 dyadic multimodal recordings including EEG, ECG, EMG, respiration and kinematic data synchronously recorded from two tennis table players (DOI: 10.5281/zenodo.17471747) during cooperative and competitive table tennis played at easy and hard levels.
6. A new method for removing motion-related artefacts from EEG recordings.
7. A set of analytical methods and tools to analyze the multimodal data at the neural, cognitive-behavioral, and social levels.
10. Preliminary results on the kinematic, cardiac and neural features of cooperation and competition during table tennis.
The following scientific results were achieved:
1. A novel mobile EEG system with integrated electronics for the novel dry sports caps mounting novel dry sports electrodes (the Flower electrodes).
2. A novel multimodal body-network sensors and electronics for the synchronous registration of EEG, ECG and respiration effort signals.
3. An inter-device synchronization system via wired and wireless communication media for the multimodal body-network sensors and motion capture system with integrated EMG sensors.
4. Definition of a new study protocol to perform dyadic studies on non-verbal joint actions performed through free full body movements occurring face to face, in cooperative and competitive modes, and at two difficulty levels.
5. A dataset of 36 dyadic multimodal recordings including EEG, ECG, EMG, respiration and kinematic data synchronously recorded from two tennis table players (DOI: 10.5281/zenodo.17471747) during cooperative and competitive table tennis played at easy and hard levels.
6. A new method for removing motion-related artefacts from EEG recordings.
7. A set of analytical methods and tools to analyze the multimodal data at the neural, cognitive-behavioral, and social levels.
10. Preliminary results on the kinematic, cardiac and neural features of cooperation and competition during table tennis.
EMBRACE draws together innovations in biomedical engineering, neuroscience, signal processing, and social psychology, going beyond the state-of-the-art in all these fields.
Our approach combines innovations in technology (a new mobile dry electrode EEG system and a novel multimodal alignment system), analytical methods (new methods for motion-related artefact removal and for multi-level analysis of multimodal data), and study paradigms (the novel table tennis paradigm) that will permit to gain unprecedented understanding of the human social brain during cooperative and competitive joint action.
Innovation in engineering regards the new EMBRACE dry sport flower electrodes that permit recording brain activity while subjects freely move with no time or space restrictions, featuring high signal quality, stability and comfort, short mounting times, long recording times, high space and time resolutions, robust and light-weight electronics, optimized skin-sensor contact, robust modular high-density caps with personalized shape and size. Furthermore, the novel multimodal alignment system offers wireless simultaneous acquisitions of EEG, EMG, ECG, respiration effort and kinematic signals from two subjects during free full body movements.
Innovation in analytical solutions regards methods for the automated correction of artefacts due to gross body movements in the neurophysiological data acquired during free full body movement joint action, and new analytical methods for the analysis of the multimodal dyadic data collected during table tennis gameplay. These analytical solutions combine hyperbrain functional connectivity, microstate analysis, Graph Theory and Network Physiology approaches that permit exploiting the richness of information contained in the collected multimodal data and investigating the mechanisms underpinning joint action at three levels of analysis: neural, cognitive-behavioral, and social.
Innovation in study design: Our new study protocol fully exploits the innovative technology developed within the project. For the first time we could investigate the complex neurophysiological processes underpinning face-to-face non-verbal joint action by using turn-based cooperative and competitive tasks that involve free full body movements. By introducing difficulty levels in the performance of the joint action task, we could also study how these processes adjust to variable external conditions.
The exploitation of the EMBRACE results will bring benefits to European industries in nano-electronics, medical electronics, and even entertainment electronics, with the potential extension of EEG technology markets to products like intelligent prostheses or brain computer interfaces. Consequently, employment opportunities will increase, especially for young engineers, physicists, neuroscientists and behavioral psychologists.
The results of the EMBRACE project can also be exploited to improve teamwork in organizations and work settings where social behavior is a main feature, and to advance the effectiveness of the many task-related goals that are achieved through human coordination and interaction.
Finally, the international mobility and dissemination of the goals and achievements of the EMBRACE project contributed to sharing cultures and knowledge with the scientific community and to promote communication on the importance of research in biomedical engineering to society.
Our approach combines innovations in technology (a new mobile dry electrode EEG system and a novel multimodal alignment system), analytical methods (new methods for motion-related artefact removal and for multi-level analysis of multimodal data), and study paradigms (the novel table tennis paradigm) that will permit to gain unprecedented understanding of the human social brain during cooperative and competitive joint action.
Innovation in engineering regards the new EMBRACE dry sport flower electrodes that permit recording brain activity while subjects freely move with no time or space restrictions, featuring high signal quality, stability and comfort, short mounting times, long recording times, high space and time resolutions, robust and light-weight electronics, optimized skin-sensor contact, robust modular high-density caps with personalized shape and size. Furthermore, the novel multimodal alignment system offers wireless simultaneous acquisitions of EEG, EMG, ECG, respiration effort and kinematic signals from two subjects during free full body movements.
Innovation in analytical solutions regards methods for the automated correction of artefacts due to gross body movements in the neurophysiological data acquired during free full body movement joint action, and new analytical methods for the analysis of the multimodal dyadic data collected during table tennis gameplay. These analytical solutions combine hyperbrain functional connectivity, microstate analysis, Graph Theory and Network Physiology approaches that permit exploiting the richness of information contained in the collected multimodal data and investigating the mechanisms underpinning joint action at three levels of analysis: neural, cognitive-behavioral, and social.
Innovation in study design: Our new study protocol fully exploits the innovative technology developed within the project. For the first time we could investigate the complex neurophysiological processes underpinning face-to-face non-verbal joint action by using turn-based cooperative and competitive tasks that involve free full body movements. By introducing difficulty levels in the performance of the joint action task, we could also study how these processes adjust to variable external conditions.
The exploitation of the EMBRACE results will bring benefits to European industries in nano-electronics, medical electronics, and even entertainment electronics, with the potential extension of EEG technology markets to products like intelligent prostheses or brain computer interfaces. Consequently, employment opportunities will increase, especially for young engineers, physicists, neuroscientists and behavioral psychologists.
The results of the EMBRACE project can also be exploited to improve teamwork in organizations and work settings where social behavior is a main feature, and to advance the effectiveness of the many task-related goals that are achieved through human coordination and interaction.
Finally, the international mobility and dissemination of the goals and achievements of the EMBRACE project contributed to sharing cultures and knowledge with the scientific community and to promote communication on the importance of research in biomedical engineering to society.