The COBRA network aimed to train researchers to explore the linguistic, cognitive, and brain mechanisms underlying conversation in both human-human and human-machine interactions. The project focused on mechanisms like interactive alignment and prediction, which facilitate fluent conversations. This interdisciplinary project involved 11 academic research centers and 4 industry partners, offering PhD students training in neuroscience, linguistics, and computational modeling. The core of the project was centered around 9 Work Packages (WP):
WP1: Alignment and Prediction in Conversation
This work package focused on understanding how speakers align their mental states and predict their partner’s responses during conversation. Experiments showed asymmetrical adjustments in speech perception and production and revealed how neural alignment corresponds to conversational alignment. Key findings include:
- Neural alignment is enhanced by prediction mechanisms during interaction.
- Phonetic and prosodic entrainment occurs even in human-robot interactions.
WP2: Linguistic Features of Alignment and Prediction
This WP explored how specific linguistic features (e.g. prosody, discourse markers) impact alignment and prediction. Key insights include:
- Discourse markers significantly influence conversation structure and predict shifts in topic.
- Alignment at the discourse level affects conversational fluency, especially in task-oriented dialogues.
WP3: Methods, Tools, and Databases
Researchers developed tools for analyzing conversational interactions in dual-speaker settings. Several multilingual datasets were created, and tools like the DeepCobra model improved the detection of speech convergence and alignment. Notable results include:
- Data collection tools for behavioral and EEG studies in conversational interactions.
- Innovative approaches for speech synthesis and turn-taking cues.
WP4: Improving Human-Machine Interaction
This WP applied alignment and prediction mechanisms to artificial agents, aiming to improve conversational AI systems. Findings include:
- Human interlocutors align similarly with both anthropomorphic and machine-like robots.
- Backchannels (e.g. “uh-huh”) are critical for natural conversation in AI systems.
WP5: Training Program
The training events provided ESRs with interdisciplinary knowledge and hands-on experience in cutting-edge research methodologies. The events, including training events, workshops and hackathons, fostered collaborative learning between academia and industry.
WP6: Dissemination and Exploitation
COBRA effectively disseminated its findings through scientific publications, workshops, participation in international conferences, and divulgate dissemination through blogs and science events for the general public. Furthermore, a digital infrastructure for both internal and external communication which comprises the Cobra website was set up, in addition to a Zenodo Community and a Twitter account.
Notable activities include:
- Many key publications in international, peer-reviewed journals (and many more work currently in preparation for publication)
- An international and interdisciplinary workshop on conversational (mis)alignment held in Aix-en-Provence including world leaders on the topics of alignment and conversation.
- Participation in multiple international conferences where PhD students presented their research.
Finally, WP7 dealt with COBRA's management strategy and procedures, and WP8 and WP9 related to ethics requirements and open research data (for which an open data plan was set up and is hosted at: amp.opidor.tr) respectively.
