Periodic Reporting for period 1 - BraSILL (Brain Correlates of Socially Interactive Language Learning)
Reporting period: 2022-10-01 to 2025-03-31
Second language (L2) learning plays a central role in enabling individuals to participate in multilingual societies, yet adults show marked variability in their success, especially in spoken communication. Crucially, most language learners aim to use the L2 in meaningful interaction, and practicing the language in social contexts has been shown to support more advanced proficiency. Despite this, the mechanisms through which social interaction facilitates L2 learning remain poorly understood.
This project set out to investigate how social, cognitive, and neurophysiological processes jointly support L2 learning, with a focus on how adults respond to social cues such as gaze direction and performance feedback during word learning. The overarching hypothesis was that socially situated learning contexts engage specific cognitive and neural mechanisms that scaffold attention, memory, and learning. To examine this, the project combined methods from cognitive neuroscience (EEG, pupillometry, eye-tracking), psycholinguistics, and virtual reality (VR) to create immersive and experimentally controlled learning environments.
The use of VR was aimed gaining precise insight into how interactive, embodied, and multimodal social processes affect learning. These insights have broad relevance, not only for L2 education, but also for areas such as rehabilitation and inclusive learning, where responsiveness to social cues can strongly influence outcomes.
The expected impact lies in advancing theoretical models of L2 learning by integrating social, cognitive, and neural perspectives, while also offering methodological innovations that bridge ecological validity and experimental control. In alignment with EU priorities on lifelong learning, digital innovation, and the integration of social sciences and humanities, the project contributes new tools and frameworks for understanding the dynamic, interactive nature of second language development.
This project set out to investigate how social, cognitive, and neurophysiological processes jointly support L2 learning, with a focus on how adults respond to social cues such as gaze direction and performance feedback during word learning. The overarching hypothesis was that socially situated learning contexts engage specific cognitive and neural mechanisms that scaffold attention, memory, and learning. To examine this, the project combined methods from cognitive neuroscience (EEG, pupillometry, eye-tracking), psycholinguistics, and virtual reality (VR) to create immersive and experimentally controlled learning environments.
The use of VR was aimed gaining precise insight into how interactive, embodied, and multimodal social processes affect learning. These insights have broad relevance, not only for L2 education, but also for areas such as rehabilitation and inclusive learning, where responsiveness to social cues can strongly influence outcomes.
The expected impact lies in advancing theoretical models of L2 learning by integrating social, cognitive, and neural perspectives, while also offering methodological innovations that bridge ecological validity and experimental control. In alignment with EU priorities on lifelong learning, digital innovation, and the integration of social sciences and humanities, the project contributes new tools and frameworks for understanding the dynamic, interactive nature of second language development.
The project resulted in four major scientific contributions that advanced our understanding of the cognitive and social mechanisms underlying adult second language (L2) learning. These include three completed empirical studies, one comprehensive theoretical review, and one ongoing application development initiative.
The first empirical study, The effects of reliable social feedback on language learning: insights from EEG and pupillometry, investigated how the reliability and social content of feedback affect word learning. Using a combination of EEG and pupillometry, participants learned novel auditory words and received feedback that varied in both accuracy and format (video-based social feedback versus symbolic static cues). Behavioral results showed that learning was more effective when feedback was reliable, regardless of its format. However, neurophysiological markers such as the stimulus-preceding negativity, late positive complex, and pupil dilation revealed differential processing of social feedback. These findings suggest that social feedback engages distinct mechanisms during both anticipation and processing. This study provides insight into how learners interpret and integrate feedback during language learning at both behavioral and neural levels.
The second empirical study, Alpha and beta desynchronization during consolidation of newly learned words, examined changes in oscillatory brain dynamics associated with the retrieval of newly learned word meanings. Participants learned new words across multiple days and performed overt and covert naming tasks during EEG recording sessions. Following successful learning, lexical retrieval was associated with a significant decrease in alpha and lower beta power, consistent with neural signatures of lexical access and memory integration. These results help characterize the temporal dynamics of how new L2 vocabulary becomes consolidated and accessible after learning.
The third empirical study, Eyes on the prize: gaze-guided learning of foreign words in social VR interactions, involved the development and implementation of a novel virtual reality environment designed to simulate naturalistic second language instruction. Participants learned foreign words from a virtual teacher whose gaze was either informative or uninformative with respect to the target object. Eye-tracking and behavioral data showed that participants learned more effectively when the teacher’s gaze reliably indicated the referent and when they followed that gaze. This result demonstrates that, like children, adults are sensitive to referential gaze cues during word learning. The study also highlights the potential of immersive VR to enable experimental manipulation of social interaction under controlled but ecologically valid conditions. Developing this platform required extensive technical work and collaboration with external VR specialists to create lifelike agents with programmable, contingent gaze behavior.
In parallel with the empirical work, the project produced a comprehensive theoretical review, Social interaction shapes and boosts second language learning: virtual reality can show us how, which synthesizes current research on socially interactive second language learning and outlines how immersive virtual reality can advance the field. Drawing on theoretical frameworks from developmental psychology, cognitive neuroscience, and second language acquisition, the review proposes that VR offers a uniquely powerful means of investigating how joint attention, feedback, co-presence, and embodied interaction support language development. By integrating findings across traditionally separate disciplines, it establishes a coherent research agenda for understanding L2 learning as a fundamentally social and dynamic process.
The second empirical study has been published in Neuroimage. The first and third empirical studies and the review paper are currently under review in high-impact journals ranked among the top in cognitive neuroscience and learning research.
Finally, the project initiated an applied collaboration with the educational technology start-up Minicoders to co-develop an interactive VR-based language learning game for adolescents. While still under development, this initiative translates core scientific insights from the project into an engaging, socially responsive learning platform. The work involved hiring additional technical personnel with expertise in game development and VR programming and contributed to the project's broader interdisciplinary and translational aims.
Together, these five contributions demonstrate scientific innovation and technical achievement. By combining psycholinguistic and neurophysiological methods with immersive virtual environments, the project provides new evidence and new tools for understanding how social interaction shapes second language learning in adults.
The first empirical study, The effects of reliable social feedback on language learning: insights from EEG and pupillometry, investigated how the reliability and social content of feedback affect word learning. Using a combination of EEG and pupillometry, participants learned novel auditory words and received feedback that varied in both accuracy and format (video-based social feedback versus symbolic static cues). Behavioral results showed that learning was more effective when feedback was reliable, regardless of its format. However, neurophysiological markers such as the stimulus-preceding negativity, late positive complex, and pupil dilation revealed differential processing of social feedback. These findings suggest that social feedback engages distinct mechanisms during both anticipation and processing. This study provides insight into how learners interpret and integrate feedback during language learning at both behavioral and neural levels.
The second empirical study, Alpha and beta desynchronization during consolidation of newly learned words, examined changes in oscillatory brain dynamics associated with the retrieval of newly learned word meanings. Participants learned new words across multiple days and performed overt and covert naming tasks during EEG recording sessions. Following successful learning, lexical retrieval was associated with a significant decrease in alpha and lower beta power, consistent with neural signatures of lexical access and memory integration. These results help characterize the temporal dynamics of how new L2 vocabulary becomes consolidated and accessible after learning.
The third empirical study, Eyes on the prize: gaze-guided learning of foreign words in social VR interactions, involved the development and implementation of a novel virtual reality environment designed to simulate naturalistic second language instruction. Participants learned foreign words from a virtual teacher whose gaze was either informative or uninformative with respect to the target object. Eye-tracking and behavioral data showed that participants learned more effectively when the teacher’s gaze reliably indicated the referent and when they followed that gaze. This result demonstrates that, like children, adults are sensitive to referential gaze cues during word learning. The study also highlights the potential of immersive VR to enable experimental manipulation of social interaction under controlled but ecologically valid conditions. Developing this platform required extensive technical work and collaboration with external VR specialists to create lifelike agents with programmable, contingent gaze behavior.
In parallel with the empirical work, the project produced a comprehensive theoretical review, Social interaction shapes and boosts second language learning: virtual reality can show us how, which synthesizes current research on socially interactive second language learning and outlines how immersive virtual reality can advance the field. Drawing on theoretical frameworks from developmental psychology, cognitive neuroscience, and second language acquisition, the review proposes that VR offers a uniquely powerful means of investigating how joint attention, feedback, co-presence, and embodied interaction support language development. By integrating findings across traditionally separate disciplines, it establishes a coherent research agenda for understanding L2 learning as a fundamentally social and dynamic process.
The second empirical study has been published in Neuroimage. The first and third empirical studies and the review paper are currently under review in high-impact journals ranked among the top in cognitive neuroscience and learning research.
Finally, the project initiated an applied collaboration with the educational technology start-up Minicoders to co-develop an interactive VR-based language learning game for adolescents. While still under development, this initiative translates core scientific insights from the project into an engaging, socially responsive learning platform. The work involved hiring additional technical personnel with expertise in game development and VR programming and contributed to the project's broader interdisciplinary and translational aims.
Together, these five contributions demonstrate scientific innovation and technical achievement. By combining psycholinguistic and neurophysiological methods with immersive virtual environments, the project provides new evidence and new tools for understanding how social interaction shapes second language learning in adults.
This project produced several results that go beyond the current state of the art in second language learning research. It provides novel empirical evidence that social cues, particularly gaze and feedback, play a measurable role in shaping the cognitive and neurophysiological mechanisms involved in adult word learning. By combining EEG, pupillometry, and eye-tracking within controlled experimental paradigms, the project advances our understanding of how attention, motivation, and memory are influenced by the social dynamics of learning.
The project also introduced new experimental methodologies that enhance the ecological validity of language learning research. The immersive virtual reality environments developed during the project allowed for precise manipulation of interactive features such as gaze behavior and social presence, while maintaining naturalistic engagement. These methodological contributions enable future research that more closely resembles real-world learning situations while preserving experimental control.
In addition, the theoretical review synthesizes insights from developmental psychology, cognitive neuroscience, second language acquisition, and human-computer interaction into a coherent research agenda. It positions socially interactive learning as a central component of successful language acquisition. This work encourages a shift in how both experimental designs and theoretical models are constructed in the field.
The project's findings have broader implications for language education and rehabilitation. A better understanding of how social engagement supports learning can inform the design of more effective and personalized instructional strategies, including those aimed at individuals with specific learning or communication needs. For example, multimodal feedback and joint attention cues may improve intervention protocols for people with language impairments or learning differences. The virtual environments developed here also provide a framework for future studies of engagement and co-presence in neurodiverse populations.
To ensure further development and impact, several priorities have been identified. These include sustained interdisciplinary collaboration with experts in virtual reality and clinical research, further validation of immersive learning paradigms across languages and learner profiles, and funding for follow-up studies exploring the long-term outcomes of socially embedded language learning. While the project was not designed for commercial application, its results are likely to inform future translational research and inspire innovation in educational technologies.
In summary, the project offers theoretical and methodological advances that significantly deepen our understanding of how social interaction supports second language learning. It lays the groundwork for future research at the intersection of neuroscience, language sciences, and immersive, interactive technology.
The project also introduced new experimental methodologies that enhance the ecological validity of language learning research. The immersive virtual reality environments developed during the project allowed for precise manipulation of interactive features such as gaze behavior and social presence, while maintaining naturalistic engagement. These methodological contributions enable future research that more closely resembles real-world learning situations while preserving experimental control.
In addition, the theoretical review synthesizes insights from developmental psychology, cognitive neuroscience, second language acquisition, and human-computer interaction into a coherent research agenda. It positions socially interactive learning as a central component of successful language acquisition. This work encourages a shift in how both experimental designs and theoretical models are constructed in the field.
The project's findings have broader implications for language education and rehabilitation. A better understanding of how social engagement supports learning can inform the design of more effective and personalized instructional strategies, including those aimed at individuals with specific learning or communication needs. For example, multimodal feedback and joint attention cues may improve intervention protocols for people with language impairments or learning differences. The virtual environments developed here also provide a framework for future studies of engagement and co-presence in neurodiverse populations.
To ensure further development and impact, several priorities have been identified. These include sustained interdisciplinary collaboration with experts in virtual reality and clinical research, further validation of immersive learning paradigms across languages and learner profiles, and funding for follow-up studies exploring the long-term outcomes of socially embedded language learning. While the project was not designed for commercial application, its results are likely to inform future translational research and inspire innovation in educational technologies.
In summary, the project offers theoretical and methodological advances that significantly deepen our understanding of how social interaction supports second language learning. It lays the groundwork for future research at the intersection of neuroscience, language sciences, and immersive, interactive technology.