Periodic Reporting for period 1 - Lullabyte (Unravelling the Effects of Music on Sleep through Musicology, Neuroscience, Psychology and Computer Science)
Reporting period: 2022-11-01 to 2024-10-31
Music exerts strong effects on the human brain, as evidenced by both subjective emotional reactions and overt changes in neurophysiology: lullabies are known in all cultures and times as an effective sleep aid for children and adults. However, there is still little scientific understanding how music and sounds affect the brain before and during sleep, and which musical or sonic qualities are particularly effective from both a subjective and an objective perspective. It hence requires a highly interdisciplinary approach that bridges the knowledge of musicology focusing on subjective music perception, cultural practice and historical contexts, and empirical neuroscience researching on measurable means of sound perception and sleep.
In Lullabyte, a team of musicologists, neuroscientists, cognitive scientists, psychologists and computer scientists aim to elucidate the neural mechanisms underlying the sleep-inducing effects of music. For this, we analyse musical features and their effect on sleep quality by leveraging controlled laboratory studies, large sleep data sets from home recordings, and machine learning and data science strategies as well as asking for subjective impressions, cultural connections and expectation concerning music for sleep to elucidate which specific musical features affect sleep physiology. Within and based on this highly multidimensional and interdisciplinary research, we organise a comprehensive research and training program for the Lullabyte network on the intersection of sleep research, musicology and computer science.
The doctoral network Lullabyte trains ten Doctoral Candidates in this radically interdisciplinary research field of music and sleep, in which they investigate the effects of music and sound on the brain’s transition from wakefulness to sleep and during sleep. The research projects hereby range from exploring neurophysiological details of auditory processing in the thalamico-cortical system over changes in sleep structure induced by different kinds of music, to psychological and musicological analyses, as well as developing machine learning strategies to analyze EEG and algorithmically generate novel, neuroscience-deduced music in real-time.
In Lullabyte, a team of musicologists, neuroscientists, cognitive scientists, psychologists and computer scientists aim to elucidate the neural mechanisms underlying the sleep-inducing effects of music. For this, we analyse musical features and their effect on sleep quality by leveraging controlled laboratory studies, large sleep data sets from home recordings, and machine learning and data science strategies as well as asking for subjective impressions, cultural connections and expectation concerning music for sleep to elucidate which specific musical features affect sleep physiology. Within and based on this highly multidimensional and interdisciplinary research, we organise a comprehensive research and training program for the Lullabyte network on the intersection of sleep research, musicology and computer science.
The doctoral network Lullabyte trains ten Doctoral Candidates in this radically interdisciplinary research field of music and sleep, in which they investigate the effects of music and sound on the brain’s transition from wakefulness to sleep and during sleep. The research projects hereby range from exploring neurophysiological details of auditory processing in the thalamico-cortical system over changes in sleep structure induced by different kinds of music, to psychological and musicological analyses, as well as developing machine learning strategies to analyze EEG and algorithmically generate novel, neuroscience-deduced music in real-time.
During the first 24 months (November 2022 to October 2024), Lullabyte has been thoroughly set-up, selected and onboarded all ten Doctoral Candidates, and was able to organize two training camps, the first one from 16.-20. September 2023 (with a welcome night on 15th) at RADBOUNUMC in Nijmegen, NL, the second camp from June 29 to July 6th 2024 (with welcome night on June 28 and night-long concert from July 6th to 7th) at UPF in Barcelona, ESP.
The aim of the first training camp provided basic introductions to the different disciplines, and kick-off the common DC project. Activities included general introductions to Digital Tools (USTUTT) and music (TUD) as well as a disciplinary focus on Neuroscience through talks on Sleep research (ICM), workshops to Sleep Analysis Methods (Frederik Weber and Leonore Bovy, RADBOUNDUMC), a demo and presentation of our industry partner Mentalab (Arenar had to cancel due to sickness), and a public Keynote lecture by our extended supervisory board member Prof. Dr. Gina Poe on sleep reseach in neuroscience. In addition, the camp provided a lot of time for team building among the DCs to really form one cohort of Lullabyte DCs, including a first working phase for the common PhD project, and an Artistic kick-off session.
As this camp was intended to focus also on onboarding of the then 9 DCs into Lullabyte, just a few presentations were open to external audiences such as the Keynote and the industry partner’s presentation.
The second camp, where we could welcome all PIs and all DCs on site, was targeted on music and sonification, including lectures by UPF on music AI as well as MIR, UNIFR on Sleep research, Endel on research in context of a start-up, and CNRS on working between Cognitive Science and Computer Science, and USTUTT on AI. In addition, two workshops took place, one with composer-in-residency Natasha Barrett on 3D audio and with KTH on sonification. Further guest speakers were Felix Pastor (UPF) on music theory and keynote speaker Prof. Dr. Gustavo Deco (UPF) on Neuroscience. Our industry partner, Starlab’s Aureli Soria-Frisch, gave an opening lecture. The Lullabyte PhDs were also able to work together on their common PhD project, explored the first poster session with the state of the art of all research projects, and enjoyed an exclusive reception for informal talks with the Lullabyte PIs.
All lectures as well as the PhD’s poster presentation were open to other PhD students who could be registered for the camp. As special public events, the second camp also hosted two artistic interventions: a UPF demo of the Reactable, and a sleep at PHONOS, including a 2,5h piece by Natasha Barrett and an improv session throughout the night with Barcelona based musicians A. Faraldo, H. Lizarazu and A. Reviriego.
Furthermore, the first conference where Lullabyte can serve as main collaborator, SoniHED’25 is upcoming on January 29, 2025. At SoniHED, which is hosted at and co-organized by KTH, a collaborative paper written by the 10 DCs was accepted under peer review and will be presented at the conference; in addition, three Lullabyte PIs are invited as keynote speakers.
On the organizational level, the project has completed 9 of 17 the planned milestones until October 2024, additional 7 milestones have been completed to date; 16 of the 17 deliverables due during the first 24 months, one deliverable was postponed to July 2025.
All DCs have successfully started into their research projects and we can observe a vivid collaboration among the DCs that goes beyond the scheduled secondments, which also have started in their various combinations.
The aim of the first training camp provided basic introductions to the different disciplines, and kick-off the common DC project. Activities included general introductions to Digital Tools (USTUTT) and music (TUD) as well as a disciplinary focus on Neuroscience through talks on Sleep research (ICM), workshops to Sleep Analysis Methods (Frederik Weber and Leonore Bovy, RADBOUNDUMC), a demo and presentation of our industry partner Mentalab (Arenar had to cancel due to sickness), and a public Keynote lecture by our extended supervisory board member Prof. Dr. Gina Poe on sleep reseach in neuroscience. In addition, the camp provided a lot of time for team building among the DCs to really form one cohort of Lullabyte DCs, including a first working phase for the common PhD project, and an Artistic kick-off session.
As this camp was intended to focus also on onboarding of the then 9 DCs into Lullabyte, just a few presentations were open to external audiences such as the Keynote and the industry partner’s presentation.
The second camp, where we could welcome all PIs and all DCs on site, was targeted on music and sonification, including lectures by UPF on music AI as well as MIR, UNIFR on Sleep research, Endel on research in context of a start-up, and CNRS on working between Cognitive Science and Computer Science, and USTUTT on AI. In addition, two workshops took place, one with composer-in-residency Natasha Barrett on 3D audio and with KTH on sonification. Further guest speakers were Felix Pastor (UPF) on music theory and keynote speaker Prof. Dr. Gustavo Deco (UPF) on Neuroscience. Our industry partner, Starlab’s Aureli Soria-Frisch, gave an opening lecture. The Lullabyte PhDs were also able to work together on their common PhD project, explored the first poster session with the state of the art of all research projects, and enjoyed an exclusive reception for informal talks with the Lullabyte PIs.
All lectures as well as the PhD’s poster presentation were open to other PhD students who could be registered for the camp. As special public events, the second camp also hosted two artistic interventions: a UPF demo of the Reactable, and a sleep at PHONOS, including a 2,5h piece by Natasha Barrett and an improv session throughout the night with Barcelona based musicians A. Faraldo, H. Lizarazu and A. Reviriego.
Furthermore, the first conference where Lullabyte can serve as main collaborator, SoniHED’25 is upcoming on January 29, 2025. At SoniHED, which is hosted at and co-organized by KTH, a collaborative paper written by the 10 DCs was accepted under peer review and will be presented at the conference; in addition, three Lullabyte PIs are invited as keynote speakers.
On the organizational level, the project has completed 9 of 17 the planned milestones until October 2024, additional 7 milestones have been completed to date; 16 of the 17 deliverables due during the first 24 months, one deliverable was postponed to July 2025.
All DCs have successfully started into their research projects and we can observe a vivid collaboration among the DCs that goes beyond the scheduled secondments, which also have started in their various combinations.
Since the start of Lullabyte, we could reach a lot of media attention, including coverage from international outlets such as Neuroscience News and local, regional and national media (TV reports, numerous radio interviews, and frequent print media mentions), being particularly interested the innovative blend of science and art, and highlighting the novel approach to exploring music’s impact on sleep.
Lullabyte is achieving a tangible impact across academic, social, and industrial landscapes. The network’s press coverage reflects its success which significantly enhances the project’s visibility.
A major strength of Lullabyte is to provide a bridge to the artistic sector. Music for sleep has become a rising market also for composers, and events such as the Dresden and Barcelona sleep concerts have garnered widespread attention, demonstrating the network’s ability to merge cutting-edge research with public engagement.
This opens up potential for various new collaborations on different fields that go beyond what we initially planned, e.g. a collaboration with the German National Music Council to foster music as a cultural good which has most recently started, but also fostering ideas and interests of some of the DCs to work towards exploitable products for a broader public market in the field of music/sound and sleep/relaxation and health.
Lullabyte is achieving a tangible impact across academic, social, and industrial landscapes. The network’s press coverage reflects its success which significantly enhances the project’s visibility.
A major strength of Lullabyte is to provide a bridge to the artistic sector. Music for sleep has become a rising market also for composers, and events such as the Dresden and Barcelona sleep concerts have garnered widespread attention, demonstrating the network’s ability to merge cutting-edge research with public engagement.
This opens up potential for various new collaborations on different fields that go beyond what we initially planned, e.g. a collaboration with the German National Music Council to foster music as a cultural good which has most recently started, but also fostering ideas and interests of some of the DCs to work towards exploitable products for a broader public market in the field of music/sound and sleep/relaxation and health.