Periodic Reporting for period 2 - REPERTORIUM (Researching and Encouraging the Promulgation of European Repertory through Technologies Operating on Records Interrelated Utilising Machines)
Période du rapport: 2024-03-01 au 2025-12-31
The REPERTORIUM project harnessed artificial intelligence and digital technologies to transform the accessibility, understanding, and dissemination of European classical and medieval music heritage. The project developed a suite of AI tools to digitise ancient and classical manuscripts, preserve European musical heritage, and create state-of-the-art sound processing technologies, including immersive audio and complex stem separation. Vast collections of music manuscripts held in monasteries, libraries, and archives remain underutilised, locked in physical formats that preclude broad public engagement and scholarly exploration. REPERTORIUM set out to digitise, catalogue, and create comprehensive databases of European art-music works, starting with the Solesmes archive of Gregorian chant.
The project's goals included developing advanced AI tools for accurate digitisation of handwritten scores in both classical and medieval neumatic notations, and deploying innovative Sound Source Separation and Spatial Audio technologies tailored for classical music. These technologies aimed to unlock new possibilities for music education, research, and entertainment through immersive streaming experiences and interactive digital concerts.
REPERTORIUM placed strong emphasis on societal and economic impact. By making Europe's musical heritage more accessible online, the project supported cultural preservation and educational initiatives. The integration of spatial audio and immersive concert experiences sought to rejuvenate the appeal of classical music and foster new audiences. The involvement of social sciences and humanities ensured that technological innovations were grounded in musicology, historiography, and cultural studies, making outcomes relevant and meaningful to stakeholders ranging from academics and musicians to the general public.
The project's goals included developing advanced AI tools for accurate digitisation of handwritten scores in both classical and medieval neumatic notations, and deploying innovative Sound Source Separation and Spatial Audio technologies tailored for classical music. These technologies aimed to unlock new possibilities for music education, research, and entertainment through immersive streaming experiences and interactive digital concerts.
REPERTORIUM placed strong emphasis on societal and economic impact. By making Europe's musical heritage more accessible online, the project supported cultural preservation and educational initiatives. The integration of spatial audio and immersive concert experiences sought to rejuvenate the appeal of classical music and foster new audiences. The involvement of social sciences and humanities ensured that technological innovations were grounded in musicology, historiography, and cultural studies, making outcomes relevant and meaningful to stakeholders ranging from academics and musicians to the general public.
REPERTORIUM combined artificial intelligence with advanced spatial audio technologies to transform how European musical heritage is accessed, studied, and experienced. Its ambition was not only to digitise the past, but to reactivate it, bridging medieval and classical sources with contemporary digital innovation.
In manuscript digitisation, the project processed more than 350,000 images from historical collections. Nearly 1,000 medieval manuscript pages and over 1,300 eighteenth-century opera arias were manually annotated to create high-quality training material for AI systems. Optical Music Recognition (OMR) tools achieved 92.47% accuracy for musical note recognition, while Music Information Retrieval (MIR) tools were trained and validated on real historical sources. The resulting chant indexing system reached approximately 80% accuracy and was integrated into a fully operational musicological workspace available for public reuse. In total, 1,250 manuscripts were catalogued, more than 156,000 chants from 138 manuscripts were indexed, and 159 new Cantus IDs were created for previously unrecorded chant variants, establishing a sustainable and interoperable European research infrastructure connected to major international repositories.
In parallel, REPERTORIUM advanced the state of the art in orchestral Sound Source Separation under realistic recording conditions. The project demonstrated reliable section-level separation and effective close-microphone debleeding, integrated into a browser-based immersive streaming platform enabling interactive rehearsal, "minus-one" performance modes, and spatial remixing. Two Sound Field Reconstruction algorithms achieved an "Excellent" rating in subjective MUSHRA evaluations.
The immersive platform represented a major technological milestone. Designed as a scalable, web-native system, it combined sound-field reconstruction, Higher-Order Ambisonics rendering, 3DoF and 6DoF interaction, multi-camera 360° video, and AI-driven source manipulation within a single environment. Public demonstrations with the London Philharmonia, the Brandenburg Symphony Orchestra, and the Lithuanian National Philharmonic confirmed both technical maturity and artistic relevance. Six promotional concerts were held across Europe, and the project exceeded its dissemination targets, producing 25 scientific publications and reaching audiences through 31 conferences and 12 workshops.
Beyond technological innovation, the project generated tangible cultural impact. Recovered Gregorian chant repertoire was brought back to life through live performance and commercial recording, while previously unpublished eighteenth-century operatic works were edited and reintroduced into concert programmes. Clear exploitation pathways were defined for immersive streaming services and educational minus-one tools, with strong potential for scale-up through European innovation funding.
In manuscript digitisation, the project processed more than 350,000 images from historical collections. Nearly 1,000 medieval manuscript pages and over 1,300 eighteenth-century opera arias were manually annotated to create high-quality training material for AI systems. Optical Music Recognition (OMR) tools achieved 92.47% accuracy for musical note recognition, while Music Information Retrieval (MIR) tools were trained and validated on real historical sources. The resulting chant indexing system reached approximately 80% accuracy and was integrated into a fully operational musicological workspace available for public reuse. In total, 1,250 manuscripts were catalogued, more than 156,000 chants from 138 manuscripts were indexed, and 159 new Cantus IDs were created for previously unrecorded chant variants, establishing a sustainable and interoperable European research infrastructure connected to major international repositories.
In parallel, REPERTORIUM advanced the state of the art in orchestral Sound Source Separation under realistic recording conditions. The project demonstrated reliable section-level separation and effective close-microphone debleeding, integrated into a browser-based immersive streaming platform enabling interactive rehearsal, "minus-one" performance modes, and spatial remixing. Two Sound Field Reconstruction algorithms achieved an "Excellent" rating in subjective MUSHRA evaluations.
The immersive platform represented a major technological milestone. Designed as a scalable, web-native system, it combined sound-field reconstruction, Higher-Order Ambisonics rendering, 3DoF and 6DoF interaction, multi-camera 360° video, and AI-driven source manipulation within a single environment. Public demonstrations with the London Philharmonia, the Brandenburg Symphony Orchestra, and the Lithuanian National Philharmonic confirmed both technical maturity and artistic relevance. Six promotional concerts were held across Europe, and the project exceeded its dissemination targets, producing 25 scientific publications and reaching audiences through 31 conferences and 12 workshops.
Beyond technological innovation, the project generated tangible cultural impact. Recovered Gregorian chant repertoire was brought back to life through live performance and commercial recording, while previously unpublished eighteenth-century operatic works were edited and reintroduced into concert programmes. Clear exploitation pathways were defined for immersive streaming services and educational minus-one tools, with strong potential for scale-up through European innovation funding.
REPERTORIUM moved decisively beyond the state of the art by advancing AI-based OMR and MIR for medieval and classical manuscripts, enabling large-scale interoperable chant indexing, developing orchestral Sound Source Separation under realistic recording conditions, delivering web-based 3DoF/6DoF immersive audio streaming, and integrating AI-driven separation into immersive concert experiences.
The project established the first scalable AI-driven workflow for medieval chant indexing, combining digitisation, structured encoding, and MIR into a coherent pipeline that transformed fragmented archival collections into FAIR-compliant, interoperable cultural data infrastructures. In orchestral SSS, it clarified realistic performance limits while delivering validated production workflows suitable for immersive and educational use. The fully browser-based immersive platform demonstrated that spatial audio streaming and AI-driven source manipulation can be deployed without proprietary applications, lowering entry barriers for orchestras and cultural institutions. Moving beyond research prototypes, REPERTORIUM defined concrete exploitation pathways for its technologies while ensuring continued open access to cultural heritage data.
The project established the first scalable AI-driven workflow for medieval chant indexing, combining digitisation, structured encoding, and MIR into a coherent pipeline that transformed fragmented archival collections into FAIR-compliant, interoperable cultural data infrastructures. In orchestral SSS, it clarified realistic performance limits while delivering validated production workflows suitable for immersive and educational use. The fully browser-based immersive platform demonstrated that spatial audio streaming and AI-driven source manipulation can be deployed without proprietary applications, lowering entry barriers for orchestras and cultural institutions. Moving beyond research prototypes, REPERTORIUM defined concrete exploitation pathways for its technologies while ensuring continued open access to cultural heritage data.