Performance Precision Annotator (PPA): AI-Assisted Annotation and Analysis of Music Performance

Informations projet

PPA

N° de convention de subvention: 101106400

DOI

10.3030/101106400

Projet clôturé

Date de signature de la CE 30 Mars 2023

Date de début 1 Juillet 2023

Date de fin 31 Juillet 2024

Financé au titre de

• Marie Skłodowska-Curie Actions (MSCA)

Coût total

Aucune donnée

Contribution de l’UE

€ 99 450,52

Coordonné par

UNIVERSITAT FUR MUSIK UND DARSTELLENDE KUNST WIEN
Austria

Periodic Reporting for period 1 - PPA (Performance Precision Annotator (PPA): AI-Assisted Annotation and Analysis of Music Performance)

Période du rapport: 2023-07-01 au 2024-07-31

Résumé du contexte et des objectifs généraux du projet

Music scholars doing performance analysis are interested in examining performance characteristics such as style and expressiveness. Typically, they collect multiple performance recordings of the same piece and compare their nuances. For example, they will often compare performance recordings of the same piece from different decades to discover historical trends; if the performances were by the same artist, then they could use these recordings to study this artist’s stylistic changes over time.

Currently, music scholars often conduct such research by manually annotating musical elements in performance recordings, such as downbeats, pitch, and note onset times. However, such manual annotation approach can be tedious and inefficient. More importantly, existing tools do not incorporate musical score information, often a vital element when analyzing performances. This project aims to create a software tool to facilitate music performance analysis. Provided with a digital musical score and a performance of that score, the software automatically generates meaningful AI-driven annotations about note onsets and tempo, fully integrated with the score. Beyond the software, this project also gathers evidence of the usefulness of such tools.

In conclusion, from our user study, our developed tool demonstrated meaningful value for research in music performance analysis. The user study also provided important insights for future refinement of such tools.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

We developed and published an open-source software tool designed to assist music performance analysis. This tool has an easy-to-use graphical user interface displaying and visualizing the score, the audio, and the annotations in intuitive ways. The automatic annotation functionality is accomplished by an AI component in the software that runs an audio-to-score alignment algorithm. Given a digital musical score and a recording thereof, the algorithm automatically finds the onset time of each note in the score. Based on these onset times and the score information, the software calculates the local tempo played at each note and displays an intuitive tempo curve to show tempo fluctuations throughout the performance. Moreover, for any misaligned notes, the software allows users to correct them intuitively by dragging the onset indicators, with the assistance of highlighted notes in the score and text labels representing the position of selected notes. Users can then export these validated onsets and tempo annotations to an Excel file for further analyses. These annotations are valuable to performance-related research as timing is an essential aspect in music expressiveness.

We also conducted a user study interviewing 15 prospective users. The feedback was broadly positive, confirming this tool’s value for research in music performance analysis, and also provided important insights for future refinement of such tools.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

This project mainly results in an innovative software tool that assists music performance analysis, as well as valuable feedback and insights on further development of such digital tools.

the user interface of the software application