Periodic Reporting for period 2 - MUSAiC (Music at the Frontiers of Artificial Creativity and Criticism)
Período documentado: 2022-04-01 hasta 2023-09-30
Artificial intelligence (AI) is impacting a growing number of domains in ways both beneficial and detrimental. It is even showing surprising impacts in the Arts, provoking questions fundamental to philosophy, law, and engineering, not to mention practices in the Arts themselves. MUSAiC is an interdisciplinary research venture confronting questions and challenges at the frontier of the AI disruption of music. It aims to analyze, criticize and fundamentally broaden the AI transformation of three interrelated music practices: 1) listening, 2) composition and performance, and 3) analysis and criticism. For each practice, and grounded in two specific music traditions (Irish and Swedish), MUSAiC will document and critically analyze the impacts of and ethical issues surrounding AI. MUSAiC will formulate and implement the first music pedagogy for AI, the lack of which continues to result in the creation of AI systems that have only a surface knowledge of music. From this pedagogy, MUSAiC will develop new holistic methods for understanding and benchmarking AI, and improving them and their application. It will implement and test novel AI systems that dynamically adapt to specific users as “digital apprentices”, aiming to bring human-AI music partnerships to new levels of fruitfulness. The outcomes of MUSAiC will facilitate applications of AI to music in robust and responsible ways, impacting a wide variety of stakeholders. It will not only prepare music practitioners and audiences of the present (human and artificial) for new ways of listening, working, appraising, and developing the art form, but will also pave the way for analyzing, criticizing and broadening the AI transformation of the other Arts. Three overarching objectives include both humanistic and technological dimensions. The first objective is to survey the cultural impact and criticism of AI, especially applied within creative practices. The second objective is to develop music pedagogy for machines, which includes the evaluation of the resulting systems. The third objective is to develop and explore novel co-creative partnerships with AI. Within each of these are specific threads of work contributing to the domain of AI and music (and Art in general), as well as engaging different communities, from developers to practitioners to the public. The questions and challenges addressed by MUSAiC are uniquely situated at intersections of music and computer science. The project moves beyond studying existing AI technologies in music practice, to involving artists and their practices for music culture production in the development, use and analysis of new AI technologies from the bottom up. MUSAiC is humanistic in the sense that it focuses on how AI technology impacts culture and culture production, e.g. How can AI partnerships facilitate and hinder music creation? MUSAiC is scientific in the sense that it answers fundamental questions about the nature of AI technology, e.g. What does an AI perceive when listening to music? MUSAiC is technological in the sense that it addresses questions of improving AI technology and its application, e.g. How can we build an AI that listens to music, and improve its understanding of music in unique domains with limited data?
Over its first 30 months, MUSAiC has conducted work within each of the three overarching objectives. For the first objective, it has surveyed AI partnerships with a focus on impact and ethics. We have taken a close look at the relationship between authenticity and AI authorship in folk music (specifically Irish traditional music), as well as identifying problems in cross-cultural applications of ethics frameworks. MUSAiC has contributed to public-facing ventures, such as the poetic research blog, "Tunes from the Ai Frontiers" (https://tunesfromtheaifrontiers.wordpress.com) as well as submissions to the AI Song Contest and Human+Machine song contest. Another achievement in this first objective is an experimental study of listener bias within the context of Irish traditional music practice. Finally, MUSAiC has organized several important events. The first "AI Music Creativity" conference (Oct. 2020) was the kick-off event for the project, and has since run each year in different countries (Austria for 2021, Japan for 2022, and UK for 2023). In Nov. 2022, MUSAiC organized a well-attended three-day festival in Stockholm, Sweden, devoted broadly to AI and art. Within the second objective, MUSAiC has been working on teaching machines about music and evaluating their success. We have been investigating attention-based models applied to symbolic music, and deep neural networks applied to music instrument sound synthesis. MUSAiC has also organized three public challenges, each eliciting submissions from researchers around the world applying machine learning to idiosyncratic styles of music. The 2020 challenge focused on the Irish double jig, and is described in a brief video documentary (https://youtu.be/KSoSyoEx6hc). Within the third objective, MUSAiC has been researching the artificial criticism of music. Specific work has approached the analysis of music generated by models trained on Irish traditional music. The PI was also the lead guest editor of a special journal issue (published Feb. 2022) consisting of nine articles (13 submitted) on the topic of music creativity and AI, many articles of which touch upon the major themes of MUSAiC. Members of the MUSAiC team are also guest editing a special issue of another journal devoted the same topic (20 submissions currently under review).
So far in the project, at least four major contributions have been made. First, the MUSAiC project has united two international research communities: the Music Metacreation Workshop (2012-2019) and the Computational Simulation of Music Creativity conference (2016-2018). The MUSAiC project organized the first AI Music Creativity conference in 2020 bringing together these communities -- and has been organized every year since. The second major contribution of MUSAiC is its cross-cultural analysis of the values and ethics surrounding applications of AI to music. This has resulted in several conference papers (one best conference paper award), a journal article, and a book chapter. The third major contribution is our music listening study, which shows how one's bias in liking an artifact is modulated by context. This enriches the current literature on the subject exhibiting a variety of contradictory conclusions; and our further work (currently being assembled for journal submission) analyzes the power of our experiment, performs the experiment online for the "general" population, and provides even more evidence for our thesis. The fourth major contribution is the identification of a lack of formal study of the emerging "AI music" ecosystem (music generated entirely by AI and being distributed into the music ecosystem via streaming platforms like Spotify and YouTube). We propose a new field of study, "musaicology", which seeks to analyze AI music along many dimensions. This work brings together nearly 20 authors from a wide variety of disciplines (engineering, musicology, computational creativity, and media studies), and is certain to make a major impact.