Hybrid Machine Learning – Optimization techniques To Generate Structured Music Through Morphing And Fusion

Información del proyecto

MorpheuS

Identificador del acuerdo de subvención: 658914

DOI

10.3030/658914

Proyecto cerrado

Fecha de la firma de la CE 5 Marzo 2015

Fecha de inicio 1 Junio 2015

Fecha de finalización 31 Mayo 2017

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 183 454,80

Aportación de la UE

€ 183 454,80

Coordinado por

QUEEN MARY UNIVERSITY OF LONDON
United Kingdom

Periodic Reporting for period 1 - MorpheuS (Hybrid Machine Learning – Optimization techniques To Generate Structured Music Through Morphing And Fusion)

Período documentado: 2015-06-01 hasta 2017-05-31

Resumen del contexto y de los objetivos generales del proyecto

State-of-the-art generation/improvisation systems include the Continuator, OMax, and Mimi. These systems generate music that sounds good on a low (note-to-note) level but lacks the critical structure and direction necessary for long term coherence. In this project, we have addressed this challenge by creating a system that generates music compositions based on structural templates. Our novel approach deploys machine learning methods in an optimization context to morph existing musical pieces into new ones and to fuse disparate styles. We have developed a novel hybrid framework that combines the strength of optimization techniques with machine learned models to generated music with long-term structure such as recurrent patterns. In doing so, we have also developed a mathematical model for calculating tonal tension of music, which proved to be a useful tool in generating affective music.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

One of the main contributions of this work is the development of a multidisciplinary technique for generating musical pieces with structure and long-term coherence. The MorpheuS system implements a novel mathematical model for calculating tonal tension in music. This model is then used for generating new music with a prespecified, or learned, tension profile. The generated new (affective) music is advances the state-of-the art, as it is optimized to contain repeated patterns and themes of a template piece by using a state-of-the-art pattern detection algorithm. Musical compositions created by MorpheuS have been performed by Prof. Elaine Chew in Stanford, London, Brighton and Cambridge.

Finally, we explored the power of deep learning methods for capturing musical style. These models which, include a semantic vector space model and long-short term models that implements a novel musically inspired image representation, confirm the huge potential of deep learning in this field. The researcher also organized the first International Workshop on Deep Learning and Music and Machine Learning, joint with the International Joint Conference of Neural Networks in Anchorage, US. The presence of 50+ registered participants and speakers from both Google and Pandora, testify to the potential of deep learning techniques for music composition.

In order to facilitate the music research of music analysts and music cognition researchers, and demonstrate the developed tension model, an interactive multimodal website, called IMMA, was developed to visualise both audio and score characteristics in sync with their score and audio performance. We plan to further extend this platform to a widely used music analysis platform and score repository.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

The MorpheuS system is the first system to generate music with a specific musical tension, thus making it perfect for future implementations in computer games, film, stock music for advertising. As the generated music is one of the first to contain a long-term structure and recurring patterns in this unique way, it offers new opportunities for the digital music industry and the field of automatic composition. When generated music contains such a long term structure, it can be integrated in our daily lives and become a valid form of music composition for laymen through smartphones or online applications.

Tension of the famous Tristan chord in the Spiral Array