Periodic Reporting for period 1 - BioMusic (BioMusical Instrument)
Okres sprawozdawczy: 2018-05-01 do 2019-10-31
The BioMusical Instrument project created a product prototype of a wearable digital musical instrument based on biosignals from the performer’s muscles. It allows musicians to perform electronic sounds from bodily gestures. Muscle tension is sensed by innovative new electromyogram sensing hardware packaged in an ergonomic housing and coupled to wireless communication. Sophisticated machine learning methods developed in the ERC MetaGesture Music and H2020 Rapid-Mix projects track musician gesture and create meaningful relationships with computer-based synthesized sound. The BioMusical Instrument enables “hands-free” performance of electronic music. The visualization of body states makes it a useful training tool in music pedagogy, and the sonification of physiological signals allow the instrument to be used in other domains. The BioMusical Instrument is the first product to combine EMG sensing, machine learning, and advanced audio signal processing. With partners in hardware development, the music technology industry, and the culture sector, we have benchmarked and evaluateed the product prototype. The project has resulted in: 1. a technique for using the EMG to design arm gestures to trace spectromorphologies of sound, 2. a multi-modal, multi-dimensional signal feature mapping technique to associate multiple dimensions of muscle data from live gesture to audio metadata in the target sound, 3. a hardware prototype produced in 20 examples of an advanced EMG system for artistic performance. After the end of project, this has been used by artists Alexandra Murray Leslie and Melissa Logan in a major museum exhibition in Susch Switzerland. The techniques from 1., 2., and 3. were used in musical concert performances by the PI at the State Musical Instrument Museum in Berlin, the CNRS in Marseille France, and concerts at the Vancouver New Music Festival, in New York, and Stockholm.