Skip to main content

Improving the eXperience of musicians and spectators with digital Musical Instruments using 3D interfaces and mixed-reality technologies


Art and science intersect often and in many ways. In particular, new technologies are hacked for the purpose of building musical instruments. As a result, more and more novel digital musical instruments come out of laboratories and companies and are used in public performances. However, the variety and quality of chosen interfaces, sound synthesis methods and mappings between gestures and sounds often prevent people from understanding and enjoying those instruments as much as familiar acoustic or electric instruments based on physic principles.
The purpose of this project is to improve the experience of both novices musicians and spectators with digital musical instruments, by combining work on 3D graphical musical interfaces and advanced musical techniques by the Fellow with research on mixed-reality displays and audience experience by the Host Institute.
A first phase will focus on highlighting the impact of musicians gestures and the behaviour of instruments using 3D representations spatially merged with the instruments. The effects on the experience of spectators during public performances will then be evaluated.
A second phase will explore the concept of adaptive mappings by means of graphical and musical level-of-detail and differentiated views using multi-user displays in order to ensure an enjoyable playing experience for all musicians regardless of their expertise level. A long-term user study will be conducted in order to evaluate the impact of the proposed solutions on musicians experience during the learning process.
Both phases will involve dissemination parts including workshop with artists, academic publications and workshops, demonstrations and concerts for the general public, release of software components under accessible licenses, and online guides for building the hardware components.

Call for proposal

See other projects for this call


Beacon House Queens Road
BS8 1QU Bristol
United Kingdom
Activity type
Higher or Secondary Education Establishments
EU contribution
€ 221 606,40
Administrative Contact
Coombs Julie (Mrs.)