Community Research and Development Information Service - CORDIS


MUSMAP addresses methodological and technological challenges in the development of multi-modal data acquisition and analysis methods for the construction of auditory-motor patterning schemes that enable the modelling of instrumental playing technique in music performance through the development and application of novel technologies to (a) automatic control of sound synthesis and (b) cognitive science research on sensory-motor interaction. The case study is violin playing: motion and sound analysis techniques will used to identify and describe auditory and motor sequences from a purposely constructed database of classical violin performances.

In particular, the research to be carried out in MUSMAP is driven by three main objectives:
O1 - To utilize multi-modal data acquisition and analysis methods to develop and validate auditory-motor pattern representation models by creating and systematically studying an extensive set of violin performance multi-modal recordings of trained musicians.
O2 - To apply auditory-motor pattern representations to design a system for automatic control of violin sound synthesis presenting a closed-loop, feed-forward architecture by which a motor sequencing component is updated in response to perceptual features of synthesized sound.
O3 - To apply auditory-motor pattern representations to propose novel, high-quality sound processing technologies through which meaningful audio perceptual attribute manipulations are driven by motor primitives, enabling auditory-motor remapping of recorded or real-time violin performances. The technologies will be presented to cognitive (neuro) science experts in an attempt to perform novel auditory feedback manipulation experiments in the context of auditory-motor interaction research from the perspective of cognitive science.

While O1 and O2 belong to the expertise of the fellow, the collaboration-driven experiments to be proposed as part of O3 represent a field-opening opportunity for the fellow to explore promising applications of advanced sound and music technologies to fundamental research in cognitive science.

Since the beginning of the project, work has been successfully devoted to four specific tasks:
1 - Development a framework for non-intrusive capture of motor control and audio signals relevant for violin performance technique
2 - Design and implementation of a bowed string physical modeling synthesis system suitable for automatic control from motor control signals
3 - Design of a sound analysis / transformation system capable of motor-driven manipulation of violin audio perceptual attributes, both off-line and real-time.
4 - Study and simulation of the relationship between bowing controls and violin sound attributes by multi-modal analysis of recorded violin performance.

Research results achieved during the reporting period include:
- Creation of a recording and data processing protocol for combined acquisition of audio and motion capture data from violin performances.
- Generation of two multi-modal databases (one with acoustic instruments and another with one electric instrument) of violin exercises for their use in developing models of auditory-motor patterning.
- Design and implementation of efficient digital models of bow-string interaction via a thermal friction model, and bridge admittances and body radiation patterns via modal analysis/synthesis. The digital sythesis models work in real-time and produce highly realistic sound
- Design of a technique to design recursive filters in parallel form as a basis for admittance and radiation modeling, and for sound transformation by low-delay equalization.
- Design of a system for off-line and real-time manipulation of violin bridge velocity signals, controlled by motor control signals; so far the system is conceptualized around bow pressing force manipulations.


Eva Martin, (Head of the Research Services)
Tel.: +34935422140


Life Sciences
Record Number: 187560 / Last updated on: 2016-08-22
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top