Technology Enhanced Learning of Musical Instrument Performance

Informations projet

TELMI

N° de convention de subvention: 688269

DOI

10.3030/688269

Projet clôturé

Date de signature de la CE 10 Novembre 2015

Date de début 1 Février 2016

Date de fin 31 Janvier 2019

Financé au titre de

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Coût total

€ 2 617 425,00

Contribution de l’UE

€ 2 617 425,00

Coordonné par

UNIVERSIDAD POMPEU FABRA
Spain

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Demonstrators, pilots, prototypes (8)

First version of TELMI System Platform
Second version of the data acquisition systems prototypes

Based on user feedback about usability and practical concerns during the recordings of the reference database as well as the student progress following, a second version of improved prototypes will be delivered.

RCM Event
Final version of TELMI System Platform
Final platforms of the data acquisition systems

The final prototypes of the measuring systems combining different modalities and fully integrated with the database, user interface and feedback systems.

First working prototypes of the measuring systems

The first working prototypes of the measuring system will be available to start recording violin masters for the reference database. This means that the database system from WP6-Integration should be available as well. Includes systems based on EMF sensors, IR Cameras, and Audio.

Public Database

This deliverable will contain the details about the setup of a public multimodal database of musical performances available for consultation and research. This multimodal recordings‘ database will allow to browse the multimodal recordings using a powerful html5 web based visualizer as well as allow the users to download the recordings for their research analysis. This database will include all data gathered during the recordings done during the whole project.

Final public event

Websites, patent fillings, videos etc. (2)

Start-up public event
Website of the Project

Website of the Project (first public version Month 6 and regular updates)

Open Research Data Pilot (1)

Data Management Plan

Documents, reports (2)

Final feature definition and computation algorithms
User-evaluations of each technology in each scenario completed

Publications

Peer reviewed articles (12)

Machine learning and music generation

Auteurs: José M. Iñesta, Darrell Conklin, Rafael Ramírez
Publié dans: Journal of Mathematics and Music, Numéro 10/2, 2016, Page(s) 87-91, ISSN 1745-9737
Éditeur: Taylor and Francis Ltd.
DOI: 10.1080/17459737.2016.1216369

EEG-Based Analysis of the Emotional Effect of Music Therapy on Palliative Care Cancer Patients

Auteurs: Rafael Ramirez, Josep Planas, Nuria Escude, Jordi Mercade, Cristina Farriols
Publié dans: Frontiers in Psychology, Numéro 9, 2018, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2018.00254

Seeking reproducibility: Assessing a multimodal study of the testing effect

Auteurs: M. Beardsley, D. Hernández-Leo, R. Ramirez-Melendez
Publié dans: Journal of Computer Assisted Learning, Numéro 34/4, 2018, Page(s) 378-386, ISSN 0266-4909
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/jcal.12265

Mapping between dynamic markings and performed loudness: a machine learning approach

Auteurs: Katerina Kosta, Rafael Ramírez, Oscar F. Bandtlow, Elaine Chew
Publié dans: Journal of Mathematics and Music, Numéro 10/2, 2016, Page(s) 149-172, ISSN 1745-9737
Éditeur: Taylor and Francis Ltd.
DOI: 10.1080/17459737.2016.1193237

Evaluation of a Sound Quality Visual Feedback System for Bow Learning Technique in Violin Beginners: An EEG Study

Auteurs: Angel David Blanco, Rafael Ramirez
Publié dans: Frontiers in Psychology, Numéro 10, 2019, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2019.00165

Automatic Assessment of Tone Quality in Violin Music Performance

Auteurs: Sergio Giraldo, George Waddell, Ignasi Nou, Ariadna Ortega, Oscar Mayor, Alfonso Perez, Aaron Williamon, Rafael Ramirez
Publié dans: Frontiers in Psychology, Numéro 10, 2019, Page(s) 334, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2019.00334

Bowing Gestures Classification in Violin Performance: A Machine Learning Approach

Auteurs: David Dalmazzo, Rafael Ramírez
Publié dans: Frontiers in Psychology, Numéro 10, 2019, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2019.00344

Analysis of Movement Quality in Full-Body Physical Activities

Auteurs: Radoslaw Niewiadomski, Ksenia Kolykhalova, Stefano Piana, Paolo Alborno, Gualtiero Volpe, Antonio Camurri
Publié dans: ACM Transactions on Interactive Intelligent Systems, Numéro 9/1, 2019, Page(s) 1-20, ISSN 2160-6455
Éditeur: Association for Computing Machinery (ACM)
DOI: 10.1145/3132369

The EyeHarp: A Gaze-Controlled Digital Musical Instrument

Auteurs: Zacharias Vamvakousis, Rafael Ramirez
Publié dans: Frontiers in Psychology, Numéro 7, 2016, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2016.00906

Enriched Multimodal Representations of Music Performances: Online Access and Visualization

Auteurs: Esteban Maestre, Panagiotis Papiotis, Marco Marchini, Quim Llimona, Oscar Mayor, Alfonso Perez, Marcelo M. Wanderley
Publié dans: IEEE MultiMedia, Numéro 24/1, 2017, Page(s) 24-34, ISSN 1070-986X
Éditeur: Institute of Electrical and Electronics Engineers
DOI: 10.1109/MMUL.2017.3

A Machine Learning Approach to Discover Rules for Expressive Performance Actions in Jazz Guitar Music

Auteurs: Sergio I. Giraldo, Rafael Ramirez
Publié dans: Frontiers in Psychology, Numéro 7, 2016, ISSN 1664-1078
Éditeur: Frontiers Research Foundation
DOI: 10.3389/fpsyg.2016.01965

A machine learning approach to ornamentation modeling and synthesis in jazz guitar

Auteurs: Sergio Giraldo, Rafael Ramírez
Publié dans: Journal of Mathematics and Music, Numéro 10/2, 2016, Page(s) 107-126, ISSN 1745-9737
Éditeur: Taylor and Francis Ltd.
DOI: 10.1080/17459737.2016.1207814

Conference proceedings (27)

A computational approach for measuring performance quality in violin tones

Auteurs: Sergio Giraldo; Rafael Ramirez; George Waddell; Aaron Williamon
Publié dans: Abstracts of the International Symposium in Performance Science (ISPS 2017), Reijavik, Iceland., Numéro 4, 2017
Éditeur: ISPS
DOI: 10.5281/zenodo.2579194

A realtime feedback learning tool to visualize sound quality in violin performances

Auteurs: Sergio Giraldo; Rafael Ramirez; George Waddell; Aaron Williamon
Publié dans: Numéro 4, 2017
Éditeur: MML
DOI: 10.5281/zenodo.2580053

Computational modelling of timbre dimensions for automatic violin tone quality assessment

Auteurs: Sergio Giraldo; Rafael Ramirez; George Waddell; Aaron Williamon
Publié dans: Proceedings of the 2018 TIMBRE is a many splendored thing Conference, Montreal, Canada., Numéro 2, 2018
Éditeur: McGill
DOI: 10.5281/zenodo.2579163

Enhancing Music Learning with Smart Technologies

Auteurs: Rafael Ramirez, Alfonso Perez, George Waddell, Aaron Williamon, Corrado Canepa, Simone Ghisio, Ksenia Kolykhalova, Maurizio Mancini, Erica Volta, Gualtiero Volpe, Sergio Giraldo, Oscar Mayor
Publié dans: Proceedings of the 5th International Conference on Movement and Computing - MOCO '18, 2018, Page(s) 1-4, ISBN 9781-450365048
Éditeur: ACM Press
DOI: 10.1145/3212721.3212886

Detection and Regression of Violin Vibrato from Scores and Mobile Devices Rendering for Didactic Purposes

Auteurs: Furkan Yesiler, Rafael Ramirez
Publié dans: Proceedings of the Sound and Music Computing Conference, 2018
Éditeur: SMC

Automatically measuring biomechanical skills of violin performance - an exploratory study

Auteurs: Erica Volta, Maurizio Mancini, Giovanna Varni, Gualtiero Volpe
Publié dans: Proceedings of the 5th International Conference on Movement and Computing - MOCO '18, 2018, Page(s) 1-4, ISBN 9781-450365048
Éditeur: ACM Press
DOI: 10.1145/3212721.3212840

Automatic assessment of violin performance using dynamic time warping classification

Auteurs: Sergio Giraldo, Ariadna Ortega, Alfonso Perez, Rafael Ramirez, George Waddell, Aaron Williamon
Publié dans: 2018 26th Signal Processing and Communications Applications Conference (SIU), 2018, Page(s) 1-3, ISBN 978-1-5386-1501-0
Éditeur: IEEE
DOI: 10.1109/SIU.2018.8404556

A Machine Learning Approach to Classification of Phonation Modes in Singing.

Auteurs: Matteo Lionello, Rafael Ramirez
Publié dans: 2018
Éditeur: SMC

Air violin: a machine learning approach to fingering gesture recognition

Auteurs: David Dalmazzo, Rafael Ramirez
Publié dans: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education - MIE 2017, 2017, Page(s) 63-66, ISBN 9781-450355575
Éditeur: ACM Press
DOI: 10.1145/3139513.3139526

Neural Correlates of Bow Learning Technique

Auteurs: Angel David Blanco; Rafael Ramirez
Publié dans: 10th International Workshop on Machine Learning and Music, 2017
Éditeur: MML

Bowing modeling for violin students assistance

Auteurs: Fabio J. M. Ortega, Sergio I. Giraldo, Rafael Ramirez
Publié dans: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education - MIE 2017, 2017, Page(s) 60-62, ISBN 9781-450355575
Éditeur: ACM Press
DOI: 10.1145/3139513.3139525

Acquisition of Violin Instrumental Gestures Using an Infrared Depth Camera

Auteurs: Zacharias Vamvakousis; Alfonso Perez Carrillo; Rafael Ramirez
Publié dans: Proceedings of the 15th Sound and Music Computing Conference (SMC2018), Limassol, Cyprus, 4-7 July, 2018
Éditeur: SMC
DOI: 10.5281/zenodo.1422548

Automated Analysis of Postural and Movement Qualities of Violin Players

Auteurs: Erica Volta, Gualtiero Volpe
Publié dans: 2019 International Workshop on Multilayer Music Representation and Processing (MMRP), 2019, Page(s) 56-59, ISBN 978-1-7281-1649-5
Éditeur: IEEE
DOI: 10.1109/mmrp.2019.8665374

Phrase-Level Modeling of Expression in Violin Performances

Auteurs: Fabio J. M. Ortega, Sergio I. Giraldo, Rafael Ramirez
Publié dans: 10th International Workshop on Machine Learning and Music, 2017
Éditeur: mml

A Machine Learning Approach to Violin Bow Technique Classification - a Comparison Between IMU and MOCAP systems

Auteurs: David Dalmazzo, Simone Tassani, Rafael Ramírez
Publié dans: Proceedings of the 5th international Workshop on Sensor-based Activity Recognition and Interaction - iWOAR '18, 2018, Page(s) 1-8, ISBN 9781450364874
Éditeur: ACM Press
DOI: 10.1145/3266157.3266216

Evaluation of audio-based feedback technologies for bow learning technique in violin beginners

Auteurs: Angel Blanco, Rafael Ramirez
Publié dans: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education - MIE 2017, 2017, Page(s) 41-43, ISBN 9781-450355575
Éditeur: ACM Press
DOI: 10.1145/3139513.3139520

Estimation of violin bowing features from Audio recordings with Convolutional Networks

Auteurs: A. Perez-Carrillo, H. Purwins
Publié dans: Conference and Workshop on Neural Information Processing Systems (NIPS), 2017
Éditeur: Neural Information Processing Systems Foundation, Inc.

A multimodal corpus for technology-enhanced learning of violin playing

Auteurs: Gualtiero Volpe, Ksenia Kolykhalova, Erica Volta, Simone Ghisio, George Waddell, Paolo Alborno, Stefano Piana, Corrado Canepa, Rafael Ramirez-Melendez
Publié dans: Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter - CHItaly '17, 2017, Page(s) 1-5, ISBN 9781-450352376
Éditeur: ACM Press
DOI: 10.1145/3125571.3125588

Informing bowing and violin learning using movement analysis and machine learning

Auteurs: E. Volta, P. Alborno, and G. Volpe
Publié dans: Proceedings of the 10th International Workshop on Machine Learning and Music, 2017
Éditeur: MML
DOI: 10.13140/rg.2.2.24907.49443

Jazz ensemble expressive performance modeling

Auteurs: Bantula, Helena; Giraldo, Sergio Iván; Ramírez, Rafael,1966-
Publié dans: Numéro 1, 2016
Éditeur: International Society for Music Information Retrieval (ISMIR)
DOI: 10.5281/zenodo.1415876

Exploiting multimodal integration in adaptive interactive systems and game-based learning interfaces

Auteurs: Erica Volta, Gualtiero Volpe
Publié dans: Proceedings of the 5th International Conference on Movement and Computing - MOCO '18, 2018, Page(s) 1-4, ISBN 9781-450365048
Éditeur: ACM Press
DOI: 10.1145/3212721.3212849

Acquisition of Human Body Kinematic and Dynamic Features in Violin Performances with Kinect

Auteurs: Pablo Fernandez Blanco, Alfonso Perez-Carrillo, Mario Ceresa
Publié dans: First International Workshop on Motor Learning For Music Performance, Numéro 1, 2017, 2017
Éditeur: Aalborg University Copenhagen

Statistical models for the indirect acquisition of violin bowing controls from audio analysis

Auteurs: Alfonso Perez-Carrillo
Publié dans: Proceedings of Meetings on Acoustics, Numéro 29, 2017, 2017, Page(s) 035003
Éditeur: Acoustical Society of America
DOI: 10.1121/2.0000402

ESTIMATION OF BOWING PARAMETERS IN VIOLIN PLAYING FROM AUDIO ANALYSIS

Auteurs: Alfonso Perez-Carrillo
Publié dans: Abstracts of the International Symposium on Performance Science 2017, Numéro 1, 2017, 2017, Page(s) 192, ISBN 9789-935937803
Éditeur: Iceland Academy of the Arts

JAZZ ENSEMBLE EXPRESSIVE PERFORMANCE MODELING

Auteurs: Helena Bantula, Sergio Giraldo and Rafael Ramirez
Publié dans: 17th International Society for Music Information Retrieval Conference, 2016., Numéro Ago 12, 2016, 2016
Éditeur: International Society for Music Information Retrieval

A Genetic Approach for Evaluation of Computational Models for Expressive Music Performance in Jazz Guitar

Auteurs: Sergio Giraldo and Rafael Ramirez
Publié dans: 9th International Workshop on Machine Learning, Numéro September 23, 2016, 2016
Éditeur: MML

HEXAPHONIC GUITAR TRANSCRIPTION AND VISUALISATION

Auteurs: Iñigo Angulo Ortegui, Sergio Giraldo, Rafael Ramirez
Publié dans: Second International Conference on Technologies for Music Notation and Representation, Numéro 27-29 May 2016, 2016
Éditeur: Anglia Ruskin University

Non-peer reviewed articles (1)

Musical neurofeedback for treating depression in elderly people

Auteurs: Rafael Ramirez, Manel Palencia-Lefler, Sergio Giraldo, Zacharias Vamvakousis
Publié dans: Frontiers in Neuroscience, Numéro 9, 2015, ISSN 1662-453X
Éditeur: Frontiers
DOI: 10.3389/fnins.2015.00354

Book chapters (1)

Violin Timbre Navigator: Real-Time Visual Feedback of Violin Bowing Based on Audio Analysis and Machine Learning

Auteurs: Alfonso Perez-Carrillo
Publié dans: MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II, Numéro 11296, 2019, Page(s) 182-193, ISBN 978-3-030-05715-2
Éditeur: Springer International Publishing
DOI: 10.1007/978-3-030-05716-9_15