Whole-Body Interaction Learning for Dance Education

Informations projet

WhoLoDancE

N° de convention de subvention: 688865

DOI

10.3030/688865

Projet clôturé

Date de signature de la CE 28 Octobre 2015

Date de début 1 Janvier 2016

Date de fin 31 Decembre 2018

Financé au titre de

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Coût total

€ 3 332 585,00

Contribution de l’UE

€ 3 332 585,00

Coordonné par

LYNKEUS
Italy

Periodic Reporting for period 2 - WhoLoDancE (Whole-Body Interaction Learning for Dance Education)

Période du rapport: 2017-07-01 au 2018-12-31

Résumé du contexte et des objectifs généraux du projet

WhoLoDancE (Whole-Body Interaction Learning for Dance Education) aimed at developing and applying breakthrough technologies to dance learning, teaching and choreography, with the double goal of preserving and leveraging the intangible cultural heritage of European dances and bringing a profound innovation in the way this long-standing art is created, learnt and taught, meanwhile making its patrimony widely available to large audiences.WhoLoDancE entailed the definition of an open conceptual framework of dance movement principles among the interdisciplinary team of technologists and dance experts constituting its research consortium. the use of motion capture technologies resulted in the creation of a digital repository of more than 6,000 dance movements from the four selected dance genres: ballet, contemporary, flamenco and traditional Greek folk dances. This movement data, in the form of motion capture, audio and video files, has been thus synchronised, segmented into simpler motions and annotated with the common vocabulary of physical and expressive qualities of the WhoLoDancE framework. This core base of multimodal, enriched data constituted the base for the development and testing of a series of innovative applications for analysis, representation and re-elaboration of movement data, created and evolved through a cross-disciplinary, continuous dialogue between developers and dance professionals inside and outside the consortium, consulted through workshops, hands-on sessions, focus groups and interviews. WhoLoDancE added value is its approach based not only on the contamination of information technology into dance, but also of a variety of movement practices coming from different dance genres into each other, compenetrating each other’s practices and methods. In this case, scientific approaches entered the digital medium to establish a mutually beneficial dialogue, highlighting commonalities within different dance practices and between dance and other disciplines.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

16 tools were achieved with different levels of maturity:some of them are almost market-ready, others are protypes that require further R&D before being user-ready. They include the library of dance movements and its web-based interface (WhoLoDancE movement library and annotator), its extension into an educational platform at the service of dance teaching (WhoLoDancE educational platform) and different tools for data curation, such as Movement quality annotation by comparison tool and Segmentation tool, besides a Low-end VR platform allowing for visualisation of dance data in the form of a 3D avatar on a common smartphone. A set of software libraries for music and sound analysis, utilised in other applications, include software for real-time extraction of movement qualities (Movement analysis software library), automatic annotation of dance data with movement qualities (Data-driven movement quality extraction software library) and analysis of physical and expressive qualities of sound (Tools for music analysis). Project outcomes also entail user-ready tools for movement analysis, such as a Search and similarity engine and its derived applications, able to analyse the representation of a movement and identify the most similar movements in the library; the query movement can be taken from the motion repository (Similarity search web-based application), recorded through low-cost sensors (Movement sketching tool) or through a simple smartphone or tablet camera (Real-time mobile movement search application); this same engine has also been employed to develop a smartphone-based game (Real-time motion-based collaborative mobile game) to provide practitioners with a score indication of the closeness of their performance to a given one, to help improve their performance. The Blending engine, has been specifically designed for choreographic creation: the application allows to assemble movements from the repository in a temporal sequence or superpose parts of different movements to generate new, cross-genre movements and choreographies, and visualise them in the form of an avatar before starting to practice them in the studio. Choreomorphy tool enables dancers, through the use of a motion capture suit, to visualise themselves into different avatars focusing on traces and volumetric shapes, the and Sonification tool converts expressive movement qualities into sound, providing real-time, responsive feedback on the movement without causing distraction. Lastly, the HoloLens experience, conceived as a mixed-reality, didactic application, enables a student to observe a piece performed by a dance master, imitate it and receive image and sound real-time feedback through the Microsoft HoloLens, also allowing for an additional teacher’s view with the support of a second HoloLens. Upon release of their user-ready version, the latest months of the project have been dedicated to a thorough final round of evaluation of the tools by a team of selected experts from the dance and 3D animation field in terms of usability, effectiveness, accessibility and potential added value for their everyday work, which have progressively guided the conclusive fine-tuning of the tools to their current version.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

the WhoLoDancE toolbox has confirmed its potential to bring an effective contribution for the innovation and cross-fertilisation of dance practices and methods, both in terms of paedagogic use-case applications and creative possibilities, as well as to the application of its approach for the preservation and re-elaboration of cultural heritage-related digital assets. Movements can be recorded with low-cost devices and compared with similar ones into the library, allowing a direct comparison with the ones users are currently learning, performed by a master dancer, or with similar movements sharing common qualities but pertaining to different genres and styles. This allows a more personalised learning experience where, before or after the live lesson with the teacher, a student can deeply reflect on the formal and expressive level of the dance movements of interest and translate this knowledge into his/her practice. All this contributes to the emerging of a new kind of learning, complementary to the traditional mimicking approach, which does not rely on the physical presence of the dance master, but still allows for an individual reference-based self-practice. On the other hand, the didactic applications offered by the educational platform provide dance teachers with new tools for creating personalised exercises and activities designed on the student depending on his/her experience and specific background, and the HoloLens experience tool allows the student to observe the recorded movements of a dance master in the form of an avatar and repeat the same sequence, with real-time visual and audio feedback on his/her performance per body part, while a teacher can observe the student moving inside the avatar volume and provide evidence-based feedback, making it a collective, teacher-student activity. Those are among the reasons why Balletto di Roma, the institution representing the Italian national ballet company of Rome, is now seeking to adopt the WhoLoDancE platform and its toolbox for their didactic activity, fostering its new stream of research projects, production and collaborations combining tradition with innovation, under the new artistic direction of Francesca Magnini.

Motion Capture in progress

Motion Capture - Dancers in special suits

ChoreoMorphy 1

Movement Library

ChoreoMorphy 1

Blending engine 1

Blending engine 2

Trails Model