Community Research and Development Information Service - CORDIS

H2020

RAPID-MIX Report Summary

Project ID: 644862
Funded under: H2020-EU.2.1.1.4.

Periodic Reporting for period 1 - RAPID-MIX (Realtime Adaptive Prototyping for Industrial Design of Multimodal Interactive eXpressive technology)

Reporting period: 2015-02-01 to 2016-01-31

Summary of the context and overall objectives of the project

RAPID-MIX brings cutting edge knowledge from three leading European research labs specialising in embodied interaction (Goldsmiths, IRCAM and UPF) to a consortium of five creative industries SMEs (Reactable Systems, Roli, AudioGaming, Orbe and PLUX) to create the next generation of Multimodal Interactive eXpressive systems (MIX) for the fields of music, videogames, and wellbeing.
In order to bring these innovations out of the lab and into market scenarios, directly to users, where they have true impact, RAPID MIX addresses four main technological and design objectives:

1- Build the next generation of Multimodal Interactive eXpressive (MIX) interfaces for natural and seamless interaction in individual and collaborative scenarios.
The project integrates novel wearable hardware, advanced real-time machine learning techniques, and online platforms for data storage and sharing, into a coherent and open set of APIs: the RAPID-API for Multimodal Interaction Design. These tools are demonstrated in creative domains through new and enhanced products and prototypes.
2- Accelerate the availability of next generation MIX interfaces according to the standards and needs of the creative industry
RAPID-MIX accelerates the transfer of research-based technology from lab to market by integrating innovative proof of concepts in real products, through fast product design cycles covering agile prototyping, API development and integration, and promoting a closer relationship between the 3 research labs and the 5 SMEs who directly exploit these novel/improved products.
3- Establish Design Specifications and Guidelines for the next generation of MIX interfaces.
A solid User-Centred Design (UCD) methodology is deployed considering industrial and end-user needs, production roadmap of SMEs partners, technological and manufacturing requirements, and end-user evaluation in relevant creative scenarios
4- Improve the SMEs competitiveness within the consortium, and propose a sustainable model for further innovation beyond the project and for the larger industrial European community
RAPID-MIX provides a set of technological solutions, design guidelines, and infrastructures that are coherent with the SMEs business models, and that will be incorporated in (at least) five new and enhanced products. Beyond the consortium SMEs, the RAPID-API will be aligned with broader industrial and market needs and available for the creative community through open licenses.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

"The first months of RAPID-MIX (M1-M4) were devoted to settle the basis for deploying User-Centred Design (UCD) actions during the lifelong of the project. This led to the definition of OPPP, an innovative UCD methodological framework specific to RAPID-MIX, comprised of a four (4) step process: Observation, Probes, Prototypes, Products. This framework also includes a set of techniques for user involvement, for assessing and benchmarking design requirements and needs, and it has been complemented with a summary on the state of the art of the main research domains that informed our proposed methodology.

This framework allowed us to gauge industrial and end-user needs during the subsequent months of the project (M5-M8) through participatory design activities, focus groups with consortium members (Paris, June 20-22 & London, September 8-9) a co-design workshop (Barcelona, June 16) and a workshop and hackathon within the Barcelona Music Hack Day at Sónar Festival (June 17-19). These UCD activities led to a first set of Design Guidelines for rapid prototyping.

In parallel, the strategy for developing the RAPID-API (a set of software and hardware solutions for multimodal interaction design) started with the preparation and categorization of the candidate technologies identified within the consortium (M1-M8). Project partners have been distributed in special interest groups to make a first drafts of specifications for the RAPID-API building blocks.

At the agile prototyping level, inputs from UCD actions and the API strategy led to the development of a first set of 8 early-prototypes (version 1, M8) each one accompanied by a video demonstrating its use in context (available at the project website: http://rapidmix.goldsmithsdigital.com/). A second set of of early prototypes (version 2) has been already defined and built (to be delivered on M14). Importantly, three prototypes are currently developed as products (Wearable Musical Instruments, BITalino (r)evolution & OpenSignals (r)evolution).

In M5 the integration of future MIX products started. An Integration Strategy Document was defined with the participation of all SME partners (M8), describing a first roadmap for each MIX product. This document also updates the original description of MIX products (as stated in the DoA) according to some changes in SME business needs, and to new opportunities for integration that were identified through the first UCD actions and the creation of prototypes v1. These roadmaps also contain detailed information about dependencies and timelines, and each one has undergone a review and feedback process by at least 2 partners outside of the SME.

The aforementioned Integration Strategy settled the basis for the development of 5 MIX products:
- Roli’s JUCE RAPID-MIX module collection (including signal processing, machine learning, and possibly other RAPID-MIX tools)
- AudioGaming’s AudioBlender plugin integration with Interactive Machine Learning
- Reactable’s Intelligent Music Engine (mobile music processing and gesture analysis platform)
- Orbe’s MIX web platform for designing new augmented reality and situated media experiences.
- PLUX’s Bitalino (r)evolution (rapid prototyping hardware platform), OpenSignals (r)evolution (next generation platform for data visualization and recording), and OpenSignals.net (online repository of biosignals data)

Additionally, a the data management plan was completed in M6, in order to handle research and production datasets collected in the project. An in-depth interviewing process has been carried out involving all RAPID-MIX partners, cataloguing their current data handling practices and stance towards data sharing and open access. During this process we also identified and outlined a number of potential datasets to be produced throughout the duration of the project, in accordance to the EU best practices regarding research data management and the interests of the RAPID-MIX partners.

Complementary to these actions, a strategic plan for Intellectual Property Right (IPR) management was developed and delivered to partners (M12). This document presents a framework for managing and exploiting IPR in RAPID-MIX, to facilitate technology development, sharing, access and innovation activities (through exploitation) among and for project partners.

The work around the evaluation of MIX prototypes and products in relevant creative environments started in M12. However, preparatory tasks have been developed in M5 through UCD actions regarding the evaluation of wearable and multimodal technologies existing within the consortium to gauge industrial and end-user needs. In M17 a report will describe the feedback of the tests made to the early prototypes, together with a qualified set of changes to the design guidelines that can be used to mature MIX technologies.

Finally, dissemination and exploitation actions were deployed all along the first year of the project. These actions started with the creation of a visual identity for both RAPID-MIX products and the RAPID-API, and the launching of the project website (http://rapidmix.goldsmithsdigital.com/), Youtube (https://www.youtube.com/channel/UC5aJFkgZmIF1DA2VEzjYs7w), Vimeo (https://vimeo.com/rapidmix) and Twitter (https://twitter.com/RAPID_MIX) accounts. The project website also features information about the launch of commercial products using RAPID-MIX technologies, including a new sensor product from PLUX, and a new product from ROLI. In addition a selection of videos have been produced and disseminated featuring communities engaging with RAPID-MIX technologies as part of hack days, workshops, and related activities (http://rapidmix.goldsmithsdigital.com/#section-15).
Regarding the dissemination, promotion and demonstration of the project outcomes within the creative community, during its first year RAPID-MIX has taken part in the Barcelona Music Hack Day within Sonar Festival (June 2015) and also has organized a workshop for hack day attendees. RAPID-MIX also took part of several events related to music and Do it Yourself communities, as in the case of the Maker Faire (Lisbon and Rome) and the Music Tech Fest 2015.
"

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The developments achieved within the first year of RAPID-MIX, and their wide visibility in creative communities such as music technologist, designers and makers (through workshops, hackathons and showcases) allow us to foresee a large impact within and beyond the creative industry in the years to come.

The current RAPID-MIX outcomes are already enabling better uses of ICT technologies by partner SMEs within the consortium. During this first year we have seen how our candidate technologies (interactive machine learning tools for gesture interpretation, online repositories, collaborative online communities, hardware platforms, audio synthesis and music information retrieval tools) have led to innovative prototypes and early commercial products. Once this set of solutions are publicly available through the RAPID-API, we foresee a significant positive impact in the way European creative communities and SMEs deploy their product development cycles and prototyping processes.

The RAPID-MIX user-centred design methodologies and guidelines -which are already publicly available through the project website - have been conceived to strengthen the leadership of European creative industries in the development of innovative technology products. Moreover, RAPID-MIX is already promoting the spillover of the knowledge acquired to other European industries beyond the creative domain. In this regard, PLUX’s BITalino (r)evolution (a hardware prototyping platform) and OpenSignals (r)evolution (software platform for real-time visualization and recording of physiological data) are currently being used in the research and e-health domains. In the same line, Reactable MIX (the product impulsed by the SME partner Reactable Systems)is currently bringing together real-time music remixing (a clear creative activity), and sports (running).

These achievements lead us to foresee an important impact of project’s outcomes in the short term, particularly for industrial users (programmers, developers and designers). These target users will shortly benefit from high-end, low cost hardware and software tools, easy to interface with each other. The affordances of such solutions have been already demonstrated in our first early prototypes, and their final integration into the RAPID-API will greatly optimise (cost and time wise) the fast-prototyping of multimodal interactive and expressive interfaces, reducing dependence on ground-up development.

Related information

Record Number: 186467 / Last updated on: 2016-07-12
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top