Periodic Reporting for period 1 - MusicBricks (MusicBricks: Musical Building Blocks for Digital Makers and Content Creators)
Reporting period: 2015-01-01 to 2016-06-30
Summary of the context and overall objectives of the project
Context and Objectives
#MusicBricks brings top academic research to market using a unique creative methodology. The aim is to transfer state-of-the-art ICT to Creative SMEs in order to develop novel business models. The methodology has three main components:
• #MusicBricks bundles new technologies into a toolkit that can easily be used by hackers and makers in order to create and develop new kinds of products, performances and processes.
• #MusicBricks seeds ideas at Creative Testbed events, incubates the successful projects to commercial prototype in Industry Testbeds, and showcases them to investors and commercial partners at Market Testbeds.
• #MusicBricks reconfigures the commercial framework for innovation by developing a new layer of intellectual property and using Market Adoption Readiness Levels (MARLs) rather than the traditional Technology Readiness Levels (TRLs).
This methodology provides an agile and rapid development framework that situates creativity at the centre of innovation. It brings different skill sets and expertise together through a shared interest in music, and provides a context in which collaborative making is encouraged and reinforced. #MusicBricks uses the fact that creativity, making and music function as a social glue that joins together people from different backgrounds: artists and scientists; academia and industry. The fertile ground for new types of products, processes and performances that this creates provides a template for innovation across all industry sectors. It also provides a model for new types of education and research, as the rapid knowledge transfer that results from combining different ways of thinking in practical and collaborative projects allows for models of learning that are suited to the contemporary context and technological environment.
Context
There are two major gaps in the Music Technology Value Chain that hinder innovation (see Fig.1). Music technology research is not finding its way into the hands of creative developers, inventors and SMEs who can turn that knowledge into products for the market. There is also no clear route for innovative creative music technology applications from SMEs out to the wider industry, to investors and arts funding organisations.
The #MusicBricks project took the data and technologies created as a result of research within academic institutions, and wrapped them into Application Programming Interfaces (APIs), Graphic User Interfaces (GUIs) and Tangible User Interfaces (TUIs). The project incentivised the use of these APIs, GUIs and TUIs outside of the academic environment by engaging creative developers and artists from the music technology community in a series of Creative Testbed hack events and then incubating the best ideas that emerged in that context. Those ideas were developed into commercial prototypes that could then be channeled to global markets by facilitating industry partnerships, investment or arts funding at a Market Testbed event.
In so doing, #MusicBricks successfully closed gaps in the music technology value chain and created a seamless transfer of knowledge and novel ICT technology to European Creative SMEs in order to boost their business potential, with a built-in feedback loop to guide future research, boosting knowledge and developing skills within the community, as well as generating new knowledge and statistics to inform future policy.
a) Socio-economic context
The #MusicBricks methodology was generated in response to a particular socio-economic context. In recent years the global music industry has been revolutionised by EU music technology innovation, increasing the volume of music streaming, social platforms, creative applications and novel business models. The junction of music and technology has created a unique phenomenon in Europe. Unlike other sectors where large industry players lead the way in ICT exploitation, in the music industry sector the revolution has been led by innovative music technology SMEs, with global successes such as Spotify, Deezer and SoundCloud leading the way for hundreds of successful startups. Europe is the world leader in music technology SMEs, and these attract considerable international investment.
While large music industry corporates resisted ICT innovation for a very long time and struggled to keep their business sustainable and to digitise vast collections of music, the above SME successes have shown that music technology innovation is contributing to greater citizen engagement and novel business models. The success led by European SMEs has brought a notable shift in the music industry's global strategy, its aims and objectives. The music industry focus has been fixed on digital innovation.
b) Innovation context
As an event which unites the music industry with the entire music technology ecosystem, Music Tech Fest has expanded from Europe into a global music technology network with events currently running across several continents. By developing methodologies that nurture creative collaboration, ideas seeding and incubation, Music Tech Fest has demonstrated the value of joining academia and industry, as well as art and science, in a collaborative platform. However, it had been apparent that despite the enthusiasm shown by large numbers of music technology stakeholders and the willingness to join forces, the value chain from academic research innovation to SMEs and large music industry corporates remained fragmented. In both Music Tech Fest and the more developer-focused Music Hack Day, major gaps prevented excellent research from Europe's leading music technology centres from reaching creative SMEs that could use them in music industry applications; and made it difficult for innovative ICT applications to find their way from creative music technology SMEs to the attention of large music industry corporations, investors, and arts funding organisations.
Objectives
At the heart of the Music Tech Fest and Music Hack Day innovation activities, the central objective of the #MusicBricks project was to close those gaps in the music technology value chain and create a channel for those connections to be formed, with Creative SMEs situated as the primary catalysts for innovation, built upon European technology research.
Not only did #MusicBricks succeed in closing those gaps, it also exceeded expectations in areas where objectives had not even been set. The project achieved a much higher degree of success because the methodology did not simply provide a route for a piece of research to find its way to market, but rather provided the context for a multiplicity of unanticipated results. #MusicBricks provides a platform for rapid knowledge transfer. The interoperability of the tools themselves provides for a wide array of combinations and ideas that might otherwise have been unachievable. The 11 new startup ideas and inventions that were selected to be brought to market are diverse and impact on industry sectors other than music and media entertainment.
Ten of the prototypes were presented at the Final Market Testbed where incubatees pitched their prototypes to investors and industry, and showcased their projects to early adopters and technology enthusiasts. In addition, following the final Market Testbed, the establishment of #MusicBricks as a limited company ensures several exploitation routes for the knowledge, experience, methodologies and products developed and tested during the project, as well as sustainability after the end of the EC project timeline.
By developing a unique methodology, #MusicBricks succeeded in:
• Bringing academic research to market;
• Creating a large number of promising creative SME startups;
• Providing a template for innovation across a wide range of industry sectors;
• Establishing a pedagogical framework for rapid knowledge transfer;
• Innovating new and proven layers of intellectual property;
• Creating new business models and routes to market;
• Establishing and documenting a rich and creative innovation ecosystem;
• Implementing and proving the value of MARLs for fast, iterative project development, early deployment, engagement with early adopters, and high data yield.
#MusicBricks brings top academic research to market using a unique creative methodology. The aim is to transfer state-of-the-art ICT to Creative SMEs in order to develop novel business models. The methodology has three main components:
• #MusicBricks bundles new technologies into a toolkit that can easily be used by hackers and makers in order to create and develop new kinds of products, performances and processes.
• #MusicBricks seeds ideas at Creative Testbed events, incubates the successful projects to commercial prototype in Industry Testbeds, and showcases them to investors and commercial partners at Market Testbeds.
• #MusicBricks reconfigures the commercial framework for innovation by developing a new layer of intellectual property and using Market Adoption Readiness Levels (MARLs) rather than the traditional Technology Readiness Levels (TRLs).
This methodology provides an agile and rapid development framework that situates creativity at the centre of innovation. It brings different skill sets and expertise together through a shared interest in music, and provides a context in which collaborative making is encouraged and reinforced. #MusicBricks uses the fact that creativity, making and music function as a social glue that joins together people from different backgrounds: artists and scientists; academia and industry. The fertile ground for new types of products, processes and performances that this creates provides a template for innovation across all industry sectors. It also provides a model for new types of education and research, as the rapid knowledge transfer that results from combining different ways of thinking in practical and collaborative projects allows for models of learning that are suited to the contemporary context and technological environment.
Context
There are two major gaps in the Music Technology Value Chain that hinder innovation (see Fig.1). Music technology research is not finding its way into the hands of creative developers, inventors and SMEs who can turn that knowledge into products for the market. There is also no clear route for innovative creative music technology applications from SMEs out to the wider industry, to investors and arts funding organisations.
The #MusicBricks project took the data and technologies created as a result of research within academic institutions, and wrapped them into Application Programming Interfaces (APIs), Graphic User Interfaces (GUIs) and Tangible User Interfaces (TUIs). The project incentivised the use of these APIs, GUIs and TUIs outside of the academic environment by engaging creative developers and artists from the music technology community in a series of Creative Testbed hack events and then incubating the best ideas that emerged in that context. Those ideas were developed into commercial prototypes that could then be channeled to global markets by facilitating industry partnerships, investment or arts funding at a Market Testbed event.
In so doing, #MusicBricks successfully closed gaps in the music technology value chain and created a seamless transfer of knowledge and novel ICT technology to European Creative SMEs in order to boost their business potential, with a built-in feedback loop to guide future research, boosting knowledge and developing skills within the community, as well as generating new knowledge and statistics to inform future policy.
a) Socio-economic context
The #MusicBricks methodology was generated in response to a particular socio-economic context. In recent years the global music industry has been revolutionised by EU music technology innovation, increasing the volume of music streaming, social platforms, creative applications and novel business models. The junction of music and technology has created a unique phenomenon in Europe. Unlike other sectors where large industry players lead the way in ICT exploitation, in the music industry sector the revolution has been led by innovative music technology SMEs, with global successes such as Spotify, Deezer and SoundCloud leading the way for hundreds of successful startups. Europe is the world leader in music technology SMEs, and these attract considerable international investment.
While large music industry corporates resisted ICT innovation for a very long time and struggled to keep their business sustainable and to digitise vast collections of music, the above SME successes have shown that music technology innovation is contributing to greater citizen engagement and novel business models. The success led by European SMEs has brought a notable shift in the music industry's global strategy, its aims and objectives. The music industry focus has been fixed on digital innovation.
b) Innovation context
As an event which unites the music industry with the entire music technology ecosystem, Music Tech Fest has expanded from Europe into a global music technology network with events currently running across several continents. By developing methodologies that nurture creative collaboration, ideas seeding and incubation, Music Tech Fest has demonstrated the value of joining academia and industry, as well as art and science, in a collaborative platform. However, it had been apparent that despite the enthusiasm shown by large numbers of music technology stakeholders and the willingness to join forces, the value chain from academic research innovation to SMEs and large music industry corporates remained fragmented. In both Music Tech Fest and the more developer-focused Music Hack Day, major gaps prevented excellent research from Europe's leading music technology centres from reaching creative SMEs that could use them in music industry applications; and made it difficult for innovative ICT applications to find their way from creative music technology SMEs to the attention of large music industry corporations, investors, and arts funding organisations.
Objectives
At the heart of the Music Tech Fest and Music Hack Day innovation activities, the central objective of the #MusicBricks project was to close those gaps in the music technology value chain and create a channel for those connections to be formed, with Creative SMEs situated as the primary catalysts for innovation, built upon European technology research.
Not only did #MusicBricks succeed in closing those gaps, it also exceeded expectations in areas where objectives had not even been set. The project achieved a much higher degree of success because the methodology did not simply provide a route for a piece of research to find its way to market, but rather provided the context for a multiplicity of unanticipated results. #MusicBricks provides a platform for rapid knowledge transfer. The interoperability of the tools themselves provides for a wide array of combinations and ideas that might otherwise have been unachievable. The 11 new startup ideas and inventions that were selected to be brought to market are diverse and impact on industry sectors other than music and media entertainment.
Ten of the prototypes were presented at the Final Market Testbed where incubatees pitched their prototypes to investors and industry, and showcased their projects to early adopters and technology enthusiasts. In addition, following the final Market Testbed, the establishment of #MusicBricks as a limited company ensures several exploitation routes for the knowledge, experience, methodologies and products developed and tested during the project, as well as sustainability after the end of the EC project timeline.
By developing a unique methodology, #MusicBricks succeeded in:
• Bringing academic research to market;
• Creating a large number of promising creative SME startups;
• Providing a template for innovation across a wide range of industry sectors;
• Establishing a pedagogical framework for rapid knowledge transfer;
• Innovating new and proven layers of intellectual property;
• Creating new business models and routes to market;
• Establishing and documenting a rich and creative innovation ecosystem;
• Implementing and proving the value of MARLs for fast, iterative project development, early deployment, engagement with early adopters, and high data yield.
Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far
#MusicBricks brought together key music technology research outputs from academic and research partners IRCAM, UPF, TU Wien and Fraunhofer IDMT and created a series of tools that could be easily deployed within creative testbeds, combined and developed into new projects, products and prototypes by makers, hackers and creative technologists.
As a series of APIs, GUIs and TUIs, the research excellence from top European institutions was brought outside the realm of academia to be integrated with current music technology innovations in the commercial realm in order to create innovative products and applications. The #MusicBricks methodology made the transfer from research knowledge to developer deployment more effective by embedding it within an existing growing multidisciplinary community.
At kick-off, partners planned to deploy 8 tools to the #MusicBricks Creative Testbeds. One of these tools was an innovative new Tangible User Interface for gesture-driven low-latency applications, and 7 were Application Programming Interfaces which wrapped existing tools and technologies together from within the world of music tech research in order to create specific ‘bricks’ that could be deployed in combination with others to make new hybrid applications. The first set of 8 tools was deployed by Month 5 of the project in time for the first Creative Testbed.
By Month 9 of the project, following the rising popularity of the #MusicBricks toolkit among developers and innovators, 2 further APIs were added by industry partners who contributed their IP stating that #MusicBricks added value to integration and deployment of their tools. A further industry IP - a data-driven Graphic User Interface - was released under MIT licence and embedded into the toolkit by Month 10. Several industry requests for addition of their tools were rejected due to their restrictive licences, however by the end of the project a further industry API, an industry GUI, and two further partner tools were added, making the total 15 tools in the #MusicBricks toolkit. Several original tools were amplified with further features in response to the requests from developers and innovators, and an official cloud platform has been accepted for #MusicBricks-driven innovations.
The #MusicBricks toolkit was embedded in the existing innovation ecosystems specifically around the Music Tech Fest and Music Hack Day communities and benefitted from these large and growing ecosystems by accessing some of the top global innovation minds.
Deployment through existing growing networks of creative developers, makers and innovators at Music Tech Fest and Music Hack Day ensured early adoption and engagement with the APIs, GUIs and TUIs to a much greater extent than anticipated. The tools were so popular that partners were unable to fulfil demand at events such as Music Hack Day at the Sonar Festival in Barcelona. The tools were successfully embedded in these communities and capitalised on the substantial numbers that these events attract, but as the tools became popular they also contributed to the overall growth of the innovation community.
The Creative Testbeds at hack events provided fertile ground for the seeding of ideas and attracted a variety of experts from different areas of activity, porting knowledge from music technology research centres to this community, but also joining the dots between the various areas of knowledge that the innovators brought to the testbeds.
In order to encourage the creation of new IP on top of Research IP provided by the research partners, the Consortium Agreement was re-written to include a new area of Innovation IP, and so enabled the innovators to take ownership of their layer of IP. In one case this resulted in a patent search conducted just 4 months after the innovation seed idea. Partners agreed on licensing the tools in ways that supported and encouraged Open Innovation (mostly MIT licences), and commercial licences were offered under fair and reasonable conditions. Fast knowledge transfer was incentivised through the organisation of testbeds and direct engagement from the project’s research engineers. This combined with micro-funding to ensure further prototype development. New business models were incentivised through early exposure to audiences, partners and investors, business mentoring and active brokering of industry and investor partnerships.
Micro-funding ensured that the potential of valuable seed ideas was developed and exploited. However the largest investment was in technological support from #MusicBricks research engineers which ensured fast knowledge transfer and robustness of prototypes. The combination of prompt funding and readily available knowledge support ensured a fast track to innovation with first results registering only 4 months after the first seed ideas.
11 new creative and commercial ideas were established at the Creative Testbeds. These ranged from performance concepts and new musical interfaces to tools for musicians with physical disabilities and gesture-driven collaborative remix applications.
• Dolphin: a wearable device that responds to head movement
• #FindingSomethingBondingSound - an application for audiovisual artists using EEG Brainwave reading technology as well as movement sensors
• Hi Note - a new accessible musical instrument
• GIRD - a gesture-based interactive audio and lighting system that allows audiences to remix, explore and interact with music and lights through dancing and movement
• Sound in Translation - a new way to collaborate and remix live by creating a system that responds intuitively to a perfomer
• Airstrument - an interactive instrument that allows a performer to improvise and create music using hand gestures
• Lightbeat - generates a music visualisation that controls a lighting system live.
• Enboard - an audiovisual experience for skateboarding
• Manuphonia - a wearable instrument and app that creates music from movements and gestures
• Snitch - a Bionic ear to help musicians get in tune and jam with others in the right key and tempo
• Interactive Cube - a physical interface for manipulating sound
The developers of the projects were incentivised to further develop their application or product with microfunding and mentorship from the #MusicBricks programme in an Industry Testbed.
The Industry Testbed lasted for a period of 3 to 5 months and ensured development from the 11 seed ideas created in 24-hour sprints at the Creative Testbeds to 11 advanced prototypes which could be taken to the Final Market Testbed. Industry mentoring, partnerships and investors were engaged long before the Final Market Testbed and in many cases followed the prototype development. Each incubated team benefitted from different links and inputs on a case by case basis.
Partners monitored and conducted regular interviews with incubatees at each of the Testbeds, which resulted in valuable feedback which informed the further development of the toolkit. Following the gathering and analysis of data, research teams were able to re-evaluate their tools, and introduce additional features or upgrades. At the final event for the project, incubated teams presented their products to potential customers and pitched to investors - the final step in bringing the original academic research outputs to market. As a result of the project, the #MusicBricks toolkit and methodology has established a viable route to market, closing the gap between academia and industry, and beyond the life of the funded project, #MusicBricks has been established as a Limited Company and will enable industry partners to continue to contribute toolkit components to this open platform for innovation.
Results of the #MusicBricks project
At the end of 18 months of the project, partners, incubatees, and the wider Innovation Ecosystem which has gathered around the #MusicBricks toolkit can measure the following tangible impacts on industry, the arts, research, IP and policy.
Awards
Project #FindingSomethingBondingSound received the Ars Electronica STARTS Prize Honourable Mention and has been listed in this year's State of the Art for achievements at the crossover between science and art.
Patents
A patent was filed by newly formed Swedish company Sojaner AB on the 11th of May 2016
Newly formed companies
Authors of the #MusicBricks Dolphin Project registered Sojaner AB as a Swedish company
MusicBricks LTD was registered as UK Limited Company with Stromatolite LTD as primary shareholder. This makes MusicBricks LTD a sister company to Music Tech Fest LTD, which has the same primary shareholder, and which was registered as the result of the FP7 MIReS CSA.
Music Tech Fest LTD was invited to open an office in Umeå, Sweden, following the success of the first Creative Testbed which was held there.
Tools in the #MusicBricks toolkit
The number of tools in the #MusicBricks toolkit has increased from the 8 tools originally planned to 15.
• 1 tool is a TUI
• 2 tools are GUIs
• 12 tools are APIs
• 5 tools have been added by industry partners
• 1 tool has been batch tested for commercial production (TUI)
• the tools are supported by the official cloud platform SoundCloud
Products
11 #MusicBricks product prototypes were completed:
• 11 product ideas were taken to market
• 10 product prototypes were showcased in the Market Testbed
• 8 products are tools for performance
• 3 products target the gaming industry in their roadmap
• 1 product is currently in development for the heavy machinery industry
• 1 product targets extreme sports entertainment
• 1 product is being considered for the health sector
• 5 products are being used as testbeds for scientific research
• 3 products have been released as Android apps
• 3 products are presented as Open Source software on GitHub
• 2 products are being developed as tools for education
• 1 product is targeting children’s education
• 1 product is SaaS
• 1 product idea has been filed for patenting (see above)
Exploitation opportunities
Several layers of exploitation opportunities have been identified:
1. Exploitation of the knowledge generated within European research institutions;
2. Exploitation of the #MusicBricks tools themselves;
3. The addition of new bricks to the toolkit from external third parties;
4. Exploitation of the toolkit as a commercially exploitable Innovation Ecosystem;
5. Exploitation of the 11 innovative startup product ideas;
6. Implementation of those products as platforms in themselves; and
7. Exploitation of the Creative Content generated for those platforms.
Social media
The hashtag #MusicBricks registered 5,5 million impacts in Month 18 of the project on Twitter. On Facebook the project has so far received 1249 likes.
Presentations and performances
The project has so far featured in 68 public presentations and performances:
27 by project partners
41 by incubatees
Scientific papers
The project has been published / accepted for publication in 5 peer-reviewed papers:
• A two-page publication (in English) about #MusicBricks was published in ERCIM News 101 by partner TU Wien in collaboration with partner Stromatolite, titled: “MusicBricks: Connecting Digital Creators to the Internet of Music Things”
• A two-page publication (in German) about MusicBricks in the OCG Journal (Journal of the Austrian Computer Society) was published by partner TU Wien, entitled “Klingende Bausteine für die Industrie”
• #MusicBricks is mentioned in the context of “Technology, Citizens and Social Change in the Framework of European Research and Innovation Programmes: Towards a Paradigm Shift” by Marta Arniani of partner Sigma (forthcoming).
• #MusicBricks features in “The Eear: Building a Real-Time MIR-based Instrument From a Hack” by Cárthach Ó Nuanáin, Martin Hermant, Angel Faraldo and Daniel Gomez
• G.Dubus E. Flety, R. Borghesi, F. Bevilacqua, "A tutorial on motion data processing for real-time expressive interaction with media using Inertial Measurement Units”
Book chapters
The project has generated 2 book chapters:
• Dubber, A . 2016. You have 24 hours to invent the future of music: music hacks, playful research and creative innovation. In: Wikström, P. & DeFillippi, R. eds., Business Innovation and Disruption in the Music Industry. Cheltenham: Edward Elgar Publishing. pp 211-228
• F. Bevilacqua, N.Schnell J.Françoise, E. Boyer, D. Schwarz, B. Caramiaux, "Designing Action–Sound Metaphors using Motion Sensing and Descriptor-based Synthesis of Recorded Sound Materials” inThe Routledge‘Companion to Embodied Music Interaction’ SECTION VII: Music interaction technologies and applications.
IP guidelines
The project created new guidelines for IP, by means of introducing the new layer of Innovation IP into the #MusicBricks Consortium Agreement. This document can now be provided as reference for other EU IAs.
Policy guidelines
The project introduced new policy guidelines for Market Adoption Readiness Levels (MARLs) and Innovation Ecosystems:
• Market Adoption Readiness Levels (MARLs) were developed for CAF recommendations for H2020 Programme 2016/17 from Section 2.1.2 of the #MusicBricks project proposal.
• MARLs were adopted in the set of EU recommendations by the Innovation Ecosystems Work Group of the Alliance of Internet of Things Innovation in January 2016.
• MARLs feature as the top 3 recommendations for Innovation in the CAF recommendations for the H2020 Programme 2018-2020. (See Open Innovation potential defined in D7.1)
• #MusicBricks Innovation Methodologies feature in multiple CAF recommendations of the Innovation Working Group for the H2020 Programme 2018-2020.
• #MusicBricks is referenced as best practice in 11 EU presentation documents.
Work performed during the project
#MusicBricks was divided into seven separate work packages that managed each aspect of the process involved in meeting the project's objectives:
• WP1: Management
• WP2: Communication, Dissemination and Exploitation
• WP3: Application Programming Interfaces (APIs)
• WP4: Creative User Interfaces (GUIs/TUIs)
• WP5: Creative Testbed: seeding of ideas through creative engagement
• WP6: Industry Testbed: ideas incubation in industry environments
• WP7: Market Testbed: facilitating routes to global markets
WP1
WP1 was aimed at coordinating consortium members in the smooth implementation and reporting of the activities described in the DoA. The objectives of the work package included administrative tasks such as ensuring the compliance with the Grant Agreement and the related reporting requirements, organisational support for project administration, meeting (?targets) and financial procedure, qualitative and support tasks like facilitating effective communication among partners, and ensuring the reach of project milestones and objectives.
It also included Innovation Coordination to monitor fast deployment of interfaces for use in events, wide dissemination to attract large numbers of innovators to the Testbeds, headhunting and selection of the best innovative minds to take on the testbed challenges, ensuring wide adoption of the tools in testbeds, choice of seed ideas with best potential for prototyping, evolution of prototypes and new business models, timely completion of prototypes and dissemination materials, early connections with industry stakeholders, ongoing support from research teams, and market-ready concepts for a fast track to innovation.
Through WP1 the following tasks were implemented:
• Administrative and financial management
• Consortium and review meetings
• Quality management and reporting
WP2
WP2, led by Stromatolite, focused on Communication, Dissemination and Exploitation of the #MusicBricks project, of the toolkit and of
the incubated teams. From the first creative testbed events, Stromatolite brought in commercial partners and brands to engage with developers and makers by setting hack challenges that incorporated #MusicBricks tools, and invited industry experts to act as judges and mentors for the resulting projects.
Partners arranged for teams to meet with industry mentors face to face in their offices and encouraged mentors to connect incubatees with relevant members of their networks to foster further and faster growth from idea to market-ready realisation. Partners also conducted regular meetings with incubated teams, both in person and via Skype to monitor progress and to coach and guide incubatees. At the hack event at Sonar +D in Barcelona, Stromatolite deployed staff specifically to promote the #MusicBricks tools and encourage their use in the hack projects. Through active engagement with hackers, workshops and promotional activities and printed material including colourful laptop stickers indicating the brick(s) with which the hack team was working, partners were able to encourage much greater participation and uptake of the technologies.
New areas were added by Stromatolite to the final market testbed event to ensure that incubated teams and the toolkit itself were given the best opportunities to reach markets, connect with industry and investment and gain exposure to potential customers and stakeholders - from academia to early adopters. The creation of the #MTFAmplifier business networking and pitch event, the Startup Soundpit ‘trade fair’ area and the dedicated performance slots for #MusicBricks projects at the Music Tech Fest in Berlin, as well as the introduction of a market-facing hack challenge as part of the element14 24-hour Hack Camp ensured the maximum potential and impact for exploitation by the toolkit and the incubated projects.
In addition to the engagement with partners at the testbed events, Stromatolite focused on an intensive cross-media campaign that incorporated traditional press and PR strategies, direct mail, leverage of existing networks within partner organisations and social media promotion and community management.
WP2 activities were organised in the following tasks:
• Project branding and exposure
• Communication strategy
• Participation in external events and publications
• Organisation of dissemination events
• Exploitation
WP3
Work Package 3 focused on enabling transfer of research results to creative developers and innovators in the form of several Application Programming Interfaces. End-users throughout the project were members of the creative communities that participate in prototyping (hacking and making) events. Participants of these events were mostly individuals with general technical and programming skills but not experts in audio or music technologies. Therefore the design of APIs focused on offering a clear and attractive functionality to target users, rather than present all possibilities of research algorithms.
Partners successfully identified and integrated various relevant and complementary technologies from the research partners (TUW, Fraunhofer, UPF) under the same umbrella. All technologies were widely used in the Creative Testbeds organised throughout the project. Feedback from creative users and hackers has allowed the improvement of certain aspects of the tools. For example, several users requested more real-time and easy to run tools.
WP3 activities were organised in the following tasks:
• Wrapping existing tools and technologies together
• Creating specific tool combinations for advanced features
• Extending the frameworks beyond the project
WP4
WP4 was devoted to tangible and graphical interfaces in the context of real-time music interaction. As per WP3, end-users throughout the project were members of the creative communities, musicians and makers involved in hacking events for creating new instruments and objects. Typically, these targeted users are mostly individuals with limited expertise in electronics or in digital signal processing, yet willing to include advanced motion sensors in their prototypes. The goal of this Work Package was thus to facilitate the accessibility and the use of such tangible interfaces in these growing communities of makers, digital artists and musicians.
Partners produced guidelines, hardware and software solutions for the participants of the #MusicBricks events and incubations. Importantly, the project allowed partners to significantly improve their technology offer based on the feedbacks of the various users.
Specifically, the focus of WP4 was the invention of the R-IoT (for Real-time Internet of Things for Music), which became one of the most popular technologies during all the Creative Testbeds and beyond. Based on previous IRCAM interface development on wireless motion sensors, the R-IoT was consolidated as a compact wireless motion platform with specifications tuned for expressivity by focusing on low latency and high sensitivity. Moreover, the project allowed for developing an associated software environment enabling users to easily take advantage of such inertial motion units (accelerometers, gyroscopes, magnetometers).
WP4 activities were organised in the following tasks:
• Guidelines and specifications for low cost music TUI interfaces
• Music TUI Analysis and Rendering
• Music TUI Smart Mapping
WP5
WP5 ensured early adoption of the #MusicBricks APIs, GUIs and TUIs at specially organised events which were designed to stimulate creation of seed ideas. The #MusicBricks project partners organised four core project activity events in order to disseminate the #MusicBricks toolset to the main target stakeholders and to allow for immediate and detailed observation and feedback on their use. Two of these events formed part of Music Tech Fest, an international festival and community of creative innovators; one event was held as part of Music Hack Day in Barcelona; and a fourth 'bonus' event at Waves Vienna.
At each event, hackathon participants have been given access to the #MusicBricks tools as well as to engineers able to answer related questions. The presence of industry partners setting challenges for the hackathons and sponsoring events allowed for projects to be developed that integrate the #MusicBricks tools with other technologies already available on the market. A jury made of consortium members and industry experts selected the ideas which best interpreted the challenges and made good use of the tools. The creators were offered the possibility of entering the #MusicBricks incubation phase, aimed at developing their idea into a market-ready prototype.
The work was organised as follows:
• The Creative Testbed: a series of community co-creativity collaborative disruptive creative engagement events
• Gathering of feedback and generation of new knowledge from the Creative Testbed
• Gathering of impact data and statistics from the pilot events
WP6
The #MusicBricks consortium provided a total of 662 hours in assisting, communicating with, and mentoring the incubatees plus providing valuable support and feedback. This is not including the efforts to set up, prepare and coordinate the Industry Testbed within the #MusicBricks consortium. Additional effort was provided free of charge by numerous external advisors from music tech companies, and music industry players such as labels, and business advisors.
All eleven incubated projects successfully completed the Industry Testbed presenting a final prototype and a video explaining the achievement and its usage and features. The mentorship provided by the consortium and partners helped the projects overcome some communication problems and blocker issues, so that all achieved their goals outlined in the initial roadmap. Several of the projects even progressed beyond their original roadmap and realized a prototype, already exhibiting more features than originally planned for the incubation period.
There was overwhelmingly positive feedback about the incubation process and mentoring, where the incubatees said they both learned and progressed a lot during the incubation period, thanks to the professional mentors. Even though funds were at a micro-funding level, incubatees were tremendously motivated and the results and outcomes are above expectations.
The work was organised as follows:
• Setting up residences (?) and facilitating incubation for the Industry Testbed
• Coordinating wrapper/tangibles to assist incubation development to prototype
• Recording Impact data from the Industry Testbed
WP7
As part of WP7 members of the consortium identified the most prominent business opportunities from ideas which started by innovative seeding with little commercial thought. Through a process of iteration the incubated ideas were guided to develop towards the market, and to a stage where they could be assessed for business potential. Partners advised the teams on licensing issues and market research through workshops and individual consultations, and started to learn more about their possible target group and a potential business model.
It is a very positive sign that industry partners such as Abbey Road, Philips and Native Instruments are interested in future outcomes of #MusicBricks, and that the establishment of #MusicBricks as a Limited Company after the project end will enable industry partners to continue to contribute toolkit components to this open platform.
The work was organised as follows:
• Developing a business strategy for the MI ICT Innovation Value Chain
• Engagement with global MI stakeholders and Global Markets
• Final Market Showcase
The following maps #MusicBricks results to the project objectives:
Objective 1: To leverage the excellence from research and capitalise on the current growth of the EU music technology sector
• Innovative technologies developed in 4 key EU research centres have been made available to innovators during 5 creative events;
• Results from research have been embedded in the growing music technology industry and ported beyond, transversally to other industry sectors.
Objective 2: To ensure early adopter market readiness at the point of large scale trials of the tools and data libraries generated through academic RTD in industry-relevant environments
• IP from research centres was wrapped into Application Programming Interfaces, Graphic User Interfaces and Tangible User Interfaces to enable access by innovative Digital Makers and Content Creators;
• Consortium members mentored creators during Creative and Industry Testbeds, receiving early adopter feedback, improving their tools and collecting new inputs for research;
• The addition of IP from industry exceeded the objectives of the action;
• Additional features and tools were developed by project partners in response to the market feedback.
Objective 3: To capitalise on the creative developer innovation ecosystems generated by the Music Tech Fest, NEM, and Music Hack Day global networks as a result of our initiatives
• #MusicBricks tools and challenges were widely promoted and disseminated to large innovator communities, particularly from the growing Music Tech Fest and Music Hack Day ecosystems, via social media channels, direct newsletters and through hands-on engagement during the events;
• #MusicBricks incubated projects were showcased during the events in order to further promote the tools and expose incubatees to investors and industry stakeholders
• The number of successful prototypes created has exceeded expectations.
• Community growth around the tools was higher than expected.
• Social impacts far exceeded expectations.
Objective 4: To incentivise the seeding of product or application ideas by early adopters of the technologies and in particular by the growing target groups of Digital Makers and Content Creators
• The Consortium Agreement was re-written to include a new area of Innovation IP, and in this way enable the innovators to take ownership of their layer of IP;
• Partners agreed on licensing the tools under licences that supported and encouraged Open Innovation;
• Testbeds gathered knowledge and expertise in one place;
• The availability of the project’s research engineers ensured fast knowledge transfer;
• Micro-funding ensured further prototype development;
• Early exposure to audiences, partners and investors was enabled;
• New business models were incentivised through business mentoring;
• Active brokering of industry and investor partnerships was offered throughout.
Objective 5: To ensure that first creative application prototypes receive adequate resources and technological support to develop robustness and reliability for industry / performance trials
• The resources provided by #MusicBricks allowed incubatees to cover essential expenses and remain focused on developing the prototype;
• Partners provided a total of 662 hours of technical and strategic mentoring ensuring the focus was maintained and any arising challenges received immediate attention;
• The success rate at generating technically advanced prototypes has exceeded expectations.
• Market model development rates are surprisingly high at 100%. The addition of research applications and open models was not originally anticipated.
Objective 6: To support product, application or performance development into market prototype ahead of competition in a timely fashion
• Incubations lasted for a period of 3 to 5 months and ensured development from 11 seed ideas created in 24-hour hackathons to 11 advanced prototypes;
• In one case a patent search was conducted immediately after the launch of the presentable prototype, and the patent has since been filed;
• The speed of the Fast Track to Innovation far exceeded expectations.
Objective 7: To facilitate product, application or performance deployment by providing market-led testbeds and routes to partnerships, private and public funding
• Industry players have assisted selected incubatees in discovering new markets and devising novel business plans;
• 72 potential investors and partners participated in the final market showcase;
• 5 industry technologies have been added to the toolkit
• The numbers of engagements with industry have been higher than expected.
Objective 8: to enable market feedback to inform policy and influence future research directions
• Case studies were generated from interviews with incubatees from the Creative Testbeds, the Industry Testbed and the Final Market Testbed.
• At every stage of the process valuable feedback enabled research teams to re-evaluate their tools, and introduce additional features or upgrades.
• Several partners decided to invest in providing additional resources following market feedback.
• The objective resulted in several scientific publications and academic book chapters despite not being a Research Action.
• The objective had a considerable impact on policy and guidelines already at the beginning of the project. The results of the project have been presented at ICT2015, and have heavily influenced AIOTI recommendations for Innovation Ecosystems and CAF recommendations for Innovation. Guidelines and methodologies developed through this project can be clearly traced in recently adopted EC Innovation policies, as well as in future directions of the H2020 Programme.
As a series of APIs, GUIs and TUIs, the research excellence from top European institutions was brought outside the realm of academia to be integrated with current music technology innovations in the commercial realm in order to create innovative products and applications. The #MusicBricks methodology made the transfer from research knowledge to developer deployment more effective by embedding it within an existing growing multidisciplinary community.
At kick-off, partners planned to deploy 8 tools to the #MusicBricks Creative Testbeds. One of these tools was an innovative new Tangible User Interface for gesture-driven low-latency applications, and 7 were Application Programming Interfaces which wrapped existing tools and technologies together from within the world of music tech research in order to create specific ‘bricks’ that could be deployed in combination with others to make new hybrid applications. The first set of 8 tools was deployed by Month 5 of the project in time for the first Creative Testbed.
By Month 9 of the project, following the rising popularity of the #MusicBricks toolkit among developers and innovators, 2 further APIs were added by industry partners who contributed their IP stating that #MusicBricks added value to integration and deployment of their tools. A further industry IP - a data-driven Graphic User Interface - was released under MIT licence and embedded into the toolkit by Month 10. Several industry requests for addition of their tools were rejected due to their restrictive licences, however by the end of the project a further industry API, an industry GUI, and two further partner tools were added, making the total 15 tools in the #MusicBricks toolkit. Several original tools were amplified with further features in response to the requests from developers and innovators, and an official cloud platform has been accepted for #MusicBricks-driven innovations.
The #MusicBricks toolkit was embedded in the existing innovation ecosystems specifically around the Music Tech Fest and Music Hack Day communities and benefitted from these large and growing ecosystems by accessing some of the top global innovation minds.
Deployment through existing growing networks of creative developers, makers and innovators at Music Tech Fest and Music Hack Day ensured early adoption and engagement with the APIs, GUIs and TUIs to a much greater extent than anticipated. The tools were so popular that partners were unable to fulfil demand at events such as Music Hack Day at the Sonar Festival in Barcelona. The tools were successfully embedded in these communities and capitalised on the substantial numbers that these events attract, but as the tools became popular they also contributed to the overall growth of the innovation community.
The Creative Testbeds at hack events provided fertile ground for the seeding of ideas and attracted a variety of experts from different areas of activity, porting knowledge from music technology research centres to this community, but also joining the dots between the various areas of knowledge that the innovators brought to the testbeds.
In order to encourage the creation of new IP on top of Research IP provided by the research partners, the Consortium Agreement was re-written to include a new area of Innovation IP, and so enabled the innovators to take ownership of their layer of IP. In one case this resulted in a patent search conducted just 4 months after the innovation seed idea. Partners agreed on licensing the tools in ways that supported and encouraged Open Innovation (mostly MIT licences), and commercial licences were offered under fair and reasonable conditions. Fast knowledge transfer was incentivised through the organisation of testbeds and direct engagement from the project’s research engineers. This combined with micro-funding to ensure further prototype development. New business models were incentivised through early exposure to audiences, partners and investors, business mentoring and active brokering of industry and investor partnerships.
Micro-funding ensured that the potential of valuable seed ideas was developed and exploited. However the largest investment was in technological support from #MusicBricks research engineers which ensured fast knowledge transfer and robustness of prototypes. The combination of prompt funding and readily available knowledge support ensured a fast track to innovation with first results registering only 4 months after the first seed ideas.
11 new creative and commercial ideas were established at the Creative Testbeds. These ranged from performance concepts and new musical interfaces to tools for musicians with physical disabilities and gesture-driven collaborative remix applications.
• Dolphin: a wearable device that responds to head movement
• #FindingSomethingBondingSound - an application for audiovisual artists using EEG Brainwave reading technology as well as movement sensors
• Hi Note - a new accessible musical instrument
• GIRD - a gesture-based interactive audio and lighting system that allows audiences to remix, explore and interact with music and lights through dancing and movement
• Sound in Translation - a new way to collaborate and remix live by creating a system that responds intuitively to a perfomer
• Airstrument - an interactive instrument that allows a performer to improvise and create music using hand gestures
• Lightbeat - generates a music visualisation that controls a lighting system live.
• Enboard - an audiovisual experience for skateboarding
• Manuphonia - a wearable instrument and app that creates music from movements and gestures
• Snitch - a Bionic ear to help musicians get in tune and jam with others in the right key and tempo
• Interactive Cube - a physical interface for manipulating sound
The developers of the projects were incentivised to further develop their application or product with microfunding and mentorship from the #MusicBricks programme in an Industry Testbed.
The Industry Testbed lasted for a period of 3 to 5 months and ensured development from the 11 seed ideas created in 24-hour sprints at the Creative Testbeds to 11 advanced prototypes which could be taken to the Final Market Testbed. Industry mentoring, partnerships and investors were engaged long before the Final Market Testbed and in many cases followed the prototype development. Each incubated team benefitted from different links and inputs on a case by case basis.
Partners monitored and conducted regular interviews with incubatees at each of the Testbeds, which resulted in valuable feedback which informed the further development of the toolkit. Following the gathering and analysis of data, research teams were able to re-evaluate their tools, and introduce additional features or upgrades. At the final event for the project, incubated teams presented their products to potential customers and pitched to investors - the final step in bringing the original academic research outputs to market. As a result of the project, the #MusicBricks toolkit and methodology has established a viable route to market, closing the gap between academia and industry, and beyond the life of the funded project, #MusicBricks has been established as a Limited Company and will enable industry partners to continue to contribute toolkit components to this open platform for innovation.
Results of the #MusicBricks project
At the end of 18 months of the project, partners, incubatees, and the wider Innovation Ecosystem which has gathered around the #MusicBricks toolkit can measure the following tangible impacts on industry, the arts, research, IP and policy.
Awards
Project #FindingSomethingBondingSound received the Ars Electronica STARTS Prize Honourable Mention and has been listed in this year's State of the Art for achievements at the crossover between science and art.
Patents
A patent was filed by newly formed Swedish company Sojaner AB on the 11th of May 2016
Newly formed companies
Authors of the #MusicBricks Dolphin Project registered Sojaner AB as a Swedish company
MusicBricks LTD was registered as UK Limited Company with Stromatolite LTD as primary shareholder. This makes MusicBricks LTD a sister company to Music Tech Fest LTD, which has the same primary shareholder, and which was registered as the result of the FP7 MIReS CSA.
Music Tech Fest LTD was invited to open an office in Umeå, Sweden, following the success of the first Creative Testbed which was held there.
Tools in the #MusicBricks toolkit
The number of tools in the #MusicBricks toolkit has increased from the 8 tools originally planned to 15.
• 1 tool is a TUI
• 2 tools are GUIs
• 12 tools are APIs
• 5 tools have been added by industry partners
• 1 tool has been batch tested for commercial production (TUI)
• the tools are supported by the official cloud platform SoundCloud
Products
11 #MusicBricks product prototypes were completed:
• 11 product ideas were taken to market
• 10 product prototypes were showcased in the Market Testbed
• 8 products are tools for performance
• 3 products target the gaming industry in their roadmap
• 1 product is currently in development for the heavy machinery industry
• 1 product targets extreme sports entertainment
• 1 product is being considered for the health sector
• 5 products are being used as testbeds for scientific research
• 3 products have been released as Android apps
• 3 products are presented as Open Source software on GitHub
• 2 products are being developed as tools for education
• 1 product is targeting children’s education
• 1 product is SaaS
• 1 product idea has been filed for patenting (see above)
Exploitation opportunities
Several layers of exploitation opportunities have been identified:
1. Exploitation of the knowledge generated within European research institutions;
2. Exploitation of the #MusicBricks tools themselves;
3. The addition of new bricks to the toolkit from external third parties;
4. Exploitation of the toolkit as a commercially exploitable Innovation Ecosystem;
5. Exploitation of the 11 innovative startup product ideas;
6. Implementation of those products as platforms in themselves; and
7. Exploitation of the Creative Content generated for those platforms.
Social media
The hashtag #MusicBricks registered 5,5 million impacts in Month 18 of the project on Twitter. On Facebook the project has so far received 1249 likes.
Presentations and performances
The project has so far featured in 68 public presentations and performances:
27 by project partners
41 by incubatees
Scientific papers
The project has been published / accepted for publication in 5 peer-reviewed papers:
• A two-page publication (in English) about #MusicBricks was published in ERCIM News 101 by partner TU Wien in collaboration with partner Stromatolite, titled: “MusicBricks: Connecting Digital Creators to the Internet of Music Things”
• A two-page publication (in German) about MusicBricks in the OCG Journal (Journal of the Austrian Computer Society) was published by partner TU Wien, entitled “Klingende Bausteine für die Industrie”
• #MusicBricks is mentioned in the context of “Technology, Citizens and Social Change in the Framework of European Research and Innovation Programmes: Towards a Paradigm Shift” by Marta Arniani of partner Sigma (forthcoming).
• #MusicBricks features in “The Eear: Building a Real-Time MIR-based Instrument From a Hack” by Cárthach Ó Nuanáin, Martin Hermant, Angel Faraldo and Daniel Gomez
• G.Dubus E. Flety, R. Borghesi, F. Bevilacqua, "A tutorial on motion data processing for real-time expressive interaction with media using Inertial Measurement Units”
Book chapters
The project has generated 2 book chapters:
• Dubber, A . 2016. You have 24 hours to invent the future of music: music hacks, playful research and creative innovation. In: Wikström, P. & DeFillippi, R. eds., Business Innovation and Disruption in the Music Industry. Cheltenham: Edward Elgar Publishing. pp 211-228
• F. Bevilacqua, N.Schnell J.Françoise, E. Boyer, D. Schwarz, B. Caramiaux, "Designing Action–Sound Metaphors using Motion Sensing and Descriptor-based Synthesis of Recorded Sound Materials” inThe Routledge‘Companion to Embodied Music Interaction’ SECTION VII: Music interaction technologies and applications.
IP guidelines
The project created new guidelines for IP, by means of introducing the new layer of Innovation IP into the #MusicBricks Consortium Agreement. This document can now be provided as reference for other EU IAs.
Policy guidelines
The project introduced new policy guidelines for Market Adoption Readiness Levels (MARLs) and Innovation Ecosystems:
• Market Adoption Readiness Levels (MARLs) were developed for CAF recommendations for H2020 Programme 2016/17 from Section 2.1.2 of the #MusicBricks project proposal.
• MARLs were adopted in the set of EU recommendations by the Innovation Ecosystems Work Group of the Alliance of Internet of Things Innovation in January 2016.
• MARLs feature as the top 3 recommendations for Innovation in the CAF recommendations for the H2020 Programme 2018-2020. (See Open Innovation potential defined in D7.1)
• #MusicBricks Innovation Methodologies feature in multiple CAF recommendations of the Innovation Working Group for the H2020 Programme 2018-2020.
• #MusicBricks is referenced as best practice in 11 EU presentation documents.
Work performed during the project
#MusicBricks was divided into seven separate work packages that managed each aspect of the process involved in meeting the project's objectives:
• WP1: Management
• WP2: Communication, Dissemination and Exploitation
• WP3: Application Programming Interfaces (APIs)
• WP4: Creative User Interfaces (GUIs/TUIs)
• WP5: Creative Testbed: seeding of ideas through creative engagement
• WP6: Industry Testbed: ideas incubation in industry environments
• WP7: Market Testbed: facilitating routes to global markets
WP1
WP1 was aimed at coordinating consortium members in the smooth implementation and reporting of the activities described in the DoA. The objectives of the work package included administrative tasks such as ensuring the compliance with the Grant Agreement and the related reporting requirements, organisational support for project administration, meeting (?targets) and financial procedure, qualitative and support tasks like facilitating effective communication among partners, and ensuring the reach of project milestones and objectives.
It also included Innovation Coordination to monitor fast deployment of interfaces for use in events, wide dissemination to attract large numbers of innovators to the Testbeds, headhunting and selection of the best innovative minds to take on the testbed challenges, ensuring wide adoption of the tools in testbeds, choice of seed ideas with best potential for prototyping, evolution of prototypes and new business models, timely completion of prototypes and dissemination materials, early connections with industry stakeholders, ongoing support from research teams, and market-ready concepts for a fast track to innovation.
Through WP1 the following tasks were implemented:
• Administrative and financial management
• Consortium and review meetings
• Quality management and reporting
WP2
WP2, led by Stromatolite, focused on Communication, Dissemination and Exploitation of the #MusicBricks project, of the toolkit and of
the incubated teams. From the first creative testbed events, Stromatolite brought in commercial partners and brands to engage with developers and makers by setting hack challenges that incorporated #MusicBricks tools, and invited industry experts to act as judges and mentors for the resulting projects.
Partners arranged for teams to meet with industry mentors face to face in their offices and encouraged mentors to connect incubatees with relevant members of their networks to foster further and faster growth from idea to market-ready realisation. Partners also conducted regular meetings with incubated teams, both in person and via Skype to monitor progress and to coach and guide incubatees. At the hack event at Sonar +D in Barcelona, Stromatolite deployed staff specifically to promote the #MusicBricks tools and encourage their use in the hack projects. Through active engagement with hackers, workshops and promotional activities and printed material including colourful laptop stickers indicating the brick(s) with which the hack team was working, partners were able to encourage much greater participation and uptake of the technologies.
New areas were added by Stromatolite to the final market testbed event to ensure that incubated teams and the toolkit itself were given the best opportunities to reach markets, connect with industry and investment and gain exposure to potential customers and stakeholders - from academia to early adopters. The creation of the #MTFAmplifier business networking and pitch event, the Startup Soundpit ‘trade fair’ area and the dedicated performance slots for #MusicBricks projects at the Music Tech Fest in Berlin, as well as the introduction of a market-facing hack challenge as part of the element14 24-hour Hack Camp ensured the maximum potential and impact for exploitation by the toolkit and the incubated projects.
In addition to the engagement with partners at the testbed events, Stromatolite focused on an intensive cross-media campaign that incorporated traditional press and PR strategies, direct mail, leverage of existing networks within partner organisations and social media promotion and community management.
WP2 activities were organised in the following tasks:
• Project branding and exposure
• Communication strategy
• Participation in external events and publications
• Organisation of dissemination events
• Exploitation
WP3
Work Package 3 focused on enabling transfer of research results to creative developers and innovators in the form of several Application Programming Interfaces. End-users throughout the project were members of the creative communities that participate in prototyping (hacking and making) events. Participants of these events were mostly individuals with general technical and programming skills but not experts in audio or music technologies. Therefore the design of APIs focused on offering a clear and attractive functionality to target users, rather than present all possibilities of research algorithms.
Partners successfully identified and integrated various relevant and complementary technologies from the research partners (TUW, Fraunhofer, UPF) under the same umbrella. All technologies were widely used in the Creative Testbeds organised throughout the project. Feedback from creative users and hackers has allowed the improvement of certain aspects of the tools. For example, several users requested more real-time and easy to run tools.
WP3 activities were organised in the following tasks:
• Wrapping existing tools and technologies together
• Creating specific tool combinations for advanced features
• Extending the frameworks beyond the project
WP4
WP4 was devoted to tangible and graphical interfaces in the context of real-time music interaction. As per WP3, end-users throughout the project were members of the creative communities, musicians and makers involved in hacking events for creating new instruments and objects. Typically, these targeted users are mostly individuals with limited expertise in electronics or in digital signal processing, yet willing to include advanced motion sensors in their prototypes. The goal of this Work Package was thus to facilitate the accessibility and the use of such tangible interfaces in these growing communities of makers, digital artists and musicians.
Partners produced guidelines, hardware and software solutions for the participants of the #MusicBricks events and incubations. Importantly, the project allowed partners to significantly improve their technology offer based on the feedbacks of the various users.
Specifically, the focus of WP4 was the invention of the R-IoT (for Real-time Internet of Things for Music), which became one of the most popular technologies during all the Creative Testbeds and beyond. Based on previous IRCAM interface development on wireless motion sensors, the R-IoT was consolidated as a compact wireless motion platform with specifications tuned for expressivity by focusing on low latency and high sensitivity. Moreover, the project allowed for developing an associated software environment enabling users to easily take advantage of such inertial motion units (accelerometers, gyroscopes, magnetometers).
WP4 activities were organised in the following tasks:
• Guidelines and specifications for low cost music TUI interfaces
• Music TUI Analysis and Rendering
• Music TUI Smart Mapping
WP5
WP5 ensured early adoption of the #MusicBricks APIs, GUIs and TUIs at specially organised events which were designed to stimulate creation of seed ideas. The #MusicBricks project partners organised four core project activity events in order to disseminate the #MusicBricks toolset to the main target stakeholders and to allow for immediate and detailed observation and feedback on their use. Two of these events formed part of Music Tech Fest, an international festival and community of creative innovators; one event was held as part of Music Hack Day in Barcelona; and a fourth 'bonus' event at Waves Vienna.
At each event, hackathon participants have been given access to the #MusicBricks tools as well as to engineers able to answer related questions. The presence of industry partners setting challenges for the hackathons and sponsoring events allowed for projects to be developed that integrate the #MusicBricks tools with other technologies already available on the market. A jury made of consortium members and industry experts selected the ideas which best interpreted the challenges and made good use of the tools. The creators were offered the possibility of entering the #MusicBricks incubation phase, aimed at developing their idea into a market-ready prototype.
The work was organised as follows:
• The Creative Testbed: a series of community co-creativity collaborative disruptive creative engagement events
• Gathering of feedback and generation of new knowledge from the Creative Testbed
• Gathering of impact data and statistics from the pilot events
WP6
The #MusicBricks consortium provided a total of 662 hours in assisting, communicating with, and mentoring the incubatees plus providing valuable support and feedback. This is not including the efforts to set up, prepare and coordinate the Industry Testbed within the #MusicBricks consortium. Additional effort was provided free of charge by numerous external advisors from music tech companies, and music industry players such as labels, and business advisors.
All eleven incubated projects successfully completed the Industry Testbed presenting a final prototype and a video explaining the achievement and its usage and features. The mentorship provided by the consortium and partners helped the projects overcome some communication problems and blocker issues, so that all achieved their goals outlined in the initial roadmap. Several of the projects even progressed beyond their original roadmap and realized a prototype, already exhibiting more features than originally planned for the incubation period.
There was overwhelmingly positive feedback about the incubation process and mentoring, where the incubatees said they both learned and progressed a lot during the incubation period, thanks to the professional mentors. Even though funds were at a micro-funding level, incubatees were tremendously motivated and the results and outcomes are above expectations.
The work was organised as follows:
• Setting up residences (?) and facilitating incubation for the Industry Testbed
• Coordinating wrapper/tangibles to assist incubation development to prototype
• Recording Impact data from the Industry Testbed
WP7
As part of WP7 members of the consortium identified the most prominent business opportunities from ideas which started by innovative seeding with little commercial thought. Through a process of iteration the incubated ideas were guided to develop towards the market, and to a stage where they could be assessed for business potential. Partners advised the teams on licensing issues and market research through workshops and individual consultations, and started to learn more about their possible target group and a potential business model.
It is a very positive sign that industry partners such as Abbey Road, Philips and Native Instruments are interested in future outcomes of #MusicBricks, and that the establishment of #MusicBricks as a Limited Company after the project end will enable industry partners to continue to contribute toolkit components to this open platform.
The work was organised as follows:
• Developing a business strategy for the MI ICT Innovation Value Chain
• Engagement with global MI stakeholders and Global Markets
• Final Market Showcase
The following maps #MusicBricks results to the project objectives:
Objective 1: To leverage the excellence from research and capitalise on the current growth of the EU music technology sector
• Innovative technologies developed in 4 key EU research centres have been made available to innovators during 5 creative events;
• Results from research have been embedded in the growing music technology industry and ported beyond, transversally to other industry sectors.
Objective 2: To ensure early adopter market readiness at the point of large scale trials of the tools and data libraries generated through academic RTD in industry-relevant environments
• IP from research centres was wrapped into Application Programming Interfaces, Graphic User Interfaces and Tangible User Interfaces to enable access by innovative Digital Makers and Content Creators;
• Consortium members mentored creators during Creative and Industry Testbeds, receiving early adopter feedback, improving their tools and collecting new inputs for research;
• The addition of IP from industry exceeded the objectives of the action;
• Additional features and tools were developed by project partners in response to the market feedback.
Objective 3: To capitalise on the creative developer innovation ecosystems generated by the Music Tech Fest, NEM, and Music Hack Day global networks as a result of our initiatives
• #MusicBricks tools and challenges were widely promoted and disseminated to large innovator communities, particularly from the growing Music Tech Fest and Music Hack Day ecosystems, via social media channels, direct newsletters and through hands-on engagement during the events;
• #MusicBricks incubated projects were showcased during the events in order to further promote the tools and expose incubatees to investors and industry stakeholders
• The number of successful prototypes created has exceeded expectations.
• Community growth around the tools was higher than expected.
• Social impacts far exceeded expectations.
Objective 4: To incentivise the seeding of product or application ideas by early adopters of the technologies and in particular by the growing target groups of Digital Makers and Content Creators
• The Consortium Agreement was re-written to include a new area of Innovation IP, and in this way enable the innovators to take ownership of their layer of IP;
• Partners agreed on licensing the tools under licences that supported and encouraged Open Innovation;
• Testbeds gathered knowledge and expertise in one place;
• The availability of the project’s research engineers ensured fast knowledge transfer;
• Micro-funding ensured further prototype development;
• Early exposure to audiences, partners and investors was enabled;
• New business models were incentivised through business mentoring;
• Active brokering of industry and investor partnerships was offered throughout.
Objective 5: To ensure that first creative application prototypes receive adequate resources and technological support to develop robustness and reliability for industry / performance trials
• The resources provided by #MusicBricks allowed incubatees to cover essential expenses and remain focused on developing the prototype;
• Partners provided a total of 662 hours of technical and strategic mentoring ensuring the focus was maintained and any arising challenges received immediate attention;
• The success rate at generating technically advanced prototypes has exceeded expectations.
• Market model development rates are surprisingly high at 100%. The addition of research applications and open models was not originally anticipated.
Objective 6: To support product, application or performance development into market prototype ahead of competition in a timely fashion
• Incubations lasted for a period of 3 to 5 months and ensured development from 11 seed ideas created in 24-hour hackathons to 11 advanced prototypes;
• In one case a patent search was conducted immediately after the launch of the presentable prototype, and the patent has since been filed;
• The speed of the Fast Track to Innovation far exceeded expectations.
Objective 7: To facilitate product, application or performance deployment by providing market-led testbeds and routes to partnerships, private and public funding
• Industry players have assisted selected incubatees in discovering new markets and devising novel business plans;
• 72 potential investors and partners participated in the final market showcase;
• 5 industry technologies have been added to the toolkit
• The numbers of engagements with industry have been higher than expected.
Objective 8: to enable market feedback to inform policy and influence future research directions
• Case studies were generated from interviews with incubatees from the Creative Testbeds, the Industry Testbed and the Final Market Testbed.
• At every stage of the process valuable feedback enabled research teams to re-evaluate their tools, and introduce additional features or upgrades.
• Several partners decided to invest in providing additional resources following market feedback.
• The objective resulted in several scientific publications and academic book chapters despite not being a Research Action.
• The objective had a considerable impact on policy and guidelines already at the beginning of the project. The results of the project have been presented at ICT2015, and have heavily influenced AIOTI recommendations for Innovation Ecosystems and CAF recommendations for Innovation. Guidelines and methodologies developed through this project can be clearly traced in recently adopted EC Innovation policies, as well as in future directions of the H2020 Programme.
Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)
Technical innovation beyond State of the Art
#MusicBricks has provided engineering teams with the opportunity to gather direct feedback from creative users. For the research labs in the consortium specialising in music technology, it has been important to observe and get inputs also from outside the academic community. This type of feedback has been shown to be extremely useful for designing and improving the development of research tools.
An example is the audio analysis library Essentia developed by UPF. This library has been extended and adapted following some feature and other requests by the creative users. For example, the EssentiaRT~ modules (http://mtg.upf.edu/technologies/EssentiaRT~(opens in new window)) which is a real-time implementation of the library, has been widely tested and used in the project hackathons. Additionally, a number of new sound synthesis algorithms have been added after requests from musicians and sound designers who wanted to use the library also to generate and process sounds (https://github.com/MTG/essentia/tree/master/src/algorithms/synthesis(opens in new window)).
The important improvements to the readiness of #MusicBricks' tangible interfaces technology allows for a wide deployment of these technologies in several new communities beyond music technology. Specifically, the possibility of producing low-cost and compact wireless sensors such as the R-IoT enables creators to reach new markets and applications. In particular, these tools allow for new approaches in education, from school to higher education, for computing and science teaching. It is also anticipated that the fields of physical rehabilitation and sport training will benefit from such technologies that can be used to provide useful feedback to users on their movements.
The team at TU Wien translated the Rhythm Patterns music analysis library (http://www.ifs.tuwien.ac.at/mir/musicbricks/index.html#RPextract(opens in new window)) to the popular programming language Python and provided it as an open source project (https://github.com/tuwien-musicir/rp_extract(opens in new window)). Based on the feedback gathered during the incubation process, they optimized the runtime behaviour, enabled processing of live audio and extended the command line interface for improved configurability. To facilitate rapid prototyping, pre-trained models for genre classification have been provided. Further, questions asked by the incubatees have been accumulated into an improved documentation and online tutorials. The integration of the library into the Python Package Index (Pypi) has been prepared and will be completed in near future.
In response to feedback at hack events and by incubatees, Fraunhofer improved the usability and documentation of the provided #MusicBricks tools (http://tinyurl.com/MusicBricksIDMT(opens in new window)). They have also delivered two additional tools to the programme, which fill the gap between the existing tools. The first of these is the Real-Time Pitch Shifting library, a C++ library that allows for the pitch of audio material to be adjusted independently of the tempo. The other is the Goatify tool, an executable that automatically replaces the main melody in a song with a given sample. The tool is delivered with free sound samples (including that of a goat, hence the name) drawn from #MusicBricks partner UPF's Freesound tool (www.freesound.org). Inspired by the results from the testbed hack events, Fraunhofer organised an internal hackathon for engineers and scientists who would not normally interact in this way thus encouraging them to experiment using this methodology and explore novel ideas and possibilities. Fraunhofer has also developed a demo kit containing code samples and practical demonstrations of each of their tools for the use of developers and hackers.
Industry innovation beyond State of the Art
#MusicBricks has innovated a unique methodology of seeding ideas through creative testbeds, incubating through industry testbeds and taking to market through market testbeds. The project has not just demonstrated the efficacy of such a methodology, but has achieved results that exceed expectations.
One of the innovations of #MusicBricks that brings it beyond the State of the Art is that it creates a toolkit from cutting edge research that makes the individual pieces interoperable. By combining the 'bricks' in different ways, new and unanticipated inventions can emerge, resulting in genuinely disruptive innovation that would not have been possible simply by attempting to take a single research output to market.
#MusicBricks has created guidelines for open innovation for business models. By embracing open development as a strategy for product improvement and dissemination, the project has demonstrated ways in which making tools for development available on platforms like GitHub could be leveraged as a commercial strategy for uptake, early user feedback and data.
The project has discovered and documented seven potential layers of exploitation and routes to market. As an innovation ecosystem, the methodology not only fosters disruptive innovation and rapid knowledge transfer, but also creates a unique and entirely new business model with 7 layers:
1. Exploitation of the knowledge generated within European research institutions;
2. Exploitation of the #MusicBricks tools themselves;
3. The addition of new bricks to the toolkit from external third parties;
4. Exploitation of the toolkit as a commercially exploitable Innovation Ecosystem;
5. Exploitation of the 11 innovative startup product ideas;
6. Implementation of those products as platforms in themselves; and
7. Exploitation of the Creative Content generated for those platforms.
#MusicBricks has demonstrated that rather than simply providing a platform for new products in the entertainment sector, using creativity as the starting point for innovation has profound implications across all industry verticals. Inventions developed in a context that focuses on playful experimentation in the world of music and technology, can generate serious and disruptive innovation in other realms.
Social innovation beyond State of the Art
#MusicBricks both represents and fosters the formation of Innovation Ecosystems. By bringing together academic research, creative development, market dissemination and industry expertise, the platform is a unique methodology that considers innovation to be a process that pools assets, resources and understandings.
#MusicBricks has employed an innovative approach to assessing readiness of the technologies, and making the transition from traditional Technology Readiness Levels (TRLs) to Market Adoption Readiness Levels (MARLs) that allow for low-risk technological innovation, rapid market deployment, data and user feedback, and an iterative approach to development that yields powerful results at an early stage of development and reconfigures the relationship between consumers and manufacturers. The feedback loops engage early adopters to help shape the new products and services, leverage social interaction and make use of the expertise of the community that may lie beyond the development team.
In order to incentivise creative developers, the consortium has created an entirely new layer of intellectual property that can be deployed across a range of other projects and programmes. By adding Innovation IP to the existing Background IP and Research IP, projects may take the results of academic research to market in a way that fairly reflects the creative work contributed by the development team of hackers and makers who invent the market-facing application or product from the original research outputs.
The #MusicBricks project situates music as a "social glue" that brings together people with different expertise, specialisms and backgrounds in a context of shared understanding, where otherwise they might not have had occasion to work together or even meet. The methodology calls for complementary skillsets, interdisciplinarity and hands-on problem solving, resulting in rapid knowledge transfer and a common ground for practical collaboration.
Impact on academic and marketing research, and policy makers
Innovation Actions are short and output oriented projects, focusing on activities directly aiming at producing plans and arrangements or designs for new, altered or improved products, processes or services. For this purpose they may include activities such as prototyping, testing, demonstrating, piloting, large-scale product validation and market replication.
The #MusicBricks project has worked as an open innovation framework connecting research, market and policy making: the project activities led not only to the development of sound prototypes leveraging European music technology R&D excellence, but also to the establishment of an Innovation Ecosystem which is made sustainable after the end of the EU grant by the #MusicBricks company and the Music Tech Fest and covers the whole spectrum from research to market and policies, demonstrating how the three cannot operate in silos.
#MusicBricks testbeds provided consortium researchers with valuable feedback from early adopters involved in the testbeds. By making tools developers and adopters part of the same community, #MusicBricks enabled the opening of new R&D perspectives and confirmed assumptions such as the necessity of having tools working in real-time. On the other hand, some of the incubatees were coming from research and their work on prototypes suggested new research directions.
The academic researchers that form the #MusicBricks developer teams are able to place their research into the hands of people who will test it to its limits, and situate theoretical and intellectual results within real world environments that place unanticipated strain on those research outputs. By responding in real time to the needs and ambitions of the participants in the hackathon events, the #MusicBricks team were able to strengthen their tools and make them more flexible, more robust and ultimately more useful within a commercial environment.
The MARLs guidelines and #MusicBricks methodologies have been presented in key policy making arenas such as the AIOTI (Alliance for Internet of Things Innovation) and CAF (CONNECT Advisory Forum), as well as events like ICT2015, the largest ICT conference in Europe, the NEM Summit, European Culture Forum and Net Futures 2016.
Impact on new business models for the music industry
Deployment of the #MusicBricks toolkit, which assembled the technologies produced by research partners, had a considerable uptake by creative developers and a high product yield, but it also created the additional impact as a valuable dissemination and exploitation vehicle for external industry stakeholders who requested to add their IP to the suite. Industry players acknowledged the value of early contact with #MusicBricks adopters, followed closely the evolution of individual projects, and observed with interest the new business models presented at the Final Market Testbed.
Although it is too early to report the long-term market performance of incubated projects, valuable tendencies have been observed during the testbeds: music technology does not necessarily find an application only in the music industry, but can be applied successfully to other sectors and develop alternative business models in a variety of industry verticals. The hands-on and collective components characterising the testbeds suggested the development of products as toolkits and the assignment of an active role to users in evolving the product.
Experimentation with music technologies provides a very good test case for a wide variety of industry pilots. As expressed by Michela Magas (STRO) in her article “7 ingredients to build a successful innovation ecosystem” published as part of this project: “…it thrives on big data; it relies on cloud services; it attracts communities; it provides fast feedback loops for experimentation; it allows for quick prototyping and cheap testing of technology ideas; and it allows to port tested ideas successfully to other industry verticals”.
The tangible value of the project is not only in a growing toolkit of actual tools, or a growing suite of products and creative works, but in the fact that the methodology implemented has identified multiple novel exploitation routes, and can be ported over to other, larger industry testbeds across a wide range of verticals.
The #MusicBricks project proposal made the observation that disruptive and creative engagement with music is inexpensive compared to other media such as film-making or game design. Easy, large-scale access to tools and creative engagement by a mass-market of music makers contributes to a global democratisation of music. It is also a low-risk activity which can result in large scale experiments at early stages of deployment and technology readiness. As a result, music provides an excellent context for creative development and playful experimentation with an immediacy of expression as well as a way of exploring and capitalising on the creative and commercial possibilities inherent in the connection of new tools and methodologies with the new generation of digital makers and content creators.
Despite the seed industry of Music Technology and its key partner creative industries of Gaming and Entertainment, #MusicBricks Transversal Applications now incorporate sectors including Health, Lighting, Communications and Lifestyle though initiatives such as the Philips Transversal Experiment. Furthermore, additional target beneficiaries have emerged such as Transportation, Forestry and Agriculture as #MusicBricks-developed interfaces seek to revolutionise heavy industry sectors. The first #MusicBricks product patent registration specifically targets these sectors, and provides for an interface developed in the context of music technology experimentation to be deployed within the operation of heavy machinery.
What had not been anticipated in the proposal was the extent to which this context was the perfect ground for experimentation that could be extrapolated and deployed transversally. That is, where traditional ‘problem solving’ approaches within an industry result in incremental innovation and technical improvements, experiments with #MusicBricks provided the opportunity for disruptive innovation within industry sectors not ordinarily associated with arts and creativity, by allowing developers to engage in creative experimentation within a low-risk, low-cost and inherently ‘playful’ context.
In so doing, #MusicBricks developed a methodology which unlocked a route for data and emerging tools available as a result of the European research projects and residing within academic institutions, to get into the hands of creative developers and content makers, thereby leveraging and amplifying existing work within Europe to further contribute to the cultural and economic output - not just across the creative sector, but potentially across all industries.
Societal impact: a reinvention of music in the digital age
Members of the #MusicBricks Innovation Ecosystem share a positive attitude towards reciprocal learning and openness which goes beyond profit and competition. In such a framework, industry is an active element of the community, and does not focus solely on exploitation and capitalisation. On the other hand, there are no simple users but adopters and creators who contribute concretely to technology implementation. The community’s playful and open approach is informed by the central role of music as catalyser of talents and technologies. Music becomes thus a social glue bringing together different fields and profiles, and enabling innovations going beyond the music field.
The project found that events that had a higher focus on creativity rather than on technical skills and competence drew a higher proportion of female participants. There appears t
o be a direct correlation between the strongly creative #MTFScandi or the strongly engineering-oriented MHD Barcelona, and the percentage of female innovators present. Creativity is therefore also essential to achieve a good gender balance. While proactive attempts to engage women’s hacker groups and female developers took place across all of the creative testbeds, the Music Tech Fest hack camps were actively promoted as a collaboration between artists and scientists, and with the hack challenges framed around making and handcrafting new physical devices rather than simply around coding. As a result, these testbeds demonstrated a significantly greater gender diversity and the nature of many of the mixed team projects took on a performative and creative aspect rather than simply technical or software-focused. These collected statistics reflect the nature of the Creative Testbed events, particularly the Music Tech Fest's emphasis on creativity, and the Music Hack Day's emphasis on engineering, however both events require a strong presence of creativity at the point of seeding ideas.
The #MusicBricks project has demonstrated clearly that the process of making is itself a pedagogical methodology. By expressing ideas through experimentation with physical computing, wires and software, participants in the Creative Testbed not only showed what they were already capable of creating, but became capable of creating much more because of the rapid knowledge transfer opportunities within the intensive environment of the 24-hour event, and surrounded by engaged, curious and creative developers and artists with complementary skills. By putting their new skills immediately into practice with both a time constraint and an expectation of peer evaluation as the projects are presented on stage at the end of the hackathon event, participants learn quickly and deeply.
The activity of making and inventing within these creative contexts also provides a social glue. It removes barriers and social unease as strangers from incredibly diverse backgrounds work together to solve problems, address challenges and make something that did not exist in the world prior to their coming together and provides a space of common understanding. Bringing experts from widely different disciplines together to interact over music and technology is powerful because it provides a point of connection that allows for collaboration and sharing of ideas across a wide spectrum of technical and creative expertise. The result is an extremely rapid knowledge transfer and a collective growth in intellectual capital. Findings from one field are mapped onto others, acting as multipliers and seed ground for new kinds of ideas that can be put into practice quickly, rather than simply an additive process of skill development. Not only are new concepts and skills passed on between participants and from the project team, but that knowledge is then immediately applied, cementing that knowledge and becoming part of the participants’ core approach.
Overall impact: #MusicBricks methodology as a template for industry innovation
Since the early days of the internet, the most profound and problematic changes experienced by the commercial sector have had immediate effects on the music industries. For this reason, the music industries provide a model for ways in which industries can adapt and respond to a changing technological environment. The sector has not always responded well and many mistakes have been made along the way. In recent years, the impulse to resist innovation and change has been supplanted by the need to take the lead in this area, and the wider economy is paying close attention to experiments and developments in the music and media sectors. As a methodology, the #MusicBricks project provides an instructive template for genuine disruptive innovation for the music industries and beyond. Rather than merely 'solve problems' - an approach that can lead to incremental innovation or technological improvement, the creative experimentation of the hackathon provides the context for disruptive innovation.
While the abstract concepts contained within the open-ended hack challenges provide a seed for ideas, the purpose of the creative hackathon event is to create actual concrete projects and products. Those projects need not necessarily be intended as music business enterprises (and, in fact, seldom are considered in that way). However, as a piece of communication in response to a provocation and the embodied expression of an idea that may or may not ultimately become codified in a business plan, the experimental 'hack' is not simply an intellectual game, but a process of making things and putting them into the world. It is a material practice - and an act of thinking out loud, embodied in physical and working objects.
The opportunity for new business innovation is not always clear in this space – though that it exists is not disputed. Projects that are developed in fun or to explore a particular creative provocation may contain within it the seed of an idea that could revolutionise the way in which fans interact with music performance or it might contain a process that may be applied transversally to other sectors.
It can be observed from the #MusicBricks case studies that the mere act of experimentation contains within it the opportunity for unexpected discoveries and unintended consequences. These discoveries can provide the starting point for new business opportunities as yet unanticipated. The seed project provides a starting point for new ideas based on existing tools, materials, systems, processes and repertoire. Providing access to these building blocks facilitates innovation. Participation in the process of innovation need not (and perhaps should not) take place within the confines of an existing industry organisation. Access to new, simple and interconnecting tools such as the #MusicBricks toolkit, coupled with a range of different literacies of digital making such as those shared within ad hoc teams that form within creative hackathon events, provides a seeding ground for innovation and enterprise, and these ingredients seldom lie exclusively within the domain of a single individual or organisation.
Participants in creative hackathon events do not, by and large, set out to invent the future of music business – but through repeated experimentation, repurposing, rebuilding and playing with the TUIs, GUIs and APIs, code, electronics and content that provide the building blocks of these hackathon events – and supplied with open challenges that engage their imagination – music hackers are statistically far more likely to stumble upon the key to a new industry opportunity than is an employee of a corporation charged with the task of simply improving what already exists.
#MusicBricks has provided engineering teams with the opportunity to gather direct feedback from creative users. For the research labs in the consortium specialising in music technology, it has been important to observe and get inputs also from outside the academic community. This type of feedback has been shown to be extremely useful for designing and improving the development of research tools.
An example is the audio analysis library Essentia developed by UPF. This library has been extended and adapted following some feature and other requests by the creative users. For example, the EssentiaRT~ modules (http://mtg.upf.edu/technologies/EssentiaRT~(opens in new window)) which is a real-time implementation of the library, has been widely tested and used in the project hackathons. Additionally, a number of new sound synthesis algorithms have been added after requests from musicians and sound designers who wanted to use the library also to generate and process sounds (https://github.com/MTG/essentia/tree/master/src/algorithms/synthesis(opens in new window)).
The important improvements to the readiness of #MusicBricks' tangible interfaces technology allows for a wide deployment of these technologies in several new communities beyond music technology. Specifically, the possibility of producing low-cost and compact wireless sensors such as the R-IoT enables creators to reach new markets and applications. In particular, these tools allow for new approaches in education, from school to higher education, for computing and science teaching. It is also anticipated that the fields of physical rehabilitation and sport training will benefit from such technologies that can be used to provide useful feedback to users on their movements.
The team at TU Wien translated the Rhythm Patterns music analysis library (http://www.ifs.tuwien.ac.at/mir/musicbricks/index.html#RPextract(opens in new window)) to the popular programming language Python and provided it as an open source project (https://github.com/tuwien-musicir/rp_extract(opens in new window)). Based on the feedback gathered during the incubation process, they optimized the runtime behaviour, enabled processing of live audio and extended the command line interface for improved configurability. To facilitate rapid prototyping, pre-trained models for genre classification have been provided. Further, questions asked by the incubatees have been accumulated into an improved documentation and online tutorials. The integration of the library into the Python Package Index (Pypi) has been prepared and will be completed in near future.
In response to feedback at hack events and by incubatees, Fraunhofer improved the usability and documentation of the provided #MusicBricks tools (http://tinyurl.com/MusicBricksIDMT(opens in new window)). They have also delivered two additional tools to the programme, which fill the gap between the existing tools. The first of these is the Real-Time Pitch Shifting library, a C++ library that allows for the pitch of audio material to be adjusted independently of the tempo. The other is the Goatify tool, an executable that automatically replaces the main melody in a song with a given sample. The tool is delivered with free sound samples (including that of a goat, hence the name) drawn from #MusicBricks partner UPF's Freesound tool (www.freesound.org). Inspired by the results from the testbed hack events, Fraunhofer organised an internal hackathon for engineers and scientists who would not normally interact in this way thus encouraging them to experiment using this methodology and explore novel ideas and possibilities. Fraunhofer has also developed a demo kit containing code samples and practical demonstrations of each of their tools for the use of developers and hackers.
Industry innovation beyond State of the Art
#MusicBricks has innovated a unique methodology of seeding ideas through creative testbeds, incubating through industry testbeds and taking to market through market testbeds. The project has not just demonstrated the efficacy of such a methodology, but has achieved results that exceed expectations.
One of the innovations of #MusicBricks that brings it beyond the State of the Art is that it creates a toolkit from cutting edge research that makes the individual pieces interoperable. By combining the 'bricks' in different ways, new and unanticipated inventions can emerge, resulting in genuinely disruptive innovation that would not have been possible simply by attempting to take a single research output to market.
#MusicBricks has created guidelines for open innovation for business models. By embracing open development as a strategy for product improvement and dissemination, the project has demonstrated ways in which making tools for development available on platforms like GitHub could be leveraged as a commercial strategy for uptake, early user feedback and data.
The project has discovered and documented seven potential layers of exploitation and routes to market. As an innovation ecosystem, the methodology not only fosters disruptive innovation and rapid knowledge transfer, but also creates a unique and entirely new business model with 7 layers:
1. Exploitation of the knowledge generated within European research institutions;
2. Exploitation of the #MusicBricks tools themselves;
3. The addition of new bricks to the toolkit from external third parties;
4. Exploitation of the toolkit as a commercially exploitable Innovation Ecosystem;
5. Exploitation of the 11 innovative startup product ideas;
6. Implementation of those products as platforms in themselves; and
7. Exploitation of the Creative Content generated for those platforms.
#MusicBricks has demonstrated that rather than simply providing a platform for new products in the entertainment sector, using creativity as the starting point for innovation has profound implications across all industry verticals. Inventions developed in a context that focuses on playful experimentation in the world of music and technology, can generate serious and disruptive innovation in other realms.
Social innovation beyond State of the Art
#MusicBricks both represents and fosters the formation of Innovation Ecosystems. By bringing together academic research, creative development, market dissemination and industry expertise, the platform is a unique methodology that considers innovation to be a process that pools assets, resources and understandings.
#MusicBricks has employed an innovative approach to assessing readiness of the technologies, and making the transition from traditional Technology Readiness Levels (TRLs) to Market Adoption Readiness Levels (MARLs) that allow for low-risk technological innovation, rapid market deployment, data and user feedback, and an iterative approach to development that yields powerful results at an early stage of development and reconfigures the relationship between consumers and manufacturers. The feedback loops engage early adopters to help shape the new products and services, leverage social interaction and make use of the expertise of the community that may lie beyond the development team.
In order to incentivise creative developers, the consortium has created an entirely new layer of intellectual property that can be deployed across a range of other projects and programmes. By adding Innovation IP to the existing Background IP and Research IP, projects may take the results of academic research to market in a way that fairly reflects the creative work contributed by the development team of hackers and makers who invent the market-facing application or product from the original research outputs.
The #MusicBricks project situates music as a "social glue" that brings together people with different expertise, specialisms and backgrounds in a context of shared understanding, where otherwise they might not have had occasion to work together or even meet. The methodology calls for complementary skillsets, interdisciplinarity and hands-on problem solving, resulting in rapid knowledge transfer and a common ground for practical collaboration.
Impact on academic and marketing research, and policy makers
Innovation Actions are short and output oriented projects, focusing on activities directly aiming at producing plans and arrangements or designs for new, altered or improved products, processes or services. For this purpose they may include activities such as prototyping, testing, demonstrating, piloting, large-scale product validation and market replication.
The #MusicBricks project has worked as an open innovation framework connecting research, market and policy making: the project activities led not only to the development of sound prototypes leveraging European music technology R&D excellence, but also to the establishment of an Innovation Ecosystem which is made sustainable after the end of the EU grant by the #MusicBricks company and the Music Tech Fest and covers the whole spectrum from research to market and policies, demonstrating how the three cannot operate in silos.
#MusicBricks testbeds provided consortium researchers with valuable feedback from early adopters involved in the testbeds. By making tools developers and adopters part of the same community, #MusicBricks enabled the opening of new R&D perspectives and confirmed assumptions such as the necessity of having tools working in real-time. On the other hand, some of the incubatees were coming from research and their work on prototypes suggested new research directions.
The academic researchers that form the #MusicBricks developer teams are able to place their research into the hands of people who will test it to its limits, and situate theoretical and intellectual results within real world environments that place unanticipated strain on those research outputs. By responding in real time to the needs and ambitions of the participants in the hackathon events, the #MusicBricks team were able to strengthen their tools and make them more flexible, more robust and ultimately more useful within a commercial environment.
The MARLs guidelines and #MusicBricks methodologies have been presented in key policy making arenas such as the AIOTI (Alliance for Internet of Things Innovation) and CAF (CONNECT Advisory Forum), as well as events like ICT2015, the largest ICT conference in Europe, the NEM Summit, European Culture Forum and Net Futures 2016.
Impact on new business models for the music industry
Deployment of the #MusicBricks toolkit, which assembled the technologies produced by research partners, had a considerable uptake by creative developers and a high product yield, but it also created the additional impact as a valuable dissemination and exploitation vehicle for external industry stakeholders who requested to add their IP to the suite. Industry players acknowledged the value of early contact with #MusicBricks adopters, followed closely the evolution of individual projects, and observed with interest the new business models presented at the Final Market Testbed.
Although it is too early to report the long-term market performance of incubated projects, valuable tendencies have been observed during the testbeds: music technology does not necessarily find an application only in the music industry, but can be applied successfully to other sectors and develop alternative business models in a variety of industry verticals. The hands-on and collective components characterising the testbeds suggested the development of products as toolkits and the assignment of an active role to users in evolving the product.
Experimentation with music technologies provides a very good test case for a wide variety of industry pilots. As expressed by Michela Magas (STRO) in her article “7 ingredients to build a successful innovation ecosystem” published as part of this project: “…it thrives on big data; it relies on cloud services; it attracts communities; it provides fast feedback loops for experimentation; it allows for quick prototyping and cheap testing of technology ideas; and it allows to port tested ideas successfully to other industry verticals”.
The tangible value of the project is not only in a growing toolkit of actual tools, or a growing suite of products and creative works, but in the fact that the methodology implemented has identified multiple novel exploitation routes, and can be ported over to other, larger industry testbeds across a wide range of verticals.
The #MusicBricks project proposal made the observation that disruptive and creative engagement with music is inexpensive compared to other media such as film-making or game design. Easy, large-scale access to tools and creative engagement by a mass-market of music makers contributes to a global democratisation of music. It is also a low-risk activity which can result in large scale experiments at early stages of deployment and technology readiness. As a result, music provides an excellent context for creative development and playful experimentation with an immediacy of expression as well as a way of exploring and capitalising on the creative and commercial possibilities inherent in the connection of new tools and methodologies with the new generation of digital makers and content creators.
Despite the seed industry of Music Technology and its key partner creative industries of Gaming and Entertainment, #MusicBricks Transversal Applications now incorporate sectors including Health, Lighting, Communications and Lifestyle though initiatives such as the Philips Transversal Experiment. Furthermore, additional target beneficiaries have emerged such as Transportation, Forestry and Agriculture as #MusicBricks-developed interfaces seek to revolutionise heavy industry sectors. The first #MusicBricks product patent registration specifically targets these sectors, and provides for an interface developed in the context of music technology experimentation to be deployed within the operation of heavy machinery.
What had not been anticipated in the proposal was the extent to which this context was the perfect ground for experimentation that could be extrapolated and deployed transversally. That is, where traditional ‘problem solving’ approaches within an industry result in incremental innovation and technical improvements, experiments with #MusicBricks provided the opportunity for disruptive innovation within industry sectors not ordinarily associated with arts and creativity, by allowing developers to engage in creative experimentation within a low-risk, low-cost and inherently ‘playful’ context.
In so doing, #MusicBricks developed a methodology which unlocked a route for data and emerging tools available as a result of the European research projects and residing within academic institutions, to get into the hands of creative developers and content makers, thereby leveraging and amplifying existing work within Europe to further contribute to the cultural and economic output - not just across the creative sector, but potentially across all industries.
Societal impact: a reinvention of music in the digital age
Members of the #MusicBricks Innovation Ecosystem share a positive attitude towards reciprocal learning and openness which goes beyond profit and competition. In such a framework, industry is an active element of the community, and does not focus solely on exploitation and capitalisation. On the other hand, there are no simple users but adopters and creators who contribute concretely to technology implementation. The community’s playful and open approach is informed by the central role of music as catalyser of talents and technologies. Music becomes thus a social glue bringing together different fields and profiles, and enabling innovations going beyond the music field.
The project found that events that had a higher focus on creativity rather than on technical skills and competence drew a higher proportion of female participants. There appears t
o be a direct correlation between the strongly creative #MTFScandi or the strongly engineering-oriented MHD Barcelona, and the percentage of female innovators present. Creativity is therefore also essential to achieve a good gender balance. While proactive attempts to engage women’s hacker groups and female developers took place across all of the creative testbeds, the Music Tech Fest hack camps were actively promoted as a collaboration between artists and scientists, and with the hack challenges framed around making and handcrafting new physical devices rather than simply around coding. As a result, these testbeds demonstrated a significantly greater gender diversity and the nature of many of the mixed team projects took on a performative and creative aspect rather than simply technical or software-focused. These collected statistics reflect the nature of the Creative Testbed events, particularly the Music Tech Fest's emphasis on creativity, and the Music Hack Day's emphasis on engineering, however both events require a strong presence of creativity at the point of seeding ideas.
The #MusicBricks project has demonstrated clearly that the process of making is itself a pedagogical methodology. By expressing ideas through experimentation with physical computing, wires and software, participants in the Creative Testbed not only showed what they were already capable of creating, but became capable of creating much more because of the rapid knowledge transfer opportunities within the intensive environment of the 24-hour event, and surrounded by engaged, curious and creative developers and artists with complementary skills. By putting their new skills immediately into practice with both a time constraint and an expectation of peer evaluation as the projects are presented on stage at the end of the hackathon event, participants learn quickly and deeply.
The activity of making and inventing within these creative contexts also provides a social glue. It removes barriers and social unease as strangers from incredibly diverse backgrounds work together to solve problems, address challenges and make something that did not exist in the world prior to their coming together and provides a space of common understanding. Bringing experts from widely different disciplines together to interact over music and technology is powerful because it provides a point of connection that allows for collaboration and sharing of ideas across a wide spectrum of technical and creative expertise. The result is an extremely rapid knowledge transfer and a collective growth in intellectual capital. Findings from one field are mapped onto others, acting as multipliers and seed ground for new kinds of ideas that can be put into practice quickly, rather than simply an additive process of skill development. Not only are new concepts and skills passed on between participants and from the project team, but that knowledge is then immediately applied, cementing that knowledge and becoming part of the participants’ core approach.
Overall impact: #MusicBricks methodology as a template for industry innovation
Since the early days of the internet, the most profound and problematic changes experienced by the commercial sector have had immediate effects on the music industries. For this reason, the music industries provide a model for ways in which industries can adapt and respond to a changing technological environment. The sector has not always responded well and many mistakes have been made along the way. In recent years, the impulse to resist innovation and change has been supplanted by the need to take the lead in this area, and the wider economy is paying close attention to experiments and developments in the music and media sectors. As a methodology, the #MusicBricks project provides an instructive template for genuine disruptive innovation for the music industries and beyond. Rather than merely 'solve problems' - an approach that can lead to incremental innovation or technological improvement, the creative experimentation of the hackathon provides the context for disruptive innovation.
While the abstract concepts contained within the open-ended hack challenges provide a seed for ideas, the purpose of the creative hackathon event is to create actual concrete projects and products. Those projects need not necessarily be intended as music business enterprises (and, in fact, seldom are considered in that way). However, as a piece of communication in response to a provocation and the embodied expression of an idea that may or may not ultimately become codified in a business plan, the experimental 'hack' is not simply an intellectual game, but a process of making things and putting them into the world. It is a material practice - and an act of thinking out loud, embodied in physical and working objects.
The opportunity for new business innovation is not always clear in this space – though that it exists is not disputed. Projects that are developed in fun or to explore a particular creative provocation may contain within it the seed of an idea that could revolutionise the way in which fans interact with music performance or it might contain a process that may be applied transversally to other sectors.
It can be observed from the #MusicBricks case studies that the mere act of experimentation contains within it the opportunity for unexpected discoveries and unintended consequences. These discoveries can provide the starting point for new business opportunities as yet unanticipated. The seed project provides a starting point for new ideas based on existing tools, materials, systems, processes and repertoire. Providing access to these building blocks facilitates innovation. Participation in the process of innovation need not (and perhaps should not) take place within the confines of an existing industry organisation. Access to new, simple and interconnecting tools such as the #MusicBricks toolkit, coupled with a range of different literacies of digital making such as those shared within ad hoc teams that form within creative hackathon events, provides a seeding ground for innovation and enterprise, and these ingredients seldom lie exclusively within the domain of a single individual or organisation.
Participants in creative hackathon events do not, by and large, set out to invent the future of music business – but through repeated experimentation, repurposing, rebuilding and playing with the TUIs, GUIs and APIs, code, electronics and content that provide the building blocks of these hackathon events – and supplied with open challenges that engage their imagination – music hackers are statistically far more likely to stumble upon the key to a new industry opportunity than is an employee of a corporation charged with the task of simply improving what already exists.