Audio Commons: An Ecosystem for Creative Reuse of Audio Content

Project Information

AudioCommons

Grant agreement ID: 688382

DOI

10.3030/688382

Project closed

EC signature date 28 October 2015

Start date 1 February 2016

End date 31 January 2019

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 2 979 055,00

EU contribution

€ 2 979 055,00

Coordinated by

UNIVERSIDAD POMPEU FABRA
Spain

Periodic Reporting for period 2 - AudioCommons (Audio Commons: An Ecosystem for Creative Reuse of Audio Content)

Reporting period: 2017-08-01 to 2019-01-31

Summary of the context and overall objectives of the project

The digital value chain that connects content creators and media industries follows a traditional model that has not yet been adapted to the modern communication society. The democratisation of multimedia content creation has changed the way in which multimedia content is created, shared and (re)used all over the world. Online multimedia sharing has experienced a huge growth in the last decade, yielding significant amounts of user-generated multimedia resources shared under open licenses. At the same time, media industries need to reduce production costs in order to remain competitive. There is, therefore, an opportunity for media industries to incorporate such content into their productions, but there is a lack of technologies for easily accessing and incorporating that type of content in their creative workflows. Particularly in the case of sound and music, a huge amount of audio material is available and released under Creative Commons (CC) licenses. The aim of the AudioCommons (AC) project has been to bring CC licensed audio content to the creative industries, enabling its creation, access, retrieval and reuse in innovative ways. The key objectives were defined as follows:

- Enable the retrieval of audio content in innovative ways by bringing current state-of-the-art methods of semantic sound and music description to higher standards, and make these methods available as software packages.
- Develop and deploy the technological layer to allow the interconnection of all stakeholders that participate in an ecosystem of content, users, services and tools that create and reuse CC audio content (i.e. the Audio Commons Ecosystem or ACE).
- Create and set up the ACE, publish CC audio content through it and build tools that can consume the content and be embedded in existing creative workflows.
- Define standard procedures for new stakeholders to join and participate in the ecosystem, fostering its future growth and sustainability.

During the three years in which the action has taken place, several contributions have been made in relation to all four key objectives. Some details of these contributions are given in the next section. Further details are included in the deliverables published throughout the project and in the final report. In summary, the four key AC objectives have been met. AC has set the foundations for an open ecosystem of content, users and tools around CC audio, and made recommendations for the future growth of such an ecosystem.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

ince the beginning of the project we have carried out work addressing different core ideas related of AC. We have implemented and deployed two prototypes of the Audio Commons Mediator, which facilitate the interconnection of services in the Audio Commons Ecosystem. The second mediator prototype makes extensive use of the Audio Commons Ontology that was also developed in the project and which allows interoperability betweens services. Also related to core AC technology, we carried out research on algorithms for the automatic annotation of music pieces, music samples and also non-musical content (including novel descriptors for perceptual timbral models), and published the outputs as open source reusable tools and an online audio analysis service.

To evaluate and demonstrate the aforementioned technology developments and the concept of the ACE, a number of prototypes have been developed that can consume the content and be embedded in existing creative workflows. This includes 3 tools from industry partners (two of them to be released commercially) and more than 10 prototypes developed by academic partners and external collaborators. Most of the prototypes have also been released under open source licenses.

In addition, we have carried out work on describing the intellectual property rights management requirements that arise from the ACE as well as explored potential business models for new actors to join the ecosystem and made recommendations for its future growth and sustainability.

The work carried out has been widely disseminated in the academia, the creative industries and to the general public. We have published more than 70 research papers, contributed to more than 30 academic and industry events, organized workshops, performances and had online presence through the website and social media.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

One of the main goals of AC is to make CC audio more easily accessible in existing creative workflows. In that regard, we have developed several tools and prototypes which allow to directly search and retrieve CC content within existing creative workflows (e.g. audio plug-ins in a Digital Audio Workstation). Furthermore, the deployed Audio Commons Mediator prototypes facilitate retrieval of CC content from different content providers by exposing a single entry point that aggregates content and presents it under a common metadata scheme (which follows the Audio Commons Ontology) and API. This currently includes audio content from Jamendo, Freesound and Europeana. All together this represents a clear improvement beyond state-of-the-art for integrating CC content in creative workflows.

We identified that one of the main issues that prevent CC licenses to be more extensively used within the creative industries is the unclearness of the licensing procedures. We worked on clarifying these issues by writing documentation which describe the application of CC licenses to the audio domain, and by integrating features in the Audio Commons Mediator which facilitate licensing procedures. For example, using the Audio Commons Mediator a Digital Audio Workstation can automatically point the user to a URL for getting commercial licenses of CC content retrieved from Jamendo. This is an improvement with respect to the state-of-the-art. Nevertheless, we also envisioned a tighter integration of licensing procedures that would be based on blockchain and smart contracts. The implementation of such advanced system was not possible in the scope of the project, but we've made recommendations for the next steps of the future development of such a system.

Another important goal of the AC project is the research and development of tools for the automatic annotation of audio content. In that regard we have developed and made available a number of tools which allow the analysis of audio content and output the results using metadata properties of the Audio Commons Ontology. These tools can automatically extract musical properties such as tempo, tonality, instrumentation (and others) with state-of-the-art accuracies. Nevertheless, when combined with the confidence metrics for these properties also developed as part of AC, the accuracies improve significantly with respect to state-of-the-art. In fact, the focus confidence measures has been a novel idea introduced by the AC consortium in the academic community. Moreover, we also developed a tool for the automatic extraction of perceptual timbre characteristics (e.g. brightness, warmth, sharpness, etc.) which provides completely new ways to browse CC content and constitutes significant progress beyond state-of-the-art as no similar tools were available to date.

All contributions together represent significant steps that foster the reuse of CC content by making it more accessible and integrated, by clarifying and facilitating licensing procedures, and by providing open tools to automatically generate meaningful and reliable audio metadata.

audio-commons-logo-horizontal.jpg

audio-commons-diagram.png