Periodic Reporting for period 2 - AudioCommons (Audio Commons: An Ecosystem for Creative Reuse of Audio Content)
Reporting period: 2017-08-01 to 2019-01-31
- Enable the retrieval of audio content in innovative ways by bringing current state-of-the-art methods of semantic sound and music description to higher standards, and make these methods available as software packages.
- Develop and deploy the technological layer to allow the interconnection of all stakeholders that participate in an ecosystem of content, users, services and tools that create and reuse CC audio content (i.e. the Audio Commons Ecosystem or ACE).
- Create and set up the ACE, publish CC audio content through it and build tools that can consume the content and be embedded in existing creative workflows.
- Define standard procedures for new stakeholders to join and participate in the ecosystem, fostering its future growth and sustainability.
During the three years in which the action has taken place, several contributions have been made in relation to all four key objectives. Some details of these contributions are given in the next section. Further details are included in the deliverables published throughout the project and in the final report. In summary, the four key AC objectives have been met. AC has set the foundations for an open ecosystem of content, users and tools around CC audio, and made recommendations for the future growth of such an ecosystem.
To evaluate and demonstrate the aforementioned technology developments and the concept of the ACE, a number of prototypes have been developed that can consume the content and be embedded in existing creative workflows. This includes 3 tools from industry partners (two of them to be released commercially) and more than 10 prototypes developed by academic partners and external collaborators. Most of the prototypes have also been released under open source licenses.
In addition, we have carried out work on describing the intellectual property rights management requirements that arise from the ACE as well as explored potential business models for new actors to join the ecosystem and made recommendations for its future growth and sustainability.
The work carried out has been widely disseminated in the academia, the creative industries and to the general public. We have published more than 70 research papers, contributed to more than 30 academic and industry events, organized workshops, performances and had online presence through the website and social media.
We identified that one of the main issues that prevent CC licenses to be more extensively used within the creative industries is the unclearness of the licensing procedures. We worked on clarifying these issues by writing documentation which describe the application of CC licenses to the audio domain, and by integrating features in the Audio Commons Mediator which facilitate licensing procedures. For example, using the Audio Commons Mediator a Digital Audio Workstation can automatically point the user to a URL for getting commercial licenses of CC content retrieved from Jamendo. This is an improvement with respect to the state-of-the-art. Nevertheless, we also envisioned a tighter integration of licensing procedures that would be based on blockchain and smart contracts. The implementation of such advanced system was not possible in the scope of the project, but we've made recommendations for the next steps of the future development of such a system.
Another important goal of the AC project is the research and development of tools for the automatic annotation of audio content. In that regard we have developed and made available a number of tools which allow the analysis of audio content and output the results using metadata properties of the Audio Commons Ontology. These tools can automatically extract musical properties such as tempo, tonality, instrumentation (and others) with state-of-the-art accuracies. Nevertheless, when combined with the confidence metrics for these properties also developed as part of AC, the accuracies improve significantly with respect to state-of-the-art. In fact, the focus confidence measures has been a novel idea introduced by the AC consortium in the academic community. Moreover, we also developed a tool for the automatic extraction of perceptual timbre characteristics (e.g. brightness, warmth, sharpness, etc.) which provides completely new ways to browse CC content and constitutes significant progress beyond state-of-the-art as no similar tools were available to date.
All contributions together represent significant steps that foster the reuse of CC content by making it more accessible and integrated, by clarifying and facilitating licensing procedures, and by providing open tools to automatically generate meaningful and reliable audio metadata.