Periodic Reporting for period 2 - SAUCE (Smart Asset re-Use in Creative Environments)
Berichtszeitraum: 2019-07-01 bis 2020-12-31
The goal of SAUCE is to produce, pilot and demonstrate a set of professional tools and techniques that reduce the costs for the production of enhanced digital content for the creative industries by increasing the potential for re-purposing and re-use of content as well as providing significantly improved technologies for digital content production and management. SAUCE will contribute to make content “smarter” by making it fully adaptive, in a broad, unprecedented manner: adaptive to context (which facilitates re-use), to purpose (among or within industries), to the user (improving the viewing experience), and to the production environment (so that it’s “future-proof”). This smart content will allow companies to easily re-use it, share it with other companies, or provide it for use in other sectors.
Specifically, SAUCE has the following definite and measurable objectives:
- Create tools to unlock value in previously-created content by allowing to edit or automatically adapt its properties (e.g. appearance, scale, motion) to the current production needs.
- Create tools to allow content created in the future to be more easily re-used and re-purposed, by developing light-field technology for the creative industries in terms of capture, storage, distribution and processing.
- Create management, animation and production tools to keep digital content accessible, discoverable and malleable by using procedural techniques and high level semantic knowledge.
Achieving these three aims will allow companies to work much more cost-effectively and efficiently. It will also open up the possibility of sharing assets with other sectors – especially those that are taking up interest in virtual production or AR/VR for various scenarios where content creation can be a barrier due to its prohibitive cost.
Deliverable production has been satisfactory: almost all deliverables have been produced on time, are well written and professionally presented.
Scenario Requirements helped identify the key use cases and scenarios for a smart search and transformation framework. DNEG produced a report (D2.3) which documented their findings and proposed a new generalised asset descriptor that would provide the basis and foundation for subsequent work packages.
The primary goals of New Technologies for Asset Creation have been progressively accomplished during the first half tenure of the project. USAAR with inputs from both internal and external experts have done very successful lightfields productions, ‘Elements’ and ‘Unfolding’. The ‘Elements’ shoot was the very first shoot done within the SAUCE project with unstructured content (fire and smoke). ‘Unfolding’ assets include different versions of the voices of a cello composition written and performed by a professional cello player, specifically for this project.
Unlocking Existing Assets: Taking the findings from WP2, DNEG delivered a working, fully functional, proof of concept for the smart search framework, which included data architecture, technical architecture and a number of reference implementations.
In Transforming Assets, DNEG focused on two main areas. The first was developing and demonstrating an extendable and composable transformation framework, which like the search framework describe in WP4, provides a means for transformation services and capabilities to be advertised to the search framework and chained together. The second focused on developing tools to alter of animation timing and trajectory. The concepts used to develop these tools provided a framework which was extended to support asset synthesis and the synthesis of new material.
Semantic Animation Production: The common theme of the tasks within WP6 revolve around methodology to generate ‘smart assets’ and unlock existing assets for re-use furthering the state-of-the-art within the animation industry. To meet these goals, months 1 to 18 within the SAUCE project have seen the completion of one task (WP6T2) and the on-going development of four tasks (WP6T1/WP6T3/WP6T5/WP6T6) to satisfy the goals outlined in the SAUCE project.
Intelligent Storage of Assets primarily focuses on the creation of a type agnostic and geolocatable asset store. The primary work completed within this WP is the separation of the front and back end of Flix and to enable the backend server to be used as a stand alone application. This will form the base of the asset store. As part of this separation and refactoring we created a rich api to allow for extensions/plugins to allow it to house the partners work.
For Experimental Production, Evaluation and Innovation Assessment, Filmakademie planned, prepared and realized a lightfield production under realistic movie production conditions. The ‘Unfolding’ production was shot in a professional studio at Saarländischer Rundfunk in cooperation with the Director of Photography (DoP) Matthias Bolliger. In the preparation phase, a virtual previsualization was created. The developed 3D scene was used to plan shots and timings as well as the overall look. In addition, the material was used to test algorithms developed within SAUCE even before the actual footage was captured. After the shoot in January 2019, the postproduction was carried out at Filmakademie.