Promoting Creativeness in Augmented Media Services

Project Information

ProCAMS

Grant agreement ID: 644460

DOI

10.3030/644460

Project closed

EC signature date 12 December 2014

Start date 1 March 2015

End date 30 November 2016

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 1 311 540,25

EU contribution

€ 996 847,00

Coordinated by

FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
Germany

Periodic Reporting for period 1 - ProCAMS (Promoting Creativeness in Augmented Media Services)

Reporting period: 2015-03-01 to 2016-11-30

Summary of the context and overall objectives of the project

New media content is commonly generated as a combination of real life videos and computer generated objects that are augmented on top of the real video, or rendering entire scenes by fusing a 3-D model of the scene and object texture taken from the video. In both cases, rendering and augmentation relies on special video analysis performed on the original video.
Intelligent Video Analytics (IVA) algorithms are mathematically complex and their software implementations require high-end programming skills. It takes a significant developmental effort to make IVA implementations robust against changing appearance of real life videos. Ideally, each video analytics scenario would require a specialized implementation of respective IVA algorithm.
This specialized nature of IVA implementations makes the video analytics market inherently fragmented and the price of single implementations high. Consequently, the high amount of specialized algorithms cannot be fully utilized by the end user looking for best possible IVA solution. This hinders a huge amount of small and medium size companies, which dominate the video content generation industry.
ProCAMS’s main objective was to develop a web-based repository of I.V.A solutions that can be used for more advanced and cost efficient production of novel content by creative industry SMEs. Further technical objectives comprise:
- Implement demonstration use cases - automatic control of video quality; detection and tracking of objects of interest; reconstruction of surface orientation & light direction; face and mood detection; automatic annotation of video content.
- Demonstrate the added value for content creators
- Customise WEB repository and establish online media services
- Develop business model for the IVA web repository

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The project has started with the existing prototype of the video repository implemented by the partner LYNC in cooperation with PUCK and Fraunhofer. This prototype was entirely re-designed during the project span, to adapt blepo repository best to the needs of the content producers, as were defined by the project creative partners.
The blepo.net platform exhibits an engaging, easy to navigate and responsive user design. Users can upload IVA solutions and test them against own videos. blepo.net supports a self-hosting option allowing IVA developers to run their solutions on their local hardware. Users looking for specific IVA solutions may publish their requests on blepo.net in a form of IVA projects. These comprise a task description, desired implementation details and a set of sample videos for which the solution is wanted. IVA developers may submit solutions to the published projects. Blepo.net supports a public and private type for project publication. IVA solutions submitted to a public project are available to any blepo user and can be tested against user’s own videos. If the project is private, the submitted solution available solely to the project owner. Blepo.net functionalities are provided on a variety of mobile devices and has been designed with security and scalability in mind.
Project’s creative SME partners have defined 22 projects for which dedicated I.V.A solutions were solicited via blepo.net. I.V.A contest was launched on 1.12.2015 (closed on 10.05.2016) on blepo.net and as the catalyst for searching of efficient I.V.A solutions.
The IVA contest was communicated online over social and professional networks and through repeated personal invitations via email to about 600 recipients. Despite of this broad communication, only 21 developers have subscribed to the contest, of which only 2 developers have submitted actual IVA solutions. None of these could however be used in the workflow of the project creative partners.
Despite of the difficulty in attracting the IVA developers the project consortium has managed to provide the needed I.V.A solutions (either self-developed using open source or adopted the purchased ones), for all the defined media use cases:
• Automatic quality control of media assets during ingest in an OTTV and broadcasting environment
• Generation of interactive online videos
• Video tracking for 360 immersive video generation
• Face clustering tool for online smart videos
• Detection of facial expressions for customer feedback monitoring
• Perspective surface and light reconstruction for augmentation of computer generated objects.
Based on the developed I:V.A solutions several online media services were established on blepo.net to support video broadcasters and content producers. Each creative partner could demonstrate the added value of the solicited and self-developed I.V.A tools for its demonstration use case and extend its own product portfolio with additional media services.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Blepo.net is unique in terms of its functionality. No comparable online platforms are available that allow their users processing their videos in the cloud. Only still image processing is offered online, which is a much less challenging task.
Blepo.net offers innovative possibilities for IVA developers, which can upload their IVA solutions as binary demo modules on the platform. These solutions become available to blepo users for testing against their own videos. This unique functionality is supported by blepo cloud engine which runs IVA solutions in a sandbox on cloud virtual machines every time when a user triggers the “test IVA solution” functionality. Further innovative “self-hosted functionality allows the developers to run their solutions the developers own server.
For the content creative community and content producing SMEs blepo offers unique online media services. These are supported by the blepo cloud engine and allow the user video upload, video processing and download of the processing result, i.e. the result video and the processing file record in xml / txt/ html formats.
The main impact expected by the project was “to establish a communication between the IVA developers and video content producers as well as a trading platform for selecting, benchmarking, and requesting of IVA solutions and services tailored to specific needs of the content creating companies”. The developed blepo platform serves as a communication hub between the I.V.A. developing and content creating communities, whereas the implemented blepo functionalities provide technical means to request, select and benchmark the I.V.A. solutions. Furthermore the ProCAMS project has provided a high added value for the consortium creative SMEs, and by that has contributed to the expected impact in full.
However the imminent impact of the blepo platform will be a limited one due to the current structure of the video analytics market, which humpers the exchange of I.V.A solutions because of the existing gap in mind sets – (content creators can poorly formalize their needs), and price of I.V.A solutions offered on the market and solicited by the content producers. No of-the-shelf IVA solutions are available on the market blepo.net projects published the ProCAMS partners.
As the result of this structural market problem the project could gain only limited traction with the IVA developers. However, our conducted market analysis allowed to quantify a huge market of about 228B$ and to identify a promising, double digit growth market opportunity, which can be profitably addressed by blepo.net in the future.

Blepo poster