Skip to main content

Repurposing and enriching images for immersive storytelling through smart digital tools

Periodic Reporting for period 1 - FotoInMotion (Repurposing and enriching images for immersive storytelling through smart digital tools)

Reporting period: 2018-01-01 to 2019-06-30

The amount of digital content available to creative industries is growing exponentially, driven by the ubiquitous use of smartphones and the proliferation of social media: (1) there is a tremendous increase in the amount of photographic content (more than 1.8 billion photos are uploaded to social media platforms each day); (2) the ongoing transformation of factual, entertainment and social media publishers and platforms from textual and photo-centric format to video-driven format (more than 400 hours of video are being uploaded to YouTube each minute); (3) the increasing impact of 3D and virtual reality for providing immersive storytelling experiences, offering new ways of audience engagement and monetization for content creators in the upcoming years.
Acknowledging the above, the following critical questions become imminent in both content production and dissemination contexts: how to repurpose this massive amount of content; what kind of innovative tools are most suitable for this process; and finally, how these tools can offer new forms of monetization possibilities for creative industries' professionals.
FotoInMotion, sets to solve these critical questions and provide an innovative solution to the repurposing of content by offering automated tools for innovative contextual data extraction, object recognition, creative transformation, editing and text animation as well as state of the art 3D conversion options that allow content creators to transform their photos into highly engaging spatial and three-dimensional video experiences.
FotoInMotion is focusing on three major creative industries sectors: photojournalism to develop interactive photo driven stories; fashion, by opening up new forms of marketing, product placement and event coverage; and festivals, by enabling PR and publicity managers to communicate the festival experiences and engage audiences through immersive communication and repurposing festival archives. Professionals and experts from these three creative industries will continuously explore and test the FotoInMotion technological outcomes in order to achieve results of highest level in terms of quality, performance and innovation.
The FotoInMotion consortium launched its activities by performing an extensive analysis of the innovation tools and trends in the video and image processing marketplace along with features, pricing and platform analysis, enabling the team to reach to a common understanding of the currents status and, therefore, conclude to a joint vocabulary between end users and technical partners. that extensive analysis allowed the end users to develop a concrete and full set of user requirements, which were then “translated” into technical requirements by all technical partners.
In parallel with the end users and based on user requirements, technical partners have initiated experimentation of state-of-the-art Machine Learning-based platforms and algorithms for the extraction and identification of visual features. Based on these results, it defined a first functional specification of the visual analysis and classification component (iCAT) and of relevant APIs for its integration within the complete FotoInMotion system architecture. In parallel, it selected and adopted a set of neural networks and started to configure, parametrise, train and fine-tune such Neural Networks to analyse photographs and identify features relevant for the FotoInMotion use cases. That lead to the development of the first version of the FotoInMotion image annotation tool (AAT). Such tool receives the output of the image analysis and features extraction tools and enables the user to enhance and/or augment the automatically obtained tags. In turn, it provides new training material and data to assess, validate, re-train and fine-tune the mentioned image analysis tools.
The technical partners have been also working towards developing and testing the first set of 2D image editing quality assistance tools, used by end users to prepare images to be processed by the FotoInMotion application. This refers to actions like cropping, changing the color balances and apply transformations on images. In parallel, the team has also developed and tested the first set of video effects, to assist narration and, consequently, produce a 3D video.
All technical work has been capitalized into the development of the first web-version of the FotoInMotion application, which will be pilot tested by the end users. This web platform is based on web-services architecture and in a user-friendly User Interface based on requests received by the pilot users and current trends. Adding to the above, the first version of the mobile application, with limited functionality, has been developed however more work will be implemented in the following months.
For the FotoInMotion project, we expect that in the field of creative industries, the project will enhance the way people use photography to portray stories and furthermore all photography related fields. For the art industries, in terms of exhibitions, it will give creatives many options to create multi-media projects from traditional photography – it will enhance the way people interact with art and give creatives the chance to re-enhance old photography projects. Likewise, in the photo-journalism field, notably for the coverage of live events, it will offer the journalist the possibility to very quickly create an appealing multimedia story and publish it online in social media networks, thus potentially engaging more audience. Such creation will make use of QdepQ’s algorithm which is currently the commercial state of the art in 2D-3D technology. Recently, the interest of academic research is peaking in this field and in FotoInMotion, we hope to use this head start to stay in the state-of-the-art commercial solution on 2D-3D conversions. FotoInMotion will offer the possibility of applying the 3D effects to photographs in an automatic and informative way on an object base, by profiting of the automatic region-based image analysis being developed in the project. Concepts of interest are being automatically assigned to regions and objects in the photographs, enabling the creative process to selectively apply editing effects when creating the multimedia story. Moreover, current online solutions for storytelling are usually based on fixed templates selected by the user. In FotoInMotion we use contextual information to learn a suggested story, making the workflow easier and more specific to the content, while the user is still in full control.
FotoInMotion logo