Placing the audience within the story
Imagine if we could view video like we view the real world: turn our gaze to a detail and examine it, turn our heads around to see what’s happening on the other side of the room, or open a door to see what’s behind it. The EU-funded Hyper360 project has delivered a software suite for creating novel video content that would enable us to do just that – and more. “It offers an end-to-end toolkit for combining 360° and 3D free viewpoint video with interactive items, creating engaging, immersive experiences,” says Angelo Manfredi, Hyper360 project coordinator and project manager at Engineering, the software company which hosted the project. The software tools developed by the project team cover all steps from creation to viewing: recording, post-production and delivery.
The full picture
To record 360° video, the solution enables directors to capture sets of large and overlapping images from the scene with multiple types of cameras which are then combined and transformed using fisheye lens technology to create a spherical view. Through the 3D capturing system, they can acquire complete 3D representations of objects or people from all angles. These are then processed by a 3D reconstruction algorithm to embed them into the 360° scenery, where they can be managed and animated to create lifelike avatars. Interactive elements are added during the post-production phase. “These so-called hotspots can be labels, icons, text or links to other media,” Manfredi explains. Viewers can use a hotspot to carry out different actions, for instance accessing more information about the item, switching to another scene or interacting with an avatar.
Built-in customisation
Consumers will be able to enjoy this new immersive type of content through computer screens, Smart TVs, mobile phones or VR headsets. The way they interact with the video will be adapted to each type of device – and to each individual viewer: “With permission from the user, Hyper360 can collect user behaviour data and remember viewing choices to build a personalised experience.” Giving audiences the freedom to select the views they are interested in makes creating a narrative a more complex task. “We need to rethink storytelling,” says Manfredi. “3D ‘mentors’ are one technique directors can use to add guidance for audiences in free viewpoint settings.” These ‘mentors’ are 3D avatars with which the viewer can interact to be guided through the video. Audiences are increasingly turning to video-on-demand (VOD) services giving them more control over what they watch, with the number of VOD streaming users expected to reach 189.9 million by 2025. Adding elements of gaming to traditional entertainment, Hyper360 is taking user choice one step further by giving them a say in how the narrative unfolds. While this new type of product is not expected to replace current streaming content, it creates new perspectives for a broad range of audiovisual products. “360° interactive video is for instance a good fit for interactive documentaries, dynamic virtual tours or actionable advertising,” Manfredi adds. A number of publicly available demos give a sneak preview of the solution’s possible uses. The full suite has been successfully trialled in two large-scale pilots with major broadcasters RBB in Germany and RTI in Italy.
Keywords
Hyper360, interactive video, interactive, 360° video, 3D, VOD, video-on-demand, free viewpoint, avatar, fisheye, advertising
