Skip to main content
EnglishEN
European Commission logo print header

intelligent 3D content extraction and manipulation for film and games

Project description


Intelligent Content and Semantics
New film techniques to put the ‘real’ back into realism

As special effects, and particularly animations, become more important in film-making, believability and realism are what producers and game developers strive for.

Now European research on automated three-dimensional modelling could make it easier and cheaper for them to provide more realistic and exciting films and games for consumers.

Researchers in the i3DPOST project aim to develop a system to capture the 3-D composition of film sets as an aid to post-production work.

At present the director works with a principal camera which takes the front-on shot the audience in the cinema sees.

After shooting, post-production staff then use a variety of tools from tape measures to laser scanners to build up a 3-D image of each shot so they can then add details which are not there, or change aspects of the shot.

Adding alien heads

This could be as simple as adding a shadow where there was none before, or adding or removing a piece of furniture.  At a more complex level, it might involve removing actors entirely or perhaps putting alien heads on actors’ bodies.

The three-year project, which kicked off in January this year, could result in automating that 3-D imaging process, making it less expensive and more effective than the current manual system.

The automated system would also add the 3-D component at the time of filming rather than afterwards. The project’s scientific coordinator, Simon Robinson, explains the researchers main focus will be on using additional cameras during filming to capture sets from different angles, working in conjunction with new software that the researchers will develop.

Bonus for games developers

As a result, not only film-makers will benefit, but game developers will also be able to speed up the production and marketing of games based on films. Instead of having to start from scratch with their own 3-D modelling when the film is complete, developers will have the same digital information to work with as the film-makers – and at the same time.

“The extra cameras must be unobtrusive so as not to disrupt the filming process, and costs need to be kept down, so we will have to work out how many additional cameras are enough and how sophisticated they need to be,” he says.

The project already has a head start on this aspect thanks to one of its partners, French production house BUF Compagnie, which already captures shots from multiple angles and uses them to recreate manually sets in post production. The challenge facing the partners is to create software to automate the process.

Capturing moving objects

The other major challenge is to capture moving objects, particularly actors.

“The capture of human performance is tricky, and capturing facial expressions is particularly difficult,” Robinson says.

If all goes according to plan, the project will develop a prototype system which can be commercialised by the partners. Using advances in 3-D data capture, 3-D motion estimation and media semantics, it will aim to provide top-quality 3D content in a structured format for use in different media platforms.

Then film audiences and game players can expect to see more realistic 3-D products from Europe’s producers and developers.

i3Dpost will improve quality and reduce the cost of high-level media production by applying intelligent technologies to the extraction of structured 3D content models from video. This will enable the increasingly automatic manipulation and re-use of characters, with changes of viewpoint and lighting. The research will advance the state of the art in 3D video production, 3D motion estimation, post-production tools and media semantics. The result will be film quality 3D content in a structured form, with semantic tagging, which can be manipulated in a graphic production pipeline and used across different media platforms.
Integration of multiple view 3D video analysis with on-set production will allow the creation of video quality actor and set models. Postproduction tools will be extended to robustly separate and manipulate scene elements. On-set capture of actor performance will be instrumented in a structured form for processing and rendering in a conventional graphics production pipeline whilst maintaining the visual quality of faces, body, and clothing movement. The representation of performance will enable the modification in postproduction of camera view, movement, and lighting to render novel content.
Structured representation will facilitate the retargeting of actor performance to different platforms for production of highly realistic characters in games enabling a new generation of interactive entertainment. Multiple view 3D video analysis will result in extensions to the OFX plug-in standard. Users will be media industry professionals in film and TV production and postproduction, games development, and advertising.

Fields of science

Programme(s)

Topic(s)

Call for proposal

FP7-ICT-2007-1
See other projects for this call

Funding Scheme

CP - Collaborative project (generic)

Coordinator Contact

Abigail Bowman Ms

Coordinator

THE FOUNDRY VISIONMONGERS LTD
EU contribution
€ 796 019,00
Address
GOLDEN SQUARE 5
W1F 9HT LONDON
United Kingdom

See on map
Region
London Inner London — West Westminster
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Administrative Contact
Bill Collis (Dr)
Links
Total cost
No data

Participants (5)

Share this page

Download

Last update: 16 July 2019

My Booklet

Permalink: https://cordis.europa.eu/project/id/211471

European Union, 2024