Final Report Summary - GOLEM (Realistic Virtual Humans)
This is a two-page summary. For a complete list of activities, publications, dissemination activities or examples of immediate impact, please refer to the project's website:
http://www.cps.unizar.es/golem/
For technical information, contact Dr. Diego Gutierrez (Universidad de Zaragoza):
diegog@unizar.es
For administrative information, contact Elena Portero (Universidad de Zaragoza):
eportero@unizar.es
OBJECTIVES:
The overall objective of GOLEM has been to share knowledge between Industry and Academia in order to carry out multidisciplinary research that results in technology that radically improves the visual appearance and behavior of virtual characters (avatars), while streamlining the production pipeline and keeping them customizable and affordable. Virtual humans can have a relevant impact in (1) industries (such as movies or games), (2) academia (by advancing the state of the art in the related research fields) and (3) neuroscience (by using the avatars in novel experiments about presence in virtual environments). This requires a multidisciplinary approach involving human-computer interaction, computer graphics and neuroscience.
CHALLENGES:
To achieve these goals, the specific challenges are two-fold: First, the development of a novel, optimized pipeline that covers capture, synthesis and animation of realistic virtual humans in real time, to enhance current state-of-the-art techniques to create virtual humans in fields such as game production or cinematography. Building on the previous challenge, the second one is conducting novel research on presence and embodiment in virtual environments.
RESULTS:
The GOLEM project has achieved remarkable results during its four-year life span; there are significant advances in all aspects involved in its development. Some examples include Sketch Express, an interface to create and edit facial expressions easily (Miranda et al. 2011, 2012); speech and lip-sync contributions to the state of the art (Freitas et al. 2012, Serra et al. 2012); an ultra-realistic real-time rendering algorithm for human skin (Jimenez et al. 2010, 2012); or advances in presence and embodiment in virtual environments (Borland et al. 2013, Blom et al. 2013). There is a total of 42 peer-reviewed publications in journals and conferences from the GOLEM project.
IMPACT:
Some of the results developed in GOLEM have already had an immediate impact in industry. Universidad de Zaragoza developed the most advance real-time simulation of human skin, and it is already being used by several game companies (such as Activision), games engines (such as Unigine or Unreal Engine 3) and other research institutions (such as the University of Southern California's Institute for Creative Technologies). Also from Zaragoza, the latest work in stylized capture and reconstruction of human hairstyles is currently being used by Disney. Last, work developed by the University of Porto, Face In Motion and Microsoft on markerless facial capture and speech synthesis is being used by the last two.
From a societal perspective, several advances in avatar production from Porto have been used in research on autism, and the Universitat de Barcelona has carried out research on virtual environments and their potential application to study presence, embodiment or more concrete aspects like anger management. The avatars developed by Zaragoza have also been used in the context of another FP7 project, VERVE, whose goal is to fight social exclusion of the elderly due to issues like memory loss or fear of falling.
DISSEMINATION
Besides publications, many other dissemination activities have been carried out throughout the project. Some highlights include the GOLEM Workshop, carried out in Zaragoza at the end of the project, open to all, where people involved in GOLEM secondments had the chance to present and discuss their latest results. The workshop also had to two invited external speakers, including Henrik Wann Jensen from the University of California San Diego, who won a Technical Achievement Award from Hollywood for his work on simulation of skin, which lead to the creation of Gollum in Lord of the Rings. The skin work from Zaragoza was also presented in top venues, like SIGGRAPH or the Game Developers Conference (both twice, in 2013 and 2014). Other recent dissemination activities include a talk by Prof. John P. Lewis (The Matrix trilogy, Avatar...) in Porto, a Seminar Talk in Virtual Skin at the YOUWIN I Saloon on Videogame and Digital Entertainment (Zaragoza), seminar talks by Crytek developers in Porto, a workshop in Barcelona (by Zaragoza and Microsoft), or the 6th NGO Day in Lisbon, by Microsoft. All these events were open to attendees outside the consortium.
CONCLUSIONS:
Despite the large number of advances, there is still work ahead. The most important is the creation of a unified framework where all the technology developed can be applied, from acquisition to final render. This is mostly an engineering effort, although a considerable one. Integrating the ultra-realistic virtual humans into existing virtual reality applications poses another interesting problem, since the high degree of realism of the characters will need to be matched by everything else in the simulation. Last, we believe our virtual humans can be instrumental in future studies about the uncanny valley.