PRESENT sets out to create virtual digital humans––sentient agents––that look entirely naturalistic, demonstrate emotional sensitivity, establish an engaging dialogue, add sense to the experience, and act as trustworthy guardians and guides in the interfaces for AR, VR, and more traditional forms of media.
As a reminder, the project’s Objectives have been to:
1- Research and develop new methods of creating sentient agents with highly realistic facial and bodily appearance;
2- Research and develop ways of endowing the agent with a high degree of emotional sensitivity as the basis for interactions with users, with lifelike behaviours that evolve over time;
3- Research and test paradigms for interaction with a highly realistic digital character in augmented and virtual environments, with security and trust in the identity of the virtual agent;
4- Specify and develop tools and a pipeline for the efficient creation of sentient agents, with APIs to integrate different functional components and use the agent in interfaces on a range of devices;
5- Integrate the system components and deploy them on both high-end professional and consumer level devices;
6- Create a range of use cases, test and evaluate experiences and interfaces incorporating the prototype sentient agent on different consumer-level and professional devices;
7- Demonstrate and promote concepts and prototypes to the media, entertainment and creative industries, and communicate results to the public.
Objective 1 has been fully achieved, with results that go well beyond initial expectations. Realism of face and body appearance has been enhanced by unexpected realism in hair and clothes (with the application of AI) and a translation pipeline has made it possible to convert complex offline rendered shading graphs to simplified real-time compatible shading models giving a very high degree of realism.
In relation to Objective 2, the project has successfully researched and developed ways of endowing the agent with a high degree of emotional sensitivity as the basis for interactions with users. Original research has been carried out and demonstrated in the final months on evolving behaviour and speech response.
Objective 3: interaction with a highly realistic digital character with security and trust in the identity of the virtual agent has been successfully demonstrated, in a limited flat-screen environment.
Objective 4: tools and a pipeline for the creation of sentient agents, with APIs to integrate different functional components in interfaces on a range of devices, have been developed and demonstrated in a range of scenarios. The introduction of photorealistic agent creation technologies into professional pipelines and use is ahead of expectations
Objective 5: system components have been integrated and deployed them on both high-end professional and consumer level devices, typically using UE as a common platform
Objective 6, to create a range of use cases, test and evaluate experiences and interfaces incorporating the prototype sentient agent on different consumer-level and professional devices has been properly achieved.
Objective 7, to demonstrate and promote concepts and prototypes to the media, entertainment and creative industries, and communicate results to the public has been achieved. Professional demonstrations at FMX were well received and to quote D8.4 “The results of the Experimental Productions and their evaluation exceed the expectations. The appreciation of the artistic and the technical concepts and results of the Delirious Departure performances was high, the same is true for the therapeutic value of Soulhacker and Adam.” The integration of some tools and processes into professional media creation pipelines has been well ahead of expectation.
