Periodic Reporting for period 1 - CLIPE (Creating Lively Interactive Populated Environments)
Período documentado: 2020-03-01 hasta 2022-02-28
The project addresses the core challenges of designing new techniques to create and control interactive virtual characters, benefitting from opportunities open by the wide availability of emergent technologies in the domains of human digitization and displays, as well as recent progresses of artificial intelligence.
The primary objective of CLIPE is to train a generation of innovators and researchers in the field of virtual characters simulation and animation and help bringing Europe at the forefront of the field. The research objective of CLIPE is to design the next‐generation of VR‐ready characters. CLIPE is addressing the most important current aspect of the problem, which is making the characters capable of:
• behaving more naturally
• interacting with real users sharing a virtual experience with them
• being more intuitively and extensively controllable for virtual worlds designers.
The project aims to expose young researchers to the state‐of‐the‐art research work in the areas, allowing them to develop character simulation techniques beyond the current frontiers, such as: machine learning to provide characters with new abilities based on examples, procedural animation techniques to design large‐scale virtual population of characters, perceptual studies to evaluate character simulation quality and communication abilities. CLIPE also extends its partnership to key industrial actors of populated virtual worlds, giving students the ability to explore new application fields and start collaborations beyond academia. Finally, a thorough training program will equip 15 young researchers (ESRs) with many of the skills required to succeed in the very competitive industry/start‐ up world if they decide to venture there.
The CLIPE research program fulfils 4 main Research Objectives, oriented toward providing new tools and methods to enable communities of experts or non-expert users create high-end virtual characters, and design narrative immersive populated environments:
1. To develop methods for generating and manipulating character meshes as well as for retargeting and stylizing motion and appearance, to make these actions achievable by non-expert users.
2. To equip virtual characters with interaction and response capabilities - unscripted and personalised.
3. To develop the methods that will allow intuitive and accessible interfaces for creating complex 3D environments involving several virtual characters (or crowds of characters), assigning them roles and specific behaviours.
4. Develop techniques for robust, realistic and scalable populated immersive virtual as well as augmented environments.
The primary objective of CLIPE is to train a generation of innovators and researchers in the field of virtual characters simulation and animation and help bringing Europe at the forefront of the field. The research objective of CLIPE is to design the next‐generation of VR‐ready characters. CLIPE is addressing the most important current aspect of the problem, which is making the characters capable of:
• behaving more naturally
• interacting with real users sharing a virtual experience with them
• being more intuitively and extensively controllable for virtual worlds designers.
The project aims to expose young researchers to the state‐of‐the‐art research work in the areas, allowing them to develop character simulation techniques beyond the current frontiers, such as: machine learning to provide characters with new abilities based on examples, procedural animation techniques to design large‐scale virtual population of characters, perceptual studies to evaluate character simulation quality and communication abilities. CLIPE also extends its partnership to key industrial actors of populated virtual worlds, giving students the ability to explore new application fields and start collaborations beyond academia. Finally, a thorough training program will equip 15 young researchers (ESRs) with many of the skills required to succeed in the very competitive industry/start‐ up world if they decide to venture there.
The CLIPE research program fulfils 4 main Research Objectives, oriented toward providing new tools and methods to enable communities of experts or non-expert users create high-end virtual characters, and design narrative immersive populated environments:
1. To develop methods for generating and manipulating character meshes as well as for retargeting and stylizing motion and appearance, to make these actions achievable by non-expert users.
2. To equip virtual characters with interaction and response capabilities - unscripted and personalised.
3. To develop the methods that will allow intuitive and accessible interfaces for creating complex 3D environments involving several virtual characters (or crowds of characters), assigning them roles and specific behaviours.
4. Develop techniques for robust, realistic and scalable populated immersive virtual as well as augmented environments.
The CLIPE project was kicked off in March 2020. The CLIPE team is implementing 8 multi-disciplinary work packages (WPs), 42 deliverables and 15 milestones. Specifically, 4 technical WPs advance the state-of-the-art in animation, virtual reality, and artificial intelligence (WP2-5); 1 WP focuses exclusively on the training of ESRs (WP6); 1 on dissemination of results (WP7) and 1 WP involves the project’s management (WP8). Also, a WP on Ethical Requirements is foreseen (WP1).
The core of the scientific work performed by the 15 recruited Early-Stage Researchers (ESRs) is conducted within WPs 2–5.
In relation to networking and training activities (WP6), a total of such three activities have been implemented by the time of this report preparation, two of which were conducted within the first 24 months of the project (1st Networking event, 1st training workshop), and the third one (2nd training workshop) at the end of March 2022.
Specifically, in February 2020 the 1st Networking event was organised by UCL with the participation of one (at least) representative from each beneficiary as well as the 13 ESRs that were recruited until then. The CLIPE partner organisations were invited to give presentations on their companies and the challenges in their sectors and discuss with the ESRs the current limitations faced by companies within the animation and virtual character industries. The objective of the 1st Networking event was to present CLIPE’s main research plan, training goals and training programme to ESRs; help each ESR develop his/ her own CDP from the beginning of their doctoral studies; encourage the networking and interaction between the ESRs, the senior academics and the partner organisations in a non-formal setting; develop the necessary scientific writing/ presentation skills for ESRs to enhance their career prospects; and organise the ESRs’ secondments to academic and partner organisations in order to enhance the implementation of CLIPE’s research objectives and integrate the real-case challenges and industrial problems into the ESRs’ research work. As the implementation of CLIPE progresses, the research of each ESR is more clearly defined and documented in their CDPs.
The CDP progress was further discussed during training workshop 1, which took place between the 13th-17th of September 2021. Training workshop 1 had the form of a Summer School, when students had less coursework and the academics involved did not have a heavy work schedule, in order to maximise the training impact.
In addition, the second training workshop took place between the 28th of March until the 2nd of April 2022, and it was organized by INRIA, at Rennes, France. The training workshop was dedicated to ESRs’ training, who physically attended the event, while most of their advisors joint remotely for the CPD sessions and the ESRs’ presentations of their scientific work.
A total of 25 deliverables and 6 milestones are completed for the first periodic period. The deliverables were submitted either on time or with (small) deviations to the initial planning and the EC was properly notified. The main reasons for the deviations were the difficulties associated with the lockdown and COVID-19 related restrictions and the delays in the recruitment of some ESRs, whose role is vital for the implementation of the deliverables. In addition, a total of ten scientific publications has been submitted/published, in addition to several dissemination activities that took place be all the beneficiaries.
The core of the scientific work performed by the 15 recruited Early-Stage Researchers (ESRs) is conducted within WPs 2–5.
In relation to networking and training activities (WP6), a total of such three activities have been implemented by the time of this report preparation, two of which were conducted within the first 24 months of the project (1st Networking event, 1st training workshop), and the third one (2nd training workshop) at the end of March 2022.
Specifically, in February 2020 the 1st Networking event was organised by UCL with the participation of one (at least) representative from each beneficiary as well as the 13 ESRs that were recruited until then. The CLIPE partner organisations were invited to give presentations on their companies and the challenges in their sectors and discuss with the ESRs the current limitations faced by companies within the animation and virtual character industries. The objective of the 1st Networking event was to present CLIPE’s main research plan, training goals and training programme to ESRs; help each ESR develop his/ her own CDP from the beginning of their doctoral studies; encourage the networking and interaction between the ESRs, the senior academics and the partner organisations in a non-formal setting; develop the necessary scientific writing/ presentation skills for ESRs to enhance their career prospects; and organise the ESRs’ secondments to academic and partner organisations in order to enhance the implementation of CLIPE’s research objectives and integrate the real-case challenges and industrial problems into the ESRs’ research work. As the implementation of CLIPE progresses, the research of each ESR is more clearly defined and documented in their CDPs.
The CDP progress was further discussed during training workshop 1, which took place between the 13th-17th of September 2021. Training workshop 1 had the form of a Summer School, when students had less coursework and the academics involved did not have a heavy work schedule, in order to maximise the training impact.
In addition, the second training workshop took place between the 28th of March until the 2nd of April 2022, and it was organized by INRIA, at Rennes, France. The training workshop was dedicated to ESRs’ training, who physically attended the event, while most of their advisors joint remotely for the CPD sessions and the ESRs’ presentations of their scientific work.
A total of 25 deliverables and 6 milestones are completed for the first periodic period. The deliverables were submitted either on time or with (small) deviations to the initial planning and the EC was properly notified. The main reasons for the deviations were the difficulties associated with the lockdown and COVID-19 related restrictions and the delays in the recruitment of some ESRs, whose role is vital for the implementation of the deliverables. In addition, a total of ten scientific publications has been submitted/published, in addition to several dissemination activities that took place be all the beneficiaries.
In Europe, a large economical sector centred on the fields of culture and entertainment focuses on developing applications that involve human characters. Creating and bringing to life virtual characters entails following well-identified steps organized in animation pipelines: creating 3D human shapes, making them animate-able, and programming them to move, behave, act, react and interact with the environment and the user to follow scenarios. These steps require both artistic and technical expertise, which prevents the amateur general public from engaging in character animation. In CLIPE we aim to push the state of the art in all the aforementioned steps of the pipeline. Already novel techniques have been proposed that enable for example easier facial animation or learning the behavior of crowds. Many other novel contributions are in development.
Training young researchers to create better virtual characters is also a very important target of this project, opening large avenues for innovation and transfer to the industry. This could potentially happen, either through the interaction of the ESRs and our industrial partners or by the ESRs taking advantage of the entrepreneurial training they receive in CLIPE and moving on to develop their own technology start-ups.
Training young researchers to create better virtual characters is also a very important target of this project, opening large avenues for innovation and transfer to the industry. This could potentially happen, either through the interaction of the ESRs and our industrial partners or by the ESRs taking advantage of the entrepreneurial training they receive in CLIPE and moving on to develop their own technology start-ups.