Periodic Reporting for period 2 - CLIPE (Creating Lively Interactive Populated Environments)
Período documentado: 2022-03-01 hasta 2024-08-31
The project addressed the core challenges of designing new techniques to create and control interactive virtual characters, benefitting from opportunities open by the wide availability of emergent technologies in the domains of human digitization and displays, as well as recent progresses of artificial intelligence.
The primary objective of CLIPE was to train a generation of innovators and researchers in the field of virtual character simulation and animation and help bring Europe to the forefront of this dynamic field. The research objective of CLIPE was to design the next-generation of VR-ready characters that could seamlessly integrate into immersive virtual environments.
CLIPE addressed the most important current aspect of the problem, which was making the characters capable of:
• behaving more naturally, by developing sophisticated character behaviors that more closely resembled human actions and expressions.
• interacting with real users, engaging in interactions with virtual characters sharing a virtual experience.
• being more intuitively and extensively controllable for virtual worlds designers.
The project provided young researchers with unique opportunities to engage in cutting-edge research, exploring innovative approaches such as:
- Machine learning, to provide characters with new abilities based on examples,
- Procedural animation techniques, to design large‐scale virtual population of characters,
- Perceptual studies, to evaluate the effectiveness of character simulations quality and communication capabilities.
CLIPE also extended its partnership to key industrial partners of populated virtual worlds, providing invaluable real-world experience for the researchers, giving students the ability to explore new application fields and start collaborations beyond academia.
Finally, the project implemented a comprehensive training program designed to equip 15 young researchers (ESRs) with many of the skills necessary to excel in the very competitive industry/start‐ up world if they decided to venture there.
The CLIPE research program achieved significant outcomes in several key areas oriented toward providing new tools and methods to enable communities of experts or non-expert users create high-end virtual characters, and design narrative immersive populated environments:
1. Character Creation and Manipulation: The project developed new tools and methods for generating and manipulating character meshes as well as for retargeting and stylizing motion and appearance, to make these actions achievable by non-expert users.
2. Intelligent Virtual character interaction: CLIPE successfully equipped virtual characters with sophisticated interaction and response capabilities, enabling more natural and engaging interactions with users.
3. User Friendly Authoring Tools: The project developed intuitive and accessible interfaces for creating complex 3D environments populated with virtual characters (or crowd of characters) facilitating the assignment of roles and specific behaviors.
4. Building Scalable and Immersive Worlds: The project advanced the development of robust, realistic, and scalable populated immersive virtual and augmented environments, enabling more immersive and engaging user experiences.
The primary objective of CLIPE was to train a generation of innovators and researchers in the field of virtual character simulation and animation and help bring Europe to the forefront of this dynamic field. The research objective of CLIPE was to design the next-generation of VR-ready characters that could seamlessly integrate into immersive virtual environments.
CLIPE addressed the most important current aspect of the problem, which was making the characters capable of:
• behaving more naturally, by developing sophisticated character behaviors that more closely resembled human actions and expressions.
• interacting with real users, engaging in interactions with virtual characters sharing a virtual experience.
• being more intuitively and extensively controllable for virtual worlds designers.
The project provided young researchers with unique opportunities to engage in cutting-edge research, exploring innovative approaches such as:
- Machine learning, to provide characters with new abilities based on examples,
- Procedural animation techniques, to design large‐scale virtual population of characters,
- Perceptual studies, to evaluate the effectiveness of character simulations quality and communication capabilities.
CLIPE also extended its partnership to key industrial partners of populated virtual worlds, providing invaluable real-world experience for the researchers, giving students the ability to explore new application fields and start collaborations beyond academia.
Finally, the project implemented a comprehensive training program designed to equip 15 young researchers (ESRs) with many of the skills necessary to excel in the very competitive industry/start‐ up world if they decided to venture there.
The CLIPE research program achieved significant outcomes in several key areas oriented toward providing new tools and methods to enable communities of experts or non-expert users create high-end virtual characters, and design narrative immersive populated environments:
1. Character Creation and Manipulation: The project developed new tools and methods for generating and manipulating character meshes as well as for retargeting and stylizing motion and appearance, to make these actions achievable by non-expert users.
2. Intelligent Virtual character interaction: CLIPE successfully equipped virtual characters with sophisticated interaction and response capabilities, enabling more natural and engaging interactions with users.
3. User Friendly Authoring Tools: The project developed intuitive and accessible interfaces for creating complex 3D environments populated with virtual characters (or crowd of characters) facilitating the assignment of roles and specific behaviors.
4. Building Scalable and Immersive Worlds: The project advanced the development of robust, realistic, and scalable populated immersive virtual and augmented environments, enabling more immersive and engaging user experiences.
The CLIPE project commenced in March 2020, with the implementation of eight multidisciplinary work packages (WPs). These WPs, strategically designed to achieve project objectives, included 44 deliverables and 15 key milestones.
Four technical WPs (WP2-5) advanced the state-of-the-art in animation, virtual reality, and artificial intelligence. WP6 was exclusively dedicated to the training and development of Early-Stage Researchers (ESRs). WP7 focused on disseminating the project's findings to the broader research community, while WP8 oversaw the project's effective management. Furthermore, WP1 was dedicated to addressing Ethical Requirements, ensuring responsible research practices throughout the project.
The core of the scientific work performed by the 15 recruited Early-Stage Researchers (ESRs) was conducted within WPs 2–5.
Concerning networking and training activities (WP6), five events were successfully implemented. Three occurred within the first 24 months. The fourth event was hosted by MPI in April 2023, and the final event was integrated into the Eurographics 2024 conference in April 2024. The final CLIPE workshop, showcased the project's achievements. ESRs presented their PhD research creating a platform for knowledge exchange and networking.
The CLIPE project disseminated its research through 55 conference presentations, 96 scientific publications, and presentations at prestigious venues like Eurographics and SIGGRAPH. This extensive dissemination increased project visibility and significantly contributed to advancing virtual reality technology. CLIPE has significantly advanced VR/AR technologies through collaborative research, achieving impactful outcomes:
• Advanced Motion Modeling: developed a novel method for representing motion, utilizing advanced machine learning techniques. This innovative approach has broad applications, including enhanced video analysis, improved character animation, and more accurate motion reconstruction.
• Enhanced Crowd Simulation: introduced "CCP: Configurable Crowd Profiles," a sophisticated simulator capable of accurately tracking and simulating complex crowd behavior. This valuable tool provides researchers with a powerful instrument for analyzing crowd dynamics and optimizing virtual environments.
• Streamlined Crowd Data Acquisition: The project pioneered the "One-Man-Crowd" (OMC) paradigm, a groundbreaking approach that enables the capture of complex crowd behaviors using a single participant. This innovative methodology has the potential to revolutionize crowd data collection and analysis.
• Facilitated Ubiquitous XR Collaboration: developed "Ubiq," an open-source library for seamless real-time collaboration in XR environments. This valuable resource fosters seamless multi-user interaction and is poised to become widely adopted by researchers and developers, thereby accelerating innovation within the XR community. Notably, UPC is currently collaborating with UCL to integrate novel animation methods for self-avatars into the Ubiq platform.
CLIPE demonstrates the power of interdisciplinary collaboration, achieving remarkable results through the combined efforts of researchers and industry experts.
Four technical WPs (WP2-5) advanced the state-of-the-art in animation, virtual reality, and artificial intelligence. WP6 was exclusively dedicated to the training and development of Early-Stage Researchers (ESRs). WP7 focused on disseminating the project's findings to the broader research community, while WP8 oversaw the project's effective management. Furthermore, WP1 was dedicated to addressing Ethical Requirements, ensuring responsible research practices throughout the project.
The core of the scientific work performed by the 15 recruited Early-Stage Researchers (ESRs) was conducted within WPs 2–5.
Concerning networking and training activities (WP6), five events were successfully implemented. Three occurred within the first 24 months. The fourth event was hosted by MPI in April 2023, and the final event was integrated into the Eurographics 2024 conference in April 2024. The final CLIPE workshop, showcased the project's achievements. ESRs presented their PhD research creating a platform for knowledge exchange and networking.
The CLIPE project disseminated its research through 55 conference presentations, 96 scientific publications, and presentations at prestigious venues like Eurographics and SIGGRAPH. This extensive dissemination increased project visibility and significantly contributed to advancing virtual reality technology. CLIPE has significantly advanced VR/AR technologies through collaborative research, achieving impactful outcomes:
• Advanced Motion Modeling: developed a novel method for representing motion, utilizing advanced machine learning techniques. This innovative approach has broad applications, including enhanced video analysis, improved character animation, and more accurate motion reconstruction.
• Enhanced Crowd Simulation: introduced "CCP: Configurable Crowd Profiles," a sophisticated simulator capable of accurately tracking and simulating complex crowd behavior. This valuable tool provides researchers with a powerful instrument for analyzing crowd dynamics and optimizing virtual environments.
• Streamlined Crowd Data Acquisition: The project pioneered the "One-Man-Crowd" (OMC) paradigm, a groundbreaking approach that enables the capture of complex crowd behaviors using a single participant. This innovative methodology has the potential to revolutionize crowd data collection and analysis.
• Facilitated Ubiquitous XR Collaboration: developed "Ubiq," an open-source library for seamless real-time collaboration in XR environments. This valuable resource fosters seamless multi-user interaction and is poised to become widely adopted by researchers and developers, thereby accelerating innovation within the XR community. Notably, UPC is currently collaborating with UCL to integrate novel animation methods for self-avatars into the Ubiq platform.
CLIPE demonstrates the power of interdisciplinary collaboration, achieving remarkable results through the combined efforts of researchers and industry experts.
The entertainment industry in Europe relies heavily on applications featuring human characters. Creating these characters traditionally involved complex, specialized processes. However, the CLIPE project has significantly advanced this field.
Key Achievements:
Improved Character Intelligence: CLIPE is developing AI-powered characters with more realistic and engaging interactions.
Enhanced Performance: The project aims to enable real-time rendering and interaction with high-fidelity virtual characters on consumer-grade hardware.
Increased Accessibility: CLIPE is democratizing character creation tools, making them more accessible to a wider range of users.
Potential Impact:
CLIPE's advancements have the potential to drive innovation in entertainment, education, healthcare, and social interaction.
By training a new generation of researchers and fostering industry collaborations, CLIPE has the potential to significantly impact the future of virtual ch
Key Achievements:
Improved Character Intelligence: CLIPE is developing AI-powered characters with more realistic and engaging interactions.
Enhanced Performance: The project aims to enable real-time rendering and interaction with high-fidelity virtual characters on consumer-grade hardware.
Increased Accessibility: CLIPE is democratizing character creation tools, making them more accessible to a wider range of users.
Potential Impact:
CLIPE's advancements have the potential to drive innovation in entertainment, education, healthcare, and social interaction.
By training a new generation of researchers and fostering industry collaborations, CLIPE has the potential to significantly impact the future of virtual ch