Creating Lively Interactive Populated Environments

Informations projet

CLIPE

N° de convention de subvention: 860768

DOI

10.3030/860768

Projet clôturé

Date de signature de la CE 4 Septembre 2019

Date de début 1 Mars 2020

Date de fin 31 Août 2024

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 4 009 882,68

Contribution de l’UE

€ 4 009 882,68

Coordonné par

UNIVERSITY OF CYPRUS
Cyprus

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Documents, reports (27)

Report on programming by demonstration methods. Initial demonstration.

Report on programming by demonstration methods Initial demonstration

Report on a subset of statistical body models available for VR/interactivity
Material and report from Networking Event No.1

Material and report from Networking Event No1

Material and report from Training Workshop No.4

Presentations, photos, videos, attendance lists and short evaluation report following the completion of the Training Workshop No.4

Material and report from Training Workshop No.2

Presentations, videos, photos, attendance lists and short evaluation report following the completion of Training Workshop No.2

Material and report Training Workshop No.1

Presentations photos videos attendance sheets short evaluation report after the Training Workshop No1 is completed

Report/publication on evaluation of groups and crowds
Report/publication on evaluation of individuals

Reportpublication on evaluation of individuals

Report on autonomous and adaptive communication behaviours
Report on methods for inserting virtual crowds into a real environment
Report on methods for authoring crowds (usability, control, interactivity and style)

Report on methods for authoring crowds usability control interactivity and style

Initial report on creating procedural cities with semantics
Report/publication on Evaluation studies for all WP5 ESRs

Final report/publication of ESR8

Report on algorithmic trade-offs for real-time crowd animation synthesis

Report on algorithmic tradeoffs for realtime crowd animation synthesis

Report on initial scenarios for stylising and physiology drive animations

A report with the initial scenarios for stylising and physiology drive animations will be available by M19

Report on methods to describe and generate virtual crowds using textual descriptions
Progress report on procedural simulation of crowd behaviour
Report on initial statistical models based on captures of face/hands/body

An initial report on initial statistical models based on captures of facehandsbody will be available by M20

Report on attention-driven model of interaction. Initial implementation.

Report on attentiondriven model of interaction Initial implementation

Report on crowd taxonomies and its relations with the VE
Material and report from Training Workshop No.3

Presentations, photos, videos, attendance lists and short evaluation report following the completion of Training Workshop No.3

Report/publication on evaluation of small groups
Report on preliminary adaptive/motion retargeting animation toolsets
Report on seamless integration of large crowds of virtual characters with real environments

Report on perception of virtual crowds in Mixed Reality

Report on experimental results from studies on trust formation
Material and report from Final CLIPE Conference (Workshop 5)

Presentations, photos, videos, attendance lists and short evaluation report following the completion of Workshop 5.

Report on social group formation in virtual environments

Websites, patent fillings, videos etc. (1)

CLIPE website and dedicated social media, partners' social media

Project website dedicated social media pages partners social media pages posting relevant articles and news concerning CLIPE throughout the project duration

Other (3)

Final promotional video for CLIPE and videos from each ESR

Final promotional video of CLIPE and videos from each ESR.

Presentations of research work in scientific conferences/seminars

Presentations of scientific papers in research/academic conferences and seminars

Newsletters, articles and presence in online and traditional media

Data sets, microdata, etc (1)

Publication on an initial integrated statistical model for whole body animation

Publications

Other (7)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Auteurs: Liu, Weiyang; Qiu, Zeju; Feng, Yao; Xiu, Yuliang; Xue, Yuxuan; Yu, Longhui; Feng, Haiwen; Liu, Zhen; Heo, Juyeon; Peng, Songyou; Wen, Yandong; Black, Michael J.; Weller, Adrian; Schölkopf, Bernhard
Publié dans: arXiv, Numéro 27, 2024
Éditeur: arXiv
DOI: 10.48550/arxiv.2311.06243

Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion

Auteurs: Chhatre, Kiran; Daněček, Radek; Athanasiou, Nikos; Becherini, Giorgio; Peters, Christopher; Black, Michael J.; Bolkart, Timo
Publié dans: arxiv, Numéro 18, 2023
Éditeur: arxiv
DOI: 10.48550/arxiv.2312.04466

P2C: A Paths-to-Crowds Framework to Parameterize Behaviors

Auteurs: Marilena Lemonari, Andreas Panayiotou, Nuria Pelechano, Theodoros Kyriakou, Yiorgos Chrysanthou, Andreas Aristidou, Panayiotis Charalambous
Publié dans: TechRxiv, 2024
Éditeur: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.36227/techrxiv.170654693.38725484/v1

PuzzleAvatar: Assembling 3D Avatars from Personal Albums

Auteurs: Xiu, Y., Ye, Y., Liu, Z., Tzionas, D. and Black, M.J.
Publié dans: 2024
Éditeur: arXiv
DOI: 10.48550/arxiv.2405.14869

Stable Video Portraits

Auteurs: Ostrek, M. and Thies, J
Publié dans: Computer Vision and Pattern Recognition, 2024
Éditeur: arXiv
DOI: 10.48550/arxiv.2409.18083

Do you read me? (E)motion Legibility of Virtual Reality Character Representations.

Auteurs: Brandstatter, Klara; Congdon, Benjamin; Steed, Anthony
Publié dans: 2024
Éditeur: University College London
DOI: 10.5522/04/26356180.v1

Ghost on the Shell: An Expressive Representation of General 3D Shapes

Auteurs: Liu, Zhen; Feng, Yao; Xiu, Yuliang; Liu, Weiyang; Paull, Liam; Black, Michael J.; Schölkopf, Bernhard
Publié dans: Numéro 25, 2024
Éditeur: arXiv
DOI: 10.48550/arxiv.2310.15168

Conference proceedings (29)

Dialogues For One: Single-User Content Creation Using Immersive Record and Replay

Auteurs: Klara Brandstätter, Anthony Steed
Publié dans: 29th ACM Symposium on Virtual Reality Software and Technology, 2023, Page(s) 1-11
Éditeur: ACM
DOI: 10.1145/3611659.3615695

ECON: Explicit Clothed humans Optimized via Normal integration

Auteurs: Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, Michael J. Black
Publié dans: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Éditeur: IEEE
DOI: 10.1109/cvpr52729.2023.00057

High-Fidelity Clothed Avatar Reconstruction from a Single Image

Auteurs: Tingting Liao, Xiaomei Zhang, Yuliang Xiu, Hongwei Yi, Xudong Liu, Guo-Jun Qi, Yong Zhang, Xuan Wang, Xiangyu Zhu, Zhen Lei
Publié dans: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, Page(s) 8662-8672
Éditeur: IEEE
DOI: 10.1109/cvpr52729.2023.00837

LexiCrowd: A Learning Paradigm towards Text to Behaviour Parameters for Crowds.

Auteurs: Lemonari, M., Andreou, N., Pelechano, N., Charalambous, P. and Chrysanthou, Y.,
Publié dans: The Eurographics Association, 2024
Éditeur: The Eurographics Association
DOI: 10.2312/cl.20241049

Tech: Text-guided reconstruction of lifelike clothed humans

Auteurs: Huang, Y., Yi, H., Xiu, Y., Liao, T., Tang, J., Cai, D. and Thies, J.
Publié dans: 2024
Éditeur: IEEE
DOI: 10.48550/arxiv.2308.08545

Social Crowd Simulation: Improving Realism with Social Rules and Gaze Behavior

Auteurs: Reiya Itatani, Nuria Pelechano
Publié dans: The 17th ACM SIGGRAPH Conference on Motion, Interaction, and Games, 2024, Page(s) 1-11
Éditeur: ACM
DOI: 10.1145/3677388.3696337

Soft Walks: Real-Time, Two-Ways Interaction between a Character and Loose Grounds

Auteurs: Chloé Paliard, Eduardo Alvarado, Damien Rohmer, Marie-Paule Cani
Publié dans: Eurographics Short Paper, 2021, ISBN 978-3-03868-133-5
Éditeur: The Eurographics Association
DOI: 10.2312/egs.20211019

Immersive Record and Replay for Lively Virtual Environments

Auteurs: Klara Brandstätter
Publié dans: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2023, Page(s) 979-980
Éditeur: IEEE
DOI: 10.1109/vrw58643.2023.00332

Synthesizing Environment-Specific People in Photographs

Auteurs: Ostrek, M., O'Sullivan, C., Black, M., and Thies, J.
Publié dans: Springer Cham, 2024
Éditeur: Springer
DOI: 10.48550/arxiv.2312.14579

Collaborative VR: Solving Riddles in the Concept of Escape Rooms

Auteurs: Ioannou, A., Lemonari, M., Liarokapis, F. and Aristidou, A.,
Publié dans: IMET (pp. 1-9)., 2023
Éditeur: IMET
DOI: 10.2312/imet.20231247

Is Immersive Virtual Reality in K-12 Education Ready for Primetime? Challenges, Possibilities, and Considerations

Auteurs: Christos Gkoumas, Lisa Izzouzi
Publié dans: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2023, Page(s) 541-544
Éditeur: IEEE
DOI: 10.1109/vrw58643.2023.00119

Exploring the Role of Expected Collision Feedback in Crowded Virtual Environments

Auteurs: Haoran Yun, Jose Luis Ponton, Alejandro Beacco, Carlos Andujar, Nuria Pelechano
Publié dans: 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), 2024, Page(s) 472-481
Éditeur: IEEE
DOI: 10.1109/vr58804.2024.00068

ShapeVerse: Physics-based Characters with Varied Body Shapes.

Auteurs: Vyas, B. and O’Sullivan, C.
Publié dans: 2024
Éditeur: The Eurographics Association
DOI: 10.2312/egp.20241042

Animation Fidelity in Self-Avatars: Impact on User Performance and Sense of Agency

Auteurs: Haoran Yun, Jose Luis Ponton, Carlos Andujar, Nuria Pelechano
Publié dans: 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), 2023
Éditeur: IEEE
DOI: 10.1109/vr55154.2023.00044

EMOCA: Emotion Driven Monocular Face Capture and Animation

Auteurs: Radek Danecek, Michael Black, Timo Bolkart
Publié dans: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, Page(s) 20279-20290
Éditeur: IEEE
DOI: 10.1109/cvpr52688.2022.01967

Soft Walks: Real-Time, Two-Ways Interaction between a Character and Loose Grounds

Auteurs: Paliard, C., Alvarado, E., Rohmer, D. and Cani, M.P.
Publié dans: 2021
Éditeur: Eurographics
DOI: 10.48550/arxiv.2104.10898

A new framework for the evaluation of locomotive motion datasets through motion matching techniques

Auteurs: Vicenzo Abichequer Sangalli, Ludovic Hoyet, Marc Christie, Julien Pettré
Publié dans: Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games, 2024, Page(s) 1-10
Éditeur: ACM
DOI: 10.1145/3561975.3562951

DART: Articulated Hand Model with Diverse Accessories and Rich Textures

Auteurs: Gao, Daiheng; Xiu, Yuliang; Li, Kailin; Yang, Lixin; Wang, Feng; Zhang, Peng; Zhang, Bang; Lu, Cewu; Tan, Ping
Publié dans: Numéro 8, 2022
Éditeur: NeurIPS
DOI: 10.48550/arxiv.2210.07650

Exploring the Perception of Center of Mass changes for VR Avatars

Auteurs: Vyas, B., Hoyet, L. and O'Sullivan, C.
Publié dans: 2023
Éditeur: Eurographics
DOI: 10.2312/egve.20231316

TADA! Text to Animatable Digital Avatars

Auteurs: Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang, Yangyi Huang, Justus Thies, Michael J. Black
Publié dans: 2024 International Conference on 3D Vision (3DV), 2024, Page(s) 1508-1519
Éditeur: IEEE
DOI: 10.1109/3dv62453.2024.00150

ICON: Implicit Clothed humans Obtained from Normals

Auteurs: Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, Michael J. Black
Publié dans: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, Page(s) 13286-13296
Éditeur: IEEE
DOI: 10.1109/cvpr52688.2022.01294

Pose representations for deep skeletal animation

Auteurs: Andreou, N., Aristidou, A., & Chrysanthou, Y.
Publié dans: Computer Graphics Forum (Vol. 41, No. 8, pp. 155-167), 2021
Éditeur: arxiv
DOI: 10.48550/arxiv.2111.13907

Extending the Open Source Social Virtual Reality Ecosystem to the Browser in Ubiq

Auteurs: Sebastian Friston, Ben Congdon, Nels Numan, Klara BrandstäTter, Lisa Izzouzi, Felix Thiel, Jingyi Zhang, Daniele Giunchi, David Swapp, Anthony Steed
Publié dans: The 28th International ACM Conference on 3D Web Technology, 2023, Page(s) 1-9
Éditeur: ACM
DOI: 10.1145/3611314.3615903

CCP: Configurable Crowd Profiles

Auteurs: Andreas Panayiotou, Theodoros Kyriakou, Marilena Lemonari, Yiorgos Chrysanthou, Panayiotis Charalambous
Publié dans: Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings, 2022, Page(s) 1-10
Éditeur: ACM
DOI: 10.1145/3528233.3530712

User Motion Accentuation in Social Pointing Scenario

Auteurs: Ruoxi Guo, Lisa Izzouzi, Anthony Steed
Publié dans: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2023, Page(s) 575-576
Éditeur: IEEE
DOI: 10.1109/vrw58643.2023.00130

D-if: Uncertainty-aware human digitization via implicit distribution field.

Auteurs: Yang, X., Luo, Y., Xiu, Y., Wang, W., Xu, H. and Fan, Z.
Publié dans: 2023
Éditeur: IEEE
DOI: 10.48550/arxiv.2308.08857

Behavioral Landmarks: Inferring Interactions from Data.

Auteurs: Lemonari, M., Charalambous, P., Panayiotou, A., Chrysanthou, Y. and Pettré, J., 2024.
Publié dans: 2024
Éditeur: The Eurographics Association
DOI: 10.2312/egp.20241039

Evaluation of Media-Based Social Interactions in Virtual Environments

Auteurs: Lisa Izzouzi, Anthony Steed, Ifat Yasin
Publié dans: Proceedings of the 2023 ACM International Conference on Interactive Media Experiences, 2023, Page(s) 419-422
Éditeur: ACM
DOI: 10.1145/3573381.3596501

Emotional Speech-Driven Animation with Content-Emotion Disentanglement

Auteurs: Radek Daněček, Kiran Chhatre, Shashank Tripathi, Yandong Wen, Michael Black, Timo Bolkart
Publié dans: SIGGRAPH Asia 2023 Conference Papers, 2024, Page(s) 1-13
Éditeur: ACM
DOI: 10.1145/3610548.3618183

Peer reviewed articles (13)

Lessons learnt running distributed and remote mixed reality experiments

Auteurs: Anthony Steed, Daniel Archer, Klara Brandstätter, Ben J. Congdon, Sebastian Friston, Priya Ganapathi, Daniele Giunchi, Lisa Izzouzi, Gun Woo (Warren) Park, David Swapp, Felix J. Thiel
Publié dans: Frontiers in Computer Science, Numéro 4, 2023, ISSN 2624-9898
Éditeur: Frontiers in Computer Science
DOI: 10.3389/fcomp.2022.966319

TRAIL: Simulating the impact of human locomotion on natural landscapes

Auteurs: Eduardo Alvarado, Oscar Argudo, Damien Rohmer, Marie-Paule Cani, Nuria Pelechano
Publié dans: The Visual Computer, Numéro 40, 2024, Page(s) 5029-5041, ISSN 0178-2789
Éditeur: Springer Verlag
DOI: 10.1007/s00371-024-03506-z

Pose Representations for Deep Skeletal Animation

Auteurs: N. Andreou, A. Aristidou, Y. Chrysanthou
Publié dans: Computer Graphics Forum, Numéro 41, 2023, Page(s) 155-167, ISSN 0167-7055
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/cgf.14632

Simulating real-life scenarios to better understand the spread of diseases under different contexts

Auteurs: Rafael Blanco, Gustavo Patow, Nuria Pelechano
Publié dans: Scientific Reports, Numéro 14, 2024, ISSN 2045-2322
Éditeur: Nature Publishing Group
DOI: 10.1038/s41598-024-52903-w

Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer‐Grade VR Devices

Auteurs: J. L. Ponton, H. Yun, C. Andujar, N. Pelechano
Publié dans: Computer Graphics Forum, Numéro 41, 2023, Page(s) 107-118, ISSN 0167-7055
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/cgf.14628

Authoring Virtual Crowds: A Survey

Auteurs: Marilena Lemonari, Rafael Blanco, Panayiotis Charalambous, Nuria Pelechano, Marios Avraamides, Julien Pettré, Yiorgos Chrysanthou
Publié dans: Computer Graphics Forum, Numéro 41, 2024, Page(s) 677-701, ISSN 0167-7055
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/cgf.14506

SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data

Auteurs: Jose Luis Ponton, Haoran Yun, Andreas Aristidou, Carlos Andujar, Nuria Pelechano
Publié dans: ACM Transactions on Graphics, Numéro 43, 2023, Page(s) 1-14, ISSN 0730-0301
Éditeur: Association for Computing Machinary, Inc.
DOI: 10.1145/3625264

Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic Environments

Auteurs: Eduardo Alvarado, Damien Rohmer, Marie‐Paule Cani
Publié dans: Computer Graphics Forum, Numéro 41, 2024, Page(s) 169-181, ISSN 0167-7055
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/cgf.14633

A Survey on Reinforcement Learning Methods in Character Animation

Auteurs: Ariel Kwiatkowski, Eduardo Alvarado, Vicky Kalogeiton, C. Karen Liu, Julien Pettré, Michiel van de Panne, Marie‐Paule Cani
Publié dans: Computer Graphics Forum, Numéro 41, 2024, Page(s) 613-639, ISSN 0167-7055
Éditeur: Blackwell Publishing Inc.
DOI: 10.1111/cgf.14504

The One-Man-Crowd: Single User Generation of Crowd Motions Using Virtual Reality

Auteurs: Tairan, Yin/Inria, Univ Rennes, CNRS, IRISA, France; Ludovic Hoyet/Inria, Univ Rennes, CNRS, IRISA, France; Marc Christie/ Inria, Univ Rennes, CNRS, IRISA, France; Marie-Paule Cani/Ecole Polytechnique, CNRS (LIX), IP Paris, France; Julien Pettré/Inria, Univ Rennes, CNRS, IRISA, France
Publié dans: IEEE Transactions on Visualization and Computer Graphics/28/5/2245-2255, 2021, ISSN 1941-0506
Éditeur: IEEE
DOI: 10.1109/tvcg.2022.3150507

Ubiq-exp: A toolkit to build and run remote and distributed mixed reality experiments

Auteurs: Anthony Steed, Lisa Izzouzi, Klara Brandstätter, Sebastian Friston, Ben Congdon, Otto Olkkonen, Daniele Giunchi, Nels Numan, David Swapp
Publié dans: Frontiers in Virtual Reality, Numéro 3, 2022, ISSN 2673-4192
Éditeur: Frontiers in Virtual Reality
DOI: 10.3389/frvir.2022.912078

With or Without You: Effect of Contextual and Responsive Crowds on VR-based Crowd Motion Capture

Auteurs: Tairan Yin, Ludovic Hoyet, Marc Christie, Marie-Paule Cani, Julien Pettré
Publié dans: IEEE Transactions on Visualization and Computer Graphics, Numéro 30, 2024, Page(s) 2785-2795, ISSN 1077-2626
Éditeur: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tvcg.2024.3372038

Real-Time Locomotion on Soft Grounds With Dynamic Footprints

Auteurs: Eduardo Alvarado, Chloé Paliard, Damien Rohmer, Marie-Paule Cani
Publié dans: Frontiers in Virtual Reality, 2022, ISSN 2673-4192
Éditeur: Frontiers
DOI: 10.3389/frvir.2022.801856