Affective Computing Models: from Facial Expression to Mind-Reading

Informations projet

ACMod

N° de convention de subvention: 101130271

DOI

10.3030/101130271

Date de signature de la CE 5 Decembre 2023

Date de début 1 Mars 2024

Date de fin 29 Février 2028

Financé au titre de

• Marie Skłodowska-Curie Actions (MSCA)

Coût total

Aucune donnée

Contribution de l’UE

€ 846 400,00

Coordonné par

OULUN YLIOPISTO
Finland

Periodic Reporting for period 1 - ACMod (Affective Computing Models: from Facial Expression to Mind-Reading)

Période du rapport: 2024-03-01 au 2026-02-28

Résumé du contexte et des objectifs généraux du projet

The ACMod project addresses one of the key challenges in human communication and human-computer interaction: understanding facial expressions and emotional states across different cultures. While facial expression analysis technologies have made significant progress in recent years, current systems still struggle to interpret emotions reliably in realistic and culturally diverse environments. In particular, differences in how emotions are expressed and perceived across cultures challenge the idea of universal facial expressions and limit the development of truly adaptive intelligent systems.

ACMod aims to advance the understanding and modelling of naturalistic facial behaviours by combining expertise from psychology, affective computing, computer vision, and artificial intelligence. The project focuses on reconstructing facial macro- and micro-expressions, analysing physiological and behavioural signals related to emotions, and developing autonomous 4D facial models capable of representing dynamic emotional states. To support this work, the consortium is building multimodal datasets combining facial data, depth information, and physiological signals collected in cross-cultural contexts.

The project brings together European and East Asian academic and industrial partners through the MSCA Staff Exchanges programme, promoting interdisciplinary collaboration and long-term knowledge exchange. Through secondments, workshops, joint research activities, and shared datasets, ACMod strengthens international research networks and supports the training and career development of researchers.

By improving the modelling of emotional behaviours and realistic facial interactions, ACMod aims to contribute to the next generation of human-centred technologies, including intelligent virtual agents, social robotics, affective computing systems, and augmented or virtual reality applications. The project also contributes to ongoing discussions on ethical and responsible use of emotion-aware AI technologies, including privacy, digital identity, and trustworthy human-machine interaction.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

ACMod aims to improve how intelligent systems understand and reproduce human emotions and facial behaviours in realistic and culturally diverse environments. To achieve this, the project combines expertise from psychology, artificial intelligence, computer vision, and affective computing to study how emotions are expressed through facial movements, physiological responses, and social interactions.

A central objective of the project is the development of multimodal approaches that combine facial images, dynamic 3D facial information, and physiological signals such as heart activity or muscle responses. By bringing these different sources of information together, ACMod seeks to create more reliable and human-centred models of emotion understanding than those currently used in conventional facial expression analysis systems.

The project also explores new methods for reconstructing and generating realistic 3D and 4D facial behaviours, including expressive virtual avatars capable of displaying natural emotional reactions. These developments contribute to the long-term goal of improving interaction with intelligent agents, social robots, and virtual or augmented reality systems, making them more adaptive, expressive, and socially aware.

Beyond the technological aspects, ACMod promotes extensive international collaboration and interdisciplinary knowledge exchange between European and East Asian research partners. Through shared datasets, workshops, scientific exchanges, and joint research activities, the project contributes to the development of a broader research community working on emotion-aware artificial intelligence and human-centred digital technologies.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

ACMod introduces a new interdisciplinary approach to facial behaviour modelling by combining psychological theories of emotion with advanced artificial intelligence and computer vision techniques. Rather than focusing only on visible facial expressions, the project studies how emotions are linked to both external facial behaviours and internal physiological responses, enabling a more complete understanding of human affective states.

Current facial analysis systems are often limited to standard image-based approaches and may struggle in realistic social situations or across different cultural contexts. ACMod moves beyond these limitations by integrating multiple types of information, including dynamic 3D facial data, physiological signals, and behavioural analysis. This allows the project to explore more realistic, robust, and culturally aware models of emotion recognition and expression.

The project also advances the development of realistic digital humans and expressive virtual avatars capable of reproducing subtle facial movements and emotional behaviours. Such technologies could support future applications in human-computer interaction, social robotics, healthcare, education, virtual communication, and immersive digital environments.

At the same time, ACMod places ethical and societal considerations at the very centre of the project. This is particularly important given the highly sensitive nature of facial, behavioural, and physiological data, as well as the growing capability of AI systems to infer emotional states from everyday consumer technologies. By integrating ethical reflection, responsible data governance, and external ethical oversight directly into the research process, ACMod promotes a new model for the development of trustworthy and human-centred AI systems, where technological innovation and societal safeguards evolve together.