Affective Computing Models: from Facial Expression to Mind-Reading

Información del proyecto

ACMod

Identificador del acuerdo de subvención: 101130271

DOI

10.3030/101130271

Fecha de la firma de la CE 5 Diciembre 2023

Fecha de inicio 1 Marzo 2024

Fecha de finalización 29 Febrero 2028

Financiado con arreglo a

• Marie Skłodowska-Curie Actions (MSCA)

Coste total

Sin datos

Aportación de la UE

€ 846 400,00

Coordinado por

OULUN YLIOPISTO
Finland

Periodic Reporting for period 1 - ACMod (Affective Computing Models: from Facial Expression to Mind-Reading)

Período documentado: 2024-03-01 hasta 2026-02-28

Resumen del contexto y de los objetivos generales del proyecto

The ACMod project addresses one of the key challenges in human communication and human-computer interaction: understanding facial expressions and emotional states across different cultures. While facial expression analysis technologies have made significant progress in recent years, current systems still struggle to interpret emotions reliably in realistic and culturally diverse environments. In particular, differences in how emotions are expressed and perceived across cultures challenge the idea of universal facial expressions and limit the development of truly adaptive intelligent systems.

ACMod aims to advance the understanding and modelling of naturalistic facial behaviours by combining expertise from psychology, affective computing, computer vision, and artificial intelligence. The project focuses on reconstructing facial macro- and micro-expressions, analysing physiological and behavioural signals related to emotions, and developing autonomous 4D facial models capable of representing dynamic emotional states. To support this work, the consortium is building multimodal datasets combining facial data, depth information, and physiological signals collected in cross-cultural contexts.

The project brings together European and East Asian academic and industrial partners through the MSCA Staff Exchanges programme, promoting interdisciplinary collaboration and long-term knowledge exchange. Through secondments, workshops, joint research activities, and shared datasets, ACMod strengthens international research networks and supports the training and career development of researchers.

By improving the modelling of emotional behaviours and realistic facial interactions, ACMod aims to contribute to the next generation of human-centred technologies, including intelligent virtual agents, social robotics, affective computing systems, and augmented or virtual reality applications. The project also contributes to ongoing discussions on ethical and responsible use of emotion-aware AI technologies, including privacy, digital identity, and trustworthy human-machine interaction.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

ACMod aims to improve how intelligent systems understand and reproduce human emotions and facial behaviours in realistic and culturally diverse environments. To achieve this, the project combines expertise from psychology, artificial intelligence, computer vision, and affective computing to study how emotions are expressed through facial movements, physiological responses, and social interactions.

A central objective of the project is the development of multimodal approaches that combine facial images, dynamic 3D facial information, and physiological signals such as heart activity or muscle responses. By bringing these different sources of information together, ACMod seeks to create more reliable and human-centred models of emotion understanding than those currently used in conventional facial expression analysis systems.

The project also explores new methods for reconstructing and generating realistic 3D and 4D facial behaviours, including expressive virtual avatars capable of displaying natural emotional reactions. These developments contribute to the long-term goal of improving interaction with intelligent agents, social robots, and virtual or augmented reality systems, making them more adaptive, expressive, and socially aware.

Beyond the technological aspects, ACMod promotes extensive international collaboration and interdisciplinary knowledge exchange between European and East Asian research partners. Through shared datasets, workshops, scientific exchanges, and joint research activities, the project contributes to the development of a broader research community working on emotion-aware artificial intelligence and human-centred digital technologies.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

ACMod introduces a new interdisciplinary approach to facial behaviour modelling by combining psychological theories of emotion with advanced artificial intelligence and computer vision techniques. Rather than focusing only on visible facial expressions, the project studies how emotions are linked to both external facial behaviours and internal physiological responses, enabling a more complete understanding of human affective states.

Current facial analysis systems are often limited to standard image-based approaches and may struggle in realistic social situations or across different cultural contexts. ACMod moves beyond these limitations by integrating multiple types of information, including dynamic 3D facial data, physiological signals, and behavioural analysis. This allows the project to explore more realistic, robust, and culturally aware models of emotion recognition and expression.

The project also advances the development of realistic digital humans and expressive virtual avatars capable of reproducing subtle facial movements and emotional behaviours. Such technologies could support future applications in human-computer interaction, social robotics, healthcare, education, virtual communication, and immersive digital environments.

At the same time, ACMod places ethical and societal considerations at the very centre of the project. This is particularly important given the highly sensitive nature of facial, behavioural, and physiological data, as well as the growing capability of AI systems to infer emotional states from everyday consumer technologies. By integrating ethical reflection, responsible data governance, and external ethical oversight directly into the research process, ACMod promotes a new model for the development of trustworthy and human-centred AI systems, where technological innovation and societal safeguards evolve together.