Affective Computing Models: from Facial Expression to Mind-Reading

Informazioni relative al progetto

ACMod

ID dell’accordo di sovvenzione: 101130271

DOI

10.3030/101130271

Data della firma CE 5 Dicembre 2023

Data di avvio 1 Marzo 2024

Data di completamento 29 Febbraio 2028

Finanziato da

• Marie Skłodowska-Curie Actions (MSCA)

Costo totale

Nessun dato

Contributo UE

€ 846 400,00

Coordinato da

OULUN YLIOPISTO
Finland

Periodic Reporting for period 1 - ACMod (Affective Computing Models: from Facial Expression to Mind-Reading)

Periodo di rendicontazione: 2024-03-01 al 2026-02-28

Sintesi del contesto e degli obiettivi generali del progetto

The ACMod project addresses one of the key challenges in human communication and human-computer interaction: understanding facial expressions and emotional states across different cultures. While facial expression analysis technologies have made significant progress in recent years, current systems still struggle to interpret emotions reliably in realistic and culturally diverse environments. In particular, differences in how emotions are expressed and perceived across cultures challenge the idea of universal facial expressions and limit the development of truly adaptive intelligent systems.

ACMod aims to advance the understanding and modelling of naturalistic facial behaviours by combining expertise from psychology, affective computing, computer vision, and artificial intelligence. The project focuses on reconstructing facial macro- and micro-expressions, analysing physiological and behavioural signals related to emotions, and developing autonomous 4D facial models capable of representing dynamic emotional states. To support this work, the consortium is building multimodal datasets combining facial data, depth information, and physiological signals collected in cross-cultural contexts.

The project brings together European and East Asian academic and industrial partners through the MSCA Staff Exchanges programme, promoting interdisciplinary collaboration and long-term knowledge exchange. Through secondments, workshops, joint research activities, and shared datasets, ACMod strengthens international research networks and supports the training and career development of researchers.

By improving the modelling of emotional behaviours and realistic facial interactions, ACMod aims to contribute to the next generation of human-centred technologies, including intelligent virtual agents, social robotics, affective computing systems, and augmented or virtual reality applications. The project also contributes to ongoing discussions on ethical and responsible use of emotion-aware AI technologies, including privacy, digital identity, and trustworthy human-machine interaction.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

ACMod aims to improve how intelligent systems understand and reproduce human emotions and facial behaviours in realistic and culturally diverse environments. To achieve this, the project combines expertise from psychology, artificial intelligence, computer vision, and affective computing to study how emotions are expressed through facial movements, physiological responses, and social interactions.

A central objective of the project is the development of multimodal approaches that combine facial images, dynamic 3D facial information, and physiological signals such as heart activity or muscle responses. By bringing these different sources of information together, ACMod seeks to create more reliable and human-centred models of emotion understanding than those currently used in conventional facial expression analysis systems.

The project also explores new methods for reconstructing and generating realistic 3D and 4D facial behaviours, including expressive virtual avatars capable of displaying natural emotional reactions. These developments contribute to the long-term goal of improving interaction with intelligent agents, social robots, and virtual or augmented reality systems, making them more adaptive, expressive, and socially aware.

Beyond the technological aspects, ACMod promotes extensive international collaboration and interdisciplinary knowledge exchange between European and East Asian research partners. Through shared datasets, workshops, scientific exchanges, and joint research activities, the project contributes to the development of a broader research community working on emotion-aware artificial intelligence and human-centred digital technologies.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

ACMod introduces a new interdisciplinary approach to facial behaviour modelling by combining psychological theories of emotion with advanced artificial intelligence and computer vision techniques. Rather than focusing only on visible facial expressions, the project studies how emotions are linked to both external facial behaviours and internal physiological responses, enabling a more complete understanding of human affective states.

Current facial analysis systems are often limited to standard image-based approaches and may struggle in realistic social situations or across different cultural contexts. ACMod moves beyond these limitations by integrating multiple types of information, including dynamic 3D facial data, physiological signals, and behavioural analysis. This allows the project to explore more realistic, robust, and culturally aware models of emotion recognition and expression.

The project also advances the development of realistic digital humans and expressive virtual avatars capable of reproducing subtle facial movements and emotional behaviours. Such technologies could support future applications in human-computer interaction, social robotics, healthcare, education, virtual communication, and immersive digital environments.

At the same time, ACMod places ethical and societal considerations at the very centre of the project. This is particularly important given the highly sensitive nature of facial, behavioural, and physiological data, as well as the growing capability of AI systems to infer emotional states from everyday consumer technologies. By integrating ethical reflection, responsible data governance, and external ethical oversight directly into the research process, ACMod promotes a new model for the development of trustworthy and human-centred AI systems, where technological innovation and societal safeguards evolve together.