Final Activity Report Summary - AFFECTME (Affective Multimodal Engagement) In today's society, computers are increasingly present in our daily life. Affective computing is about empowering computers with the ability to recognise as well as elicit user emotions. So far, most of the research has been concentrating on facial and verbal cues as these have long been thought to be most important. The focus of this research, instead, is on bodily expressions of affect, and how they can be automatically recognised. The main achievements of this project over the last two years are three-fold: 1. Creation of a corpus of affective postures for use by the affective computing community. A large set of postures was collected across different scenarios, from chronic pain patients during physiotherapy exercises through various types of whole-body game players. Different case studies were designed to cover the various factors that elicit a large variety of affective bodily expressions, in particular, level of interest (whether the person is immersed or not), social context (whether the task is co-operative or individual), effort (whether the task is physically demanding or not). The affective states conveyed by these postures were labelled using affective categories and various affective dimensions (e.g. arousal). The corpus is being continuously updated and is made available upon request on the project website http://www.uclic.ucl.ac.uk/people/n.berthouze/AffectME.html as well as on the EU-funded HUMAINE portal http://emotion-research.net/. 2. Development of techniques for the automatic recognition of non-acted affective postures. A novel approach to the automatic classification of these postures based on low-level descriptions of their features was developed, using various state-of-the-art computational modelling techniques. To validate this approach, a benchmark was created that is based on human recognition performance. Our system was shown to perform as well as human participants in discriminating between complex affective states such as concentration, frustration or triumph. 3. Understanding of the relation between body movement and engagement in games. A recent trend in the development of games has been to offer whole-body game controllers. To date, the relationship between the use of whole-body control and level of engagement in games has not been evaluated. In a series of case studies, we studied the emotional and social components of gaming experience when using whole-body controllers. We proposed a model for the interaction between game-controller, body movement and four known engagement components (hard fun, easy fun, emotional state and social experience). This model has important implications for the design of games (e.g. exertion games) as it provides a principled approach for designing interactions leading to specific levels of engagement. Additionally, it was shown that body movements can be used to measure quality and level of engagement of players, which should be useful to the gaming industry.
