Final Report Summary - PROPEREMO (Production and perception of emotion: An affective sciences approach)
Emotion has fascinated thinkers and scientists over the last two millennia. The ERC project “Production and perception of emotion” (PROPEREMO), directed by Klaus Scherer at the Swiss Centre for Affective Sciences at the University of Geneva, has adopted a theory-guided, interdisciplinary approach to study emotion empirically. The theoretical framework consists of Scherer’s Component Process Model of emotion (CPM) proposing that the subjective appraisal of important events causes emotion episodes consisting of dynamically interrelated changes in multiple components of an organism’s state (motivational shifts, physiological responses, motor expression, and subjective feeling). With funding from the ERC Advanced grant and other sources, the Geneva research group has achieved major advances in our understanding of both a) the production and unfolding of emotions in the individual, and b) the perception and inference of such emotions by observers in a social context, documented in numerous publications. In collaboration with researchers from many different disciplines in psychology, the social and behavioural sciences, the humanities, the medical sciences and computing and engineering science, significant progress has been made with respect to the following major issues:
What exactly is an emotion? From an evolutionary perspective, we have developed a set of minimal conditions for defining and using the term emotion: an affective episode that is directed at an event or object following an evaluation via multiple appraisals of the significance of the event for the person's well-being (intentionality). For the purpose of behavioural adaptation, this episode consists of bodily changes (arousal, expression), thoughts, and feelings that are driven by the appraisal results in a repetitive (recursive) fashion, leading to an increased coherence of the organism’s response systems. In addition, we have proposed criteria to distinguish a) normal and abnormal (pathological) emotional responses, b) utilitarian (adaptive) vs. aesthetic and epistemic emotions, and c) different facets of emotional competence and emotion dispositions.
What is the nature of the appraisal process that triggers and shapes an emotion episode? We have empirically demonstrated that the appraisal criteria theoretically predicted by the CPM all contribute to shaping the emotion process and that there are important interactions between the different appraisals, strongly mediating the final outcome. Using measures of brain activity, physiological responses, and facial/vocal/bodily expressions, we confirmed the prediction that the results of specific appraisals occur in a fixed sequence: novelty intrinsic pleasantness goal conduciveness coping potential (control and power) moral appropriateness. Using physiological measures with high temporal resolution, we have been able to establish the mental chronography of the onset of different appraisals in the temporal range of 80 to 800 ms. We have also developed a theoretical framework describing different types of valence criteria. In a large scale experiment, we have studied the effect of valence appraisals in different modalities (visual, auditory, and olfactory) on a large number of different responses. We are using the massive data sets generated in this research to examine the process of response synchronization between components (brain activity, physiological responses, motor expression, and reported feeling).
What are the precise mechanisms underlying emotion expression production and perception? Using behavioural micro-coding of facial expressions and gestures/body movements as well as digital acoustic analysis of vocal expressions, we confirmed the CPM prediction that expression patterns vary greatly for different members of emotion families (e.g. irritation vs. rage, anxiety vs. panic fear). In particular, we were able to demonstrate, for the first time, subtle differences between different types of positive emotions. We also confirmed the presumed link between emotion categories (e.g. anger, joy) and emotion dimensions (e.g. valence, power), for both vocal (pitch, loudness, perturbation, and spectral composition) and facial expression (facial muscle movements). As to emotion perception, we produced a comprehensive review of past research in this area with a special emphasis on the issue of universality versus cultural relativity. Using both actor portrayals and systematically manipulated computer synthesis of realistic facial muscle movements, we examined the type of informational cues that observers use to infer appraisals, action tendencies, and social signals, in addition to overall emotion quality, from expressions. Finally, we examined individual differences in emotion recognition ability, allowing us to construct and validate several new tests of this ability. The automatic recognition of emotion plays a major role in human-computer-interaction technology, having led to the development of an influential interdisciplinary affective computing research tradition. We have participated in these developments from the outset and have directly collaborated with a group at the Technical University of Munich (added beneficiary) on a number of studies on automatic emotion recognition in speech and singing.
How are emotions expressed in words? And what is the role of language and culture? In a massive interdisciplinary and cross-cultural study, we examined the profiles consisting of many features representing all emotion components for major emotion words in over 25 different languages (GRID study). The results, reported in a major book and several articles, show that the underlying emotional space consists of four (rather than the classic two) dimensions (valence, power/coping, arousal, and unpredictability). While these universal dimensions explain a substantial amount of the variance, there are also language- and culture-specific differences in the profiles, suggesting some degree of linguistic or cultural relativity in the naming of emotions. The latter have been and continue to be explored in case studies with languages from many different families and local variations (e.g. standard Indonesian vs. Minangkabau; mainland, Taiwan, and Singaporean Chinese; Maoris speaking Te Reo or English). We developed a short form of the GRID instrument, which has been used to examine larger numbers of affect terms (more than 80 for French and English) as well as special subsets of terms (e.g. describing achievement emotions or aesthetic emotions). It is of particular interest to study the naming of more subtle emotions, such as the aesthetic emotions evoked by music, paintings, poems and other forms of art. Here, we continued the development of our Geneva Emotional Music Scale (GEMS), and in collaboration with researchers from psychology, philosophy and literary sciences at the Freie Universität Berlin (added beneficiary), we developed and validated a new scale for the measurement of aesthetic emotions (AESTHEMOS).
What exactly is an emotion? From an evolutionary perspective, we have developed a set of minimal conditions for defining and using the term emotion: an affective episode that is directed at an event or object following an evaluation via multiple appraisals of the significance of the event for the person's well-being (intentionality). For the purpose of behavioural adaptation, this episode consists of bodily changes (arousal, expression), thoughts, and feelings that are driven by the appraisal results in a repetitive (recursive) fashion, leading to an increased coherence of the organism’s response systems. In addition, we have proposed criteria to distinguish a) normal and abnormal (pathological) emotional responses, b) utilitarian (adaptive) vs. aesthetic and epistemic emotions, and c) different facets of emotional competence and emotion dispositions.
What is the nature of the appraisal process that triggers and shapes an emotion episode? We have empirically demonstrated that the appraisal criteria theoretically predicted by the CPM all contribute to shaping the emotion process and that there are important interactions between the different appraisals, strongly mediating the final outcome. Using measures of brain activity, physiological responses, and facial/vocal/bodily expressions, we confirmed the prediction that the results of specific appraisals occur in a fixed sequence: novelty intrinsic pleasantness goal conduciveness coping potential (control and power) moral appropriateness. Using physiological measures with high temporal resolution, we have been able to establish the mental chronography of the onset of different appraisals in the temporal range of 80 to 800 ms. We have also developed a theoretical framework describing different types of valence criteria. In a large scale experiment, we have studied the effect of valence appraisals in different modalities (visual, auditory, and olfactory) on a large number of different responses. We are using the massive data sets generated in this research to examine the process of response synchronization between components (brain activity, physiological responses, motor expression, and reported feeling).
What are the precise mechanisms underlying emotion expression production and perception? Using behavioural micro-coding of facial expressions and gestures/body movements as well as digital acoustic analysis of vocal expressions, we confirmed the CPM prediction that expression patterns vary greatly for different members of emotion families (e.g. irritation vs. rage, anxiety vs. panic fear). In particular, we were able to demonstrate, for the first time, subtle differences between different types of positive emotions. We also confirmed the presumed link between emotion categories (e.g. anger, joy) and emotion dimensions (e.g. valence, power), for both vocal (pitch, loudness, perturbation, and spectral composition) and facial expression (facial muscle movements). As to emotion perception, we produced a comprehensive review of past research in this area with a special emphasis on the issue of universality versus cultural relativity. Using both actor portrayals and systematically manipulated computer synthesis of realistic facial muscle movements, we examined the type of informational cues that observers use to infer appraisals, action tendencies, and social signals, in addition to overall emotion quality, from expressions. Finally, we examined individual differences in emotion recognition ability, allowing us to construct and validate several new tests of this ability. The automatic recognition of emotion plays a major role in human-computer-interaction technology, having led to the development of an influential interdisciplinary affective computing research tradition. We have participated in these developments from the outset and have directly collaborated with a group at the Technical University of Munich (added beneficiary) on a number of studies on automatic emotion recognition in speech and singing.
How are emotions expressed in words? And what is the role of language and culture? In a massive interdisciplinary and cross-cultural study, we examined the profiles consisting of many features representing all emotion components for major emotion words in over 25 different languages (GRID study). The results, reported in a major book and several articles, show that the underlying emotional space consists of four (rather than the classic two) dimensions (valence, power/coping, arousal, and unpredictability). While these universal dimensions explain a substantial amount of the variance, there are also language- and culture-specific differences in the profiles, suggesting some degree of linguistic or cultural relativity in the naming of emotions. The latter have been and continue to be explored in case studies with languages from many different families and local variations (e.g. standard Indonesian vs. Minangkabau; mainland, Taiwan, and Singaporean Chinese; Maoris speaking Te Reo or English). We developed a short form of the GRID instrument, which has been used to examine larger numbers of affect terms (more than 80 for French and English) as well as special subsets of terms (e.g. describing achievement emotions or aesthetic emotions). It is of particular interest to study the naming of more subtle emotions, such as the aesthetic emotions evoked by music, paintings, poems and other forms of art. Here, we continued the development of our Geneva Emotional Music Scale (GEMS), and in collaboration with researchers from psychology, philosophy and literary sciences at the Freie Universität Berlin (added beneficiary), we developed and validated a new scale for the measurement of aesthetic emotions (AESTHEMOS).