FP5 project prototypes computer with emotional intelligence
Researchers have developed and prototyped a computer capable of human-computer interaction and realistic emotional expressions under a Fifth Framework Programme (FP5) project The ten partners of the ERMIS (Emotionally Rich Man-machine Intelligent System) project, funded under the IST (information society technologies) priority, focused on linguistic and paralinguistic cues in human speech in order to produce a computer character able to interpret its users' attitude or emotional state. The computer can, through the analysis of the user's speech, facial gestures and expressions, tell whether he or she is bored, angry or interested. To achieve this, the partners used technologies such as linguistic and paralinguistic speech analysis, robust speech recognition, facial expression analysis, and interpretation of the user's emotional state through hybrid, neurofussy, techniques. In an initial phase, the team extracted some 400 features of common speech, then picked around 20 to 25 of them as the most important in expressing emotion. For facial expression, some 19 were selected as the most relevant. These terms were then fed into a neural network architecture in order to be analysed. In a second phase, the input obtained was then incorporated into a prototype system with different on-screen characters, each of which is capable of reacting to and reproducing the emotional content in speech and facial expressions of human subjects. As the project partners explained, by interacting with their human users, the computer characters would attempt to make them angry, happy, sad or bored. The result of the ERMIS project is a 'sensitive artificial listener', a computer character capable of much more realistic expression of emotions in human communication than has previously existed. The project partners, which include British and French Telecom and Nokia, are now analysing these results with a view to using them according to their specific requirements. The project results have also led to a follow-on initiative - the four-year HUMAIN project funded under the Sixth Framework Programme (FP6).