Skip to main content
European Commission logo print header

Virtual ICT with Empathic Characters

Deliverables

VICTEC committed itself from the beginning to a user-centred design approach in which teachers and more specifically children - the target users - would have a key role in the design of the FearNot! demonstrator. The wisdom of this approach was underlined early in the project when the team discovered that realistic bullying scenarios could not be found in the scholarly or educational literature. Two mechanisms were used for involving users in design. One was the use of the simple one-episode scripted sketch of what the FearNot! demonstrator might look like that was created by the Minerva-funded VES project with which VICTEC collaborated. This ought to have been a part of the work programme from the start but the need for such a 'quick-and-dirty' prototype system was overlooked. The Trailer, as it was called, was used in conjunction with a questionnaire to gain feedback from a variety of groups on a variety of aspects: quality of graphics, animation, and sound; believability of characters and stories; the amount and nature of interaction; what feelings the single episode evoked both in the user and for the characters. Interesting results were gained here: children did empathise with the victim character in the Trailer and felt anger towards the bully, while teachers disliked the victim character and liked the bully character. Technical experts (mostly male and in the mid-to-late twenties) did not have strong reactions to any of the characters but tended to prefer the one female character. Children were critical of the production values of the Trailer, assessing the graphics, animation and sound as of low quality, but it appeared that the realism of the stories meant that in spite of this they found the characters believable. Stories were collected from children using the story-boarding tool Kar2ouche, kindly made available to the project by a UK-based educational software company, Immersive Education. The team noted that while the bullying incidents in the stories seemed very credible, attempts to offer a solution to the problem at the end were often less so, with a frequent use of a deus ex machine - a teacher or head teacher - who often expelled the bully. Interestingly the other source of stories lay in drama, with a number of scenarios developed by students in the University of Hertfordshire Drama Department, later scripted by the project and recorded as source material for investigating the use of speech technology for the FearNot! characters. The project regards the successful involvement of children and teachers in the design of aspects of FearNot! as an important achievement, giving the team confidence as they worked on the development of the technology that the work was relevant and useful.
The results comprise a large-scale evaluation of the software developed by the project in the UK, Germany and Portugal with a total sample of around 800 children. The results are about to be published in several psychology journals. The involvement of schools in Germany is also a key result, since the end users (children) and teachers contributed in the development of scenarios, the evaluation of the software and its implementation in school curricula. Furthermore, the development of a theoretical basis for the empathic relationship between human users and synthetic characters was achieved, and the specific role of ideomotoric empathy was explored. These specific aspects will be further investigated in further projects on both national and European level.
The SenToy had been developed by the INESC-ID partner in the earlier EU project Safira as a novel interaction device used in a duelling game in which the emotional state of the two game characters duelling had to be influenced. Conscious of the limitations of keyboard and mouse interaction, VICTEC included further development of and experiments with SenToy in the context of the FearNot! demonstrator. SenToy, is a large, soft, neutal doll-shaped toy equipped with sensors discussed at more length in D4.2.1. Children are able to make gestures with it, which can be picked up and analysed, as an emotional state, from a small set of five or six emotional states. Other documents have discussed the interaction issue surrounding the use of SenToy. The 'at a distance' stance of the child with respect to the virtual drama, in which they could see how the victim responded to their earlier advice, but not directly control their action, was adopted for several good reasons, also discussed elsewhere. In brief, the team felt that direct participation would give the child too privileged a position (since the bully could not actually hurt them), would undermine the believability of the characters (such intervention could not take place in the real-world case) and would hinder the reflective approach needed for the child to benefit from the exploration of different strategies. For these reasons, the SenToy could not be used in the same way as in the Safira application, which in any case was a game. FearNot! was definitely not intended to be a game. A further constraint lay in the number of SenToys available to the project: the original SenToy with upgraded hardware, and a second one constructed for the project. This was of course not compatible with large-scale evaluation and even producing one SenToy for use in each country was beyond the resources of the project. It was therefore decided to experiment with modes of operation rather than run any large-scale evaluation. After some discussion, the interaction mode adopted was for the child user to use the SenToy as a means of providing an emotional commentary on the action-taking place in FearNot! For this purpose, a face was added to the FearNot! Interface, which indicated which emotion the child was trying to convey. A small-scale evaluation was carried out which indicated that the children were able in most cases to produce commentary emotions, though there was some conflict between handling the SenToy and operating the keyboard to advance the action. This activity can be seen in the project video for the SenToy. The reason for choosing this function for the SenToy was that questionnaires applied after the drama capture the final feelings of the child and do not record any changes as the story progresses. It was felt to be potentially interesting to compare questionnaire results with emotions signalled in commentary form. As a result of carrying out this experiment, the team realised that another interesting use of the SenToy would be to express theory of mind capabilities - instead of the child user expressing how they felt at points in the story they could be asked to express what they thought one of the characters was feeling. The differences between the use of the SenToy in the Safira project and in VICTEC were considerable, and the team drew the lesson that new interaction devices have to be part of an overall definition of interaction modalities. FearNot! was essentially designed for keyboard interaction, inevitably given the current state-of-the-art in speech and language. Using the SenToy alongside it - even though most of the small evaluation group enjoyed it - was relatively cumbersome. The project achieved what it had intended in investigating a novel form of interaction, and the SenToy was positively received by the small set of subjects, but further work would be needed to integrate it properly with FearNot!
An achievement of the project was establishing new relationships between the variables tested in the University of Herts evaluation exercise, relating to bullying status, empathic ability and theory of mind (TOM). These results are discussed in detail in D7.2.1, but such was the amount of data collected that further results after the end of the project is expected. Results were affected both by country and by gender. UK pupils scored higher on affective and cognitive empathy than Germans, with Portuguese pupils scoring in between: UK and Portuguese pupils show more affective empathy than German pupils while Portuguese and German pupils score lower on cognitive empathy than the UK pupils. This result may relate to the emphasis that is put on Personal and Social Education (PSE) in the UK. In all countries involved in the project, female pupils have higher affective empathy scores than male pupils, in line with existing results from empathy research. The bullying questionnaire completed by both pupils and teachers, expectations confirmed that physical bullying can be more easily detected both by pupils and teachers than relational bullying and especially relational victimisation. This supports the idea that relational bullying takes place behind the back of the teacher and parts of the class. Portuguese pupils named significantly more victims of relational bullying than pupils from any other country. Portuguese results also displayed gender differences both for relational and physical bullying, with males being over-represented in the bullies and victims categories, and females being more frequently assigned to the neutrals category. Usability evaluation showed that children enjoy interacting with the software and that they empathise with the characters even though the physical appearance of the characters has been rated rather poorly, indicating that they seem to infer the emotions from situational cues rather than from the characters¿ facial and body expression. Pupils also preferred the interactive application to a teacher-led session on the topic, mainly because of matters of privacy.
The framework represents a principled way of bringing together AI and graphical components. It applies a multi-agent architecture with a clean interface between components, organised around a symbolic world model. It allows reuse of characters between visualisation systems and supports domain-specific ontologies. It has currently been implemented in two versions, one completely in java and the other with a C sharp bridge using .net. It is being reissued in other projects and allows new systems to be developed in much less time since the same components do not have to be completely reinvented each tome but can be reused.
As our major aim was to create Empathic Synthetic Characters, we considered some requirements for our agents, in particular properties such as personality, believability and empathy. In order to do so, we had to develop a bit more deeply the notions of empathy and believability in the architecture below. The capability of emergent FearNot! was entirely based on the ability of the empathic agent architecture developed by the project to produce character actions that would lead to interesting stories. Thus the architecture was a key part of technical development, applying theories of appraisal and coping behaviour from Ortony, Clore and Collins (OCC) and Lazarus respectively in an emotion-driven action-selection system. Novel aspects included the use of the emotions Fear and Hope to manage character goals and the integration of language and physical actions into one action-selection mechanism. Because of its link to the emergent narrative mechanism, development of this architecture continued right to the end of the project as part of the bottom-up engineering process required to develop. The completed architecture is a very flexible one with potential for reuse in other PSE domains than bullying: The achievement of this architecture was that it made possible the innovative unscripted approach of emergent FearNot! while retaining a generic XML-driven structure laying the basis for later reuse.
It had been foreseen in the project proposal that the other generic component of the VLE would be an authoring tool supporting the generation of both graphical and AI-based content for the Framework. Just as the Framework had to encompass both graphical and AI elements, so the authoring tool was required to handle graphical virtual worlds, 3D graphical character bodies and animations, as well as the personality traits and behaviour repertoire required internally to drive the graphical elements. It had been decided early in the project that it was unrealistic to aim this tool at teachers and students: inspection of the 2D commercial tool Kar2ouche brought home just how much development effort would be involved in doing this. The authoring system was therefore always intended for the use of experts such as project team members. The overall design of this system used XML output as a way of configuring library content: it would have been futile to try to produce a 3D graphics tool given that existing tools such as 3ds Max are already so widely used. The personality configuration element of the tool was clearly tied to the agent architecture, and for this reason, after the production of the first prototype-authoring tool early in 2003; work was suspended until the finalisation of the agent architecture. However, though the gross structure of this architecture was produced according to the project plan, the dependence of emergent narrative on the detailed structure of the agent architecture meant that smaller changes carried on right up until the production of emergent FearNot! at the end of the project. This was a result of the bottom-up testing strategy that is inevitable if emergence is being used as the main mechanism, and it created problems in finalising the output of the authoring tool. The second prototype of the authoring tool used a much more robust structure in which the library content was held in an SQL database, and the data storage, manipulation and display layers were clearly separated. The project has through this achieved a platform which can be further developed and which would serve as the basis for a more widely usable tool in a future project.
The project development team produced and delivered two prototypes of the `FearNot! demonstrator for the project: a scripted version and one driven by character interaction, the emergent version. In both cases, the structure of the demonstrator includes an introduction to the characters and the situation, and three episodes in which a bully character attempts to bully a victim character. In between each episode, the victim goes to the library and asks the child user to give them advice about what to do. The scripted version was used for all the large-scale evaluation sessions in all three countries; the emergent version was produced at the end of the project and a small-scale evaluation was carried out in Portugal only. The scripted FearNot! used text output for most of the character utterances in the version used for the evaluation, but an updated version with speech recorded for the characters in both the physical bullying and relational bullying was created for use in talks and presentations. This version also included improved graphics, with better animations and modified character bodies, developed for the emergent version produced at the same time. The Emergent version of FearNot! has no scripting of the action within an episode- the character action-selection systems allow the characters to select from their repertoire of actions on the fly. A certain amount of physical indeterminacy is also built in to this version. This means for example that if the victim is advised by the child to hit the bully back, and if the character's state of confidence is high enough to do this, sometimes the bully falls over and sometimes he does not. If he does, this impacts his level of confidence so that he desists from any further bullying in that episode. But if the bully is not seriously affected by the victim's blow then he becomes angry and is likely to hit the victim back even harder. This makes the exact sequence of events in the episode and the eventual outcome hard to predict, and the fact that this outcome has an effect on the emotional state of the characters involved means that differences propagate through the episodes.
The project succeeded in meeting its objective of building long-term relationships with schools, starting with the successful teachers' workshop organised in October 2002. This laid the basis for later work with the group of schools from which teachers attended. A second teachers' workshop was organised by the VES project already mentioned in October 2003, and many team members attended this. VES already had links with German schools in the Kassel area, and it was thanks to their support that the Bamberg partners were able to build on the contacts made with these schools to involve them also in VICTEC. The user-centred design work already discussed and the pedagogical evaluation also helped to maintain contact between the VICTEC team and associated schools. Each participating school will receive a CD containing the FearNot! software and copies of the VICTEC brochure produced as an annex (Annex 2) to this document. The large number of schools in the UK reflects the involvement of a large number of local schools in the University of Herts evaluation event in June 2004. The eCIRCUS bid already mentioned has involved this same set of schools and should this proposal be funded the existing relationships would be continued and further strengthened. COUNTRY- ASSOCIATED SCHOOLS UK: Hillside County Primary School, Helsby St Mark's C Of E Primary School, Worsley St Andrew's Methodist School, Salford St Paul's C of E School, Salford Wardley C Of E Primary School, Swinton Saint Willibrords RC primary school, Clayton Commonswood School, Welwyn Garden City Swallow Dell Primary School. Welwyn Garden City St Nicholas CE Primary School, Stevenage St Johns CE JMI School, Welwyn Garden City New Briars School, Briars Lane, Hatfield Little Heath Primary School, Potters Bar Hazel Grove Primary School, Hatfield Creswick JMI, Howlands, Welwyn Garden City Welwyn St Mary's CE primary school, Welwyn Portugal: Escola N?1 de Sao Joao do Estoril (Sao Joao do Estoril Elementary School N?1)

Searching for OpenAIRE data...

There was an error trying to search data from OpenAIRE

No results available