The DE-ENIGMA team have annotated the recordings of 50 (25 British and 25 Serbian) participants in terms of continuous emotion stated, measured in valence (positivity / negativity of the emotion) and arousal (intensity of the emotion). The annotation results were then aligned using dynamic time warping for development of the machine learning method.
Online distribution of the dataset has been shown to be impractical due to the large size of the corpus (~7 TB). Instead, data distribution is now facilitated by delivering of physical storage
medium and is stored at more than one partner to help with data storage and distribution.
The following Prototypes have been delivered since the beginning of the DE-ENIGMA project:
• Prototype 0: the first version of the technical set-up was designed to support data collection.
• Prototype 1: the first technical integration was completed and the first interactive game has was developed. It was evaluated with additional children on the autism spectrum to further reduce the risk of triggering noise sensitivities.
• Prototype 2: This is a robot that responded to simple automatically observed situations. We had envisioned a multimodal recognizer, able to interpret and react appropriately to stress, affect, and interest shown by autistic children. However, this has proven a particularly challenging task for the tech partners; we speculate this is due to the challenge of working with children with autism, who by definition display very different types of behaviours and emotions. A tablet was added so that the child had an extra means of communicating with the robot and going through the therapeutic exercises. Some autonomous robot behaviours were integrated based on the messages generated by observed situations.
• Prototype 3 we originally had envisioned to make a final iteration to the previous prototype and test that on a larger scale with autistic children. We now would like to approach that differently: We have identified a need to more fundamentally explore the notion that robots help children with autism 'because they are predictable'. This assumption is commonplace throughout human robot interaction and autism literature, but to the best of the DE-ENIGMA team’s knowledge, has never been empirically proven. In order to contribute fundamentally to robot interaction with autistic children we want to develop a third prototype, 3A, to carry out an experiment with autistic children at UK locations to test that hypothesis, we believe it will highly impact research if we are successful.
• In order to demonstrate the integrated technical capacity of the robot, we will build a 4th prototype, 3B, where there is multimodal interpretation/reasoning and where we provide teachers with insights the robot has into the children's progress through automated detection. We will user test this capability with expert teachers. This prototype will demonstrate the way the robot and the exercise adapt to the children's responses and will also provide the teacher with feedback on for instance micro-expressions, arousal, engagement, or performance of the children.
Our Dissemination partners maintain the project website and other related online dissemination. We continuously update the website content with news items, biographies of involved staff, publications and DE-ENIGMA was invited to present its work to European leaders at the Tallinn Digital Summit.
