Skip to main content

DE-ENIGMA: Multi-Modal Human-Robot Interaction for Teaching and Expanding Social Imagination in Autistic Children

Deliverables

Functional requirements, interactions and constraints

Functional requirements, interactions and constraints

Ethical clearances - update M46

Updated report from D1.1 including copies of further ethical clearances for UCL and SSA.

Report of User Requirements Analysis

Report of User Requirements Analysis: Document with overview of relevant theory, measures and models, specification of user requirements and needs as well as usage scenarios to inform the development of the DE-ENIGMA robot assisted therapy-based game. Preliminary earlier version to be agreed.

Second report on internal evaluation processes

Report on the results of internal evaluation processes during the second year of the project.

Final report on internal evaluation processes

Report on the results of internal evaluation processes during the final period of the project.

Third report on internal evaluation processes

Report on the results of internal evaluation processes during the third year of the project.

Overall Dissemination Plan

Overall Dissemination Plan

Ethical clearances - update M30

Updated report from D1.1 including copies of further ethical clearances for UCL and SSA.

DE-ENIGMA progress report YR2

Report on the general progress of the project in relation to the DoA.

Exploitation Plan

Exploitation Plan

Ethical clearances

Very concise report including copies of ethical clearances for UCL and SSA.

Data Management Plan

Data Management Plan

Ethical clearances - update M21

Updated report from D1.1 including copies of further ethical clearances for UCL and SSA.

DE-ENIGMA progress report YR3

Report on the general progress in the project in relation to the DoA.

DE-ENIGMA final progress report

Report on the general progress of the project in relation to the DoA.

DE-ENIGMA progress report YR1

Report on the general progress in the project in relation to the DoA. Preliminary earlier version to be agreed.

Description of Validation results of full integrated system

Manuscripts describing the behavioural tasks and validation with children with and without autism and the results of the clinical trial, and a manuscript describing the full integrated system.

First report on internal evaluation processes

Report on the results of internal evaluation processes during the first year of the project. Preliminary earlier version to be agreed.

Battery of Validated Behavioural Tasks

Battery of validated behavioural tasks measuring the perception, recognition, understanding and expression of socioemotional cues using robot and human stimuli. These tasks will be made freely available to researchers via our website (probably in a zip file) with documentation.

Publications

Dialogue Design for a Robot-Based Face-Mirroring Game to Engage Autistic Children with Emotional Expressions

Author(s): Pauline Chevalier, Jamy J. Li, Eloise Ainger, Alyssa M. Alcorn, Snezana Babovic, Vicky Charisi, Suncica Petrovic, Bob R. Schadenberg, Elizabeth Pellicano, Vanessa Evers
Published in: Social Robotics. ICSR 2017, Issue LNCS 10652, 2017, Page(s) 546-555
DOI: 10.1007/978-3-319-70022-9_54

An Image-based Deep Spectrum Feature Representation for the Recognition of Emotional Speech

Author(s): Nicholas Cummins, Shahin Amiriparian, Gerhard Hagerer, Anton Batliner, Stefan Steidl, Björn W. Schuller
Published in: Proceedings of the 2017 ACM on Multimedia Conference - MM '17, Issue MM'17, 2017, Page(s) 478-484
DOI: 10.1145/3123266.3123371

Affect bursts to constrain the meaning of the facial expression of the humanoid robot Zeno

Author(s): Schadenberg, B. R., Heylen, D. K. J., & Evers, V.
Published in: Proceedings of the 1st Workshop on Social Interaction and Multimodal Expression for Socially Intelligent Robots (RO-MAN 2017), Issue vol. 2059, 2018, Page(s) 30-39

Does She Speak RTT? Towards an Earlier Identification of Rett Syndrome Through Intelligent Pre-Linguistic Vocalisation Analysis

Author(s): Florian B. Pokorny, Peter B. Marschik, Christa Einspieler, Björn Schuller
Published in: Interspeech 2016, 2016, Page(s) 1953-1957
DOI: 10.21437/Interspeech.2016-520

Automatic Analysis of Typical and Atypical Encoding of Spontaneous Emotion in the Voice of Children

Author(s): Fabien Ringeval, Erik Marchi, Charline Grossard, Jean Xavier, Mohamed Chetouani, David Cohen, Björn Schuller
Published in: Interspeech 2016, 2016, Page(s) 1210-1214
DOI: 10.21437/Interspeech.2016-766

Pairwise Decomposition with Deep Neural Networks and Multiscale Kernel Subspace Learning for Acoustic Scene Classification

Author(s): E. Marchi, D. Tonelli, X. Xu, F. Ringeval, J. Deng, S. Squartini, and B. Schuller
Published in: Proceedings of the Detection and Classification of Acoustic Scenes and Events 2016 IEEE AASP Challenge Workshop, Issue DCASE 2016, 2016, Page(s) 65-69

Is Deception Emotional? An Emotion-Driven Predictive Approach

Author(s): Shahin Amiriparian, Jouni Pohjalainen, Erik Marchi, Sergey Pugachevskiy, Björn Schuller
Published in: Interspeech 2016, 2016, Page(s) 2011-2015
DOI: 10.21437/Interspeech.2016-565

Manual versus Automated: The Challenging Routine of Infant Vocalisation Segmentation in Home Videos to Study Neuro(mal)development

Author(s): Florian B. Pokorny, Robert Peharz, Wolfgang Roth, Matthias Zöhrer, Franz Pernkopf, Peter B. Marschik, Björn Schuller
Published in: Interspeech 2016, 2016, Page(s) 2997-3001
DOI: 10.21437/Interspeech.2016-1341

Deep Learning of Graph Matching

Author(s): Andrei Zanfir, Cristian Sminchisescu
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 2684-2693
DOI: 10.1109/cvpr.2018.00284

3D Human Sensing, Action and Emotion Recognition in Robot Assisted Therapy of Children with Autism

Author(s): Elisabeta Marinoiu, Mihai Zanfir, Vlad Olaru, Cristian Sminchisescu
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 2158-2167
DOI: 10.1109/cvpr.2018.00230

Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints

Author(s): Andrei Zanfir, Elisabeta Marinoiu, Cristian Sminchisescu
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 2148-2157
DOI: 10.1109/cvpr.2018.00229

Human Appearance Transfer

Author(s): Mihai Zanfir, Alin-Ionut Popa, Andrei Zanfir, Cristian Sminchisescu
Published in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018), Issue CVPR 2018, 2018, Page(s) 5391-5399

End-to-End Audiovisual Fusion with LSTMs

Author(s): Petridis, Stavros; Wang, Yujiang; Li, Zuwei; Pantic, Maja
Published in: Proceedings of The 14th International Conference on Auditory-Visual Speech Processing, Issue AVSP2017, 2017

GAGAN: Geometry-Aware Generative Adversarial Networks

Author(s): Kossaifi, Jean; Tran, Linh; Panagakis, Yannis; Pantic, Maja
Published in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Issue CVPR 2018, 2018, Page(s) 878-887

Variational Gaussian Process Auto-Encoder for Ordinal Prediction of Facial Action Units

Author(s): Stefanos Eleftheriadis, Ognjen Rudovic, Marc Peter Deisenroth, Maja Pantic
Published in: Proceedings of the Asian Conference on Computer Vision, Issue ACCV'16, 2017, Page(s) 154-170
DOI: 10.1007/978-3-319-54184-6_10

Social and Affective Robotics Tutorial

Author(s): Maja Pantic, Vanessa Evers, Marc Deisenroth, Luis Merino, Bjoern Schuller
Published in: Proceedings of the 2016 ACM on Multimedia Conference - MM '16, 2016, Page(s) 1477-1478
DOI: 10.1145/2964284.2986914

Gaussian Process Domain Experts for Model Adaptation in Facial Behavior Analysis

Author(s): Stefanos Eleftheriadis, Ognjen Rudovic, Marc P. Deisenroth, Maja Pantic
Published in: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2016, Page(s) 1469-1477
DOI: 10.1109/cvprw.2016.184

Enhancing LSTM RNN-Based Speech Overlap Detection by Artificially Mixed Data

Author(s): G. Hagerer, V. Pandit, F. Eyben, B. Schuller
Published in: Proceedings of the Audio Engineering Society Conference: 2017 AES International Conference on Semantic Audio, 2017, Page(s) 8p

Perception of Paralinguistic Traits in Synthesized Voices

Author(s): Alice Baird, Stina Hasse Jørgensen, Emilia Parada-Cabaleiro, Simone Hantke, Nicholas Cummins, Björn Schuller
Published in: Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences - AM '17, Issue 12, 2017, Page(s) 1-5
DOI: 10.1145/3123514.3123528

“Did you laugh enough today?” – Deep Neural Networks for Mobile and Wearable Laughter Trackers

Author(s): G. Hagerer, N. Cummins, F. Eyben, B. Schuller
Published in: Proceedings of the 18th Annual Conference of the International Speech Communication Association, Issue INTERSPEECH 2017, 2017, Page(s) 2044-2045

The SEILS dataset: Symbolically Encoded Scores in Modern-Early Notation for Computational Musicology

Author(s): Emilia Parada-Cabaleiro, Anton Batliner, Alice Baird, Björn W Schuller
Published in: Proceedings of the 18th International Society for Music Information Retrieval Conference, Issue ISMIR 2017, 2017, Page(s) 575-581

Deep Structured Learning for Facial Action Unit Intensity Estimation

Author(s): Robert Walecki, Ognjen Rudovic, Vladimir Pavlovic, Bjoern Schuller, Maja Pantic
Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Page(s) 5709-5718
DOI: 10.1109/cvpr.2017.605

Design of a Robot-Based Emotion-Mirroring Game to Engage Autistic Children with Emotional Expressions.

Author(s): Schadenberg B. R., Chevalier, P., Li, J. J., Ainger, E., Alcorn, A., Babovic, S., Davison, D., Charisi, V., Petrovic, S., Reidsma, D., Shen, J., Pellicano, E., Heylen, D. K. J., Evers, V.
Published in: In INSAR Conference, 2018
DOI: 10.13140/rg.2.2.34816.74247

Human Appearance Transfer

Author(s): Mihai Zanfir, Alin-Ionut Popa, Andrei Zanfir, Cristian Sminchisescu
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 5391-5399
DOI: 10.1109/cvpr.2018.00565

Social Robots in Therapy - Focusing on Autonomy and Ethical Challenges

Author(s): Pablo G. Esteban, Daniel Hernández García, Hee Rin Lee, Pauline Chevalier, Paul Baxter, Cindy Bethel
Published in: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction - HRI '18, 2018, Page(s) 391-392
DOI: 10.1145/3173386.3173562

Recognition of Echolalic Autistic Child Vocalisations Utilising Convolutional Recurrent Neural Networks

Author(s): Shahin Amiriparian, Alice Baird, Sahib Julka, Alyssa Alcorn, Sandra Ottl, Sunčica Petrović, Eloise Ainger, Nicholas Cummins, Björn Schuller
Published in: Interspeech 2018, 2018, Page(s) 2334-2338
DOI: 10.21437/interspeech.2018-1772

Robust Laughter Detection for Wearable Wellbeing Sensing

Author(s): Gerhard Hagerer, Nicholas Cummins, Florian Eyben, Björn Schuller
Published in: Proceedings of the 2018 International Conference on Digital Health - DH '18, 2018, Page(s) 156-157
DOI: 10.1145/3194658.3194693

AVEC 2018 Workshop and Challenge - Bipolar Disorder and Cross-Cultural Affect Recognition

Author(s): Fabien Ringeval, Adrien Michaud, Elvan Ciftçi, Hüseyin Güleç, Albert Ali Salah, Maja Pantic, Björn Schuller, Michel Valstar, Roddy Cowie, Heysem Kaya, Maximilian Schmitt, Shahin Amiriparian, Nicholas Cummins, Denis Lalanne
Published in: Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop - AVEC'18, 2018, Page(s) 3-13
DOI: 10.1145/3266302.3266316

Multimodal Bag-of-Words for Cross Domains Sentiment Analysis

Author(s): Nicholas Cummins, Shahin Amiriparian, Sandra Ottl, Maurice Gerczuk, Maximilian Schmitt, Bjorn Schuller
Published in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, Page(s) 4954-4958
DOI: 10.1109/icassp.2018.8462660

Flipper 2.0 - A Pragmatic Dialogue Engine for Embodied Conversational Agents

Author(s): Jelte van Waterschoot, Merijn Bruijnes, Jan Flokstra, Dennis Reidsma, Daniel Davison, Mariët Theune, Dirk Heylen
Published in: Proceedings of the 18th International Conference on Intelligent Virtual Agents - IVA '18, 2018, Page(s) 43-50
DOI: 10.1145/3267851.3267882

Are Some Emotions Harder Than Others? a Study of Autistic Children Recognising Human and Robot Facial Emotion Expressions in the UK and Serbia

Author(s): Girard. D., Alcorn, A. M., Ainger, E., Tavassoli, T. T., Babovic Dimitrijevic,S., Skendzic, S., Petrovic, S., Petrovic, V., Pellicano, E. and DE-ENIGMA Project Consortium.
Published in: International Society for Autism Research (INSAR) conference, 2018

Autism Data Goes Big: A Publicly-Accessible Multi-Modal Database of Child Interactions for Behavioural and Machine Learning Research

Author(s): The DE-ENIGMA Project Consortium
Published in: International Society for Autism Research (INSAR), 2018

Educator Practices and Perspectives on Social and Emotional Skills Teaching in UK Autism Education

Author(s): Mantinioti. S, Ainger., E, Alcorn, A. M. & Pellicano. E.
Published in: International Society for Autism Research (INSAR), 2018

Observing Interactions between Teaching Staff and Autistic Children: Who Communicates, How Often, and Why? a Cross-Cultural Comparison between UK and Serbia

Author(s): Ainger, E., Shek, Y., Alcorn, A. M. & Pellicano, E.
Published in: International Society for Autism Research (INSAR), 2018

Autism Characteristics, Adaptive Skills, and Performance in an Emotion-Recognition Teaching Programme with a Humanoid Robot

Author(s): Alcorn, A. M., Girard, D., Ainger, E., Tavassoli, T. T., Babovic Dimitrijevic,S., Skendzic, S., Petrovic, S., Petrovic, V., Pellicano, E. and DE-ENIGMA Project Consortium.
Published in: International Society for Autism Research (INSAR), 2018

Design of a Robot-Based Emotion-Mirroring Game to Engage Autistic Children with Emotional Expressions

Author(s): B.R. Schadenberg., P. Chevalier., J.J. Li., E. Ainger., A.M. Alcorn., S. Babović Dimitrijevic., D. Davison., V. Charisi., S. Petrović., D. Reidsma., J. Shen., E. Pellicano., D.K.J. Heylen., V. Evers
Published in: International Society for Autism Research (INSAR), 2018

A Framework for Joint Estimation and Guided Annotation of Facial Action Unit Intensity

Author(s): Robert Walecki, Ognjen Rudovic, Maja Pantic, Vladimir Pavlovic, Jeffrey F. Cohn
Published in: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2016, Page(s) 1460-1468
DOI: 10.1109/cvprw.2016.183

Variational gaussian process auto-encoder for ordinal prediction of facial action units

Author(s): Eleftheriadis S., Rudovic O., Deisenroth M.P., Pantic M.
Published in: Proceedings of the Asian Conference on Computer Vision 2016, 2016

Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity

Author(s): Robert Walecki, Ognjen Rudovic, Vladimir Pavlovic, Maja Pantic
Published in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Page(s) 4902-4910
DOI: 10.1109/cvpr.2016.530

Usability of a Robot’s Realistic Facial Expressions and Peripherals in Autistic Children’s Therapy

Author(s): Jamy Li, Daniel Davison, Bob Schadenberg, Alyssa Alcorn, Alria Williams, Suncica Petrovic, Snezana Babovic Dimitrijevic, Jie Shen, Liz Pellicano and Vanessa Evers
Published in: “Social Robots in Therapy and Care” workshop of the 14th Annual ACM/IEEE International Conference on Human Robot Interaction (HRI2019) Conference, 2019

Growing-Up Hand in Hand with Robots - Designing and Evaluating Child-Robot Interaction from a Developmental Perspective

Author(s): Cristina Zaga, Vicky Charisi, Bob Schadenberg, Dennis Reidsma, Mark Neerincx, Tony Prescott, Michael Zillich, Paul Verschure, Vanessa Evers
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, Issue 2017, 2017, Page(s) 429-430
DOI: 10.1145/3029798.3029804

Summary for AVEC 2017 - Real-life Depression and Affect Challenge and Workshop

Author(s): Fabien Ringeval, Björn Schuller, Michel Valstar, Jonathan Gratch, Roddy Cowie, Maja Pantic
Published in: Proceedings of the 2017 ACM on Multimedia Conference - MM '17, 2017, Page(s) 1963-1964
DOI: 10.1145/3123266.3132049

AVEC 2017 - Real-life Depression, and Affect Recognition Workshop and Challenge

Author(s): Fabien Ringeval, Maja Pantic, Björn Schuller, Michel Valstar, Jonathan Gratch, Roddy Cowie, Stefan Scherer, Sharon Mozgai, Nicholas Cummins, Maximilian Schmitt
Published in: Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge - AVEC '17, 2017, Page(s) 3-9
DOI: 10.1145/3133944.3133953

Summary for AVEC 2018 - Bipolar Disorder and Cross-Cultural Affect Recognition

Author(s): Fabien Ringeval, Björn Schuller, Michel Valstar, Roddy Cowie, Maja Pantic
Published in: 2018 ACM Multimedia Conference on Multimedia Conference - MM '18, 2018, Page(s) 2111-2112
DOI: 10.1145/3240508.3243719

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

Author(s): Alin-Ionut Popa, Mihai Zanfir, Cristian Sminchisescu
Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Page(s) 4714-4723
DOI: 10.1109/cvpr.2017.501

Variational Gaussian Process Auto-Encoder for Ordinal Prediction of Facial Action Units

Author(s): Eleftheriadis, Stefanos; Rudovic, Ognjen; Deisenroth, Marc Peter; Pantic, Maja; Lai, Shang-Hong; Lepetit, Vincent; Nishino, Ko; Sato, Yoichi
Published in: Computer Vision – ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, 154 - 170, Issue 12, 2017

AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition

Author(s): Ringeval, Fabien; Schuller, Björn; Valstar, Michel; Cummins, NIcholas; Cowie, Roddy; Tavabi, Leili; Schmitt, Maximilian; Alisamir, Sina; Amiriparian, Shahin; Messner, Eva-Maria; Song, Siyang; Liu, Shuo; Zhao, Ziping; Mallol-Ragolta, Adria; Ren, Zhao; Soleymani, Mohammad; Pantic, Maja
Published in: Issue 11, 2019

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Author(s): Ognjen Rudovic, Yuria Utsumi, Jaeryoung Lee, Javier Hernandez, Eduardo Castello Ferrer, Bjorn Schuller, Rosalind W. Picard
Published in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, Page(s) 339-346
DOI: 10.1109/iros.2018.8594177

GAGAN: Geometry-Aware Generative Adversarial Networks

Author(s): Jean Kossaifi, Linh Tran, Yannis Panagakis, Maja Pantic
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 878-887
DOI: 10.1109/cvpr.2018.00098

Helping Educators Monitor Autistic Children ’ s Progress Across Sessions : A Needfinding Study

Author(s): Schadenberg, B. R., Li, J. J., Petrovic, S., Reidsma, D., Heylen, D. K. J., & Evers, V.
Published in: In Companion ofthe 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20 Companion), 2020

Stimulation of Psychological Listener Experiences by Semi-Automatically Composed Electroacoustic Environments

Author(s): E. Parada-Cabaleiro, A. Baird, N. Cummins, B.W. Schuller
Published in: Proceedings of the 18th IEEE International Conference on Multimedia and Expo, Issue ICME 2017, 2017, Page(s) 1051-1056

Summary for AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge

Author(s): Michel Valstar, Jonathan Gratch, Björn Schuller, Fabien Ringeval, Roddy Cowie, Maja Pantic
Published in: Proceedings of the 2016 ACM on Multimedia Conference - MM '16, 2016, Page(s) 1483-1484
DOI: 10.1145/2964284.2980532

AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge

Author(s): Michel Valstar, Maja Pantic, Jonathan Gratch, Björn Schuller, Fabien Ringeval, Dennis Lalanne, Mercedes Torres Torres, Stefan Scherer, Giota Stratou, Roddy Cowie
Published in: Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge - AVEC '16, 2016, Page(s) 3-10
DOI: 10.1145/2988257.2988258

Audio-visual object localization and separation using low-rank and sparsity

Author(s): Jie Pu, Yannis Panagakis, Stavros Petridis, Maja Pantic
Published in: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, Page(s) 2901-2905
DOI: 10.1109/icassp.2017.7952687

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

Author(s): Alin-Ionut Popa, Mihai Zanfir, Cristian Sminchisescu
Published in: Proceedings of IEEE Int’l Conf. on Computer Vision and Pattern Recognition, Issue CVPR 2017, 2017

End-to-end visual speech recognition with LSTMS

Author(s): Stavros Petridis, Zuwei Li, Maja Pantic
Published in: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, Page(s) 2592-2596
DOI: 10.1109/ICASSP.2017.7952625

Large-Scale Data-Dependent Kernel Approximation

Author(s): A. Popa, M. Zanfir, C. Sminchisescu
Published in: Proceedings of the 20th International Conference on Artificial In- telligence and Statistics, Issue volume 54, 2017

Deep Structured Learning for Facial Action Unit Intensity Estimation

Author(s): R. Walecki, O. Rudovic, V. Pavlovic, B. Schuller, M. Pantic
Published in: Proceedings of IEEE Int’l Conf. on Computer Vision and Pattern Recognition, Issue CVPR 2017, 2017

Support by Participatory Sense-Making in Robot Therapy for Children with Autism

Author(s): Judith Weda, Bob Schadenberg, Jelle van Dijk
Published in: Proceedings of the 13th Student Interaction Design Research Conference, 2017

Things that Make Robots Go HMMM : Heterogeneous Multilevel Multimodal Mixing to Realise Fluent, Multiparty, Human-Robot Interaction

Author(s): Daniel Davison, Binnur Gorer, Jan Kolkmeier, Jeroen Linssen, Bob Schadenberg, Bob van de Vijver, Nick Campbell, Edwin Dertien and Dennis Reidsma
Published in: Proceedings of eNTERFACE’16 The 12th Summer Workshop on Multimodal Interfaces, Issue CTIT Workshop Proceedings WP 17-02, 2017, Page(s) 6-20

Multi-instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation

Author(s): Adria Ruiz, Ognjen Rudovic, Xavier Binefa, Maja Pantic
Published in: Proceedings of the Asian Conference on Computer Vision 2016, Issue ACCV 2016, 2017, Page(s) 171-186
DOI: 10.1007/978-3-319-54184-6_11

Automatic Classification of Autistic Child Vocalisations: A Novel Database and Results

Author(s): Alice Baird, Shahin Amiriparian, Nicholas Cummins, Alyssa M. Alcorn, Anton Batliner, Sergey Pugachevskiy, Michael Freitag, Maurice Gerczuk, Björn Schuller
Published in: Interspeech 2017, Issue Interspeech 2017, 2017, Page(s) 849-853
DOI: 10.21437/Interspeech.2017-730

The Perception of Emotions in Noisified Nonsense Speech

Author(s): Emilia Parada-Cabaleiro, Alice Baird, Anton Batliner, Nicholas Cummins, Simone Hantke, Björn Schuller
Published in: Interspeech 2017, Issue Interspeech 2017, 2017, Page(s) 3246-3250
DOI: 10.21437/Interspeech.2017-104

Introducing the Weighted Trustability Evaluator for Crowdsourcing Exemplified by Speaker Likability Classification

Author(s): S. Hantke, E. Marchi, and B. Schuller
Published in: Proceedings of the 10th Language Resources and Evaluation Conference, Issue LREC 2016, 2016, Page(s) 2156-2161

Towards Cross-lingual Automatic Diagnosis of Autism Spectrum Condition in Children’s Voices

Author(s): M. Schmitt, E. Marchi, F. Ringeval, and B. Schuller
Published in: Proceedings of the 14th ITG Conference on Speech Communication, Issue volume 267 of ITG Fachbericht, 2016, Page(s) 264-268

Speech-based Diagnosis of Autism Spectrum Condition by Generative Adversarial Network Representations

Author(s): Jun Deng, Nicholas Cummins, Maximilian Schmitt, Kun Qian, Fabien Ringeval, Björn Schuller
Published in: Proceedings of the 2017 International Conference on Digital Health - DH '17, Issue DH '17, 2017, Page(s) 53-57
DOI: 10.1145/3079452.3079492

Snore Sound Classification Using Image-Based Deep Spectrum Features

Author(s): Shahin Amiriparian, Maurice Gerczuk, Sandra Ottl, Nicholas Cummins, Michael Freitag, Sergey Pugachevskiy, Alice Baird, Björn Schuller
Published in: Interspeech 2017, Issue Interspeech 2017, 2017, Page(s) 3512-3516
DOI: 10.21437/Interspeech.2017-434

AVEC 2017 – Real-life Depression, and Affect Recognition Workshop and Challenge

Author(s): F. Ringeval, B. Schuller, M. Valstar, J. Gratch, R. Cowie, S. Scherer, S. Mozgai, N. Cummins, M. Schmitt, and M. Pantic
Published in: Proceedings of the 7th Audio/Visual Emotion Challenge and Workshop, Issue 7, 2017, Page(s) 6

An ‘End-to-Evolution’ Hybrid Approach for Snore Sound Classification

Author(s): Michael Freitag, Shahin Amiriparian, Nicholas Cummins, Maurice Gerczuk, Björn Schuller
Published in: Interspeech 2017, Issue Interspeech 2017, 2017, Page(s) 3507-3511
DOI: 10.21437/Interspeech.2017-173

The University of Passau Open Emotion Recognition System for the Multimodal Emotion Challenge

Author(s): Jun Deng, Nicholas Cummins, Jing Han, Xinzhou Xu, Zhao Ren, Vedhas Pandit, Zixing Zhang, Björn Schuller
Published in: Communications in Computer and Information Science, Issue CCPR 2016 vol 663, 2016, Page(s) 652-666
DOI: 10.1007/978-981-10-3005-5_54

Deep Recurrent Neural Network-based Autoencoders for Acoustic Novelty Detection

Author(s): Erik Marchi, Fabio Vesperini, Stefano Squartini, and Björn W. Schuller
Published in: Computational Intelligence and Neuroscience, 2016, ISSN 1687-5273
DOI: 10.1155/8483

Exploitation of Phase-Based Features for Whispered Speech Emotion Recognition

Author(s): Jun Deng, Xinzhou Xu, Zixing Zhang, Sascha Fruhholz, Bjorn Schuller
Published in: IEEE Access, Issue 4, 2016, Page(s) 4299-4309, ISSN 2169-3536
DOI: 10.1109/ACCESS.2016.2591442

Personalized machine learning for robot perception of affect and engagement in autism therapy

Author(s): Ognjen Rudovic, Jaeryoung Lee, Miles Dai, Björn Schuller, Rosalind W. Picard
Published in: Science Robotics, Issue 3/19, 2018, Page(s) eaao6760, ISSN 2470-9476
DOI: 10.1126/scirobotics.aao6760

Measuring Engagement in Robot-Assisted Autism Therapy: A Cross-Cultural Study

Author(s): Ognjen Rudovic, Jaeryoung Lee, Lea Mascarell-Maricic, Björn W. Schuller, Rosalind W. Picard
Published in: Frontiers in Robotics and AI, Issue 4, 2017, ISSN 2296-9144
DOI: 10.3389/FROBT.2017.00036

Calibrated Prediction Intervals for Neural Network Regressors

Author(s): Gil Keren, Nicholas Cummins, Bjorn Schuller
Published in: IEEE Access, Issue vol. 6, 2018, Page(s) 1-1, ISSN 2169-3536
DOI: 10.1109/ACCESS.2018.2871713

Affect bursts to constrain the meaning of the facial expressions of the humanoid robot Zeno.

Author(s): Schadenberg, B. R., Heylen, D. K. J. & Evers, V
Published in: Proceedings of the 1st Workshop on Social Interaction and Multimodal Expression for Socially Intelligent Robots (RO-MAN 2017), 2018, ISSN 1613-0073

The ASC-Inclusion Perceptual Serious Gaming Platform for Autistic Children

Author(s): Erik Marchi, Bjorn Schuller, Alice Baird, Simon Baron-Cohen, Amandine Lassalle, Helen O Reilly, Delia Pigat, Peter Robinson, Ian Davies, Tadas Baltrusaitis, Andra Adams, Marwa Mahmoud, Ofer Golan, Shimrit Fridenson-Hayo, Shahar Tal, Shai Newman, Noga Meir-Goren, Antonio Camurri, Stefano Piana, Sven Bolte, Metin Sezgin, Nese Alyuz, Agnieszka Rynkiewicz, Aurelie Baranger
Published in: IEEE Transactions on Games, 2018, Page(s) 1-12, ISSN 2475-1502
DOI: 10.1109/tg.2018.2864640

Teaching Machines on Snoring: A Benchmark on Computer Audition for Snore Sound Excitation Localisation

Author(s): K. Qian, C. Janott, Z. Zhang, J. Deng, A. Baird, C. Heiser, W. Hohenhorst, M. Herzog, W. Hemmer, and B. Schuller
Published in: Archives of Acoustics, 2018, ISSN 0137-5075
DOI: 10.24425/123918

Deep Scalogram Representations for Acoustic Scene Classification

Author(s): Zhao Ren, Kun Qian, Yebin Wang, Zixing Zhang, Vedhas Pandit, Alice Baird, Bjorn Schuller
Published in: IEEE/CAA Journal of Automatica Sinica, Issue 5/3, 2018, Page(s) 662-669, ISSN 2329-9274
DOI: 10.1109/jas.2018.7511066

Speech endpoint detection based on critical band and energy entropy

Author(s): Ting ZHANG, Ling HE, Hua HUANG, Xiaoheng LIU
Published in: Journal of Computer Applications, Issue 33/1, 2013, Page(s) 175-178, ISSN 1001-9081
DOI: 10.3724/sp.j.1087.2013.00175

Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection

Author(s): Erik Marchi, Fabio Vesperini, Stefano Squartini, Björn Schuller
Published in: Computational Intelligence and Neuroscience, Issue 2017, 2017, Page(s) 1-14, ISSN 1687-5265
DOI: 10.1155/2017/4694860

Educators' Views on Using Humanoid Robots With Autistic Learners in Special Education Settings in England

Author(s): Alyssa M. Alcorn, Eloise Ainger, Vicky Charisi, Stefania Mantinioti, Sunčica Petrović, Bob R. Schadenberg, Teresa Tavassoli, Elizabeth Pellicano
Published in: Frontiers in Robotics and AI, Issue 6, 2019, ISSN 2296-9144
DOI: 10.3389/frobt.2019.00107

A Two-Dimensional Framework of Multiple Kernel Subspace Learning for Recognizing Emotion in Speech

Author(s): Xinzhou Xu, Jun Deng, Nicholas Cummins, Zixing Zhang, Chen Wu, Li Zhao, Bjorn Schuller
Published in: IEEE/ACM Transactions on Audio, Speech, and Language Processing, Issue 25/7, 2017, Page(s) 1436-1449, ISSN 2329-9290
DOI: 10.1109/TASLP.2017.2694704

The Effect of Narrow-Band Transmission on Recognition of Paralinguistic Information From Human Vocalizations

Author(s): Sascha Fruhholz, Erik Marchi, Bjorn Schuller
Published in: IEEE Access, Issue 4, 2016, Page(s) 6059-6072, ISSN 2169-3536
DOI: 10.1109/ACCESS.2016.2604038

Advanced Data Exploitation in Speech Analysis: An overview

Author(s): Zixing Zhang, Nicholas Cummins, Bjoern Schuller
Published in: IEEE Signal Processing Magazine, Issue 34/4, 2017, Page(s) 107-129, ISSN 1053-5888
DOI: 10.1109/msp.2017.2699358

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Author(s): Ognjen Rudovic Affiliation: Massachusetts Institute of Technology, USA; Yuria Utsumi Affiliation: Massachusetts Institute of Technology, USA; Jaeryoung Lee Affiliation: Chubu University, Japan; Javier Hernandez Affiliation: Massachusetts Institute of Technology, USA; Eduardo Castello Ferrer Affiliation: Massachusetts Institute of Technology, USA; Bjorn Schuller Affiliation: Imperial College London
Published in: 2018

GAGAN: Geometry-Aware Generative Adversarial Networks

Author(s): Jean Kossaifi, Linh Tran, Yannis Panagakis, Maja Pantic
Published in: 2018