MultiModal Mall Entertainment Robot

Project Information

MuMMER

Grant agreement ID: 688147

DOI 10.3030/688147

Project closed

EC signature date 10 November 2015

Start date 1 March 2016

End date 29 February 2020

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost € 5 345 137,50

EU contribution € 4 297 835,00

Coordinated by

UNIVERSITY OF GLASGOW
United Kingdom

Deliverables

Documents, reports (12)

Report on long-term deployment study

Reporting on Tasks 7.3 and 7.4.

Business case scenarios

Reporting on Task 8.2.

Kick Off Meeting Report

Report on the project kick off meeting

Co-design of use scenarios and user requirements

Reporting on Task 12 Initial use scenarios and user requirements will be ready at M6 via the ongoing codesignprocess The deliverable will be annually updated with codesigned use scenarios until M40

Acceptance of human-interactive mobile robots in the consumer markets

Reporting on Task 1.3.

Socially appropriate, engaging, and entertaining human-robot interaction in MuMMER

Final reporting on all tasks and evaluations.

Final data management plan

The final result of Task 8.3.

Social signal processing in MuMMER

Final reporting on all tasks and evaluations.

Success metrics for human-interactive mobile robots

Reporting on Task 1.3 Initial success metrics will be ready on M6. The success metrics will be updated on M40 to be applied and validated in the field trial.

Human-aware interactive tasks and motion control in MuMMER

Final reporting on all tasks.

Report on physical setup of robot environment

Reporting on Task 7.1.

Active multimodal sensing and perception in MuMMER

Reporting on methods and evaluation for all tasks.

Other (15)

Initial components of Human-aware motion planner

First simplified version of a humanaware navigation planner to be tested on the first project software integration

Social state processing components (upgrade)

Upgraded software demonstrating Tasks 3133

Audio-visual algorithms for person tracking and characterization (baseline)

Demonstrating Tasks T21 T22 and T24

Development of technical infrastructure

Report on Task 61 and Task 62

Social state processing components (final)

Final version of software demonstrating all aspects of Tasks 3.1–3.3.

Initial WP5 software ready for integration on real robot

This Deliverable corresponds to the initial human belief manager based on Perspectivetaking and geometric reasoning as well as a more elaborate version of the humanaware navigation planner

Social state processing components (initial)

Software demonstrating Tasks 31 and 32

Final WP5 software ready for long-term study

This Deliverable corresponds to the final version of the human belief manager, a set of Interactive Human-robot motion schemes as well as the final version of Human-Aware interactive tasks and motion control sub-system.

Socially engaging interaction management component (final)

Software demonstrating Task 4.2: final trained policies for engaging multimodal social interaction.

Delivery of adapted Pepper robot

Demonstrating Task 6.4.

Algorithms for gesture and interaction aware tracking, audio-visual association, and social cue extraction

Demonstrating improvements in tasks T2.1, T2.3 and T2.4.

Socially engaging interaction management component (initial)

Software demonstrating Task 41 social belief state monitoring and initialbaseline action selection strategy

Interaction aware and active perception algorithms

Corresponding to all tasks.

Audio-visual algorithms for person tracking and characterization (upgrade)

Corresponding to all tasks

Socially engaging interaction management component (upgrade)

Upgraded software demonstrating Task 41 initial trained policies and evaluation

Demonstrators, pilots, prototypes (2)

Initial integrated robot system

The initial integrated system developed as part of Task 63

Final integrated robot system

The final system developed as part of Task 6.3.

Open Research Data Pilot (1)

Initial data management plan

The initial result of Task 83

Websites, patent fillings, videos etc. (1)

Project website and social media dissemination

As part of Task 81

Publications

Conference proceedings (70)

Deep Neural Networks for Multiple Speaker Detection and Localization

Author(s): Weipeng He, Petr Motlicek, Jean-Marc Odobez
Published in: 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, Page(s) 74-79, ISBN 978-1-5386-3081-5
Publisher: IEEE
DOI: 10.1109/ICRA.2018.8461267

The More I Understand it, the Less I Like it: The Relationship Between Understandability and Godspeed Scores for Robotic Gestures

Author(s): Amol Deshmukh and Bart Craenen and Mary Ellen Foster and Alessandro Vinciarelli
Published in: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2018
Publisher: IEEE

Supervised Gaze Bias Correction for Gaze Coding in Interactions

Author(s): R. Siegfried and J.-M. Odobez
Published in: Proceedings of the Communication by Gaze Interaction symposium (COGAIN), 2017
Publisher: COGAIN

Human-Robot Interaction Requires More Than Slot Filling - Multi-Threaded Dialogue for Collaborative Tasks and Social Conversation

Author(s): Ioannis Papaioannou, Christian Dondrup, Oliver Lemon
Published in: FAIM/ISCA Workshop on Artificial Intelligence for Multimodal Human Robot Interaction, 2018, Page(s) 61-64
Publisher: ISCA
DOI: 10.21437/ai-mhri.2018-15

Joint Localization and Classification of Multiple Sound Sources Using a Multi-task Neural Network

Author(s): Weipeng He, Petr Motlicek, Jean-Marc Odobez
Published in: Interspeech 2018, 2018, Page(s) 312-316
Publisher: ISCA
DOI: 10.21437/Interspeech.2018-1269

Real-time Convolutional Networks for Depth-based Human Pose Estimation

Author(s): Martinez-Gonzalez, Angel and Villamizar, Michael and Canevet, Olivier and Odobez, Jean-Marc
Published in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, 2018
Publisher: IEEE

How can we transition from lab to the real world with our HCI and HRI setups?

Author(s): Dondrup, Christian and Baillie, Lynne and Broz, Frank and Lohan, Katrin
Published in: Proceedings of the Fourth Workshop on Public-Space Human-Robot Interaction (PubRob 2018), 2018
Publisher: ACM SIGCHI

Incrementally Learning Semantic Attributes through Dialogue Interaction

Author(s): Andrea Vanzo, Jose L. Part, Yanchao Yu, Daniele Nardi, and Oliver Lemon
Published in: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS '18), 2018, Page(s) 865-873
Publisher: ACM

Spoken Conversational AI in Video Games-Emotional Dialogue Management Increases User Engagement

Author(s): Fraser, Jamie and Papaioannou, Ioannis and Lemon, Oliver
Published in: Proceedings of International Conference on Intelligent VirtualAgents (IVA '18),, 2018
Publisher: IVA
DOI: 10.1145/3267851.3267896

Investigating Depth Domain Adaptation for Efficient Human Pose Estimation

Author(s): Martinez-Gonzalez, Angel and Villamizar, Michael and Canevet, Olivier and Odobez, Jean-Marc
Published in: 2018 European Conference on Computer Vision - Workshops, {ECCVW} 2018, 2018
Publisher: ECCV

A Differential Approach for Gaze Estimation with Calibration

Author(s): Liu, Gang and Yu, Yu and Funes Mora, Kenneth Alberto and Odobez, Jean-Marc
Published in: 29TH BRITISH MACHINE VISION CONFERENCE (BMVC), 2018
Publisher: BMVC

Deep Multitask Gaze Estimation with a Constrained Landmark-Gaze Model

Author(s): Yu, Yu and Liu, Gang and Odobez, Jean-Marc
Published in: European Conference on Computer Vision Workshop, 2018
Publisher: ECCV

Planning Human and Robot Placements for Shared Visual Perspective, the SVP Planner

Author(s): Jules Waldhart and Aurelie Clodic and Rachid Alami
Published in: Proceedings of IROS workshop Robotic Co-workers 4.0, 2018
Publisher: IEEE

Alana: Social dialogue using an ensemble model and a ranker trained on user feedback

Author(s): Papaioannou, Ioannis and Curry, Amanda Cercas and Part, Jose L and Shalyminov, Igor and Xu, Xinnuo and Yu, Yanchao and Du{\v{s}}ek, Ondrej and Rieser, Verena and Lemon, Oliver
Published in: 2017
Publisher: Amazon

Viewing Robot Navigation in Human Environment as a Cooperative Activity

Author(s): Khambhaita, Harmish and Alami, Rachid
Published in: International Symposium on Robotics Research (ISSR 2017), 2017
Publisher: ISSR

Underworlds: Cascading situation assessment for robots

Author(s): Severin Lemaignan and Yoan Sallami and Christopher Wallbridge and Aurelie Clodic and Tony Belpaeme and Rachid Alami
Published in: Proceedings of IROS 2018, 2018
Publisher: IEEE

Shaping Gestures to Shape Personalities: The Relationship Between Gesture Parameters, Attributed Personality Traits and Godspeed Scores

Author(s): Bart Craenen and Amol Deshmukh and Mary Ellen Foster and Alessandro Vinciarelli
Published in: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2018
Publisher: IEEE

Natural Language Generation for Public-Space Social Robots

Author(s): Mary Ellen Foster
Published in: Proceedings of the 4th Workshop on Public Space Human-Robot Interaction (PubRob 2018), 2018
Publisher: ACM SIGCHI

Assessing the social criteria for human-robot collaborative navigation: A comparison of human-aware navigation planners

Author(s): Harmish Khambhaita, Rachid Alami
Published in: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2017, Page(s) 1140-1145, ISBN 978-1-5386-3518-6
Publisher: IEEE
DOI: 10.1109/ROMAN.2017.8172447

Where Can I Find a Pharmacy? - Human-Driven Design of a Service Robot's Guidance Behaviour

Author(s): Heikkila, Paivi and Lammi, Hanna and Belhassein, Kathleen
Published in: Proceedings of the Fourth Workshop on Public-Space Human-Robot Interaction (PubRob 2018), 2018
Publisher: ACM SIGCHI

Evaluating the Pertinence of Robot Decisions in a Human-Robot Joint Action Context: The PeRDITA Questionnaire

Author(s): Devin, S. and Vrignaud, C. and Belhassein, K. and Clodic, A. and Carreras, O. and Alami, R.
Published in: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2018
Publisher: IEEE

Sympathy Begins with a Smile, Intelligence Begins with a Word: Use of Multimodal Features in Spoken Human-Robot Interaction

Author(s): Jekaterina Novikova, Christian Dondrup, Ioannis Papaioannou, Oliver Lemon
Published in: Proceedings of the First Workshop on Language Grounding for Robotics, 2017, Page(s) 86-94
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/W17-2811

A domain adaptation approach to improve speaker turn embedding using face representation

Author(s): Nam Le, Jean-Marc Odobez
Published in: Proceedings of the 19th ACM International Conference on Multimodal Interaction - ICMI 2017, 2017, Page(s) 411-415, ISBN 9781-450355438
Publisher: ACM Press
DOI: 10.1145/3136755.3136800

Hybrid chat and task dialogue for more engaging HRI using reinforcement learning

Author(s): Ioannis Papaioannou, Christian Dondrup, Jekaterina Novikova, Oliver Lemon
Published in: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2017, Page(s) 593-598, ISBN 978-1-5386-3518-6
Publisher: IEEE
DOI: 10.1109/roman.2017.8172363

Do We Really Like Robots that Match our Personality? The Case of Big-Five Traits, Godspeed Scores and Robotic Gestures

Author(s): Bart Craenen and Amol Deshmukh and Mary Ellen Foster and Alessandro Vinciarelli
Published in: Proceedings of IEEE RO-MAN 2018, 2018
Publisher: IEEE

Shopping Mall Robots -- Opportunities and Constraints from the Retailer and Manager Perspective

Author(s): Marketta Niemelä, Päivi Heikkilä, Hanna Lammi, Virpi Oksman
Published in: Social Robotics, 2017, Page(s) 485-494
Publisher: Springer International Publishing
DOI: 10.1007/978-3-319-70022-9_48

Shaping Gestures to Shape Personality: Big-Five Traits, Godspeed Scores and the Similarity-Attraction Effect

Author(s): Bart Craenen and Amol Deshmukh and Mary Ellen Foster and Alessandro Vinciarelli
Published in: Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018), 2018
Publisher: ACM

Robust and Discriminative Speaker Embedding via Intra-Class Distance Variance Regularization

Author(s): Nam Le, Jean-Marc Odobez
Published in: Interspeech 2018, 2018, Page(s) 2257-2261
Publisher: ISCA
DOI: 10.21437/Interspeech.2018-1685

Introducing a ROS based planning and execution framework for human-robot interaction

Author(s): Christian Dondrup, Ioannis Papaioannou, Jekaterina Novikova, Oliver Lemon
Published in: Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents - ISIAA 2017, 2017, Page(s) 27-28, ISBN 9781-450355582
Publisher: ACM Press
DOI: 10.1145/3139491.3139500

Improving Speaker Turn Embedding by Crossmodal Transfer Learning from Face Embedding

Author(s): Nam Le, Jean-Marc Odobez
Published in: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017, Page(s) 428-437, ISBN 978-1-5386-1034-3
Publisher: IEEE
DOI: 10.1109/iccvw.2017.58

Towards the use of social interaction conventions as prior for gaze model adaptation

Author(s): Rémy Siegfried, Yu Yu, Jean-Marc Odobez
Published in: Proceedings of the 19th ACM International Conference on Multimodal Interaction - ICMI 2017, 2017, Page(s) 154-162, ISBN 9781-450355438
Publisher: ACM Press
DOI: 10.1145/3136755.3136793

Hello Pepper, May I Tickle You? - Children's and Adults' Responses to an Entertainment Robot at a Shopping Mall

Author(s): Iina Aaltonen, Anne Arvola, Päivi Heikkilä, Hanna Lammi
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 53-54, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3038362

The MuMMER project: Engaging human-robot interaction in real-world public spaces

Author(s): Mary Ellen Foster, Rachid Alami, Olli Gestranius, Oliver Lemon, Marketta Niemelä, Jean-Marc Odobez, Amit Kumar Pandey
Published in: 8th International Conference on Social Robotics, ICSR 2016, Kansas City, MO, USA, November 2016, 2016, Page(s) 753-763
Publisher: Springer International Publishing
DOI: 10.1007/978-3-319-47437-3_74

Qualitative constraints for human-aware robot navigation using Velocity Costmaps

Author(s): Christian Dondrup, Marc Hanheide
Published in: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2016, Page(s) 586-592, ISBN 978-1-5090-3929-6
Publisher: IEEE
DOI: 10.1109/ROMAN.2016.7745177

Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction

Author(s): Y. Yu, K. Funes, and J.-M. Odobez
Published in: 12th IEEE Conference on Automatic Face and Gesture Recognition, 2017
Publisher: IEEE

Head-Body Motion Coordination for Human Aware Robot Navigation

Author(s): Harmish Khambhaita, Jorge Rios-Martinez, and Rachid Alami
Published in: 9th International Workshop on Human Friendly Robotics, 29. & 30. September 2016, Genoa, Italy, 2016
Publisher: Not available

Public-space human-robot interaction in the MuMMER project

Author(s): Mary Ellen Foster
Published in: The 3rd Workshop on Public Space Human-Robot Interaction (PubRob 2016), at RO-MAN 2016, New York, NY, August 2016, 2016
Publisher: PubRob 2016

Towards building an attentive artificial listener: on the perception of attentiveness in audio-visual feedback tokens

Author(s): Catharine Oertel, José Lopes, Yu Yu, Kenneth A. Funes Mora, Joakim Gustafson, Alan W. Black, Jean-Marc Odobez
Published in: Proceedings of the 18th ACM International Conference on Multimodal Interaction - ICMI 2016, 2016, Page(s) 21-28, ISBN 9781-450345569
Publisher: ACM Press
DOI: 10.1145/2993148.2993188

Monitoring the Acceptance of a Social Service Robot in a Shopping Mall - First Results

Author(s): Marketta Niemelä, Anne Arvola, lina Aaltonen
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 225-226, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3038333

A Human-Robot Cooperative Navigation Planner

Author(s): Harmish Khambhaita, Rachid Alami
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 161-162, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3038374

Combining Chat and Task-Based Multimodal Dialogue for More Engaging HRI - A Scalable Method Using Reinforcement Learning

Author(s): Ioannis Papaioannou, Oliver Lemon
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 365-366, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3034820

Perceptual Control Theory for Engagement and Disengagement of Users in Public Spaces

Author(s): Amol Deshmukh, Mary Ellen Foster, Alessandro Vinciarelli
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 105-106, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3038364

Training an adaptive dialogue policy for interactive learning of visually grounded word meaning

Author(s): Yanchao Yu, Arash Eshghi, and Oliver Lemon
Published in: Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2016, Page(s) 339-349
Publisher: ACL

A Novel Software Combining Task and Motion Planning for Human-Robot Interaction

Author(s): Jules Waldhart and Mamoun Gharbi and Rachid Alami
Published in: AI-HRI 2016, AAAI Fall Symposium Series November 17-19, Arlington, VA, USA, 2016
Publisher: AAAI

A Social Service Robot in a Shopping Mall - Expectations of the Management, Retailers and Consumers

Author(s): Marketta Niemelä, Päivi Heikkilä, Hanna Lammi
Published in: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI '17, 2017, Page(s) 227-228, ISBN 9781-450348850
Publisher: ACM Press
DOI: 10.1145/3029798.3038301

Using General-Purpose Planning for Action Selection in Human-Robot Interaction

Author(s): Petrick, Ronald P. A.; Foster, Mary Ellen
Published in: AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (AI-HRI 2016), Issue 1, 2016
Publisher: AAAI

MuMMER: Socially Intelligent Human-Robot Interaction in Public Spaces

Author(s): Foster, Mary Ellen; Craenen, Bart; Deshmukh, Amol; Lemon, Oliver; Bastianelli, Emanuele; Dondrup, Christian; Papaioannou, Ioannis; Vanzo, Andrea; Odobez, Jean-Marc; Canévet, Olivier; Cao, Yuanzhouhan; He, Weipeng; Martínez-González, Angel; Motlicek, Petr; Siegfried, Rémy; Alami, Rachid; Belhassein, Kathleen; Buisan, Guilhem; Clodic, Aurélie; Mayima, Amandine; Sallami, Yoan; Sarthou, Guillaume
Published in: Proceedings of AI-HRI 2019, 2019
Publisher: AAAI

Semantic Spatial Representation: a unique representation of an environment based on an ontology for robotic applications

Author(s): Guillaume Sarthou, Aurélie Clodic, Rachid Alami
Published in: Proceedings of the Combined Workshop on Spatial Language Understanding (, 2019, Page(s) 50-60
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/w19-1606

The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings

Author(s): Yanchao Yu, Arash Eshghi, Gregory Mills, Oliver Lemon
Published in: Proceedings of the Sixth Workshop on Vision and Language, 2017, Page(s) 1-10
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/W17-2001

Topological localization using Wi-Fi and vision merged into FABMAP framework

Author(s): Mathieu Nowakowski, Cyril Joly, Sebastien Dalibard, Nicolas Garcia, Fabien Moutarde
Published in: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, Page(s) 3339-3344, ISBN 978-1-5386-2682-5
Publisher: IEEE
DOI: 10.1109/IROS.2017.8206171

Evaluation of the Quality of Interaction from the robot point of view in Human-Robot Interactions

Author(s): Mayima, Amandine; Clodic, Aurélie; Alami, Rachid
Published in: 1st Edition of Quality of Interaction in Socially Assistive Robots (QISAR) Workshop, Issue 11, 2019
Publisher: ICSR

Which Contingent Events to Observe for the Dynamic Controllability of a Plan

Author(s): Bit-Monnot, Arthur; Ghallab, Malik; Ingrand, Félix
Published in: Proceedings of IJCAI 2016, Issue 4, 2016
Publisher: IJCAI

Learning how to Learn: An Adaptive Dialogue Agent for Incrementally Learning Visually Grounded Word Meanings

Author(s): Yanchao Yu, Arash Eshghi, Oliver Lemon
Published in: Proceedings of the First Workshop on Language Grounding for Robotics, 2017, Page(s) 10-19
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/W17-2802

Ontologenius: A long-term semantic memory for robotic agents

Author(s): Guillaume Sarthou, Aurelie Clodic, Rachid Alami
Published in: 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2019, Page(s) 1-8, ISBN 978-1-7281-2622-7
Publisher: IEEE
DOI: 10.1109/ro-man46459.2019.8956305

A deep learning approach for robust head pose independent eye movements recognition from videos

Author(s): Rémy Siegfried, Yu Yu, Jean-Marc Odobez
Published in: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, 2019, Page(s) 1-5, ISBN 9781-450367097
Publisher: ACM
DOI: 10.1145/3314111.3319844

Reasoning on Shared Visual Perspective to Improve Route Directions

Author(s): Jules Waldhart, Aurelie Clodic, Rachid Alami
Published in: 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2019, Page(s) 1-8, ISBN 978-1-7281-2622-7
Publisher: IEEE
DOI: 10.1109/ro-man46459.2019.8956322

Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLU

Author(s): Vanzo, Andrea Bastianelli, Emanuele Lemon, Oliver
Published in: Proceedings of SIGDIAL 2019, 2019
Publisher: Association for Computational Linguistics

Simulation-based physics reasoning for consistent scene estimation in an HRI context

Author(s): Yoan Sallami, Severin Lemaignan, Aurelie Clodic, Rachid Alami
Published in: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, Page(s) 7834-7841, ISBN 978-1-7281-4004-9
Publisher: IEEE
DOI: 10.1109/iros40897.2019.8968106

Implementing an implicit false belief task by cascading situation assessment with Underworlds

Author(s): Sallami, Yoan Lemaignan, Severin Clodic, Aurelie Alami, Rachid
Published in: Workshop “Toward a framework for joint action: what about theory of mind?” at Robotics Science and System conference (RSS'18), Pittsburgh,USA, 2018
Publisher: RSS

Guiding Task through Route Description in the MuMMER Project

Author(s): Phani-Teja Singamaneni, Amandine Mayima, Guillaume Sarthou, Yoan Sallami, Guilhem Buisan, Kathleen Belhassein, Jules Waldhart, Aurélie Clodic
Published in: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 2020, Page(s) 643-643, ISBN 9781-450370578
Publisher: ACM
DOI: 10.1145/3371382.3378398

Unsupervised Representation Learning for Gaze Estimation

Author(s): Yu Yu, Jean-Marc Odobez
Published in: Proceedings of CVPR 2020, 2020
Publisher: CVPR

Shaping Robot Gestures to Shape Users' Perception - The Effect of Amplitude and Speed on Godspeed Ratings

Author(s): Amol Deshmukh, Bart Craenen, Alessandro Vinciarelli, Mary Ellen Foster
Published in: Proceedings of the 6th International Conference on Human-Agent Interaction, 2018, Page(s) 293-300, ISBN 9781-450359535
Publisher: ACM
DOI: 10.1145/3284432.3284445

Evaluation of Interacting Robots in Public Spaces Through Scientific Competitions

Author(s): Iocchi, Luca Nardi, Daniele Foster, Mary Ellen
Published in: Proceedings of the 4th Workshop on Public Space Human-Robot Interaction (PubRob 2018), 2018
Publisher: PubRob

Long-term evaluation of a shopping mall robot: a mixed-method approach to study the success of a Pepper demonstrator in a mall

Author(s): Marketta Niemela, Paivi Heikkila, Antti Tammela, Hanna Lammi, Iina Aaltonen, Anne Arvola
Published in: Oral presentation in the ICSR2019 workshop Robots in the Wild, 2019
Publisher: ICSR

The Unexpected Daily Situations (UDS) Dataset - A New Benchmark for Socially-Aware Assistive Robots

Author(s): Yoan Sallami, Katie Winkle, Nicola Webb, Severin Lemaignan, Rachid Alami
Published in: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 2020, Page(s) 427-429, ISBN 9781-450370578
Publisher: ACM
DOI: 10.1145/3371382.3378270

Face-to-face conversation - why embodiment matters for conversational user interfaces

Author(s): Mary Ellen Foster
Published in: Proceedings of the 1st International Conference on Conversational User Interfaces - CUI '19, 2019, Page(s) 1-3, ISBN 9781-450371872
Publisher: ACM Press
DOI: 10.1145/3342775.3342810

Contextual Non-verbal Behaviour Generation for Humanoid Robot Using Text Sentiment

Author(s): Deshmukh, Amol Mazel, Alexandre Foster, Mary Ellen
Published in: Proceedings of IEEE RO-MAN 2019, 2019
Publisher: IEEE

Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation

Author(s): Yuanzhouhan Cao, Olivier Canevet, Jean-Marc Odobez
Published in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, Page(s) 1089-1094, ISBN 978-1-5386-8094-0
Publisher: IEEE
DOI: 10.1109/iros.2018.8594223

Petri Net Machines for Human-Agent Interaction

Author(s): Christian Dondrup, Ioannis Papaioannou, Oliver Lemon
Published in: Proceedings of AI-HRI 2019, 2019
Publisher: AAAI

Adaptation of Multiple Sound Source Localization Neural Networks with Weak Supervision and Domain-adversarial Training

Author(s): Weipeng He, Petr Motlicek, Jean-Marc Odobez
Published in: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, Page(s) 770-774, ISBN 978-1-4799-8131-1
Publisher: IEEE
DOI: 10.1109/icassp.2019.8682655

Other (4)

Human-Human Guidance Study

Author(s): Belhassein, Kathleen and Clodic, Aurelie and Cochet, Helene and Niemela, Marketta and Heikkila, Paivi and Lammi, Hanna and Tammela, Antti
Published in: 2017
Publisher: LAAS

Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis

Author(s): Yu, Yu; Liu, Gang; Odobez, Jean-Marc
Published in: http://infoscience.epfl.ch/record/265519, Issue 5, 2019
Publisher: Idiap

Theoretical Guarantees of Deep Embedding Losses Under Label Noise

Author(s): Nam Le, Jean-Marc Odobez
Published in: 2018
Publisher: ArXiV

Social Human-Robot Interaction in the Wild.

Author(s): M. Niemela, A. Ahtinen, M. Turunen
Published in: A workshop in the Academic Mindtrek '20 conference, 2020
Publisher: Academic Mindtrek

Peer reviewed articles (4)

A Mass-Produced Sociable Humanoid Robot: Pepper: The First Machine of Its Kind

Author(s): Amit Kumar Pandey, Rodolphe Gelin
Published in: IEEE Robotics & Automation Magazine, Issue 25/3, 2018, Page(s) 40-48, ISSN 1070-9932
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/MRA.2018.2833157

HeadFusion: 360 Head Pose Tracking Combining 3D Morphable Model and 3D Reconstruction

Author(s): Yu Yu, Kenneth Alberto Funes Mora, Jean-Marc Odobez
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 40/11, 2018, Page(s) 2653-2667, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/TPAMI.2018.2841403

Improving speech embedding using crossmodal transfer learning with audio-visual data

Author(s): Nam Le, Jean-Marc Odobez
Published in: Multimedia Tools and Applications, Issue 78/11, 2019, Page(s) 15681-15704, ISSN 1380-7501
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s11042-018-6992-3

Natural language generation for social robotics: opportunities and challenges

Author(s): Mary Ellen Foster
Published in: Philosophical Transactions of the Royal Society B: Biological Sciences, Issue 374/1771, 2019, Page(s) 20180027, ISSN 0962-8436
Publisher: Royal Society of London
DOI: 10.1098/rstb.2018.0027

Book chapters (5)

Enriching the Human-Robot Interaction Loop with Natural, Semantic, and Symbolic Gestures

Author(s): Katrin Solveig Lohan, Hagen Lehmann, Christian Dondrup, Frank Broz, Hatice Kose
Published in: Humanoid Robotics: A Reference, 2017, Page(s) 1-21
Publisher: Springer Netherlands
DOI: 10.1007/978-94-007-7194-9_136-1

Separating Representation, Reasoning, and Implementation for Interaction Management: Lessons from Automated Planning

Author(s): Mary Ellen Foster, Ronald P. A. Petrick
Published in: Dialogues with Social Robots: Enablements, Analyses, and Evaluation, 2017, Page(s) 93-107
Publisher: Springer Singapore
DOI: 10.1007/978-981-10-2585-3_7

Should a Robot Guide Like a Human? A Qualitative Four-Phase Study of a Shopping Mall Robot

Author(s): Päivi Heikkilä, Hanna Lammi, Marketta Niemelä, Kathleen Belhassein, Guillaume Sarthou, Antti Tammela, Aurélie Clodic, Rachid Alami
Published in: Social Robotics - 11th International Conference, ICSR 2019, Madrid, Spain, November 26–29, 2019, Proceedings, Issue 11876, 2019, Page(s) 548-557, ISBN 978-3-030-35887-7
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-35888-4_51

A Social Robot in a Shopping Mall: Studies on Acceptance and Stakeholder Expectations

Author(s): Marketta Niemelä, Päivi Heikkilä, Hanna Lammi, Virpi Oksman
Published in: Social Robots: Technological, Societal and Ethical Aspects of Human-Robot Interaction, 2019, Page(s) 119-144, ISBN 978-3-030-17106-3
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-17107-0_7

Semantic-Based Interaction for Teaching Robot Behavior Compositions Using Spoken Language

Author(s): Victor Paléologue, Jocelyn Martin, Amit Kumar Pandey, Mohamed Chetouani
Published in: Social Robotics - 10th International Conference, ICSR 2018, Qingdao, China, November 28 - 30, 2018, Proceedings, Issue 11357, 2018, Page(s) 421-430, ISBN 978-3-030-05203-4
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-05204-1_41

Thesis and dissertations (1)

A New Formulation and Resolution Schemes for Planning Collaborative Human-Robot Tasks

Author(s): Waldhart, Jules
Published in: 2019
Publisher: Universite Toulouse, INSA Toulouse