Developing crowd-friendly robots
Robots are increasingly deployed to deliver products and services, and are more frequently encountering crowds, creating logistical and safety challenges. Delivery robots are usually programmed to reach their destination quickly, causing minimal disruption. Whereas, mobility assistance robots have to safely navigate dynamic environments and actually enable interaction. The EU-supported project CROWDBOT has developed technologies capable of ensuring safe close contact with people. “Safe robots previously kept their distance from people, but as robots become a part of daily life, this approach won’t be fit for purpose anymore,” explains project coordinator Julien Pettre. Following three principal work streams – robot sensing, crowd prediction and robot control – CROWDBOT has demonstrated its own approaches through novel experimentation. “Our paradigm shift can now inform the technical choices of software developers,” adds Pettre.
Diverse scenarios
Project partners developed demonstrators of various robot functionalities in their labs, before integrating them into two test scenarios. Firstly, mobility assistance robotics. This was of interest because: the robotic elements are not always obvious, robotic behaviour is usually determined jointly by users and programming, and proximity to people is a prerequisite. UCL integrated their technologies into powered wheelchairs, with EPFL using a new Japanese prototype robot called Qolo, where users stand up on the robot. Qolo was equipped with a 2-level control programme which adjusted trajectory to avoid collisions, but could also limit impact forces, with contact detected by sensors picking up gesture, sound or touch signals. The robot was tested in the streets and outdoor markets of Lausanne, Switzerland, and through never-before-performed experiments, such as crash tests between Qolo and dummies. Secondly, SBRE, Locomotec and ETHZ tested service robots. While Locomotec worked with their own robot, Cuybot, ETHZ and SBRE worked with the humanoid robot Pepper, designed to interact with people in public. Pepper was equipped with additional sensing and processing capabilities, an integrated on-board tracker and improved localisation. It was tested in various dynamic scenarios, including a lab Christmas party, to explore its interaction – including gesture, touch and speech – with humans, during crowd navigation. Despite limitations imposed by COVID-19, Pepper was also integrated into virtual reality testing scenarios. The result was improvements to Pepper’s design and control functions. “These unique experiments allowed us to extensively explore a wide range of interactions, safely – impossible in real life. They gave us insights which save time and money,” says Pettre. For both scenarios, CROWDBOT used a combination of camera and lidar sensors integrated with trackers. To accurately predict crowd behaviour, deep learning techniques were developed to extrapolate from 2D tracking data generated by the movement of people surrounding the robots. By the project’s end, the team were starting to assess full body tracking.
Anticipating policy debates
We only need think of autonomous vehicles to realise that robots in public spaces will soon become a reality, inevitably sparking regulation debates. “Previously, there was little knowledge about robot crowd navigation, or it was about effective mitigation. By actually prioritising interaction, we focused on robot reactions,” says Pettre. “But how crowds then react to robots will depend on understanding of their purpose, important for policymaking.” Most of the project’s codes and data sets are already available from the project’s website. Project partners have already integrated project learning on board their robots. For example, EPFL has adopted collision control measures, while RWTH has improved sensing capabilities. “While we worked with existing robots, the next level would be to design a better one from scratch,” concludes Pettre.
Keywords
CROWDBOT, robot, navigation, crowd, sensors, wheelchairs, safety, Pepper
