As a final staget, the consortium has successfully achieved the final milestone: “Robot can handle some exceptions (for instance the technician needs to go back to collect more tools) and generalize to new events.”
Key breakthroughs in the area of Grasping and Manipulation include:
• New humanoid robot incorporating two compliant 8 DoF arms, under-actuated hands, holonomic platform, a head with 5 cameras and a functional control architecture for integration of sensorimotor skills, learning and reasoning abilities.
• Novel methods for grasping objects and maintenance tools by combining visual and haptic sensing with model-based and data-driven machine learning approaches
• Methods for grasping small and challenging objects with under-actuated hands
• Methods for manipulating large objects in collaboration with humans where the robot decides how to grasp an object or tool depending on human actions.
• Learning techniques ranging from explorative learning to teaching or coaching by humans.
The ability to test this use case in real-world environments of Ocado’s highly automated warehouse was fundamental to the success.
ARMAR-6’s versatile artificial intelligence capabilities allow it to act in situations that are not foreseen at programming time. This means it’s capable of autonomously performing maintenance tasks in industrial facilities. It can recognise it’s collaboration partners' need of help and offer assistance. The ability to teach how to see in 3D without the requirement of fully supervised 3D training data will revolutionise the field of Machine Learning.
Key breakthroughs in the area of Task Understanding for Proactive Collaboration include:
Learning:
- Algorithms for gathering knowledge, from activities and actions sequence recognition to tools segmentation and context classification.
Reacting:
- Novel architectures for Help Recognition for a fast interaction and assistance via anticipation and forecasting;
- Advancements in guaranteeing safety - a state-dependent dynamical system to provide solutions for compliant interaction with humans;
- Reactive motion planning which enables the robot to react to human/environment interaction forces.
Scene Recognition:
- New algorithms for dynamic 3D scene understanding that can build geometric and semantic 3D maps of the environment even when objects move independently from the camera.
- Pioneering supervised and unsupervised approaches for 3D reconstruction of human poses and semantic scene understanding that can learn from just 2D labels or even no labels at all.
With collaborative robots, trust and adoption are key: what you build needs to fit into the natural ways people work, which includes being able to respond in human-suited timescales and in ways that are meaningful to humans. This means that developments in natural language are crucial to usability. Key breakthroughs made by the consortium members in this interaction include:
- Creation of a speech interface between humans and robots that is solely based on all-neural models: all-neural automatic speech recognition, all-neural dialog modelling and all-neural speech synthesis.
As part of the project dissemination, the project has produced the final project video:
https://youtu.be/-KF5XSSTn_o(opens in new window).
