Learning from demonstration (LfD) is a paradigm that allows robots to autonomously learn from demos to perform new tasks through human demonstrations, which can bridge robotics and AI techniques to promote robot manipulability and robot programming feasibility. This project addresses two problems of LfD technology: 1) physical differences between robotic arm/gripper system and human arm/hand in manipulation, such as most grippers are not as soft and deformable as human fingertips; 2) learning skills with failure reasoning and incremental learning capabilities. From the perspective of neural motor theory, humans have stronger perceptual, cognitive, and muscular adaptive abilities that can sense and recognise temperature, pressure, vibration, texture, and shape of the touched object and adjust muscle impedance to changes in the environment, which is difficult for the robot. The differences between humans and robots are challenging, but there is hope to solve them to some extent using advanced robotics and AI techniques.
L3TD aims to solve the two challenging problems by developing a new tele-demonstration interface and proposing some new theoretical innovations on incremental learning and few-shot learning, and applying them to robot manipulation in fields such as nuclear industry and medical assistance with five separated objectives, which can be summarised into the following three aspects
Aspect 1 ( Objective 1 and Objective 2): Establish a teleoperation interface with a new facility to obtain a human demonstration data set. The interface operates in a "human-in-loop" control mode that allows humans to make immediate decisions from the first perspective, and the multimodal demonstration data is collected to be stored in a demonstration dataset and managed in a condensed form by the hierarchical labels with primitive capabilities.
Aspect 2 ( Objective 3 and Objective 4): Learning primitive skills and programming primitive skills (PS) for few-shot tasks. New theories of PS learning and PS programming are explored based on the learning methods such as improved meta-learning, reinforcement learning, and broad learning, etc., to achieve failure reasoning and adaptation to tasks with zero/few shots, respectively.
Aspect 3 ( Objective 5): Experimental Verification. We choose typical actions such as grasping objects and approaching in medical assistance scenes to verify the effectiveness of the proposed methods using data collected from the demonstration system.
