Periodic Reporting for period 2 - SAHR (Skill Acquisition in Humans and Robots)
Reporting period: 2019-04-01 to 2020-09-30
Humans have a remarkable capacity for fine manipulation and unique ability to acquire such skills through practice as well as through observation of more skilled practioneers. The best example is craftsmanship. Crafts demand high dexterity and proficiency at a craft is attained only after years of practice. The SAHR project aims at offering more insight on how humans acquire these skills and to use this knowledge to support the development of new learning algorithms for robots. To this end, SAHR develops novel approaches to skill acquisition in robots that follow stages of skill acquisition in humans. To inform the design of these new learning algorithms, we follow a group of apprentices at watchmaking. Watch making is an ideally suited craft as it requires very precise control of both hands’ fingers in coordination. Analysis of human acquisition skills aims to achieve a better understanding of how humans overcome their sensory-motor noise and the sheer complexity of controlling simultaneously many degrees of freedom. We develop a framework to enable robots to acquire similarly complex manipulation skills, through a two-stage process. First, a trial and error process allows to identify the task constraints and to determine a motor program satisfying these constraints. Second, an optimization process is launched to refine and optimize the skill to reach an economy of effort. Production of the skill becomes automated. To automate the skill, we use Dynamical Systems (DS), closed-form control laws that offer inherent robustness to perturbation, hence ideally suited to generate the required run-time high-speed adaptation.
In parrallel, SAHR pursued fundamental advances in machine learning with application to robot control. The notion of bifurcation in dynamical systems was exploited to embed simultaneously periodic and discrete dynamics of motion in a single control law. This technique was applied to replicate swift switching from polishing to pick and place as displayed in watchmaking with a robot arm manipulator. Additionally, novel approaches to identify automatically control laws through the use of spectral clustering and manifold learning were developed.
Finally, the project made important efforts to ensure reproducibility of results. To this end, a benchmark of bimanual coordination in robot control was devised and published. The benchmark will serve to document progresses in the project but also more generally in robotics. Additionally, the longitudinal study's data have been made available on the project’s website alongside code for the analysis and modeling of the data.
Analysis of the data have led us to propose a novel taxonomy of hand postures that differ from hand postures documented in existing taxonomies. We also offer, for the first time, a taxonomy of bimanual coordinated hand postures that reveals interesting information on the role than handedness plays in finger coordination. While handedness is well documented for arm movements, its role in fine manipulation has received much less attention.
Watchmaking is a craft known for its complexity. Through its longitudinal study of watchmaking apprenticeship, the SAHR project generates a unique and the first of its kind detailed recordings of this craft. These data will provide a historical account of the craft, should this craft come to disappear one day.
The SAHR project contributes to theoretical and experimental advances in machine learning and robotics. It pushes forward the use of dynamical systems as a mode to control robots. While learning dynamical systems was so far performed in a fully supervised manner, results from SAHR show that unsupervised learning techniques can also be used for the identification of the systems. This is advantageous, as the approach does not necessitate explicit labelling which is time consuming and relies on expert data.
The SAHR project bet that bifurcations, a well-known property of dynamical systems, could be used to good advantage to embed multiple control laws in a single system. This departs from traditional hybrid systems approaches. Results show that it is indeed possible to learn bi-stable systems with particular dynamics leading to fixed point attractors and other dynamics leading to limit cycles. The approach is, however, restricted to linear dynamics and efforts in the second part of the project will be dedicated to extend this to more complex non-linear dynamics.
The SAHR project’s ultimate goal is to devise a learning scheme to enable robots to automatically model a new skill through practice and to automate this skill through the use of closed-form construct via dynamical systems. Such a scheme will enable robots to acquire and adapt their manipulation skills. The SAHR project will aim to demonstrate the general applicability of this scheme by having robots transfer fine insertion and polishing skills to different materials and at different scales.