Periodic Reporting for period 2 - I.AM. (Impact Aware Manipulation by Dexterous Robot Control and Learning in Dynamic Semi-Structured Logistic Environments) Reporting period: 2021-07-01 to 2022-12-31 Summary of the context and overall objectives of the project Robotics is revolutionizing logistics applications. However, robot manipulation abilities are still far from human level of dexterity. Today, robots are not yet able to perform dynamic manipulation tasks such as filling densely and quickly a box with items or swiftly grasping heavy boxes from a pallet. The current boundary of state-of-the-art robot control is reached when contact is established at non-zero speed and simultaneously at multiple locations and when objects are heavy with respect to the robot own mass (e.g. lifting a 10 Kg box with two arms establishing contact at a speed higher than 0.1 m/s). Contacts established in these conditions lead to impacts, physical interactions characterized by fast changes in robot and object velocities and, at impact times, large peaks of the interaction forces. Current limitations in performing dynamic manipulation tasks are caused by the inability to reliably predict the effect of robot-object impacts as well as use these predictions for learning, sensing, and control impact manipulation motions. As a response to these challenges, I.AM. will (1) validate robot-object impact models to predict the result of a collision, (2) learn dynamic motions with impacts to achieve user specified goals, (3) provide robust sensing of robot-object post-impact velocities, contact forces and contact state, and (4) allow for robust robot control of dynamic manipulation tasks. Finally, I.AM. will (5) demonstrate this impact aware manipulation technology in relevant logistics scenarios with socio-economic impact. Regulation surrounding construction and logistics is becoming even more restrictive. Regulations that guarantee continuous improvement of working conditions has always been the aim of the trade unions. Action has been taken in this direction. For example, working time reduction, total weight limit to be handled within a working shift, height limitations in the loading of container (impact on the capacity utilization rate). These regulatory restrictions provide an incentive to develop robotic technology to ensure that workers are safe and working within regulations. In the logistic sector, the job of workers that do take-put processes is physically demanding work, where operators need handle items up to 15 kg, in 4 hour shifts. I.AM. will create a better business case for automating more processes in the logistics domain. Automation of up to 5% more processes in typical retail and parcel distribution centers and even up to 45% more processes in typical E-commerce warehouses are estimated by Vanderlande to have a sound business case through I.AM technology. With this increased automation of take-put processes that involves lifting of items and cases up to 15kg, the amount of injuries in the work force will be significantly lowered. Furthermore, I.AM. will trigger the attention of several robotic manufacturers on the potentials of developing an impact aware manipulation technology for robots and thus create new market and job opportunities. Because I.AM. builds on different robotic domains, a multi-level impact is expected: (1) Improved impact modeling and simulation will allow to increase the performance of next generation torque-controlled robots, by allowing reliable simulations with impacts considered in an early engineering stage and guiding design decisions; (2) better understanding and modeling of dynamic interaction will help to increase the performance on consolidated applications currently addressed with simpler static/quasi static manipulation; (3) Improved planning and sensing will allow implementing new tasks of high dynamic performance opening the way to new and emerging applications; (4) Greater availability and easy adoption of simulation technology thanks to simple interfacing with controller libraries. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far Several activities have been conducted in concert by the I.AM. project partners to ensure reaching the project objectives and demonstrating the validation scenarios.In particular, under the lead ot the TU/e, the consortium has created procedures to estimate object-enviroment and robot-environment impact dynamics and impact model parameters from motion capture and robot encoder data. Furthermore, a procedure to identify stiffness and damping properties of a suction cup gripper under partial vacuum from motion capture data has been devised. Procedures to learn a dynamical system to perform swift motions such as tossing, hitting, and grabbing of relative heavy objects have been developed by EPFL. Furthermore, a method to obtain more accurate robot-enviroment dynamic intereaction forces and faster impact detection (from hundreds of second to a few milliseconds) has been proposed and validated by TUM, based on data from the joint encoders, joint torque, and an additional external IMU positioned on the end-effector. An impact-aware QP robot control framework has been developed, including learning procedures to estimate dynamics properties of soft objects to perform impacts without violating hardware limitations, by CNRS. Algoryx has been key in developing a composite software which allows to represents the dynamic and geometric properties of simulation scene while also allowing for integration with QP robot control as well as in supporting TU/e in creating a data collection and storage procedure for impact motions. Some of the collected data has already been openly released, encouraging research on impact-aware robotics. Supported by all I.AM. project partners, Smart Robotics and Vanderlande have been instrumental for the setting up of a laboratory on TU/e campus for the TOSS And BOX scenarios, provided with a large conveyor belt, two robot arms (UR10 and Panda), and a motion capture system; Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) Most of the research work that has been reported in the first period of the project goes beyond what is the current state of the art in impact-aware robotics. The expectation for the end of the project is that the research conducted on impact-aware modeling, learning, sensing, and control will turn into solid methods and software components with the potential of becoming part of the software suite offered by some of the I.AM. project partners and also provide important insights for robot hardware redesign for impact-aware manipulation. To provide a concrete example, we expect that the model of the suction cup in holding and release phases and the corresponding procedure to estimate its parameters from motion capture data will become a valuable software module to provide a physics simulator with realistic physics of robot-object interaction for autonomous manipulation in logistics. Vanderlande Innovation Lab at TU/e campus
