Periodic Reporting for period 2 - AERIAL-CORE (AERIAL COgnitive integrated multi-task Robotic system with Extended operation range and safety) Reporting period: 2020-12-01 to 2022-05-31 Summary of the context and overall objectives of the project The main objective of AERIAL-CORE is the development of core technology modules and an integrated aerial cognitive robotic system that will have unprecedented capabilities on the operational range and safety in the interaction with people for applications such as the inspection and maintenance of large infrastructures. The project will integrate aerial robots with different characteristics to meet the requirements of: (1) Long range and local very accurate inspection of the infrastructure;(2) Maintenance activities based on aerial manipulation involving force interactions; and (3) Aerial co-working safely and efficiently helping human workers in inspection and maintenance. AERIAL-CORE technology modules will be based on Cognitive Mechatronics and will apply cognitive capabilities to aerial morphing in order to combine long range endurance and accurate observations, manipulation involving force interactions, and co-working with humans. The project will develop:(1) Cognitive functionalities for aerial robots including perception based on novel sensors, such as event cameras, and data fusion techniques, learning, reactivity, fast on-line planning, large scale mapping and teaming with multiple aerial robots; (2) Aerial platforms with morphing capabilities, to save energy in long range flights and perform a very accurate inspection; (3) Cognitive aerial manipulation capabilities, including manipulation while flying, while holding with one limb, and while hanging or perching to improve accuracy and develop greater forces; (4) Cognitive safe aerial robotic co-workers capable of helping humans working at height; and(5) Integrated aerial robotic system for the inspection and maintenance of large infrastructures. The higher TRL results of (1)-(4) and the integrated system (5) will be demonstrated in electrical power system inspection and maintenance, which is an application with a huge economic impact that also has implications in the safety of workers and in wildlife conservation. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The Project started December 1, 2019. In the first year the specifications and the cognitive architecture have been developed. The specifications are divided in three main groups: long range, manipulation and aerial co-worker. Moreover, application use cases have been defined.The architecture is based on hybrid paradigms. The design allows fast execution of the functional layer while bringing the capabilities of deliberative systems for more complex tasks and safe modes with human in the loop. The architecture is also able to integrate multiple aerial robotic systems. The research in cognitive functionalities has been started including learning methods, sensor data fusion, accurate tracking of electrical lines, autonomous landing/perching, multi-sensor large scale mapping and multi-UAV collaboration for long-endurance operation and development of simulators to test the above functionalities. Research in new aerial platforms with cognitive capabilities also started, including vehicles with both fixed-wing and rotary wing capabilities, fixed-wing vehicles capable of perching, morphing omnidirectional multi-rotor systems and fixed wing-flapping wing hybrid vehicles.Moreover the project started the research in arms and end-effectors for aerial manipulation, including end-effectors for holding/grabbing and manipulation, as well as robotic manipulators with force interactions. We also developed new aerial manipulators and developed the first cognitive perception and planning functionalities. In parallel augmented cognitive human-machine interfaces are being developed. The aerial-co-working research also started including the consideration of physical interaction, the cognition of human activities working in the lines, the human-aerial co-worker cognitive interaction and the coordination of a team of aerial co-workers.The above research involved the development of the first prototypes and experiments in the electrical lines. Thus, we performed the first experiments in 15 Kv lines, including physical contact with the line in tension.The Dissemination and Exploitation activities have already started including publications, organization of workshops, presentation in conferences including many keynotes, exhibitions, presentation to industries and educational presentations. The first Thesis with project results have been also presented. We have also defined the Industrial Advisory Committee consisting of 22 experts from 17 companies including aerial platform integrators, technology developers and end-users. This committee provided feedback about the specification and first developments of the project. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The cognitive architecture for the inspections and maintenance with one and multiple aerial robots, involving long range inspection, safe manipulation and co-working is completely new.We are also progressing beyond the state of the art in the development of cognitive functionalities for inspection and maintenance with aerial robots including: (1) new learning methods; (2) sensor data fusion with industrial sensors; (3) new accurate tracking systems combining visual and electromagnetic; (4) fully autonomous robust landing and perching; (5) new multi-resolution, multi-sensor 3D mapping capable of building accurate large scale maps ; and (6) application of multi-UAV cooperative techniques for the inspection of electrical grids with heterogeneous aerial robots.AERIAL-CORE is also developing very innovative UAVs including: (1) multi-segmented folding wings inspired by insect wings that have been incorporated in a new vehicle; (2) new perching systems for fixed-wing UAVs; (3) novel multi-rotor omni-directional vehicle that can morph to save energy; (4) Innovative platforms combining flapping wing for maneuvering and conventional fixed wing to save energy. We are also innovating in the development of end-effectors and robotic arms to perform maintenance tasks in electrical lines including: (1) end-effectors for the installation and removal of specialized devices such as wire separators, bird diverters and charging stations that will be used by the drones (2) bio-inspired soft actuators; (3) robotic manipulators with force interactions to perform impulsive-like forces in electrical linesMoreover, several innovative aerial manipulation technologies to perform the above-mentioned tasks in the electrical line have been identified. (1) New free flight manipulators with dual long-reach arms, manipulators grabbing at the same time the conductors and manipulation perched on the line; (2) Autonomous perception functionalities of the devices to be manipulated in the electrical line; New autonomous mission planning and trajectory planning techniques. On the other hand, we are developing intuitive and user-friendly interfaces based on human-body motion that will overcome any other teleoperation technology in aerial manipulation.Finally, we are also progressing beyond the state of the technology in the aerial co-working. This includes: (1) physical interaction of aerial co-workers by means of omnidirectional multi-rotor platforms; (2) identification of the activities of human operators working in the line by means of learning; (3) interaction of the aerial co-workers based on body gestures of the human operator; and 4) continuous assistance and monitoring of the workers in the electrical line by means of a team of aerial robots. AERIAL-CORE system architecture with multiple aerial robots Aerial manipulator touching a live-line Mapping experiment in the ATLAS demonstration scenario Aerial co-worker development: handover Fixed-wing UAV images for electrical power line inspection End-effector for installing wire separators Aerial manipulator with dual arms for the maintenance of electrical power lines Autonomous detection of power line elements Design of new arms for aerial manipulation
