Community Research and Development Information Service - CORDIS

H2020

AEROWORKS Report Summary

Project ID: 644128
Funded under: H2020-EU.2.1.1.5.

Periodic Reporting for period 1 - AEROWORKS (Collaborative Aerial Robotic Workers)

Reporting period: 2015-01-01 to 2016-06-30

Summary of the context and overall objectives of the project

The application domain that inspired the AEROWORKS research and innovation activities is that of civil infrastructure services, and its detectable growing necessity for high automation, improved Quality of Services (QoS) and capital-saving maintenance cycles, while retaining –or maximizing– safety and reliability. In order to clearly define the research activities and technological developments, and confine them within focused and specific boundaries, the project consortium has gathered experts from academia, robotics innovation enterprises, as well as key end-users, and aims to provide solutions to the specific –yet wide and challenging– area of civil infrastructure (power generation and distribution, oil & gas, water supply etc.) inspection and maintenance operations.

In general Inspection, repair and maintenance works are complex operations, executed in several cycles and typically require time-consuming, costly and risky procedures, involving highly trained personnel who employ largely non-unified and non-repeatable methods. A variety of methods and approaches are adopted to address the challenges of infrastructure maintenance. Specialized personnel perform visual inspection, nondestructive testing and maintenance tasks using scaffolds, roping or even manned helicopters in order to obtain access to the sites of interest.

The AEROWORKS project introduces the concept of “Collaborative Aerial Robotic Workers”. The goal is to develop a team of collaborative aerial systems equipped with advanced environmental perception and 3D reconstruction, active aerial manipulation, intelligent task planning, and multi-agent collaboration capabilities. Such a team of Aerial Robotic Workers (ARWs) will be capable of autonomously inspecting infrastructure facilities and acting in order to execute a maintenance task by means of aerial manipulation, and exploiting multi-robot collaboration.

Furthermore, the project aims to investigate the emerging scientific challenges of decentralized multi-robot collaboration, path-planning, control for aerial collaborative manipulation, aerial manipulator design, autonomous localization, as well as cooperative environmental perception and reconstruction. Emphasizing on technological innovation, AEROWORKS aims to investigate an approach with very promising returns in infrastructure inspection, repair & maintenance, leading to big savings in costs, while maximizing personnel/asset safety. With such a potential impact, AEROWORKS is at the forefront of bringing Robotics to the basis necessary in real applications, where they can make a difference.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

WP1 - Management

LTU is coordinating AEROWORKS and it is responsible for the tuning of all the activities within the project, the financial and progress status reporting as well as the coordination of the consortium meetings and handling the communication with the European Commission. Moreover, regarding the management, LTU has worked towards the early identification of potential issues and worked intensively for resolving them, while keeping the cooperation spirit and the working tempo in the project as high as possible.

WP2 – System Specifications

In this WP all the activities have been carried according to the DoA, while all the corresponding deliverables have been submitted as it was planned. This WP has been the base for all the developments of AEROWORKS since it contained all the specifications and the plans for the rest of the WPs. In this WP the initial and end-user specifications for the real life application of AEROWORKS and the corresponding concept of the ARW have been analyzed in an end-user approach in order to achieve the maximum impact.

WP3 – Dexterous Aerial Manipulation

To develop the concept of aerial robotic worker the manipulation system plays an important role for its effectiveness and overall performance. To this end, within T3.1 the project partners defined the specifications for the manipulation system derived from WP2 and the application scenarios. Additional technical and scientific requirements were highlighted from each partner through a questionnaire to further define the design constraints of the system. Consequently, multiple concepts have been proposed by the partners were the tree main have been completed and prototyped, by UT, LTU and ETH-ASL.
Task 3.2:
T3.2 started at M7 and focused on the testing of the manipulators by UT, LTU and ETH-ASL to understand the disturbances introduced to ARW from the operation of the manipulation system. During this period at first integration week the proposed systems were tested on bench and compared to get a better insight of their capabilities (weaknesses and strengths). Preliminary tests have also been done during flight operations to analyze their performance. Additionally, extensive simulations of the dynamic model (UAV and arm) were conducted to examine the interaction of the manipulator motion during a maneuver of the UAV.
Task 3.3:
This task started in M11. The project at this phase was focused on the design of end effectors capable for co-manipulation, as well as in the interaction of multiple agents through the forces-torques applied on their arms.

WP4 – Collaborative Perception, Mapping and Vision for Manipulation

Task 4.1:
The project focused on real-time visual-inertial navigation for single ARW. To this end, a new monocular visual-inertial odometry algorithm called ROVIO (Robust Visual Inertial Odometry) has been developed and demonstrated great performance and robustness for aerial robot applications.
Task 4.2:
In this task results on real-time dense scene reconstruction from a computationally constraint platform for inspection operations have been presented. Furthermore, initial work on developing a place recognition algorithm able to perform on scenarios with viewpoint changes has been conducted.
Task 4.3:
The collaborative pose estimation concept has been investigated with two agents flying to improve the accuracy in position and orientation of the manipulator for inspection and maintenance tasks.
Task 4.4:
Initial work on developing vision-based algorithms for manipulation. The preliminary work has been conducted by mounting a camera on a fixed base manipulator

WP5 – Aerial Robotic Workers Development and Control

Task 5.1:
The development of the aerial robot according to the specification of WP2 has been conducted. Navigation logic state machine and ROS user interface has been developed and thoroughly tested. Additionally, new electronics for enhanced control of the ARW have been developed.
Task 5.2:
The coupled model of the UAV and manipulator has been investigated.
A simulation framework (RotorS) based on Gazebo simulator and Bullet physics engine has been developed and currently is available in open source for aerial robots simulation. Control strategies and manipulators design within AEROWORKS has been successfully simulated within RotorS framework. The simulator features a clean interface similar to the real system
Task 5.3:
A baseline model-based controller based on linear Model Predictive Control (MPC) combined with external disturbances observer has been implemented and employed in most flight tests. This controller provides reliable and robust trajectory tracking for the aerial robot. Moreover, a fast nonlinear MPC for attitude tracking has been successfully implemented and tested also in the case of failure of one propeller.
Task 5.4:
Simulations have been carried out in RotorS simulator to examine the disturbances in the coupled system.

WP6 – Collaborative Planning and Control for Inspection and Aerial Manipulation

Task 6.1: The major contributions in this task consist the autonomous inspection of a large scale known structure for an ARW to decide whether maintenance is required. Furthermore autonomous exploration algorithm of an unknown or partially known environment has been developed with reduced computational demands. The agent follows a path that explores an extensive part of the unknown area. The obtained map can be used afterwards for inspection missions. Another path planner approach uses geodesic metric for creating the trajectory of an ARW footprint.
Task 6.2: The work done for this task is a continuation of T6.1. A distributed collaborative control framework for optimizing the coverage performance of a UAV swarm has been developed. The proposed strategy leads to coverage swarm configuration based on local information from neighboring UAVs. Moreover, centralized and a distributed model predictive control (MPC) schemes for cooperative motion control of Unmanned Aerial Vehicles were proposed to avoid collision with each other. An efficient coverage and inspection planner has been designed to realize 2D coverage by a team of ARW in emergency cases like limited power or harsh environments. Preliminary results on attitude synchronization of unit vectors presented showed that it could be used in multi agent control.
Task 6.3: Two cases were examined within this task. Load lifting using cables and manipulation with robotic arms. Initially, a controller has been designed to lift a load by an ARW using cable guaranteeing that the cable remained under tension. Mathematical models were developed to estimate the corresponding forces-torques applied on the end effector of two UAVs.
Task 6.4: While Tasks 6.1-6.3 focus on designing the continuous feedback allowing the realization of a motion or action task, in Tasks 6.4 and 6.5 these low-level controllers are abstracted as propositions such as “go to point A”, “grasp this object” or “request help from another ARW”. Then a high-level planning has to be designed to satisfy some specification son these abstract tasks. Most efforts were focused on the development of an efficient mission planning mechanism for a team of ARWs.
Task 6.6:
An initial contribution towards including heterogeneity in the designed method is the previously mentioned result on collaborative manipulation using robotic arms. This result involves several agents equipped with rigid manipulators cooperating to manipulate a single object under uncertainty in the kinematic and dynamic parameters of the agents and the object as well as heterogeneity in the agents' models and sensors

WP7 – System Integration and Evaluation

To better synchronize their efforts is has been decided among the consortium to establish integration weeks where all partners have the chance to integrate with other and improve the coordination of the project. During the first half of the project 2 integration weeks have been realized. The 1st week was held at ETH Zurich by V4R, while the 2nd was organized by UT in Enschede.
Regarding 1st week within WP3 the goal was to:
- align the ongoing work of the partners in terms of hardware and software interfaces
- test the manipulators on a bench setup in order to discover advantages and disadvantages of each ARW solution
- support other WPs and hardware development by performing preliminary flights
- understand the issues related to aerial manipulation and tool handling
- evaluate performance of manipulators during autonomous and manual flights

The three manipulators proposed by he partners have been initially configured and evaluated on a bench test in order to discover the advantages and disadvantages of each manipulation structures in terms of some indicators.

Additionally, there was an effort to interrelate WP3 and WP5 with preliminary flight tests with the ARW, to understand the influence of the dynamics of the manipulator on the control of the UAV and vice versa. Experiments have been conducted to have a preliminary evaluation for physical interaction of the aerial manipulator.

Within WP6 collaborative pose estimation has been partly integrated with the aerial platforms. Moreover, a single robot load lifting scenario was experimented. Next a leader follower scenario was considered where the leader is given a trajectory and the follower has to mimic the leader’s behavior. Finally, an area coverage algorithm based on Guaranteed Voronoi principle was implemented in simulations.

Within WP4 the visual inertial SLAM system called ROVIO was integrated to the platform. Additionally, the densification module for T4.2 was testsed with an obstacle avoidance module.

Within WP5 the ASCTEC Neo platform has been delivered and used for the flying tests. A appart from the close collaboration between WP3 and WP5 regarding the manipulation, a model predictive controller with external disturbance observer was running onboard for position reference tracking of the UAV. Furthermore, multiple simulation studies were performed:

• Dynamics of Aerial-manipulators
• Gazebo simulation of aerial manipulators
• Collaborative area coverage
• Collaborative Mapping

Regarding 2nd integration week the work was majorly focused on aerial manipulation. Efforts were made to integrate the end-effector, manipulator and controller to perform first tooled experiments.

The main goals for WP3 were:

• Active tool handling
• Test interaction capabilities of aerial manipulators

Within WP4 an ARW equipped with the visual sensor and running SLAM mapped the area for navigation and perform a light dense reconstruction in real-time (T4.2). Preliminary vision for manipulation experiments consisted the mechanical interface with in-hand-camera and the remote manipulation

In WP5 the aim was to control the ARW using MPC in combination with external disturbance observer and show behavior in contact with the environment. Additionally, preliminary tests of fast collision avoidance using depth sensor data were performed.

In WP6 simulations for collaborative load lifting were performed. Additionally, preliminary results on coverage algorithm “Efficient Coverage and Inspection Planner” (ECIP) that realized 2D coverage and inspection task by a team of ARWs as fast as possible.

WP8 – Dissemination, Promotion and Exploitation

AEROWORKS project participates in pilot initiative on open research data. Therefore discussions among partners have started to coordinate data sharing. D8.3 contains data management plan for open research data and is continuously revised from partners to define the sharing policies. Regarding dissemination actions project members have participated in various events to promote AEROWORKS concept. Moreover, AEROWORKS proposed special session on Aerial Manipulation in the 24th Mediterranean Conference on Control and Automation (MED).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

AEROWORKS have created progress beyond the State of the art in all the corresponding technological WPs. In short the project has introduced the totally novel concept of Aerial Robotic Worker (ARW) equipped with 3 totally novel aerial manipulators. The UAV for this experimentations was the NEO, an AEROWORKS developed UAV. For controlling the UAV as well as the UAV and manipulator combined system, a set of novel control and estimation schemes have been developed. These schemes were considering not only the case of a single ARW but also the case of multiple ARWs interacting with the infrastructure or performing coverage tasks. For adding robustness to the ARWs novel algorithms for visual navigation and environmental perception have been produced and experimentally evaluated where applicable. These components of the project have been integrated in 2 integration meetings that lasted 1 week each and the project have created significant novel integration activities towards the inspection and interaction with an infrastructure. Furthermore, AEROWORKS have initiated the process of evaluation the ARWS in real life working conditions, towards the final selected demonstration scenarios, which are the inspection and interaction with a wind turbine and a combustion chamber.

Related information

Record Number: 192955 / Last updated on: 2016-12-16
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top