Community Research and Development Information Service - CORDIS

H2020

SWARMs Report Summary

Project ID: 662107
Funded under: H2020-EU.2.1.1.7.

Periodic Reporting for period 1 - SWARMs (Smart and Networking UnderWAter Robots in Cooperation Meshes)

Reporting period: 2015-07-01 to 2016-06-30

Summary of the context and overall objectives of the project

In the near future the oceans will supply a substantial part of human and industrial needs: the oil and gas industry will move to ever deeper waters and renewable energy will be harvested from the seas in offshore wind farms and with tidal, current and wave energy converters. Furthermore minerals such as cobalt, nickel, and copper, rare earths, silver and gold will be mined from the seafloor (deep sea mining). To this end, new offshore and port infrastructure will need to be built, maintained and repaired.
Nowadays, the major part of subsea and offshore operations is done by divers in dangerous missions. This is true for all construction, repair and maintenance work in shallow waters, be it in the offshore renewable energy sector, the port infrastructure sector or dealing with subsea cables and pipelines. Since the number of available divers is limited and their deployment is very expensive, the dependency on their work represents a real threat to the offshore industry. The SWARMs project aims to solve this problem by extending the use of unmanned underwater vehicles (autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs)). This will be achieved by enabling AUVs and ROVs to collaborate in a cooperative mesh, thus increasing the reliability of AUV/ROV operations. Since in such a collaborative mesh vehicles can combine their functionalities, new applications become feasible. Additionally the SWARMs project will increase the autonomy of AUVs and improve the usability of ROVs by introducing new operator assistance functions for intuitive control. This will substantially reduce training times for operators which nowadays take up to five years. In contrast to present day subsea machinery, the SWARMs system will be very versatile, allowing seamless integration of new robots from different manufacturers. There is thus no need any more for tailor-made systems without any re-usability. This will increase the cost-efficiency of AUVs/ROVs and make them accessible to industries with less financial strength than the oil and gas industry. This way, by reducing the costs of offshore operations and making new applications possible, SWARMs will increase Europe’s competitiveness in the offshore and subsea sector. Moreover, the technologies developed within this project will also give Europe a head start in the emerging deep sea mining business where new subsea technologies will also be needed.
In detail, in order to achieve these goals we will develop:
- An intelligent human-machine interaction and control tool, including protocols for human-machine shared autonomy, task planning and operator assistance functions.
- A situation awareness/environment characterization platform to combine information coming from sensors (acoustic, vision) on different vehicles. This will allow a 3D representation of the environment. A vision based classification system will be designed for object identification and tracking.
- Intelligent coordination and decision-making algorithms for mission re-planning and re-synchronization to cope with the harsh and unpredictable subsea conditions.
- A robust communication network to allow messaging between cooperating vehicles (based on acoustic transmitters)
- A semantic middleware to enable robots from different manufacturers to collaborate and combine their capabilities. This will be based on the concept of robots as a service.
- A methodology for the efficient design, validation and verification of save underwater operations.

Early in the project there will be a first testing phase at the PLOCAN facility on the Canary Islands and, towards the end of the project, SWARMs’ achievements will be demonstrated in two field tests in the Black Sea and along the Norwegian coastline in different scenarios:
- Offshore maintenance operations in offshore wind parks and oil platforms: for example, the scouring around the foundations of offshore wind turbines will be repaired by refilling sediment in the washed-out areas and then levelling the seafloor. This will require the collaboration of AUVs capable to do bathymetry and heavy subsea machinery moving the sediment. As the development of heavy machinery is out of the scope of this project, this use case will be demonstrated on a smaller scale using ROVs instead of heavy machines.
- Monitoring of pollution due to hydrogen sulphide and oil spills: deploying a swarm of collaborating robots will enable the surveillance of a large area in short time.
- Support to construction processes in subsea environments: levelling of the seafloor prior to the construction of a new pier, support during the installation of pipelines.
Besides enabling future growth and job creation in the European blue economy, SWARMs will also help protecting the environment by making it possible to detect pollution and holding polluters responsible. Science and education can also profit from the SWARMs approach as it will make the use of AUVs/ROVs less cost intensive and more versatile, thus opening up new possibilities for universities and research institutions.

OVERALL OBJECTIVES:
The aim of the SWARMs project is to expand the use of AUVs/ROVs and facilitate the creation, planning and execution of maritime and offshore operations, thereby making autonomous operations a viable option for new and existent industries. This will reduce the operational cost, increase the safety of tasks assigned to divers and contribute to overtake the current factors that threaten the expansion of the offshore sector. The approach is to design and develop an integrated platform for a new generation of autonomous underwater operations, as a set of Software/Hardware components, adopted and incorporated into the current generation of underwater vehicles. This will enable execution of both simple and complex industrial operations by using cooperating heterogeneous vehicles. Autonomy, cooperation, robustness, cost-effectiveness, and reliability of the operations will be improved in terms of the activities’ sophistication involved in a mission. The SWARMs approach is underpinned by:
(1) A distributed, integrated and coordinated set of SW/HW components which enable that AUVs/ROVs, from different manufactures, can share (integrate) functionalities (robot features) in a transparent way, regardless both of the physical location as well as the tasks and activities to be carried out in the operations. This allows using the best functionality of each underwater robot when accomplishing and commanding (remote operation) sophisticated operations by means of distributing these among them. This will make the creation of new underwater vehicles to fit the particular operational needs unnecessary, guaranteeing the cost-effectiveness of AUVs/ROVs and the operational capabilities associated to achieve simple or complex underwater operations.
(2) A distributed set of intelligent components for perception, decision-making and environment recognition capable of assisting the vehicles in characterizing the working environment, including artefacts, and improving the autonomy, robustness, safety and reliability of the cooperating underwater vehicles involved in industrial operations.
(3) Demonstration of the SWARMs approach in different real-world scenarios belonging to end-user partners and depending on the type of operations for: (1) offshore maintenance operations of subsea infrastructures – pipes, cables, oil platforms, mine deposits and wind farms (a) by monitoring, inspecting and repairing the appearance of cracks, scouring, deteriorations, fouling, corrosions and affected surrounding, (b) by monitoring pollution and rising levels of hydrogen sulphide, and (c) by detection, inspection, and tracking of plumes from spill or emission; and (2) support to construction on subsea environments (d) for performing bathymetry and levelling operations on berms; (e) installation and maintenance of subsea pipeline and templates.
(4) Improve the communications technologies as a base of cooperation and information exchange amongst AUVs/ROVs, as well as the sensing (vision and acoustic) technologies used in the demonstration of the SWARMs demonstrator scenarios.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In the firs reporting period covered by this report (M1-M12), the following tasks have been carried out:
Within WP1 Task 1.1 has been developed according to the foreseen objectives and the deliverable D1.1 has been delivered as planned and has been accepted. Within WP2, several tasks have been started during this period. Task 2.1 has developed a methodology for analysing an autonomous operation, and D2.1 was delivered. In Task 2.2, the design guidelines for a future SWARMs user interface have been created and included in D2.3 which was delivered as scheduled. Task 2.3 is also progressing, following a similar approach as Task 2.1, and D2.5 was delivered in M12 as planned. Task 2.4 has just started in the last month of the present reporting period. Task 2.6 has been carried out in close cooperation with Tasks 2.1 and 8.1 in order to define the demonstrators use cases to test the developed methods from previous tasks. Within this task, D2.11 has been delivered as scheduled including the results obtained. Finally, in Task 2.7 deliverable D2.7 was delayed (from M12 to early M13), including the communication protocol of the semantic middleware with the individual ROVs/AUVs and manipulators.
Regarding WP3 the preliminary SWARMs architecture has been defined, highlighting the three main functional components: Mission Management Tool (MMT), middleware system, and robot system. As a way to define the architecture solution, discussions started from existing developments for coordination of underwater, but also aerial and ground, swarms/groups of robots and from the specifications provided by WP2 and scenarios defined by WP8. The four tasks of this WP (T3.1-T3.4) started as planned and one deliverable has been finished as scheduled: D3.1 (M9).
Several tasks within WP4 have been started in this period as well. In this period, a high resolution interferometric side-looking sonar was selected and the integration of the 3D bathymetric sonar into the small ECA AUV A9 was completed (Task 4.1). As for Task 4.2, an initial plan for the purpose of underwater middle size map making has been defined. Regarding Task 4.3, a preliminary state of the art review of suitable algorithms was conducted. Within T4.4, the initial version of the general Context Awareness framework, including the key components and respective interaction was finished and included in D4.10. Finally, within Task 4.5 the stereovision module of the Naiad AUV has been implemented, a plan for integration of the stereovision system with the Naiad has been defined and the initial versions of the software for vision data has been developed and tested. The following deliverables were finished and submitted as scheduled: D4.1(M12), D4.4(M12), D4.7(M12), D4.10(M12) and D4.12(M12).
Related with the communication and networked objectives, the four tasks within WP5 (T5.1-T5.4) have started. The general architecture of the communication network was defined on the base of information coming from storyboards analysis, mission definition, available vehicles fitted with specific sensors and equipment. Moreover, depending on the type of mission and tasks, the involved vehicles can work close together or move quite far one from the others. These requirements impose several constraints and need different sub-network to operate in parallel. One deliverable within this WP has been submitted as scheduled: D5.4.
The three tasks of WP6 started as planned (T6.1-T6.3), with the following achievements (although no deliverable has been completed during this period, following the schedule): ROS has been selected as robot environment for integration and several components have been already defined (Robot Planner, Robot Supervisor, Robot Monitor and Robot Interface).
The work done in WP7 started with Task 7.1 where the partners summarized the requirements for their individual storyboards, delivering a document comprising general concepts of the underwater vehicle and manipulator arm. In Task 7.2 a simulation environment solution was analysed and selected (compatible with the ROS solution selected in other WPs) and vehicles, manipulator, sensors and underwater environment on the chosen software were implemented. Task 7.4 produced some sketches and solved the basic flow and display of information in the different content categories, keeping in track the works in conceptual design of an intuitive input device, as well as start working on a user interface simulation prototype. Two deliverables were finished in this reporting period: D7.1 (delayed one month, from M7 to M8) and D7.2 delivered on time (M12).
Regarding the demonstrators included in WP8, within T8.1 three different demonstration scenarios were described in detail giving information about the capacities, logistics, characteristics, equipment and infrastructure of each of them. Furthermore, a series of storyboards were delivered describing all the use cases stage by stage. The results were included in D8.1 finished as scheduled (M6). Deliverable D8.2 was also finished as planned (M9), including a complete matrix describing the availability of the vehicles for each one of the demo sites, a functionality matrix describing a series of missions for each technology to be tested during the early trials, a large description of each missions and detailed description of each vehicle involved in SWARMs project. In addition, within Task 8.3 the missions delivered in T8.2 were extended into detailed mission explanations. In addition, a trial area characterization has been delivered in order to inform all the participant where and how to proceed during the early trials.
Finally, in WP9, the project website was developed and made available online. In addition, some additional actions were also carried out during this period towards the promotion of the project in relevant events. Furthermore, a general overview of the project, its core topics and initial developments has been made available online in the format of a Newsletter in SWARMs website. Regarding the exploitation plans, several partners initiated a further refined internal assessment process of the envisaged exploitable outcomes resulting from the project resulting in the exploitation plans presented in deliverable D9.7. An assessment has been carried out on existing and evolving standardization outcomes released by relevant standardization bodies, committees and working groups, on SWARMs related topics, that can also be eventually influenced, at least to some extent, by the project’s outcomes. Finally, the initial assessment of potential interactions among other research projects has been compiled in deliverable D9.1, where the results achieved in the first stage of T9.3 were presented. Within this reporting period, the following deliverables were completed: D9.1 (M1), D9.2 (M7, delayed one month), D9.3(M10), D9.7 (M13, delayed a couple of weeks), D9.9 (M13, delayed a couple of weeks) and D9.10(M12).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

SWARMs concept is based on the open innovation paradigm, assuming that firms can and should use a mix of external and internal ideas, and internal and external paths to market, to advance their technology . Certainly, SWARMs proposes a new approach to design safer Cyber Physical Systems (autonomous underwater systems) by integrating a platform of AUV/ROVS to operate in different environmental conditions. SWARMs approach aims at building innovation by improving the already available technologies and making the off-the-shelf technologies operating in an integrated and autonomous manner. SWARMs project addresses the System Engineering challenge of the ‘Robotics 2020 Strategic Research Agenda for Robotics in Europe’ and contributes to the integration and deployment of systems composed of multiple robots, and the interaction between the system and the environment.
SWARMs will strengthen the European industrial competitiveness and growth by reducing costs for offshore and subsea construction, repair and maintenance tasks. For example, this will allow Europe to extend its port infrastructure, which is important, given the ever increasing globalization. Also, offshore infrastructure will be less cost-intensive to construct and maintain, thus allowing Europe to develop its offshore energy sector, installing offshore wind parks or striving into more innovative ways to harvest energy from the ocean, such as ocean current and tidal turbines or wave energy converters. In this way, SWARMs will also improve Europe’s innovation capacities. Moreover, cost-reduction due to the SWARMs approach will also help the oil and gas industry in the Black and North Seas evolve. Both aspects will help Europe to meet its energetic needs.

Given the increasing world population it will become more and more important in the future to take full benefit of the resources of the oceans and SWARMs will help Europe to stay at the technological forefront of this development while at the same time protecting this vulnerable environment.The project is aligned with the Action Plan for a Maritime Strategy in the Atlantic area by targeting: ‘Priority 1-Promote entrepreneurship and innovation’: by increasing competitiveness and improving capacities in the blue technology sector; and ‘Priority 2- Protect, secure and develop the potential of the Atlantic marine and coastal environment’, by proposing a new approach to the management of offshore operations and marine environmental monitoring’.
SWARMs approach, once tested within the project, can be adopted and replicated in different sectors under the umbrella of the Integrated Maritime Policy, including monitoring Good Environmental Status indicators identified in the MSFD. SWARMS’ approach could also contribute to the Integrated Maritime Surveillance improving monitoring activities and reducing the environmental impact of more traditional monitoring activities dependent on fuel.
From a societal point of view, SWARMS will contribute to strengthen employment and EU competitiveness in the maritime and offshore operations domain and in the blue economy sector in general, in line with the objectives set out by the Europe 2020 strategy. In the short-term this will be achieved by giving the opportunity to Master and PhD students involved in the project to become involved in the industry process, gain familiarity with the industry environment and become more competent and competitive.
SWARMs’ approach will also aim to sensibly reduce the human risk-factor associated with offshore operations. This will be achieved by reducing the employment of divers for difficult long operations in extreme environmental conditions, thus contributing to increasing the safety of the operations. At the same time, personnel will be required by SWARMs to remotely command the systems deployed thus improving the job profiles required and guaranteeing employment sustainability.

Related information

Record Number: 192960 / Last updated on: 2016-12-16