Periodic Reporting for period 2 - SeaClear (SEarch, identificAtion and Collection of marine Litter with Autonomous Robots) Reporting period: 2021-07-01 to 2022-12-31 Summary of the context and overall objectives of the project Today's oceans contain 26-66 million tons of waste, with approximately 94% located on the seafloor. So far, collection efforts have focused mostly on surface waste, with only a few local efforts to gather underwater waste, always using human divers. No solution exists that exploits autonomous robots for underwater litter collection; the SeaClear project will develop the first. We will create a mixed team of Unmanned Underwater, Surface, and Aerial Vehicles – UUVs, USVs, UAVs – to find and collect litter from the seabed and from the water column, focusing on coastal areas since that is where waste inflow concentrates. The UAVs and an inspection UUV map the litter, aiming to establish correlations between surface and underwater litter. A collection UUV then gathers litter, using a combined suction-gripper manipulator. The end goal is to operate the robots autonomously, without remote human intervention. When fully operational, the SeaClear system aims to detect and classify underwater litter with 80% success rate, and collect it with a 90% success rate; all this at 70% reduced cost compared to human divers.The SeaClear system will be displayed at four demo sites where an autonomous robot system will be verified: one system for the purpose of cleaning ports in the Hamburg port area on two demo site locations with the end-user Hamburg Port Authority (Germany), another in the area of Dubrovnik, namely near Lokrum Island and one in the area of the Mali Ston Bay, with Regional agency DUNEA (Croatia) as end-user for these areas. With these four demo sites, SeaClear has an overview in completely different sectors, given the fact that we have a pilot site of port location emphasizing maritime industries and two other locations representing protected nature areas, one from the tourism sector and the other from the mariculture sector, namely the shellfish industry. Including all of the mentioned areas, the SeaClear system will face different waste fractions, both from inland and sea origin, and obstacles that need to be solved in order for such a system to be fully functional. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The SeaClear consortium has successfully achieved all its objectives as stated in the DoA until now (Month 36) with significant progress in system, software, and hardware development as well as in development of use cases, outreach, and community building. Important highlights include: A cost effectiveness study based on the current development status has been set up benchmarking the robotic system with divers, proving SeaClears positive impact in terms of cost and performance; the robot hardware in the system (USV, inspection UUV, collection UUV, gripper, collection basket, and interfaces) is at a highly advanced stage of development, with many components already fully ready: USV, collection UUV, the UAV itself; the fully novel gripper and an advanced prototype of the collection basket. Initial versions of all key software components are developed and most of them already (individually) verified on the robotic system or with real data: litter detection, mapping, and control of the robots. Significant results here include the UUV pose estimation using the UAV with experimental verification, improvements of the digital twin via modelling the tether influence, the creation of a new SeaClear trash-dataset of the demonstration sites, the experimental validation of the aerial mapping and litter detection despite glare on the ocean surface, a qualitative experimental validation of camera- and sonar-based mapping, model-free reinforcement learning and model-based planner strategies for active mapping, an improved image-enhancement and sensor fusion pipeline for underwater object classification via convolutional neuronal networks, improvements on the mixed model-based and learning control strategy for grasping objects with the collection ROV and the transfer of the visual servo control to the automated basket approach. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The key innovation in SeaClear is that our project is the first to develop an autonomous, robotic solution for cleaning litter off the sea floor. This involves significant advances beyond the state of the art in litter classification, machine learning for mapping, and cooperative and shared control.We are already making a significant impact with the public, with two popularization videos, two highly successful press releases, and many outreach talks, among others. Stakeholders have started being involved in a community-of-practice in Dubrovnik. We have established connections with several existing companies and projects on related topics. In addition, we have worked out a business plan and a data management plan.Regarding system integration, the project concept was analyzed together the main system components that were defined, and the appropriate server backbone has been designed and implemented. The SOC has been tested with each individual SeaClear robot for commanding the robots to specific waypoints, while the SeaClear Service Layers has enabled monitoring the robots over the WebUI. Individual components and some software interfaces that enable the flow of information from the robotic system to the shore operators and SeaClear clients and backwards have been tested during preliminary trials. Alongside the network infrastructure has been validated. Several individual hardware and software components of the SeaClear robotic system have been tested and validated during test campaigns in Dubrovnik, Hamburg and Marseille. We have significantly improved the web and social media presence of the project, issued highly impactful press releases, represented the project at many outreach events, and started up our exploration steering board; in addition to many other dissemination and exploitation activities. Initial experimental tests were conducted on all three demo sites which proved to be very useful for the system setup. During these trials data needed for litter detection and verification of estimation and control algorithms was gathered and documented. Moreover, we are currently in the phase of planning our first demonstration, bound to happen in May-June 2023. The progress made in the reporting period provides very good prospects that we will be able to successfully carry out the work and realize the objectives of the SeaClear project. In addition, the fact that the end-users are involved in all steps of the project significantly increases the chances of successful deployment of the system after the end of the project. Visualization of SeaClear system
