Periodic Reporting for period 2 - REMARO (Reliable AI for Marine Robotics)
Okres sprawozdawczy: 2022-12-01 do 2025-04-30
To advance autonomy and reliability of underwater robot applications, Europe needs research-level experts in Artificial Intelligence for marine robotics, experts who master reliability methods for functional safety engineering and can communicate with safety certification bodies, end-users, and entrepreneurs. The European Training Network on Reliable AI for Marine Robotics REMARO trains 15 early stage researchers (ESRs) who will push forward the underwater robotics industry in Europe and grow the European blue economy. The overall goal and objectives of REMARO focuses on reliable AI—the robotic brain—for autonomous robots performing inspection, surveying, and maintenance tasks under water. REMARO ESRs work at the intersection of two disciplines: AI and software reliability. For this purpose, the network consists of 11 beneficiaries and 3 partner organizations that excel both in scientific and industrial leadership in these disciplines. The ESRs are exposed to both the robotics AI body of knowledge (e.g. AI methods for vision, localization, knowledge representation, planning, and control), and software reliability methods and tools (e.g. design patterns, testing and estimation methods, fault tolerance and reliability analysis, verification methods for safe control).
Throughout the duration of the project REMARO has worked on reliable AI methods for perception, reliable AI methods for knowledge representation, planning and mission execution, and tools and methods for safety, risk assessment, and self-diagnosis. The network has published results on sonar scene reconstruction, object detection, active learning, deep-learning-based pose estimation, and recognition of underwater species. REMARO ESRs have become experts on underwater simulation environments, underwater knowledge representation, automated runtime diagnosis and reconfiguration (meta-control), and risk averse planning under uncertainty. Other ESRs produced methods for testing underwater robot controllers, analysis of neural network controllers for safety, analysis of knowledge graphs and knowledge bases for correctness, and generating underwater test data for perception. All these methods, even those that are general, have been demonstrated in maritime applications, some in the actual maritime environments.
REMARO has organized nine training events, and several research workshops including two public events at established venues in robotics (ICRA) and formal methods (ETAPS). The trainings covered theoretical aspects (control theory), robot software engineering, robot software architecture, robot dynamics and control, perception, deliberation, but also transferable aspects such as ethics and handling stress in the dissertation process. They involved site visits to EIVA in Skandeborg/Denmark, DFKI in Bremen/Germany and DNV in Oslo/Norway, Ocean Scan in Porto, giving the ESRs exposition to the breadth of marine safety and engineering industry. REMARO has co-sponsored a prestigious PhD school on Probability in Computer Science (PICS) jointly with the European Association of Theoretical Computer Science, European Association for Theory and Practice of Software, ACM SIGPLAN, and ACM SIGSOFT. All ERSs have completed the training program, and five have already handed in their dissertations while the others are on track to completion.
Our online repositories include eight released marine datasets and ten open-source software artifacts. More than fifty research papers have been prepared. Most are published and available under an open-access agreement.
REMARO has organized nine training events, and several research workshops including two public events at established venues in robotics (ICRA) and formal methods (ETAPS). The trainings covered theoretical aspects (control theory), robot software engineering, robot software architecture, robot dynamics and control, perception, deliberation, but also transferable aspects such as ethics and handling stress in the dissertation process. They involved site visits to EIVA in Skandeborg/Denmark, DFKI in Bremen/Germany and DNV in Oslo/Norway, Ocean Scan in Porto, giving the ESRs exposition to the breadth of marine safety and engineering industry. REMARO has co-sponsored a prestigious PhD school on Probability in Computer Science (PICS) jointly with the European Association of Theoretical Computer Science, European Association for Theory and Practice of Software, ACM SIGPLAN, and ACM SIGSOFT. All ERSs have completed the training program, and five have already handed in their dissertations while the others are on track to completion.
Our online repositories include eight released marine datasets and ten open-source software artifacts. More than fifty research papers have been prepared. Most are published and available under an open-access agreement.
The key contributions of REMARO lie in bringing the AI methods and reliability to the underwater sector, which is in dire need of developing reliable autonomy, if we want to understand the ocean's ecosystems and build safe underwater infrastructures. The results of two ESRs have been already productized by EIVA. OceanScan MST is working on productizing sonar-based perception components. One ESR is opening a consulting company in the marine sector, and two continue to work in R&D in marine robotics and AI. We highlight selected scientific breakthroughs below:
Active learning for training models for pipeline following and diagnostics,
Several data sets and benchmarks for vision-based underwater localization algorithms,
A novel approach for estimating depth maps using conditional Generative Adversarial Networks (cGANs),
New methods for sonar-based object detection and guidance with side-scan sonars,
Development, enhancement, and benchmarking of dynamic models and maneuver/turning models using CFD-Derived Hydrodynamic Coefficients,
Meta-control, self-adaptation and safety assurance methods for autonomous underwater vehicles,
Semantic digital twins for marine robotics,
Testing frameworks for addressing the simulation-to-reality gap for testing perception (based on neural radial fields),
Test generation from hybrid system control models for AUVs dynamics and maneuvers,
Robust adversarial retraining enhancing robustness of perception models against noise,
Planning under uncertainty with statistical risk assessment for the marine domain,
Testing methods for knowledge-graph based decision making systems for robots.
Most of these developments are continued past the network operating part through productization, use in further research and for training MSc graduates in the domain.
Active learning for training models for pipeline following and diagnostics,
Several data sets and benchmarks for vision-based underwater localization algorithms,
A novel approach for estimating depth maps using conditional Generative Adversarial Networks (cGANs),
New methods for sonar-based object detection and guidance with side-scan sonars,
Development, enhancement, and benchmarking of dynamic models and maneuver/turning models using CFD-Derived Hydrodynamic Coefficients,
Meta-control, self-adaptation and safety assurance methods for autonomous underwater vehicles,
Semantic digital twins for marine robotics,
Testing frameworks for addressing the simulation-to-reality gap for testing perception (based on neural radial fields),
Test generation from hybrid system control models for AUVs dynamics and maneuvers,
Robust adversarial retraining enhancing robustness of perception models against noise,
Planning under uncertainty with statistical risk assessment for the marine domain,
Testing methods for knowledge-graph based decision making systems for robots.
Most of these developments are continued past the network operating part through productization, use in further research and for training MSc graduates in the domain.