Periodic Reporting for period 2 - Drones4Safety (Inspection Drones for Ensuring Safety in Transport Infrastructures)
Période du rapport: 2021-12-01 au 2023-05-31
The Drones4Safety project aims to increase the safety of the European civil transport system by building a cooperative, autonomous, and continuously operating drone system that will be offered to railway and bridge operators to inspect their transportation infrastructure accurately, frequently, and autonomously.
Project main objective:
1- To develop a solution for harvesting energy for continuous drone inspection,
2- To increase inspection efficiency by developing AI algorithms,
3- To provide a platform for a collaborative drone operation for inspections,
4- To develop a failsafe inspection drone,
5- To build a cloud-based AI system for autonomous navigation.
By using the Drones4Safety charging, collaborating and inspection development approach, it is possible to inspect a vast portion of the European transport infrastructure autonomously, frequently, and at least 10 times cheaper than the current inspection solutions that are done by helicopters. The Drones4Safety system provides a time- and cost-saving method, featuring an enormous increase in inspection efficiency, measurable on several parameters:
A. time needed to collect and send enough information to allow assessment by an expert group,
B. feasibility of inspections compared to situations in which today it is not possible to visually inspect the structure,
C. costs of deploying expert personnel versus costs of deploying a swarm of autonomous drones,
D. coverage area - time needed to evaluate a large number of structures,
E. higher accuracy during inspections.
Project main objective:
1- To develop a solution for harvesting energy for continuous drone inspection,
2- To increase inspection efficiency by developing AI algorithms,
3- To provide a platform for a collaborative drone operation for inspections,
4- To develop a failsafe inspection drone,
5- To build a cloud-based AI system for autonomous navigation.
By using the Drones4Safety charging, collaborating and inspection development approach, it is possible to inspect a vast portion of the European transport infrastructure autonomously, frequently, and at least 10 times cheaper than the current inspection solutions that are done by helicopters. The Drones4Safety system provides a time- and cost-saving method, featuring an enormous increase in inspection efficiency, measurable on several parameters:
A. time needed to collect and send enough information to allow assessment by an expert group,
B. feasibility of inspections compared to situations in which today it is not possible to visually inspect the structure,
C. costs of deploying expert personnel versus costs of deploying a swarm of autonomous drones,
D. coverage area - time needed to evaluate a large number of structures,
E. higher accuracy during inspections.
The Project started on June 1, 2020. In the first 18 months, the drone system specifications and architecture have been defined and a preliminary description and analysis of the two use-cases (bridges and railways inspections) to verify and validate the platform has been developed, together with a list of selected test sites where the defined use-cases may be executed.
Preparation of the business model for the Drones4Safety (D4S) project in order to show the project’s feasibility from a technical and economic/financial sides.
The design of the system architecture follows top-down design methodologies with well-defined system interfaces to assure a flexible system design and seamless integration between the subsystem components of the D4S system. In addition, the architecture is designed to include multiple drones for swarming operations. The architecture assures the safety of the platform by separating the drone control system into two parts, high and low-level flight controls to guarantee a stable autonomous system design.
The research in drone autonomous navigation and grasping on overhead cables have been started by a global survey on lightweight sensors for cable detections with deep evaluations of the sensor findings. A flexible drone system has been developed using advanced heterogeneous chips (MPSoC) and sensors.
The research in AC and DC recharging started with a deep understanding of the environment in which the drone will be operating, which is followed by a design and a development of lightweight harvesting mechanisms for recharging from powerlines and railways.
The research for AI for fault detection has been started by:
• creating and selecting datasets of the common defects in railways and bridges;
• choosing a platform to host the 3D models and the inspection pictures (with labels) of the desired infrastructure for inspection;
• developing machine learning (ML) algorithms for autonomous fault detection in the gathered pictures. In particular, the ML activity was mainly focused on bridges for which both supervised and unsupervised ML techniques have been investigated and exploited.
The research in the drone swarm started by defining the swarm functionalities, developing and testing parts in a simulated environment, and specifying the communication infrastructure between the drones as well as between drones and the cloud services.
The research in mission control and navigation started by defining the cloud infrastructure for monitoring and controlling the drones. The first prototype of the cloud services (monitoring, control, and automatic route calculations) has been designed, developed, and tested.
The developed drone prototypes are continuously being tested at the novel powerline setup at SDU, Odense. The powerline setup provides a controlled testing environment for the project results.
The Dissemination and Exploitation activities already started including publications, presentations in conferences, the organization of workshops including joint workshops with other H2020 projects in MG-02-08-2019 call, presentations to industries, and educational institutes. Master theses with project results have been also presented.
The Industrial Advisory Board has been established and is consisting of 11 experts from 11 companies and research institutes. The experts are end-users, AI, energy harvesting, drone design, and technology developers.
Preparation of the business model for the Drones4Safety (D4S) project in order to show the project’s feasibility from a technical and economic/financial sides.
The design of the system architecture follows top-down design methodologies with well-defined system interfaces to assure a flexible system design and seamless integration between the subsystem components of the D4S system. In addition, the architecture is designed to include multiple drones for swarming operations. The architecture assures the safety of the platform by separating the drone control system into two parts, high and low-level flight controls to guarantee a stable autonomous system design.
The research in drone autonomous navigation and grasping on overhead cables have been started by a global survey on lightweight sensors for cable detections with deep evaluations of the sensor findings. A flexible drone system has been developed using advanced heterogeneous chips (MPSoC) and sensors.
The research in AC and DC recharging started with a deep understanding of the environment in which the drone will be operating, which is followed by a design and a development of lightweight harvesting mechanisms for recharging from powerlines and railways.
The research for AI for fault detection has been started by:
• creating and selecting datasets of the common defects in railways and bridges;
• choosing a platform to host the 3D models and the inspection pictures (with labels) of the desired infrastructure for inspection;
• developing machine learning (ML) algorithms for autonomous fault detection in the gathered pictures. In particular, the ML activity was mainly focused on bridges for which both supervised and unsupervised ML techniques have been investigated and exploited.
The research in the drone swarm started by defining the swarm functionalities, developing and testing parts in a simulated environment, and specifying the communication infrastructure between the drones as well as between drones and the cloud services.
The research in mission control and navigation started by defining the cloud infrastructure for monitoring and controlling the drones. The first prototype of the cloud services (monitoring, control, and automatic route calculations) has been designed, developed, and tested.
The developed drone prototypes are continuously being tested at the novel powerline setup at SDU, Odense. The powerline setup provides a controlled testing environment for the project results.
The Dissemination and Exploitation activities already started including publications, presentations in conferences, the organization of workshops including joint workshops with other H2020 projects in MG-02-08-2019 call, presentations to industries, and educational institutes. Master theses with project results have been also presented.
The Industrial Advisory Board has been established and is consisting of 11 experts from 11 companies and research institutes. The experts are end-users, AI, energy harvesting, drone design, and technology developers.
The drone system design and architecture including cutting-edge technologies such as the heterogeneous platform with unconventional sensors (mmWave and Magnetometers) with the latest Robot Operating System (ROS2) are completely new to use in the drone field.
The lightweight energy harvesting mechanism from DC lines is novel and promising from the business point of view, being the DC railway lines the most extended and old ones, thus enabling a DIaaS market.
The development of AI algorithms to detect defects with few training data thanks to:
• The additional use of synthetic 3D models and pictures to produce extra datasets to train AI supervised models;
• The exploitation of AI unsupervised models to be used in combination with supervised techniques thus enhancing detection capabilities (e.g. supervised models used for asset detection, then unsupervised for detecting if a defect is there or not and supervised models again to assign a category to detected defects)
The algorithms for the drone swarm system and its security are novel to use in the drone domain. The path planning component adds swarm control in a reliable manner taking into account obstacle and collision avoidance. To minimize the complexity of the system a simple approach to formation flying using leader-follows scheme has been designed and validated. We have contributed to modelling of cybersecurity threats of the multi-drone system highlighting the most important security risk of the unmanned aerial system.
Finally, in cloud services using the latest open-source platforms for building the system infrastructure and developing advanced search algorithms to run on the cloud:
• Extant algorithms for the vehicle routing problem are unable to account for the degree of freedom that drones have. IThe ability of drones to fly from any coordinate to another enables drones to be routed more flexibly and efficiently than other vehicles.
• Continuous integration and deployment of microservices allow for creating an extensible and easily portable cloud platform for drone-cloud communication and the off-loading of processing from edge devices such as the drones’ onboard computers.
Regulatory assessment activities showed that autonomous flights and swarm operations are not yet covered by the regulation. The work done partially demonstrated the feasibility and safety of such kind of operations and this may substantially contribute to both the technological advancement and the regulatory developments in the field.
The lightweight energy harvesting mechanism from DC lines is novel and promising from the business point of view, being the DC railway lines the most extended and old ones, thus enabling a DIaaS market.
The development of AI algorithms to detect defects with few training data thanks to:
• The additional use of synthetic 3D models and pictures to produce extra datasets to train AI supervised models;
• The exploitation of AI unsupervised models to be used in combination with supervised techniques thus enhancing detection capabilities (e.g. supervised models used for asset detection, then unsupervised for detecting if a defect is there or not and supervised models again to assign a category to detected defects)
The algorithms for the drone swarm system and its security are novel to use in the drone domain. The path planning component adds swarm control in a reliable manner taking into account obstacle and collision avoidance. To minimize the complexity of the system a simple approach to formation flying using leader-follows scheme has been designed and validated. We have contributed to modelling of cybersecurity threats of the multi-drone system highlighting the most important security risk of the unmanned aerial system.
Finally, in cloud services using the latest open-source platforms for building the system infrastructure and developing advanced search algorithms to run on the cloud:
• Extant algorithms for the vehicle routing problem are unable to account for the degree of freedom that drones have. IThe ability of drones to fly from any coordinate to another enables drones to be routed more flexibly and efficiently than other vehicles.
• Continuous integration and deployment of microservices allow for creating an extensible and easily portable cloud platform for drone-cloud communication and the off-loading of processing from edge devices such as the drones’ onboard computers.
Regulatory assessment activities showed that autonomous flights and swarm operations are not yet covered by the regulation. The work done partially demonstrated the feasibility and safety of such kind of operations and this may substantially contribute to both the technological advancement and the regulatory developments in the field.