At project start, there was a rough idea of how to design an Earth observation system to achieve the above objectives, as well as a general idea that such a system is useful for monitoring railways or geographically similar distributed infrastructures.
We identified further use cases for such an infrastructure earth observation system - road, dyke and power line monitoring - and created specific requirements for our pilot use case, the monitoring of railway tracks, as well as a demonstration scenario to test these requirements. While the final project goal is to setup a satellite constellation, the first step is to develop and integrate our system into an airborne prototype to demonstrate the feasibility of the concept. To keep our developments as compatible as possible for use in a satellite, we have expanded our use case requirements accordingly. This is to ensure that our solution is also suitable for the communication bandwidth and computing capacity of a spacecraft and can withstand the environmental conditions in space.
Using the above, we updated our system design. This is now partitioned into the flight platform and a ground system. The latter has a user interface component as well as a processing platform, which communicates with the interface to accept tasks and present results. The processing platform is set up using standard components -- containers to encapsulate software for processing steps, a message passing middleware to let these components exchange information, and an S3 storage to store bulk data. For both the user interface and the platform infrastructure we developed working prototypes and verified most of them using data from measurement flights. On the non-user end the ground system communicates with our flight platform. This will contain an on-board processing computer, which we specified and made an engineering model of, which we environmentally tested to check that the same computer would also work in space even if it will only be mounted on our demonstrator airplane used in this project. We updated the design of that flight platform to include the sensing equipment (navigation systems, nadir/oblique RGB cameras, SAR), in two pods mounted below each of the planes wings. We designed and assembled the wing pods to include acquisition computers for the sensing equipment and developed synchronization between the SAR or cameras, respectively, and the navigation systems. We also tested the individual hardware components, i.e. prior to integration into the platform, and calibrated sensors in order to ensure useful data is recorded after plane take-off.
One of the main ideas is to use sensors with different properties to observe the infrastructure and to find obstructions on, in our pilot use case, railway tracks. To do so, we developed and implemented processing steps from sensor data acquisition to the output of obstacle detections. For the RGB images, these are the determination of the regions of interest based on map data, geo-referencing of images, image masking and the detection of tracks in those images to find anomalies by comparing detections to railway track maps. For SAR, the steps are image formation in pre-determined regions of interest to produce SAR images corresponding to geo-referenced tiles, as well as the detection of changes of these SAR image tiles against references images. Finally these almost independent processing chains are joined by fusing the detections from the SAR and the RGB chains. For the individual processing steps, as well as for the software infrastructure to run them on the on-board platform, we developed prototypes, which are currently being integrated into the on-board processing system.
We also recorded datasets. This resulted in a preliminary RGB image dataset showing railway tracks with various anomalies captured on a very large and very diverse model railway, RGB images of railway tracks from UAV flights, as well as a small dataset of overlapping SAR and VIS images of railway tracks, some with obstacles placed onto them.
