Periodic Reporting for period 2 - SURPRISE (SUper-Resolved comPRessive InStrument in the visible and medium infrared for Earth observation applications)
Okres sprawozdawczy: 2021-07-01 do 2023-06-30
Earth Observation (EO) has many benefits, like its low operation cost, large availability and global continuous coverage. With respect to optical missions, applications are still limited by two main issues: revisit time and spatial resolution. While Low Earth Observation (LEO) missions can achieve resolutions better than 100 m, their revisit time typically stands at several days, limiting their capacity to monitor dynamic events. Geostationary (GEO) missions, instead, provide data on an hourly basis, but their spatial resolution is currently limited to 1 km typically, insufficient to fully address local phenomena. By studying a new concept of instruments with medium spatial resolution and near continuous revisit (hourly), the SURPRISE project could pave the way to a major breakthrough in EO.
SURPRISE’s main objective is to implement a demonstrator of an EO payload - working in the Visible - Near Infrared (VNIR) and in the Medium InfraRed (MIR) and conceived to operate from GEO platform - with enhanced performance in terms of spatial resolution, on-board data processing and encryption functionalities. This goal is achieved by using Compressive Sensing (CS) implemented via a Spatial Light Modulator (SLM) technology.
CS and SLM are used to devise a super-resolution architecture used to increase the number of pixels of the image observed by the instrument’s fore-optics, without increasing the number of detector’s elements. The acquisition of several measurements - each corresponding to the integrated value of the image modulated via suitable SLM patterns - is used to reconstruct the super-spectral image with a higher number of pixels. By applying CS techniques, the image can be reconstructed by acquiring a total number of measurements smaller than the number of pixels of the reconstructed image.
CS offers further advantages for handling large amounts of data and fast on-board processing for information extraction, as well as native encryption on top of native compression. SLM technology, at the core of the CS architecture, is also addressed by: reworking commercial SLM and testing current European SLM in relevant environment; outlining a development roadmap for a European SLM suitable for space.
In the whole, the SURPRISE action provided unprecedented insights on the implementation of super-resolved CS EO instrumentation thanks to: the construction of a demonstrator in the VNIR and MIR able to achieve a super-resolution factor of 32x32, the study of a European SLM technology suitable for space and the analysis of the impact of CS instrumentation on EO applications, with an emphasis on fire monitoring and ocean colour.
SURPRISE’s main objective is to implement a demonstrator of an EO payload - working in the Visible - Near Infrared (VNIR) and in the Medium InfraRed (MIR) and conceived to operate from GEO platform - with enhanced performance in terms of spatial resolution, on-board data processing and encryption functionalities. This goal is achieved by using Compressive Sensing (CS) implemented via a Spatial Light Modulator (SLM) technology.
CS and SLM are used to devise a super-resolution architecture used to increase the number of pixels of the image observed by the instrument’s fore-optics, without increasing the number of detector’s elements. The acquisition of several measurements - each corresponding to the integrated value of the image modulated via suitable SLM patterns - is used to reconstruct the super-spectral image with a higher number of pixels. By applying CS techniques, the image can be reconstructed by acquiring a total number of measurements smaller than the number of pixels of the reconstructed image.
CS offers further advantages for handling large amounts of data and fast on-board processing for information extraction, as well as native encryption on top of native compression. SLM technology, at the core of the CS architecture, is also addressed by: reworking commercial SLM and testing current European SLM in relevant environment; outlining a development roadmap for a European SLM suitable for space.
In the whole, the SURPRISE action provided unprecedented insights on the implementation of super-resolved CS EO instrumentation thanks to: the construction of a demonstrator in the VNIR and MIR able to achieve a super-resolution factor of 32x32, the study of a European SLM technology suitable for space and the analysis of the impact of CS instrumentation on EO applications, with an emphasis on fire monitoring and ocean colour.
The SURPRISE team first outlined the demonstrator’s architecture, its specifications and completed the design, also studying in detail the impact of SLM specifications and diffraction effects. All components were identified and procured, mostly among Commercial Off-The-Shelf (COTS) parts. Image simulations were used to assess the demonstrator’s expected performances, while reconstruction algorithms were developed with different approaches (Total Variation and deep learning) and their performances compared. A procedure to rework COTS SLM for operation in the MIR was consolidated. SLM operation was studied to outline the specifications for a space-oriented European SLM, able to operate also in the MIR. Current European SLM technology was tested in relevant environment.
In the second period, all sub-systems were constructed and the demonstrator assembled and comprehensively tested on different targets by acquiring CS measurements. Images were reconstructed from CS measurements by using the optimised algorithms, also with different compression ratios and super-resolution factors. EO applications that could most benefit from SURPRISE-like concept were identified - namely, fire monitoring and ocean colour –and user requirements used to outline two SURPRISE-like EO payloads, each targeting a specific application. CS-enabled encryption and on-board processing were studied. Development roadmaps for the European space-oriented SLM and for the SURPRISE-like payload were finally drafted.
The main results achieved at the end of the SURPRISE action include:
• state-of-the-art CS demonstrator with super-resolution up to 32x32 in the VNIR and MIR;
• detailed studies of SLM diffraction effects in the MIR;
• consolidated procedure for reworking COTS SLM;
• environmental testing on current European SLM technology and feasibility study with development roadmap for a European SLM for space;
• optimised reconstruction algorithms, with analysis to enable native encryption and on-board processing;
• two SURPRISE-like payload concepts, one for fire monitoring and the other for ocean colour – with relevant development roadmap.
Dissemination was achieved via: SURPRISE website and social media, 5 webinars with stakeholders (ESA, EUMETSAT, national Space Agencies of SURPRISE countries, end users communities), 4 project videos, participation to conferences with publication of 15 papers.
Exploitation was pursued for 3 identified key exploitable results (target scanning system, European SLM, CS demonstrator), with paths tailored to their different levels of innovation and maturity, with also the support of the H2020 Results Booster.
In the second period, all sub-systems were constructed and the demonstrator assembled and comprehensively tested on different targets by acquiring CS measurements. Images were reconstructed from CS measurements by using the optimised algorithms, also with different compression ratios and super-resolution factors. EO applications that could most benefit from SURPRISE-like concept were identified - namely, fire monitoring and ocean colour –and user requirements used to outline two SURPRISE-like EO payloads, each targeting a specific application. CS-enabled encryption and on-board processing were studied. Development roadmaps for the European space-oriented SLM and for the SURPRISE-like payload were finally drafted.
The main results achieved at the end of the SURPRISE action include:
• state-of-the-art CS demonstrator with super-resolution up to 32x32 in the VNIR and MIR;
• detailed studies of SLM diffraction effects in the MIR;
• consolidated procedure for reworking COTS SLM;
• environmental testing on current European SLM technology and feasibility study with development roadmap for a European SLM for space;
• optimised reconstruction algorithms, with analysis to enable native encryption and on-board processing;
• two SURPRISE-like payload concepts, one for fire monitoring and the other for ocean colour – with relevant development roadmap.
Dissemination was achieved via: SURPRISE website and social media, 5 webinars with stakeholders (ESA, EUMETSAT, national Space Agencies of SURPRISE countries, end users communities), 4 project videos, participation to conferences with publication of 15 papers.
Exploitation was pursued for 3 identified key exploitable results (target scanning system, European SLM, CS demonstrator), with paths tailored to their different levels of innovation and maturity, with also the support of the H2020 Results Booster.
The SURPRISE action has provided substantial insight for improving the capability of future space optical systems in many respects:
• Novel generation of optical systems: the demonstrator showed the possibility to construct a CS instrument able to produce super-resolved images up to a factor 32 x32 in the VNIR and MIR. The data acquired are natively compressed. The demonstrator mimics the operation of an EO payload from geostationary platform. To the author’s knowledge, this is the first demonstrator ever implemented featuring all these characteristics. Further, diffraction effects of SLM were also studied in detail, posing a sound basis for the development of CS-based instrumentation in the IR.
• A new approach to data acquisition, compression and encryption: the demonstrator and its testing have provided a meaningful insight for a novel, CS approach to data acquisition enjoying native compression and encryption, with a very convenient tradeoff between security and complexity.
• Data processing and computational burden: the action demonstrated that there is not the need of a specific compression and encryption hardware, reducing the amount of computations onboard and the need to handle large amount of data. Onboard capabilities analysis yielded better performances than in a conventional instrument, yielding high true positive and true negative rates.
Moreover, the action has contributed to strengthening the European scientific and industrial competitiveness in cutting-edge technologies: the demonstrator is presently at the forefront of CS research, while the SLM feasibility study and environmental tests poses a sound basis for European competitiveness in this cutting-edge technology.
Finally, the impact analysis on EO applications showed that the specifications of the SURPRISE-like GEO payload could provide clear benefits for Ocean Colour and Fire Monitoring, mainly in terms of spatial resolution together with revisit time, with potential large benefits for future EO application products, from environmental risk management to scientific investigation of high temperature events, from global change studies to climatological studies.
• Novel generation of optical systems: the demonstrator showed the possibility to construct a CS instrument able to produce super-resolved images up to a factor 32 x32 in the VNIR and MIR. The data acquired are natively compressed. The demonstrator mimics the operation of an EO payload from geostationary platform. To the author’s knowledge, this is the first demonstrator ever implemented featuring all these characteristics. Further, diffraction effects of SLM were also studied in detail, posing a sound basis for the development of CS-based instrumentation in the IR.
• A new approach to data acquisition, compression and encryption: the demonstrator and its testing have provided a meaningful insight for a novel, CS approach to data acquisition enjoying native compression and encryption, with a very convenient tradeoff between security and complexity.
• Data processing and computational burden: the action demonstrated that there is not the need of a specific compression and encryption hardware, reducing the amount of computations onboard and the need to handle large amount of data. Onboard capabilities analysis yielded better performances than in a conventional instrument, yielding high true positive and true negative rates.
Moreover, the action has contributed to strengthening the European scientific and industrial competitiveness in cutting-edge technologies: the demonstrator is presently at the forefront of CS research, while the SLM feasibility study and environmental tests poses a sound basis for European competitiveness in this cutting-edge technology.
Finally, the impact analysis on EO applications showed that the specifications of the SURPRISE-like GEO payload could provide clear benefits for Ocean Colour and Fire Monitoring, mainly in terms of spatial resolution together with revisit time, with potential large benefits for future EO application products, from environmental risk management to scientific investigation of high temperature events, from global change studies to climatological studies.