Periodic Reporting for period 1 - MENELAOS_NT (European Training Network (ETN) on Multimodal Environmental Exploration Systems – Novel Technologies)
Okres sprawozdawczy: 2020-01-01 do 2021-12-31
In the context of rapid changes of our planet Earth, explosive population growth, accelerating industrialization, increasing exploitation of resources, serious natural hazards, recent natural disasters, and, importantly, the growing number of complex systems interacting with their environments, MENELAOS_NT addresses the key issue of scientifically and technologically exploring “the world” on micro- and macroscopic scale. The main goal is to provide scientists with deeper insights, better understanding and more information to understand and monitor the basic processes and environments in order to predict and possibly control critical evolutions. This would support decision makers with more sophisticated and reliable information for deciding on sustainable measures. MENELAOS_NT implements this by sophisticated interdisciplinary research combined with excellent training for the most promising ESRs in the field, facilitated by a well-balanced intersectoral consortium.
MENELAOS_NT addresses societal key challenges, e.g. sustainable agriculture and forestry, bioeconomy, environmental changes, resource efficiency, as well as protecting freedom and security of the European society. MENELAOS_NT applies Novel Technologies (NT) to realize multimodal – multi sensor data fusion to optimally combine the information “partitions”, delivered by different sensors (in-situ/remote, optical/non-optical) on different scales, with different resolutions and with different reliability. Traditional remote observation, as well as local (in-situ based) approaches have demonstrated to be insufficient to cope with the ample but very tailored information need of the decision-making bodies.
The “Big Data” approach to data acquisition appears feasible and popular, but comes at the cost of exponentially-growing data sets. In order to promote sustainable information generation, the state of the art of analogue to digital conversion, followed by denoising, interpreting and extracting information must give way to directly acquire information instead of data. The novel discipline of Computational Sensing describes the capacity of directly acquiring focused information rather than simply data. Prominent representatives of this discipline are the novel theories of Compressed Sensing (CS) and Compressed Learning (CL). MENELAOS_NT leverages these disruptive techniques to distill information at sensing.
MENELAOS_NT addresses societal key challenges, e.g. sustainable agriculture and forestry, bioeconomy, environmental changes, resource efficiency, as well as protecting freedom and security of the European society. MENELAOS_NT applies Novel Technologies (NT) to realize multimodal – multi sensor data fusion to optimally combine the information “partitions”, delivered by different sensors (in-situ/remote, optical/non-optical) on different scales, with different resolutions and with different reliability. Traditional remote observation, as well as local (in-situ based) approaches have demonstrated to be insufficient to cope with the ample but very tailored information need of the decision-making bodies.
The “Big Data” approach to data acquisition appears feasible and popular, but comes at the cost of exponentially-growing data sets. In order to promote sustainable information generation, the state of the art of analogue to digital conversion, followed by denoising, interpreting and extracting information must give way to directly acquire information instead of data. The novel discipline of Computational Sensing describes the capacity of directly acquiring focused information rather than simply data. Prominent representatives of this discipline are the novel theories of Compressed Sensing (CS) and Compressed Learning (CL). MENELAOS_NT leverages these disruptive techniques to distill information at sensing.
In the following we summarize the progress on each work package (WP). Acronyms: RR1: first research report, FP1: presentation at the first annual Fall Presentations, CP1: research paper in a conference.
WP1: Novel Sensors and Systems:
Deliverables corresponding to the RR1, FP1, and CP1 within WP1 featured contributions of the ESR 1. All have been timely submitted.
WP2: Sensor Information Flow and Integration:
Deliverables corresponding to the RR1, FP1, and CP1 within WP2 featured contributions of the ESRs 2-6. All have been timely submitted.
WP3: Novel Approaches to Sensor Data Processing:
Deliverables corresponding to the RR1, FP1, and CP1 within WP3 should feature the contributions of the ESRs 7-11. Nevertheless, the scientific contributions of the ESRs 10 and 11 are still missing due to recruitment delays. First versions of all deliverables excluding the missing contributions have been timely submitted.
WP4: High Level Information Mining:
Deliverables corresponding to the RR1, FP1, and CP1 within WP4 feature the scientific contributions of the ESRs 12-15. All have been timely submitted.
WP5: Doctoral Training:
a) Generation of career development plans (CDPs).
b) Setup of e-learning facilities.
c) Collecting all available courses.
d) Completion and evaluation of the FP1.
e) Organization of network-wide events. Major events: FP1 and Mid-term Summer School (MSS).
f) Coordination and supervision of 12 secondments.
WP6: Communication, Exploitation & Dissemination:
a) Dissemination to scientific community: CP1 and the implication of external scientists in network-wide dissemination events (FP1, MSS). In the web: ResearchGate, ZENODO, and openAire.
b) Dissemination to industrial community: agreement on preparing a talk for the “Radar in action” series and active dissemination in LinkedIn.
c) Dissemination to general public: participation in the ERN 2021. Social media: LinkedIn, Twitter, and COLCHIS blog.
d) Formulation of an exploitation strategy
WP7: Management & Administration:
a) Consortium Management: Kick-off meeting. Definition of main boards and committees. Definition and update of a Consortium Agreement. Submission of the progress report and associated mid-term check.
b) Submission and update of the “Comprehensive Risk Plans”.
c) Recruitment of all ESRs.
WP8: Open Research Data Pilot (ORDP):
Generation and maintenance of a data management plan (DMP). To date, no modification to the first version has been deemed necessary and the latter remains valid.
WP9: Ethics requirements:
The network has timely submitted the two deliverables foreseen within this WP (NEC and DU).
WP1: Novel Sensors and Systems:
Deliverables corresponding to the RR1, FP1, and CP1 within WP1 featured contributions of the ESR 1. All have been timely submitted.
WP2: Sensor Information Flow and Integration:
Deliverables corresponding to the RR1, FP1, and CP1 within WP2 featured contributions of the ESRs 2-6. All have been timely submitted.
WP3: Novel Approaches to Sensor Data Processing:
Deliverables corresponding to the RR1, FP1, and CP1 within WP3 should feature the contributions of the ESRs 7-11. Nevertheless, the scientific contributions of the ESRs 10 and 11 are still missing due to recruitment delays. First versions of all deliverables excluding the missing contributions have been timely submitted.
WP4: High Level Information Mining:
Deliverables corresponding to the RR1, FP1, and CP1 within WP4 feature the scientific contributions of the ESRs 12-15. All have been timely submitted.
WP5: Doctoral Training:
a) Generation of career development plans (CDPs).
b) Setup of e-learning facilities.
c) Collecting all available courses.
d) Completion and evaluation of the FP1.
e) Organization of network-wide events. Major events: FP1 and Mid-term Summer School (MSS).
f) Coordination and supervision of 12 secondments.
WP6: Communication, Exploitation & Dissemination:
a) Dissemination to scientific community: CP1 and the implication of external scientists in network-wide dissemination events (FP1, MSS). In the web: ResearchGate, ZENODO, and openAire.
b) Dissemination to industrial community: agreement on preparing a talk for the “Radar in action” series and active dissemination in LinkedIn.
c) Dissemination to general public: participation in the ERN 2021. Social media: LinkedIn, Twitter, and COLCHIS blog.
d) Formulation of an exploitation strategy
WP7: Management & Administration:
a) Consortium Management: Kick-off meeting. Definition of main boards and committees. Definition and update of a Consortium Agreement. Submission of the progress report and associated mid-term check.
b) Submission and update of the “Comprehensive Risk Plans”.
c) Recruitment of all ESRs.
WP8: Open Research Data Pilot (ORDP):
Generation and maintenance of a data management plan (DMP). To date, no modification to the first version has been deemed necessary and the latter remains valid.
WP9: Ethics requirements:
The network has timely submitted the two deliverables foreseen within this WP (NEC and DU).
ESR 1: The progress beyond the state of the art (SotA) comprises a ToF macro-pixel structure that allows to acquire multi-frame data in one shot, while resonant demodulation annihilates the harmonic content.
ESR 2: The progress beyond SotA comprises a 4T-APS event camera with in-pixel circuitry for high dynamic range with CDS included, providing ON/OFF events by frame differencing.
ESR 3: Progress beyond SotA includes methods for accurate and long-range dense ToF imaging from few measurements has been demonstrated by means of CS, including low-density sensing matrix design.
ESR 4: As opposed to the SotA in ToF imaging, where a powerful illumination system is required, 3D ToF estimation has been demonstrated without a dedicated illumination source, exploiting VLC or LiFi.
ESR 5: Beyond SotA is the development of an NN architecture that takes a single out-of-focus image as input, and simultaneously estimates a depth map and restores the all-in-focus image.
ESR 6: Beyond SotA is a method for radar band fusion that relies on a CS algorithm (ISTA), rather than spectral estimation, to estimate missing data within a frequency gap.
ESR 7: Beyond SotA is the processing, analysis, and interpretation of medium and high-resolution SAR images as well as geophysical measurements in order to estimate the Doppler centroid image.
ESR 8: Differently from the SotA, the ESR has applied the optical flow method to solve the coregistration problem for airborne SAR data and studied CS-based tomographic SAR for the first time.
ESR 9: Progress beyond SotA incudes an algorithm to improve the reconstruction capability of sparse reconstruction algorithms for MIMO radar by optimizing the coherence of the sensing matrix.
ESR 12: Progress beyond SotA incudes the study of different perception low-cost sensors in outdoor environments and the accuracy evaluation of ToF cameras such as Azure Kinect.
ESR 13: Beyond SotA is the introduction of a normalization approach for LiDAR data within the Scan to BIM framework that takes geometry, scanlines, and intensity of point clouds into account.
ESR 14: Methods beyond SotA include a hybrid GAN that leverages spectral angular distances for cloud removal and a classification approach based on probabilistic distances.
ESR 15: A major contribution beyond SotA is the development of complex-valued deep architectures for SAR data processing and the application of applied semantic data mining techniques to EO data.
ESR 2: The progress beyond SotA comprises a 4T-APS event camera with in-pixel circuitry for high dynamic range with CDS included, providing ON/OFF events by frame differencing.
ESR 3: Progress beyond SotA includes methods for accurate and long-range dense ToF imaging from few measurements has been demonstrated by means of CS, including low-density sensing matrix design.
ESR 4: As opposed to the SotA in ToF imaging, where a powerful illumination system is required, 3D ToF estimation has been demonstrated without a dedicated illumination source, exploiting VLC or LiFi.
ESR 5: Beyond SotA is the development of an NN architecture that takes a single out-of-focus image as input, and simultaneously estimates a depth map and restores the all-in-focus image.
ESR 6: Beyond SotA is a method for radar band fusion that relies on a CS algorithm (ISTA), rather than spectral estimation, to estimate missing data within a frequency gap.
ESR 7: Beyond SotA is the processing, analysis, and interpretation of medium and high-resolution SAR images as well as geophysical measurements in order to estimate the Doppler centroid image.
ESR 8: Differently from the SotA, the ESR has applied the optical flow method to solve the coregistration problem for airborne SAR data and studied CS-based tomographic SAR for the first time.
ESR 9: Progress beyond SotA incudes an algorithm to improve the reconstruction capability of sparse reconstruction algorithms for MIMO radar by optimizing the coherence of the sensing matrix.
ESR 12: Progress beyond SotA incudes the study of different perception low-cost sensors in outdoor environments and the accuracy evaluation of ToF cameras such as Azure Kinect.
ESR 13: Beyond SotA is the introduction of a normalization approach for LiDAR data within the Scan to BIM framework that takes geometry, scanlines, and intensity of point clouds into account.
ESR 14: Methods beyond SotA include a hybrid GAN that leverages spectral angular distances for cloud removal and a classification approach based on probabilistic distances.
ESR 15: A major contribution beyond SotA is the development of complex-valued deep architectures for SAR data processing and the application of applied semantic data mining techniques to EO data.