RapidAI4EO - Advancing the State-of-the-Art for Rapid and Continuous Land Monitoring

Project Information

RapidAI4EO

Grant agreement ID: 101004356

DOI

10.3030/101004356

Project closed

EC signature date 16 October 2020

Start date 1 January 2021

End date 31 March 2023

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies – Space

Total cost

€ 1 498 987,50

EU contribution

€ 1 498 987,50

Coordinated by

VLAAMSE INSTELLING VOOR TECHNOLOGISCH ONDERZOEK N.V.
Belgium

Periodic Reporting for period 2 - RapidAI4EO (RapidAI4EO - Advancing the State-of-the-Art for Rapid and Continuous Land Monitoring)

Reporting period: 2022-01-01 to 2023-03-31

Summary of the context and overall objectives of the project

Between January 2021 and March 2023 Planet Labs, VITO Remote Sensing, Vision Impulse, IIASA and Serco worked together in the H2020 project RapidAI4EO with the ambition to contribute enabling new and better ways of measuring and understanding the human footprint on our planet. The project idea was built on the convergence of two exciting trends in remote sensing and artificial intelligence: the explosion in satellite sensing constellations, both government and commercial, and incredible advances in Artificial Intelligence. The overarching goal of the project was to advance the state-of-the-art in deriving time-critical and location-specific insights into dynamic land surface processes by leveraging these trends, in particular, to enable faster and finer-grained classification and change detection of land use and land cover in Europe. To this aim we have created and released the RapidAI4EO corpus which is the largest and most comprehensive labelled, multi-source and multi-scale, spatiotemporal training dataset. This open dataset is expected to stimulate further development of new spatiotemporal land monitoring applications. We have implemented and benchmarked a number of novel Machine Learning (ML) architectures based on Deep Learning (DL) for patch-based change detection and pixel-based land cover mapping that can realise the full potential of new combined data sources. The project contributes to improving our understanding of land cover and has demonstrated a large scale, end-to-end process to continuously monitor and update LULC products.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have created the RapidAI4EO corpus which is the most complete and dense, labelled, spatiotemporal, analysis-ready training dataset for modelling the dynamics of land covers (LC). It combines the higher spectral resolution of Sentinel-2 (12 bands) with the higher spatial (3m) and temporal resolution (5-daily observations) of cloud-free, harmonised Planet Fusion imagery at 500,000 patch locations. Locations were sampled to account for CLC classes and spatial distribution, as well as country representation, across the entire territory of the EU. The corpus is available to the community on Radiant Earth’s Source Cooperative (https://rapidai4eo.source.coop/) a key platform for ML datasets in the EO sector ensuring long-term access to this unique dataset. The corpus has the potential to improve various Copernicus Land Monitoring Service products, mainly because of its enhanced spatial and temporal resolution, and can be augmented with new labels, making it a versatile resource which can be used for other applications especially those which benefit from high cadence timeseries of imagery.
Furthermore, we have built and experimented with ML architectures based on the latest developments in Artificial Intelligence (DL) for patch-based change detection and pixel-based LC mapping that can realise the full potential of the combined data sources and temporal cadence of observations contained in the corpus. We have demonstrated our ability to detect change on a quarterly and land cover on a monthly basis. The introduction of the temporal dimension in both the change detection and LC classification was improving model performance. We experimented with both supervised and unsupervised methods. The developed methods were able to highlight LC changes at patch level, especially when combined with other techniques (e.g. Computer Vision methods, pixel-based filtering) and domain knowledge. They hold potential for integration in operational workflows as these support faster updates of existing LULC products but further work is required to facilitate wider adoption of these methods.
We presented the project and its results at multiple key conferences in the EO and ML domain (BiDS, IGARSS, ISPRS, ESA Living Planet) and released additional datasets to the community on open access platforms (reference dataset, LC maps). Furthermore, we released two demonstrations: (1) mimicking the human annotator workflow and (2) surfacing patch-level LULC maps and change detection heatmaps that resulted from the supervised ML work.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The RapidAI4EO dataset is revolutionary in both its spatiotemporal dimension and quality (multisource, harmonised, cloud-free, analysis ready). Before its advent, there were no available annotated large scale data sets that provided the high cadence, multi year time series necessary to drive the development of new AI architectures that can fully exploit the new data streams provided by multiple constellations.
The current CLC product has a Minimum Mapping Unit of 25 ha and 100m resolution. We have developed LC mapping and change detection solutions that can improve the spatial resolution of the product by 10-20x. The temporal updates of the current product are every 6 years and our solution can efficiently drive quarterly updates (quarterly heat maps of change, up to monthly land cover maps).
RapidAI4EO enables more accurate measurements from space in support of several of the SDGs thanks to the much higher temporal cadence and spatial resolution. It can provide change detection maps for the entire European continent which has an enormous potential for various sectors, enabling continuous environmental monitoring, monitoring of urban expansion, early alerts for deforestation, ploughing of protected permanent grasslands, and other abrupt or gradual environmental changes. Having an automated mapping approach to map different types of urban tissue or forest types, can be a game changer and help countries to automatically update their cadaster or better determine and monitor the amount of woody biomass and assess their carbon stock. Delivering continuous observation and mapping capabilities has an enormous potential for further scientific discoveries and to understand, anticipate and address the potential consequences of human activities on the planet and its climate. There is already evidence that high cadence, high resolution EO measurements have led to the discovery of previously unknown phenomenology.

RapidAI4EO LULC Mapping

RapidAI4EO Unsupervised Change Detection workflow

RapidAI4EO Corpus - Locations

RapidAI4EO Corpus - Imagery

RapidAI4EO Supervised Change Detection workflow