Big data set the basis of developing EO foundation models. During the first phase of ThinkingEarth, we introduced the following datasets and benchmarks supporting our foundation models:
-SSL4EO-S12-ML, which is a global multi-label land-cover land-use classification dataset combining multispectral and Synthetic Aperture Radar (SAR) imagery with open land-cover products to create scene-level multi-label annotations;
-SSL4EO-S, which integrates ~15TB data from all Copernicus Sentinel missions and the Copernicus DEM GLO-30 product, offering a large-scale, multi-modal dataset to advance unified EO foundation models with a potential connection to weather and climate;
-Kuro Siwo, which is a global multi-temporal manually annotated SAR dataset for rapid flood mapping, consisting of 43 global flood events;
-FoMo-Bench, which is the first unified forest monitoring benchmark, spanning 15 datasets with diverse data modalities and tasks, with a total volume exceeding 10TB;
-GAIA, which is a large-scale vision-language dataset designed to bridge the gap between remote sensing imagery and natural language understanding.
Using innovative self-supervised and cross-modal learning techniques to improve performance and generalizability, we developed the first version of ThinkingEarth's Copernicus foundation models:
-SupCon and SoftCon leverage the SSL4EO-S12-ML dataset to guide self-supervised pretraining using multi-label land cover annotations;
-FoMo-Net is a pre-training paradigm for learning to process all of the most common modalities in the Remote Sensing domain with a single, sensor-agnostic foundation model. Its pre-training scheme contains rich multi-sensor information, both paired and unpaired, from most parts of the world;
-In our work on Remote Sensing Vision-Language models we focused on Contrastive Language-Image Pre-training (CLIP), an open-vocabulary foundation model, which achieves high accuracy across many image classification tasks and is often competitive with a fully supervised baseline without being explicitly trained. We introduce a novel approach to align RS imagery modalities with the visual and textual capabilities of CLIP, enhancing its zero-shot capability in EO.
Towards the creation of a graph representation of the Earth, we developed a robust framework with baseline implementations of leading models for medium-range weather forecasting, enabling optimized training workflows and efficient parallelization of data and model computations across multiple GPUs. We investigated a range of model architectures and implemented architectural enhancements to improve forecasting accuracy on ERA5 climate data. We applied causal analysis methods for capturing teleconnections. We used Direct Effect Analysis (DEA), a framework for constructing causal representations of environmental variables by isolating the direct effects of specific drivers while controlling for confounders. We demonstrated DEA's utility through a case study of ENSO’s direct effects on NDVI in Africa, highlighting its broader applicability to environmental and policy-related challenges.
In UC1 "Distributed solar energy production forecasting and demand management" we developed a Multi-Layer Cloud Motion Vector forecasting method and a Deep-Learning Emulator for radiative transfer modelling. We released the first prototypes in the form of interactive online applications for Lower Austria State and Metropolitan Athens Region.
In UC2a "Biodiversity monitoring in urban environment" we enhanced the spatial resolution of Sentinel-2 imagery using DL-based super-resolution. We developed a web-based decision support system prototype in the form of an online platform that visualizes Sentinel-2 super-resolved images and biodiversity indices, aiming to support urban planners with actionable insights on green/blue infrastructure.
In UC2b "Forest biomass monitoring" we developed a first prototype version of a cloud-based tool that enables scalable remote monitoring of remote forest carbon stocks, providing an accessible platform for stakeholders in the carbon credit industry to verify and track high-quality carbon credit at both local and regional levels.
In UC3 "Causal inference for food insecurity analysis" we compiled a multi-source dataset harmonizing climate, economic, and conflict data from the period 2025-2022 across 56 Somali districts. We estimated Average Treatment Effects using X-Learner to quantify the average impact of cash-based transfers on food insecurity.
Lastly, we prepared three spotlight applications within the first cycle of the project, targeting the convenient evaluation of the task-agnostic Copernicus foundation models using three different Sentinel modalities: flood mapping with Sentinel-1 SAR imagery, land use land cover classification with Sentinel-3 OLCI imagery, and air pollutants estimation with Sentinel-5P products about atmospheric constituents.
