Periodic Reporting for period 3 - CENTURION (COPERNICUS DATACUBE / AI DATACUBESERVICES FOR SOCIETY, INDUSTRY AND NEW MARKET GENERATION)
Okres sprawozdawczy: 2023-05-01 do 2024-08-31
Earth Observation (EO) satellites have the collective capacity to continuously monitor the entire globe. However, despite the availability of huge (often freely available) volumes of data, less than 5% is currently being analysed. Part of the problem is that this data often requires considerable preparation since it is obtained using numerous types of instruments operating over a range of spatial and temporal resolutions, while the offered products employ a selection of geographical projections. Such preparation is even more challenging considering the massive volumes involved. The required preprocessing and subsequent analysis may, therefore, require technical and scientific expertise not normally held by users for whom the data would be of greatest value.
The CENTURION project aimed to help remedy this by improving upon the analysis and management capabilities employed for terabyte to petabyte-scale volumes of data. This was to be achieved by combining two technologies: datacubes and artificial intelligence as a service, for scalable, flexible, and easy-to-use analytics and data fusion with ensured consistency.
Datacubes are built from large assets of homogenised Earth data leading to Analysis Ready Data (ARD). The employed datacube technology is rasdaman, which includes features such as a standards-based datacube query language, a highly optimized “green computing” engine, the integration of arbitrary external code, and location-transparent federation. The AI processing is provided with functional modular applications, collectively termed AI Knowledge Packs (AI KPs). The AI KPs would allow those with greater technical and scientific knowledge to augment and expand upon them by including their own analysis methods, leading to a “library” of pre-packaged exploitable modular AI applications, while still being accessible to non-experts.
The expected benefits would include a set of readily exploitable AI KPs covering a wide spectrum of real-life use cases. These use cases involved ARD generation, agricultural weather indices, the characterization of maritime shipping dynamics, near-real-time monitoring of forest dynamics, and near-real time flood zone mapping. The platform would be exploitable by current EOD users, while attracting new users who may not have previously employed such data because of a perceived or actual lack of expertise.
The CENTURION project aimed to help remedy this by improving upon the analysis and management capabilities employed for terabyte to petabyte-scale volumes of data. This was to be achieved by combining two technologies: datacubes and artificial intelligence as a service, for scalable, flexible, and easy-to-use analytics and data fusion with ensured consistency.
Datacubes are built from large assets of homogenised Earth data leading to Analysis Ready Data (ARD). The employed datacube technology is rasdaman, which includes features such as a standards-based datacube query language, a highly optimized “green computing” engine, the integration of arbitrary external code, and location-transparent federation. The AI processing is provided with functional modular applications, collectively termed AI Knowledge Packs (AI KPs). The AI KPs would allow those with greater technical and scientific knowledge to augment and expand upon them by including their own analysis methods, leading to a “library” of pre-packaged exploitable modular AI applications, while still being accessible to non-experts.
The expected benefits would include a set of readily exploitable AI KPs covering a wide spectrum of real-life use cases. These use cases involved ARD generation, agricultural weather indices, the characterization of maritime shipping dynamics, near-real-time monitoring of forest dynamics, and near-real time flood zone mapping. The platform would be exploitable by current EOD users, while attracting new users who may not have previously employed such data because of a perceived or actual lack of expertise.
The CENTURION high-level architecture and solution/project roadmap (outlining the vision and describing the solutions to be built and the time-phased delivery of the required technical requirements/product features) were developed. This led to the CENTURION platform's initial components being designed, developed, and assembled into a prototype demonstrator by the project partners OPT/NET BV and rasdaman GmbH. The resulting demonstrator allowed the production and storage of Sentinel-2 datacubes in the rasdaman datacube system, and their retrieval with significant savings in time and energy due to a reduction in the required computing resources. In addition, rasdaman’s datacube deployment was federated with the CreoDIAS datacube service (managed by project partner CloudFerro), yielding petabytes of Sentinel, Copernicus Land Monitoring Service, Copernicus Climate Change Service, and other datacubes readily available to the CENTURION service.
Another outcome was a new visualisation and EO image sampling scheme developed by project partner Universitatea Politehnica din Bucuresti. It used convolutional neural networks for up-sampling low-resolution spectral bands from Sentinel-2 up to 10-m spatial resolution, tested across a variety of environments. Such visualisation allows actions like data analytics, semantic class separation, detecting patterns and anomalies, and the design of improved prediction models.
Other forms of information exploitable by the envisaged CENTURION platform include the acquisition and analysis of data arising from maritime Automatic Identification Signals (AIS) and social media newsfeeds, the latter used within the context of humanitarian crises in Europe caused by natural disasters and by Russia’s war against Ukraine. Specialised collection systems for such data were designed and implemented by OPT/NET with more than 28 thousand newsfeed data objects collected and stored in the CENTURION platform’s prototype.
The project set out to identify the technical and commercial requirements of potential customers. The best that the CENTURION platform would have offered was as a Platform as a Service to build EO applications which would then be used by customers via the CENTURION marketplace. This would have involved monthly base and royalty fees, with the EO data exploited under Software as a Service or Insights as a Service business models. Furthermore, to make the results of the project better known and to enhance interest in Copernicus products in general, an Earth Data Helix was established by project partner CrowdHelix to cluster parties with a potential interest in the CENTURION platform.
Another outcome was a new visualisation and EO image sampling scheme developed by project partner Universitatea Politehnica din Bucuresti. It used convolutional neural networks for up-sampling low-resolution spectral bands from Sentinel-2 up to 10-m spatial resolution, tested across a variety of environments. Such visualisation allows actions like data analytics, semantic class separation, detecting patterns and anomalies, and the design of improved prediction models.
Other forms of information exploitable by the envisaged CENTURION platform include the acquisition and analysis of data arising from maritime Automatic Identification Signals (AIS) and social media newsfeeds, the latter used within the context of humanitarian crises in Europe caused by natural disasters and by Russia’s war against Ukraine. Specialised collection systems for such data were designed and implemented by OPT/NET with more than 28 thousand newsfeed data objects collected and stored in the CENTURION platform’s prototype.
The project set out to identify the technical and commercial requirements of potential customers. The best that the CENTURION platform would have offered was as a Platform as a Service to build EO applications which would then be used by customers via the CENTURION marketplace. This would have involved monthly base and royalty fees, with the EO data exploited under Software as a Service or Insights as a Service business models. Furthermore, to make the results of the project better known and to enhance interest in Copernicus products in general, an Earth Data Helix was established by project partner CrowdHelix to cluster parties with a potential interest in the CENTURION platform.
The ultimate focus of CENTURION was to maximise the value of Copernicus EOD for European society and industry while producing long-lasting benefits for the study of the earth system and human society. The CENTURION platform would have achieved this by lowering the barriers to non-technical specialists in big data to exploit EOD by the development of its more user-centric functionality.
Advances beyond the state of the art were achieved in several areas:
● Advancements in EOD datacube query functionality enhancements like 4D corridor and curtain queries, as well as infrastructure support such as datacube security, and quota and billing management.
● The integration of AI (specifically, neural networks) into standards-based datacube query languages, thereby bringing both AI and datacubes together.
● The extension of AI to operate on substantially larger data volumes than currently possible.
These advancements extended the exploitation of EOD, offering the potential for new growth opportunities, not only in scientific activities but also in commercial ventures. Specifically, the output of the CENTURION project accelerates Copernicus knowledge creation, paving the way for new business models. The results achieved have raised the TRL and become part of the partners’ products and will be enhanced further beyond the project.
The rasdaman and Constructor University team contributed to standards in INSPIRE, ISO, and OGC. This included the rasdaman platform being acknowledged by the European Commission as the (currently single) Good Practice for INSPIRE coverage services. They also contributed, as editor, to several geographical information ISO specifications (19123-1 and 19123-3), which were adopted as International Standards in 2023.
The concepts and product designs developed by OPT/NET during CENTURION have been incorporated into several commercial projects and innovative research and collaboration proposals, with a combined monetary value exceeding 1M EUR. OPT/NET was also awarded the Best EU Innovation in AI Pitch by Dealflow.eu in 2023, positioning the company on a positive trajectory in regard to the monetisation and commercialisation of the developed technology. Furthermore, an official collaboration has been announced between rasdaman and Airbus, with rasdaman also winning two German mapping agencies as customers. In addition, it has filed a patent on federated datacube technology.
Advances beyond the state of the art were achieved in several areas:
● Advancements in EOD datacube query functionality enhancements like 4D corridor and curtain queries, as well as infrastructure support such as datacube security, and quota and billing management.
● The integration of AI (specifically, neural networks) into standards-based datacube query languages, thereby bringing both AI and datacubes together.
● The extension of AI to operate on substantially larger data volumes than currently possible.
These advancements extended the exploitation of EOD, offering the potential for new growth opportunities, not only in scientific activities but also in commercial ventures. Specifically, the output of the CENTURION project accelerates Copernicus knowledge creation, paving the way for new business models. The results achieved have raised the TRL and become part of the partners’ products and will be enhanced further beyond the project.
The rasdaman and Constructor University team contributed to standards in INSPIRE, ISO, and OGC. This included the rasdaman platform being acknowledged by the European Commission as the (currently single) Good Practice for INSPIRE coverage services. They also contributed, as editor, to several geographical information ISO specifications (19123-1 and 19123-3), which were adopted as International Standards in 2023.
The concepts and product designs developed by OPT/NET during CENTURION have been incorporated into several commercial projects and innovative research and collaboration proposals, with a combined monetary value exceeding 1M EUR. OPT/NET was also awarded the Best EU Innovation in AI Pitch by Dealflow.eu in 2023, positioning the company on a positive trajectory in regard to the monetisation and commercialisation of the developed technology. Furthermore, an official collaboration has been announced between rasdaman and Airbus, with rasdaman also winning two German mapping agencies as customers. In addition, it has filed a patent on federated datacube technology.