Periodic Reporting for period 1 - Copernicus App Lab (Stimulating wider uptake of Copernicus Services by making them available as linked open data)
Okres sprawozdawczy: 2016-11-01 do 2019-03-31
The H2020 project “Copernicus App Lab” provides a data streaming solution that bridges the gap between the scientific Earth observation (EO) community and the mobile application developers. The proof of concept exploits data from the Copernicus Land, Marine Environment, and Atmosphere Monitoring Services, enriched as linked open data to promote their inclusion into value-added downstream services and/or mobile applications.
To date, Copernicus services and data have not been officially released in formats or via tools popular among mobile app developers, nor have they been offered as linked open data. Offering easy-to-use tools to the community for linking their data to Copernicus data will make Copernicus products an increasingly essential node in the web of data, which means it will get more exposure and increased usage in more diverse societal sectors.
The main intention of the Copernicus App Lab was to foster new business opportunities by providing EO data in a format that makes the data searchable and easily accessible to new potential downstream service providers.
Considering current developments in the Copernicus ecosystem, the Copernicus App Lab project results are foreseen to become part of one or more Copernicus Data and Information Access Service (DIAS) in the future, through which all Copernicus data are now being made available. Corresponding analysis and benchmarking activities have been conducted throughout the second project period across three different DIAS with regards to cloud bursting APIs as well as data access APIs.
To date, Copernicus services and data have not been officially released in formats or via tools popular among mobile app developers, nor have they been offered as linked open data. Offering easy-to-use tools to the community for linking their data to Copernicus data will make Copernicus products an increasingly essential node in the web of data, which means it will get more exposure and increased usage in more diverse societal sectors.
The main intention of the Copernicus App Lab was to foster new business opportunities by providing EO data in a format that makes the data searchable and easily accessible to new potential downstream service providers.
Considering current developments in the Copernicus ecosystem, the Copernicus App Lab project results are foreseen to become part of one or more Copernicus Data and Information Access Service (DIAS) in the future, through which all Copernicus data are now being made available. Corresponding analysis and benchmarking activities have been conducted throughout the second project period across three different DIAS with regards to cloud bursting APIs as well as data access APIs.
A summary of the work performed and results achieved per work package is given below:
WP1: Project and innovation management
- Set-up and coordination of a project advisory board
- Establishment of an innovation management roadmap for exploitation of the project’s results, including the continuous dialogue with the Entrusted Entities and various Copernicus Data and Information Access Service (DIAS) providers.
WP2: Platform services and operations
- Establishment of an App Lab cloud architecture supporting operational deployment scenarios of the docker-based AppLab back-end components, to be operated on one or more DIAS.
- DAP deployment for CGLS data products also tested and benchmarked on DIAS providers, to prepare the App Lab consortium with the management of failover and Content Delivery Network optimizations. Measured performance is on-par compared to commercial Cloud infrastructure providers.
- Cost-benefit analysis on DAP implementation for CGLS as well as user uptake statistics showing popularity of CGLS products, and of the Leaf Area Index in particular.
WP3: Tools for enhancing, cataloguing, and publishing geospatial data-streams from the Copernicus Services
- +40 Copernicus services datasets have been semantically-enriched, published on the Streaming Data Library (SDL) and made available as linked data.
- Published online facilities to explore and discover datasets
- Published the App Lab Analytics SDK providing downstream service providers with a broad range of analytical functions in creating added-value products (e.g. enabling on-the-fly spatial and temporal aggregations or the execution of advanced analytical functions, such as anomaly detection, detect (the strength of) correlations between raw Copernicus data, etc.).
- Created several use cases, illustrating the use of raw Copernicus data as well as (derived) added-value products for societal relevant benefit areas (e.g. city marketing, ballast water management, etc.).
- Published VISual and RESTful Maps-APIs for popular Mobile App Developer-frameworks (e.g. Android, iOS, JavaScript, Node.js) to obtain (derived) datasets. Also, the linked data tools of WP4 have been integrated such that developers can seamlessly consume subsets of relevant Copernicus data as LOD.
WP4: Tools for linked EO data
- Constructed ontologies to convert Copernicus data in the RDF model
- Converted Copernicus Services data into linked EO data and interlinking them with other open geospatial data
- Created (GeoSPARQL) endpoints so that EO data can be accessed as linked data using Web standards. These endpoints also contain other relevant linked geospatial data (e.g. administrative boundaries, OpenStreetMap data) so that users can pose enriched queries against the combination of these datasets
- Search engines like Google have been enabled to treat datasets produced by Copernicus as “entities” in their own right and store knowledge about them in their internal knowledge graph.
WP5: Dissemination and user uptake
- Created and implemented a communication plan (incl. CI, project logo, website, press releases, flyers, mailings, social media etc.)
- Organised beta-testing activities such as user engagement during ESA Space App Camp 2017 and 2018, ActInSpace Hackathon 2018, Phi-Week 2018 and other workshops and paper presentations. In addition feedback was gathered via an online beta testing phase and through the engagement of LOD experts.
- Validated the feedback received, leading to further evolvements and improvements of the platform.
- Extensive outreach and dissemination: 30 conference presentations and workshops, 9 scientific papers published and almost 40 media and other online publications.
WP1: Project and innovation management
- Set-up and coordination of a project advisory board
- Establishment of an innovation management roadmap for exploitation of the project’s results, including the continuous dialogue with the Entrusted Entities and various Copernicus Data and Information Access Service (DIAS) providers.
WP2: Platform services and operations
- Establishment of an App Lab cloud architecture supporting operational deployment scenarios of the docker-based AppLab back-end components, to be operated on one or more DIAS.
- DAP deployment for CGLS data products also tested and benchmarked on DIAS providers, to prepare the App Lab consortium with the management of failover and Content Delivery Network optimizations. Measured performance is on-par compared to commercial Cloud infrastructure providers.
- Cost-benefit analysis on DAP implementation for CGLS as well as user uptake statistics showing popularity of CGLS products, and of the Leaf Area Index in particular.
WP3: Tools for enhancing, cataloguing, and publishing geospatial data-streams from the Copernicus Services
- +40 Copernicus services datasets have been semantically-enriched, published on the Streaming Data Library (SDL) and made available as linked data.
- Published online facilities to explore and discover datasets
- Published the App Lab Analytics SDK providing downstream service providers with a broad range of analytical functions in creating added-value products (e.g. enabling on-the-fly spatial and temporal aggregations or the execution of advanced analytical functions, such as anomaly detection, detect (the strength of) correlations between raw Copernicus data, etc.).
- Created several use cases, illustrating the use of raw Copernicus data as well as (derived) added-value products for societal relevant benefit areas (e.g. city marketing, ballast water management, etc.).
- Published VISual and RESTful Maps-APIs for popular Mobile App Developer-frameworks (e.g. Android, iOS, JavaScript, Node.js) to obtain (derived) datasets. Also, the linked data tools of WP4 have been integrated such that developers can seamlessly consume subsets of relevant Copernicus data as LOD.
WP4: Tools for linked EO data
- Constructed ontologies to convert Copernicus data in the RDF model
- Converted Copernicus Services data into linked EO data and interlinking them with other open geospatial data
- Created (GeoSPARQL) endpoints so that EO data can be accessed as linked data using Web standards. These endpoints also contain other relevant linked geospatial data (e.g. administrative boundaries, OpenStreetMap data) so that users can pose enriched queries against the combination of these datasets
- Search engines like Google have been enabled to treat datasets produced by Copernicus as “entities” in their own right and store knowledge about them in their internal knowledge graph.
WP5: Dissemination and user uptake
- Created and implemented a communication plan (incl. CI, project logo, website, press releases, flyers, mailings, social media etc.)
- Organised beta-testing activities such as user engagement during ESA Space App Camp 2017 and 2018, ActInSpace Hackathon 2018, Phi-Week 2018 and other workshops and paper presentations. In addition feedback was gathered via an online beta testing phase and through the engagement of LOD experts.
- Validated the feedback received, leading to further evolvements and improvements of the platform.
- Extensive outreach and dissemination: 30 conference presentations and workshops, 9 scientific papers published and almost 40 media and other online publications.
The Copernicus App Lab has established a proof-of-concept for providing the data from the Copernicus Land, Marine Environment Monitoring and the Atmosphere Monitoring Service as LOD for the take-up of Copernicus data in mobile applications including the right tools for potential future linkage by the Copernicus services or developers directly.
A brief outline is given below of the tools and technologies beyond the state-of-the-art offered by the Copernicus App Lab:
- Provision of streaming access to Copernicus data to a single-point-of-access Streaming Data Library, where developers can access data from different Copernicus services incl. a demonstrator with access to the complete data archive from PROBA-V and the Copernicus Global Land Service
- Provision of a maps API supporting multiple platforms to enable mobile developers to directly connect the information gained from Copernicus data to their downstream applications
- Provision of Copernicus data and services as linked open data along with tools for publishing, interlinking, querying and visualising Copernicus data
Quotes from users:
„Great simple service that makes it easy to fetch data“
“Interlinking datasets is easy in comparison to other technologies”
“The volume and the variety of EO data that can be interlinked and collectively processed through the Copernicus App Lab Linked Data Tools is unique.”
“I liked the analytics part quite a bit. Customisable spatial averaging, anomaly detection, and especially correlation functions are most useful for EO-based disaster management, either for hazard estimation or early warning.”
“I was thrilled by the analytics component of the tool. The expert mode is intuitive and very convenient to use for some analytics. These are tools that do not exist in COTS s/w such as ArcGIS, ERDAS IMAGINE or ENVI. One would have to write several Python scripts to make most that the App Lab analysis performs with a few clicks.”
A brief outline is given below of the tools and technologies beyond the state-of-the-art offered by the Copernicus App Lab:
- Provision of streaming access to Copernicus data to a single-point-of-access Streaming Data Library, where developers can access data from different Copernicus services incl. a demonstrator with access to the complete data archive from PROBA-V and the Copernicus Global Land Service
- Provision of a maps API supporting multiple platforms to enable mobile developers to directly connect the information gained from Copernicus data to their downstream applications
- Provision of Copernicus data and services as linked open data along with tools for publishing, interlinking, querying and visualising Copernicus data
Quotes from users:
„Great simple service that makes it easy to fetch data“
“Interlinking datasets is easy in comparison to other technologies”
“The volume and the variety of EO data that can be interlinked and collectively processed through the Copernicus App Lab Linked Data Tools is unique.”
“I liked the analytics part quite a bit. Customisable spatial averaging, anomaly detection, and especially correlation functions are most useful for EO-based disaster management, either for hazard estimation or early warning.”
“I was thrilled by the analytics component of the tool. The expert mode is intuitive and very convenient to use for some analytics. These are tools that do not exist in COTS s/w such as ArcGIS, ERDAS IMAGINE or ENVI. One would have to write several Python scripts to make most that the App Lab analysis performs with a few clicks.”