Periodic Reporting for period 2 - openEO (openEO - a common, open source interface between Earth Observation data infrastructures and front-end applications)
Okres sprawozdawczy: 2018-10-01 do 2020-11-30
Therefore, objectives of the openEO project have been to build a common, open source interface for standardising the communication between users and back-ends, and to develop interfaces for various clients and back-ends. openEO APIs were developed to standardise process chains via the programming libraries for Python, R, and JavaScript, by using web editors or via mobile applications. Furthermore, APIs have been developed to translate these standardised commands into the local syntaxes of chosen service providers (back-ends at VITO, EODC, mundialis, Sinergise, EURAC, JRC, CREODIAS, WEkEO DIAS and a proxy to Google Earth Engine). Serving as template for the connection of further back-ends, the existing open source APIs will simplify the use of cloud-based EO processing engines, allow switching between cloud-based back office providers and comparing them, and enable reproducible, open EO science. Thereby, openEO reduces the entry barriers for the adaptation of cloud computing technologies by a broad user community and paves the way for the federation of EO data infrastructure capabilities.
Other objectives have been to implement pre-defined processes for all aspects of the EO data life cycle and to develop use cases as proof-of-concept. 140+ processes have been implemented to find, access, and to process EO data. Additionally, user-defined functions can be processed on selected back-ends, using the openEO API. Five use cases have been developed to prove the usefulness of the developed processes and to validate the openEO interfaces.
Standards of the endpoints metadata and user interfaces were analysed to allow the further development of openEO benefitting all partners. For involving the opinion of external users in these processes, several ways for communicating with the consortium have been established. A Zenodo community was created to provide all scientific publications and accepted project deliverables to the users. Additionally, during the second review period the consortium actively spread information about the developed standard, participating in or hosting 26 international conferences / workshops / seminars / hackathons or summer schools. Bilateral talks were organised with pilot users / external users or service providers about the use of openEO standard or the implementation of its APIs on additional platforms. The input of these discussions has been integrated into the further development. openEO APIs were further developed for CREODIAS and WEkEO DIAS as proof of concept.
In parallel with constantly upgrading the API in various code versions, five use cases have been developed. This helped with the definition of 140+ pre-defined processes from all aspects of the EO data life cycle, which were developed throughout the project to create a unified openEO syntax. These processes were implemented by the participating back-ends, as possible. Furthermore, the principle of processing user-defined functions has been discussed and implemented on several back-ends. An openEO API v1.0 was published in July 2020, representing the final version of the openEO project, but not the final version of the API. The state of repositories were constantly published in the open source platform GitHub.
140+ pre-defined functions were developed to be used for the standardised communication. These functions cover following aspects:
- Authentication and user data management
- Query of EO data and processes available at the endpoints
- Band calculations and time series statistics
- Filtering and aggregation of specific dimensions
- UDF execution
- Zonal statistics
- Upload and use of vector data
- EO Data download
User-defined functions can be transferred via openEO to various back-ends for execution, allowing the execution of tailored workflows. Use cases have been implemented for defined pilot users to i) to validate the openEO standard and its potential to allow comparability between service providers, and to ii) be further used by at least the pilot users.
The openEO access to back-end providers represents a value adding services of EO data storage and processing services. It may empower both public and commercial users to use a standardised interface to realise EO data processing without having to deploy storage or processing facilities. This will enable them to reach out to a larger customer base. Users will be able to directly access back-end services and integrate them into their regular Python, R, or JavaScript workflows. The standardisation of a now highly fragmented market of often unique services to make them comparable will facilitate users to switch between endpoints.
Since openEO is developed and distributed under a permissive open source license, both industry and society will benefit from participating together to form openEO: There are no requirements or limitations on how the developed software can be used or redistributed. Permissive licenses do not control the license terms of future products, which build on openEO. Distributing openEO explicitly under the Apache license 2.0 will simultaneously guarantee the dissemination of the open source code without any changes in the original version.