In the last decades the European Space Agency (ESA) and the European Commission (EC), in partnership with other institutions and service operators such as EUMETSAT, have been developing a sustainable ecosystem of Earth Observation (EO) satellite missions, data providers, distributors, storage and processing centres, added value service providers and most importantly end users. Among these initiatives, Copernicus ecosystem has been consolidated through ground-breaking initiatives such as the launch of the Sentinel satellites constellation, providing operational satellite data, as well as the implementation of the Copernicus Services in different thematic areas, and is complemented by ESA’s own efforts in the Thematic Exploitation Platforms and other initiatives, making Europe the worldwide leader in the EO field.
In parallel to the improved data capabilities available in Europe, the downstream sector has been developing capabilities towards using EO to support decision making processes. From pipeline infrastructure monitoring to forestry management, EO-based services have been under development and are reaching operational status in many business domains. Satellite imagery can have enough resolution to monitor movements along borders, coast flooding, crop productivity, detect vessels, identify key urban features and infrastructures or monitor the evolution of construction works for licensing purposes.
The objective of the BETTER project was to tap into all these resources, capabilities and know-how established through past initiatives in the Space and ICT sector to develop the intermediate data handling layers (i.e. Data Pipelines), based on Big Data tools and technologies. In this way, the intermediate layers leveraged existing European EO-based services and infrastructures to: a) integrate Copernicus EO imagery and products with other high value datasets such as citizen observatories, mobile communication records, news and social media data; and b) maximise information and knowledge content within those integrated heterogeneous datasets to provide added-value EO-based products to a wide variety of users with diverse needs.
The service ecosystem constructed by the BETTER partners provides the ideal setup for the main objective of the BETTER project: the development of a Data Pipeline service layer. Data Pipelines are simply sequential components responsible for data access, integration, transformation, analytics and visualization. They are able to extract the required data or information from a wide array and large volumes of data and deliver it to different users, according to their exact specifications.
During the three years of the project, the project objectives were fully achieved by the development of BETTER as a Service based on: a) user requirements coming from 36 challenges launched by a wide range of promoters, from from international agencies to research institutes, universities and companies; b) development and integration in the Ellip platform of data pipelines that require the systematic processing of large heterogeneous EO based datasets; c) the implementation of on-demand applications that explored the data streams provided by the different developed pipelines; d) the definition of a robust business model, based on the experience gained during the project, to explore the developed pipelines or to develop new pipelines for future customers.
