Periodic Reporting for period 2 - GAIA-CLIM (Gap Analysis for Integrated Atmospheric ECV CLImate Monitoring) Reporting period: 2016-09-01 to 2018-02-28 Summary of the context and overall objectives of the project The GAIA-CLIM project aimed to establish sound methods for the characterisation of satellite-based Earth Observation (EO) data by surface-based and sub-orbital measurements (non-satellite measurements).The Copernicus program, if successful, shall enable a step-change in our ability to use and exploit environmental data to the benefit of society. A crucial component of the Copernicus framework is provision of high-quality observational datasets from satellites. These need to be calibrated and validated, requiring ancillary datasets from in-situ and other sources of sufficiently high quality and quantity to robustly characterise sensor performance and radiative-transfer modelling. The challenges to rigorous satellite-data characterisation are formidable, because without traceability in the comparator measures, there is ambiguity in any comparison. The objectives of GAIA-CLIM were to play a full role in supporting Copernicus. Firstly, by taking concrete steps to improve our capabilities to exploit current non-satellite observations to characterize satellite data. Secondly, by establishing prioritized needs for further observational capacity targeted at providing the required step-change in satellite calibration and validation. This needs development of user tools, including statistical tools and the integrating capabilities afforded by data assimilation systems, to enable users to access and work with the data, including uncertainty information. GAIA-CLIM has shown how this could be approached via its ‘Virtual Observatory’ demonstrator tool for a selection of well characterised ground-based / sub-orbital measurements and satellite co-locations. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far Work carried out included:• An assessment report on a ‘system-of-systems’ approach for assigning suitability of candidate non-satellite measurements for satellite calibration and validation, followed by its application to a broad range of candidate measurement networks and the collection of associated metadata.• Development of a 3D-discovery tool for metadata from ground-based observing networks.• Model and statistically-based studies to quantifying the impacts of geographical gaps in in-situ observational capabilities.• The construction of traceability chains and ‘product traceability and uncertainty documents’ for various non-satellite instrument types as a first step towards full instrument characterisation. • Modelling and statistically-based analysis describing generic metrology aspects of an atmospheric measurement and of data comparisons.• Libraries of climatological co-location uncertainties, containing guiding material and look-up tables.• A completed assessment of new satellite missions with respect to global NWP systems (GCOM-W AMSR-2, FY-3C, F-19 SSMIS, TVZA-GY microwave imager/sounder on the Russian Meteor-M N2 satellite, and GPM GMI).• The ‘GRUAN processor’, which converts radiosonde profiles and their uncertainty to TOA radiance equivalent measurements, and subsequent efforts to monitor reference data (from GRUAN) with respect to global NWP fields.• Report detailing approach to the calibration and validation of (atmospheric state variable) EO data, and detailing proposed approach to other ECVs and associated EO data• Development of the 'Virtual Observatory' demonstrator tool, based on user requirements derived from a user survey and the feedback of two user workshops with external users, as well as the series of ‘Roadshow’ events at target user institutions. • Production of five official releases of a Gaps Assessment and Impacts Document (GAID), including an online searchable catalogue of gaps and SMART remedies.• Formulation of eleven high-level recommendations resulting from the gap analysis.As a member of the Open Data Pilot, where possible, these activities have been developed in a way that enables sustainability, exploitation, and potential future development by project partners or third parties. Principal outcomes have been collated together and are presented at http://www.gaia-clim.eu/page/results. Substantial efforts were made to engage the user community throughout the project. Presentations were made at relevant conferences and workshops, there were two user workshops and, in the final year, a series of ‘Roadshow events’ addressed key stakeholders. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) GAIA-CLIM has made significant advances in the comprehensive geographical review of capabilities and gaps in the existing surface-based and sub-orbital observing systems for the characterization of EO measurement performance. Comprehensive discovery metadata has been collected and stored in a consistent format.GAIA-CLIM supported the integration of scientific expertise on metrology and calibration of many of the principal global high-quality measurement groups and networks, as well as EU-funded projects, substantially helping to improve cohesion and better understanding of areas of mutual concern and benefit. The in-depth analysis of six measurement techniques provides a platform for the provision of improved measurement series.Work on co-location effects explored several new and novel approaches to quantify the irreducible mismatch uncertainty, leading to several publications. The project participated in an ISSI-program activity that evaluated techniques across domains. The Look-up-tables produced provide a tool available for use by interested researchers.The ‘GRUAN processor’, hosted at NWP-SAF, permits for the first time the transfer of the geophysical profile information with uncertainties to an equivalent TOA radiation measure and corresponding uncertainty. This allows future validation of level-1b satellite products. GAIA-CLIM also produced novel characterisation, via the use of reference-quality data, of a range of microwave imager-products via the use of NWP data assimilation. Results shall lead to improvements in utilisation in both NWP and reanalyses.The Virtual Observatory facility provides a demonstrator capability to interact with co-location data between satellite and non-satellite measurements. It also extends EUMETSAT’s existing co-location tools to exploit non-satellite measurements.The GAID supports future work by identifying and assessing gaps in capabilities or knowledge against user requirements, as well as evaluating their scientific and societal impacts. The recommendations document takes this detailed information to distil a set of eleven high-level recommendations.The novel approach of GAIA-CLIM was to demonstrate traceable EO Cal/Val for a number of metrologically mature ECVs, in the domains of atmospheric state and composition that will guarantee that products are assessable and intelligible to third-party users. The project findings represent a lasting legacy to the contributing observing networks, leading to improvements in data traceability and comparability of EO measurement systems.The improvement in EO instrument characterisation undertaken supported the assessment of the accuracy of EO instruments and opened opportunities for new global standards of accuracy and rigour of climate data records. This will greatly benefit the Copernicus Climate Change Services. NWP minus GRUAN differences, in both geophysical space and TOA brightness temperatures for ATMS. Example traceability chain for GRUAN temperature profiles. Conceptual illustration. View of the 3D metadata visualisation tool. Sceen shot from the Virtual Observatory for co-locations with a ground-based station (Barrow).
