Final Report Summary - IGAS (IAGOS for the GMES Atmospheric Service)
Executive Summary:
This project aimed to facilitate the use of the data stream provided by the European Research Infrastructure IAGOS (In-service Aircraft for a Global Observing System). IAGOS is establishing a distributed infrastructure for long-term observations of atmospheric composition on a global scale from an initial fleet of 10-20 long-range in-service aircraft of internationally operating airlines, providing accurate in-situ observations of greenhouse gases (GHGs), reactive gases, aerosols, and cloud particles at high spatial resolution.
IGAS served as a bridge between this data stream and the Copernicus Atmosphere Monitoring Service (CAMS), previously called the GMES Atmosphere Service, as in the acronym IGAS: IAGOS for the GMES Atmosphere Service. This work included implementing near-real-time data transfer of profile measurements upon landing, and setting the stage for the real-time transfer of measurements from the air via the E-AMDAR network. Work was undertaken to ensure that the emerging greenhouse gas (GHG) and aerosol measurements could be immediately integrated in the CAMS monitoring/forecasting system.
To facilitate the exploitation of these data by the broader scientific community, the existing IAGOS database was improved, enhancing the search and retrieval capabilities, and linking the database with the HALO database hosted by DLR and the JOIN interface to the CAMS database in Jülich. Furthermore, tools were developed to provide value-added products such as the assessment of the representation error for measurements made near airports. Flexible programs were developed to enable satellite validation based on the measurements of several different species, including collocation, profile completion, and the application of averaging kernels. This work is particularly relevant given the upcoming launch of Sentinel-5P.
As the IAGOS data stream is intended for long-term climate monitoring, able to detect trends over decadal scales, it is critical that the quality of the measurements is maintained and fully documented. To that end, IGAS established procedures for regular evaluation of the measurements, following the guidelines of the World Meteorological Organization (WMO) and drawing upon the guidance of external experts. IGAS also introduced automated comparison of IAGOS measurements with similar collocated measurements to provide rapid feedback on the quality of the data.
Finally, IGAS also fostered instrument development, targeting specific quantities that were requested by CAMS and its predecessors. These newly-developed instruments will be deployed in the future within IAGOS. Thus measurements of the IAGOS fleet remain at the cutting edge of atmospheric science, filling a defined need in the operational monitoring framework. This ensures that the IAGOS measurements will be taken up by the operational community, allowing maximal exploitation of these data.
Through these measures, IGAS has brought the measurements of IAGOS to a broader community, making the data more accessible to both operational users and the scientific community at large. At the same time, the IGAS has ensured the quality, consistency, and relevance of IAGOS measurements into the future.
Project Context and Objectives:
Faced with the specter of climate change, monitoring of atmospheric composition grows increasingly important. Not only in terms of tracking the concentration and distribution of greenhouse gases, the main drivers of global warming, but also in order to monitor atmospheric chemistry under a variable climate. Through the European Research Infrastructure IAGOS (In-service Aircraft for a Global Observing System), this is being done via the deployment of instrumentation on long-range civil aircraft operated by cooperating airlines. These measurements provide both profiles (during takeoff and landing) and long transects through the climatically relevant tropopause region. The species measured within IAGOS cover the Essential Climate Variables related to atmospheric composition, including GHGs, reactive gases, aerosols, and cloud particles, all measured continuously and autonomously along the flight track.
The overarching objective of IGAS was to improve the accessibility and exploitation of IAGOS measurements, facilitating their use in the scientific community. The Copernicus Atmosphere Monitoring Service (CAMS) in particular was targeted as an operational end user.
This work was divided into four main goals within the project:
1) Optimally linking IAGOS products to CAMS and satellite validation (WP2)
2) Implementing the near-real-time data stream (WP3)
3) Evaluating and harmonizing data quality in routine aircraft observations (WP4)
4) Enhancing the IAGOS measurement capabilities to detect key species that are important for CAMS (WP5)
Each of these goals was realized through the tasks of a given work package, as outlined below.
WP2: Enhancing the IAGOS-CAMS link, by
• Making IAGOS airborne in-situ data quickly and easily available for direct use within CAMS
• Developing the IAGOS database into a system with full semantic interoperability, leading to enhanced search and retrieval capabilities, and inclusion of the complete IAGOS-CORE and a major part of the IAGOS-CARIBIC data
• Linking the three different IGAS data services (the IAGOS database in Toulouse, the HALO database at DLR, and the JOIN interface to the MACC database in Jülich) in an interoperable manner via a web portal (data portal at iagos.sedoo.fr)
• Enhancing the use of airborne in-situ data for satellite validation activities by provision of standardised completed atmospheric profiles, co-location tools, and representativeness assessment of IAGOS data
• Developing the use of IAGOS GHG and aerosol data in the CAMS monitoring/forecasting system
WP3: Implementation of operational data transfer via
• Completing the certification of and implementing near-real-time (NRT) data transfer from IAGOS aircraft via SATCOM (SATellite COMmunication) and E-AMDAR (EUMETNET (The network of European Meteorological Services) Aircraft Meteorological Data Relay)
• Providing IAGOS data for direct use within the NRT operation of CAMS by distributing the IAGOS NRT reports over the real-time network of the WMO Information System (WIS)
WP4: Evaluation and harmonization of data quality in routine aircraft observations, by
• Establishing procedures for regular evaluation and documentation of the quality of the IAGOS measurements, following WMO (World Meteorological Organization) guidelines
• Comparing IAGOS data within the IAGOS aircraft fleet and with other existing observing systems
WP5: Enhancing the IAGOS observational capabilities, namely through
• Improving IAGOS-CORE aerosol and cloud detection capabilities by advancing instruments measuring total water, cloud droplets, ice crystals, and particle extinction as explicitly requested by GISC (GMES in-situ Coordination)
• Enhancing IAGOS-CARIBIC detection capabilities for VOCs as requested by MACC-II
• Assessing future IAGOS-CORE instrument configurations with increased flexibility
The work on each of these goals was necessary to improve the accessibility and use of the IAGOS data. The database development ensures that the data can be more easily found by users, even if they are searching from a different data portal. The tools for satellite validation open up a range of applications, here demonstrated for four species and sensors. The preparatory work to ingest the new IAGOS data streams into CAMS ensures operational uptake of these measurements.
Critical to this operational application is the automated transfer of the data in near-real-time, with CAMS targeting access within three hours of the measurement being made. The IGAS efforts to implement near-real-time data transfer are twofold: the transfer upon landing via the GSM network results in half the profiles being transmitted within this time window, and most within six hours. The use of the RTTU would ensure that all the profile measurements from IAGOS would be available to operational users in near-real-time.
The work to harmonize and evaluate the data quality within IAGOS is key for longer-term applications in climate monitoring. In order to detect small trends over time, a consistent level of data quality is required, and a full description of any changes in the standard operating procedure or calibration of the measurements is critical for their interpretation. Being able to trace the measurements to internationally recognized standards is key for the appropriate interpretation of the data, and will encourage their uptake within the community.
Finally, the instrument development contributes to the continued relevance of the European Research Infrastructure IAGOS. The developments come in direct response to requests from operational users (CAMS and its predecessors), fulfilling a recognized need. As the instrument development is taking place within the context of IAGOS, an operational application of the technical advances is clearly available.
Project Results:
WP2: Enhancing the IAGOS-CAMS link
The first task of this goal focused on database developments to improve the accessibility of the data to both CAMS and other users. For this, the metadata of the IAGOS measurements were redefined to ensure that they were compliant with CF naming conventions. Semantic information was also added to the metadata to improve the search capabilities, and the harmonized uncertainties on each measurement, as established in WP4, were incorporated into the records. Through this effort the IAGOS-CARIBIC, historic MOZAIC, and IAGOS-CORE data were integrated into the IAGOS Data Centre in Toulouse in a common format.
To facilitate the application of IAGOS and other airborne data with the fields generated from CAMS, interoperability was established between the IAGOS Database, the HALO Database at DLR, and the JOIN interface to the CAMS database in Jülich, all of which were linked with a web portal. The DLR HALO database was extended to allow machine-to-machine interoperability with the IAGOS database and the JOIN interface. The interactive visualization across servers that has been realized in JOIN was a major step forward, with the JOIN common data model (CDM) allowing flexible access to aircraft and model data, whereby datasets with many different geometries can be accessed through a common API.
The IAGOS user interface, hosted in Toulouse, France, serves as the web portal linking the three IAGOS data centres. A shared authorization between the IAGOS data centres using API-keys was established such that users do not have to log in multiple times when using data from different sources. The IAGOS user interface was also improved through pagination, a new database manager (MongoDB) and a description page for each flight. Quicklook plots have been added for browsing data, and flight tracks are plotted both in real time and when making data selections. Details are reported in deliverables D2.2 (“Interoperable service for IAGOS-core, IAGOS-CARIBIC, campaign, and model data”) and D2.3 (“Web portal linking the different individual IAGOS data services”).
Another focus of this work package was the application of the measurements for the purposes of satellite validation. Satellite validation tools using IAGOS profile and flight track data have been developed and demonstrated for a range of satellite instruments (IASI, SCIAMACHY, OMI, GOSAT), and species (CO, O3, NO¬2, CO2, and CH4). To this end, species-specific tools were developed to collocate both profile and flight track data (for CO) with satellite measurements. For profile measurements, the measured profile was then extended to cover the whole atmosphere (including the unsampled stratosphere) using another data source, such as model fields, satellite prior profiles, or independent satellite profiles. In one case (the GHG satellite validation tool) the user is allowed to choose which data source to use for column completion from a range of options, allowing for sensitivity testing. Finally, averaging kernels were applied, and statistical comparisons were carried out between the satellite measurements and collocated IAGOS measurements. Thus tools were created to enable the use of IAGOS measurements for satellite validation across species and platforms. These tools are described in detail in the deliverable D2.5 report.
Another tool developed dealt with the representation error of the carbon monoxide profile measurements in close proximity to airports. This is intended to aid in the interpretation of the measurements and comparison with modeled data at different spatial resolutions. The output from this tool is stored along with the profile measurements on the IAGOS database. For details, the reader is referred to the deliverable D2.6 report.
The final component of the task to enhance the IAGOS-CAMS link focused on the implementation of the forthcoming IAGOS-GHG data stream in the CAMS monitoring/forecasting system. Here an observation operator was developed in order to ingest GHG profiles in near-real time, and implemented in the CAMS forecast and analysis system (see deliverable reports D2.7 and D2.8).
Due to delays in the provision of the IAGOS-GHG data, the operator was tested on profile data provided by the Japanese CONTRAIL program (Comprehensive Observation Network for TRace gases by AIrLiner). The application of this observation operator for aircraft CO2 profiles allow CAMS to monitor forecast and analysis departures in near-real-time. This allows for the routine production of CO2 profile monitoring statistics and diagnostics within the CAMS system.
WP3: Implementation of operational data transfer
This Work Package was dedicated to the implementation of operational data transfer, such that the IAGOS measurements are available in near-real-time for CAMS and other operational users.
As a first step, data transfer was established over the mobile phone network upon landing (fully described in deliverable report D3.1). IAGOS Package One (P1) near-real-time (NRT) data profiles (atmospheric ozone mixing ratio and carbon monoxide) and coordinates are collected on-board in real time and transmitted to the IAGOS Data Centre once the aircraft has arrived at its parking stand, using a GPRS (General Packet Radio System). These profiles are then made available in BUFR format about 6 hours after the observation time on average (less for landing profiles and more for takeoff).
The profiles are averaged on a 150 m vertical resolution from ground level to an altitude of about 12 km (cruise altitude) during the ascent and descent profiles. This vertical resolution was chosen as it is still higher than that of most global models. These data profiles, unvalidated and uncalibrated, are automatically transferred to the IAGOS data server at CNRS-LA (eole.aero.obs- mip.fr). From there they are downloaded by the CAMS as soon as a new file becomes available. The CAMS system checks this server every 10 minutes and receives the new files through a secure shell data exchange.
In order to ensure that all the IAGOS profile measurements (ascent and descent) are available for CAMS within the targeted 3-hour near-real-time window, work was undertaken to implement real-time data transmission through the air via the RTTU (real-time transmission unit). Software was developed and installed into Package 1 (P1) to reduce the vertical resolution of the profile to 40 representative levels, in order to keep data transmission costs in check (see detailed description in the deliverable D3.2 report). The data are then transferred from P1 to the RTTU, which transmits them via satellite to the ground-AMDAR centre. From there they are inserted as BUFR data into the WMO Information System (WIS). This would ensure that profile measurements are transferred within 30 minutes of collection for takeoff and within 3 minutes of collection for landing.
The IGAS project saw the development, testing, and implementation of the P1 software for real-time applications. Furthermore, both the software and the hardware (the first RTTU) were certified for installation on board a participating aircraft. Unfortunately due to scheduling difficulties with the maintenance schedule of the aircraft, the installation of the RTTU had to be postponed until after the end of the project. However, partner CNRM confirmed its commitment to install the RTTU after the end of the IGAS project, if necessary at its own expenses.
WP4: Evaluation and harmonisation of data quality in routine aircraft observations
In this work package, a conceptual model for the QA/QC (quality assurance/quality control) of IAGOS data was established. The concept is described in deliverable D4.1 report, and its implementation is reported in D4.2. For a long-term data record, this work is seen as critical in order to ensure that any trends seen over time are not the result in changes in measurement procedures or calibrations.
The first task in this concept was the preparation and evaluation of standard operating procedures (SOPs) and factsheet for each IAGOS instrument. Furthermore, guidelines were developed for the harmonized storage of the measurements in the IAGOS database including their uncertainties and a data flagging index describing the status of the measurements.
The implementation of this QA/QC concept and its structure were tested and evaluated by external experts at workshops held at the end of the second and third years of the project, and their feedback was incorporated (D4.2). The experience obtained in this two round evaluation process demonstrated that external experts are essential in order to achieve an independent and objective assessment of the IAGOS-QA/QC concept in operation. The QA/QC documentation (including all QA/QC-protocols) has been identified as an essential part of the archived flight data, and thus has been archived as metadata at the IAGOS database.
Because the same quantity is measured by different instruments within the IAGOS fleet, one of the pillars of the QA/QC procedure is the evaluation of the data consistency internal to the IAGOS network. This can involve either two of the same instruments flying on two different aircraft or measurements made by two different kinds of instruments flying on the same aircraft.
This is done by means of an automatic application of detection of 4D coincidences within IAGOS fleet, implemented in the IAGOS central data base (details in report D4.4). Criteria of matching in time and space have been clearly defined for every compound (O3, CO, H2O, NOx, NOy, aerosols, CO2, CH4, cloud particles) based on input from the PIs of the different packages.
This procedure is currently operational for O3, CO and H2O only, as the other packages are not flying yet on board equipped aircraft. Statistical analysis for the years 2006 and 2012 shows excellent agreements between different CO instruments. The defined score is a bit lower for ozone, but the criteria are more severe and difficult to meet given that variability/reactivity of ozone is higher. Details can be found in the deliverable report D4.4. The methodology has been successfully applied to (profiles and cruise) data over the year 2013 and the results have been published in Nédélec et al., 2015 (MOZAIC-IAGOS special issue in Tellus-B).
The tool has been further developed in order to calculate and plot the statistics of all detected matching cases (i.e. coincidences in time and space). The tool has been made more more flexible such that one can specify criteria for the strictness of the match in horizontal distance, vertical distance, and difference in time. In addition, the tool is capable of finding instances where the same air mass was sampled twice even if there is a difference in time or space between the two sampling events.
In addition to these internal consistency checks, Lagrangian trajectories were used to detect measurements of common air masses from IAGOS-CORE or IAGOS-CARIBIC aircrafts and external platforms like CONTRAIL, NOAA ESRL/GMD or DLR research aircraft.
The experiences gained during the software development and first tests has been summarised in the Milestone MS45 report submitted in July 2014. It also contains a detailed description of the coincidence detection algorithm and examples for coincidences found using flights in 2006.
Testing revealed that standard IAGOS-CORE 20-day backward trajectories stored with 24 hours time-steps are not suitable for this kind of analysis, and new 7-day trajectories with hourly resolution were calculated.
The output of the coincidence detection software was discussed with the IAGOS instrument PIs. To facilitate their work, the output now includes a tabular list of all matching data points including the measured trace gas concentrations for the air mass at both ends of the Lagrangian trajectories. The results are summarized in D4.5.
In order to collect the QA/QC documentation for each IAGOS instrument during a given deployment period, a template was created jointly with the IAGOS instrument PIs. The template provides for reporting of pre- and post-deployment QA/QC as well as QA/QC information during the deployment period, but also for reporting the results of internal consistency tests (using the comparison of observations of the same quantity by different IAGOS instruments) and external consistency tests (involving instruments from outside of IAGOS). Details can be found in the deliverable report D4.3. The templates were revised based on feedback from both instrument PIs (from both IAGOS-CORE and -CARIBIC) and the external reviewers.
WP5: Enhancing the IAGOS observational capabilities
The tasks of this work package operated essentially independently of one another, targeting specific instrumental development to improve the capabilities and flexibility of the IAGOS fleet.
Task 5.1 Investigation of possible change in configuration of IAGOS setup in order to allow more flexibility in the installation of different combinations of instruments in the future
The suite of instruments operated on IAGOS-CORE aircraft is based on a modular design. Package 1 (P1) operates on all IAGOS-CORE aircraft, and measures ozone, water vapour, carbon monoxide, and cloud particle number concentration. The concept of the second unit (Package 2, or P2) was introduced during the transition from MOZAIC to IAGOS, and includes compact and highly specialized instruments for measuring specific species and properties such as nitrogen-containing compounds (P2a: NOy, P2b: NOx), aerosol particle properties (P2c) or greenhouse gases (P2d: CO2, CH4, CO, water vapour). In the final configuration of the infrastructure, one option of this second unit (Package 2, option a-d) will be installed on each aircraft.
This task considered the possible combinations of different instruments within the existing IAGOS-CORE installation. The requirements for a stand- alone data acquisition system (S-DAS), the minimum requirements of the different instruments regarding the need for gases and/or liquids, and the certification needs for changing the scientific instrumentation in the housing of P1 have been assessed.
It was found that key to any re-configuration is the development of a stand-alone data acquisition system (S-DAS). The required components of an S-DAS have been determined and in total four different options for the installation of an S-DAS have been identified. The advantages and disadvantages of each option, including certification issues, time planning and costs, were identified. Based on this assessment, reconfiguration of the IAGOS-CORE rack is not planned by IAGOS-AISBL in the near future, not least because of the costs involved. Details are reported in D5.5.
Task 5.2: Development and construction of PTR-MS
A new and improved Proton Transfer Reactor – Mass Spectrometer (named PTR-MS-2) for the detection of volatile organic compounds (VOCs) was developed to replace the presently installed PTR-MS-1 (in use since 2005). PTR-MS-2 is has been adapted from a prototype instrument that was developed (until 2015) for use on board research aircraft, with the aim of operating an identical instrument for IAGOS-CARIBIC. Owing to the reconstruction of the vacuum recipient made of nickel coated aluminium alloy and the newly-built control unit, the weight of PTR-MS-2 is 50 kg (without rack), which is much lighter than PTR-MS-1 (~100 kg). The new instrument control in PTR-MS-2 is based on a compact V25 control computer (MPI-C Mainz) and several control boards. One great advantage of PTR-MS-2 lies in its robustness and ease of operation. For instance broken boards can easily be exchanged on the fly which is an important issue when performing regular flight series each month with strict deadlines. This allows for all housekeeping data to be logged and reviewed after flights to locate technical problems.
Furthermore, the design of the inlet system of PTR-MS-2 was improved and offers a much better regulation of the drift tube pressure (PDT) compared to PTR-MS-1, i.e. 2.3(±0.02%) hPa versus 2.3±(0.3%) hPa. As the instrumental sensitivity depends quadratically on PDT, the PTR-MS-2 offers a much better precision. Besides, lab measurements point to the fact that part of the chemical background in PTR-MS-1 is caused by a Viton®-sealed proportional valve for pressure control, which (owing to the redesign of the inlet system) is not used in the PTR-MS-2. Therefore we expect a considerably lower chemical background in PTR-MS-2.
PTR-MS-2 will also expand the list of detectable VOCs. While PTR-MS-1 is only able to measure tropospheric mixing ratios of acetone, acetonitrile and (to a minor extent) methanol, PTR-MS-2 can in addition measure benzene (limit of detection (LOD): ~20 pptV), methanol (LOD: ~500 pptV) and formaldehyde (LOD: ~4 ppbV) as demonstrated in D5.6.
Recently PTR-MS-1 became available as normal lab instrument, so modified components can be tested and integrated into PTR-MS-2 gradually after successful lab tests. Among these is an improved drift tube which uses static dissipative SEMITRON® spacers as isolating elements to eliminate charging effects and a modified ion source drift region to reduce the water vapour concentration in the drift tube, improving, for instance, the detection limit of formaldehyde. In the long run, the operation of an ion funnel in the last segments of the drift tube is expected to improve ion transmission and therefore the sensitivity.
Task 5.3: Development and evaluation of a miniature cloud spectrometer with particle depolarisation detection capability able to discriminate between water, ice and other particle type suitable for future replacement of the IAGOS BCP instrument
A new improved backscatter cloud probe has been developed and tested both in the laboratory and on a number of aircraft. The new probe, (called backscatter cloud probe with depolarisation, BCPD), includes an additional channel to measure linear backscatter polarisation with the aim of providing additional information to derive better statistical representations of water-spherical and ice and ash non-spherical particles in mixed phase cloud and dust-ash environments.
The instrument was certified for flight on the FAAM Bae 146 (UK), Halo and Falcon aircraft (Germany), Cessna Citation (Univ. Wyoming) and has also been flown on a Cessna research aircraft in Japan.
A number of improvements to the optical arrangement and detector electronics provide single particle information to allow a fraction of the sample particles to be assessed using various backscatter scattering ratios of polarisation and size to assess the shape of the particles. The instrument is also able to detect particles down to significantly smaller sizes than the traditional BCP, (< 2 μm compared to 5 μm). Data acquisition on the Bae146 initially used LabView-based software but this has now been replaced by an option in the DMT PADS data acquisition software, allowing the probe to be integrated with other DMT cloud probes and data acquisition systems used on a number of different aircraft. Data analysis software has been produced allowing routine retrieval of particle size distribution data along with the core data files from the Bae146 aircraft. The instrument’s sample area has also been characterised using a new scanning droplet gun generator to improve uncertainty in sample volume calculation. The instrument has been tested in Arctic stratocumulus and convective clouds in the south of England using the BAe146 and validated against a range of research cloud spectrometers. All data have been archived at the UK CEDA and BADC web sites. Preliminary analysis of retrieved particle size distribution measurements from BCPD showed good agreement under most of the sampled mixed phase cloud conditions with the larger research instruments. A technical publication is being prepared and posters have already been presented at conferences. It was noted that the BCPD provides better overlap with the aerosol spectrometers particle size distributions under certain conditions compared with e.g. CDP. Based on a typical uncertainty analysis for spectrometers of this kind the retrieved size distributions are deemed good.
The instrument was also recently flown through a volcanic plume and water cloud downwind of the Sakurajima volcano in Japan and was able to identify the presence of ash particles within the water cloud. The UK BCPD was also deployed at a laboratory experiment at DMT to assess response to different mineral/dust articles and data are being analysed.
Laboratory experiments with BCPD and analogue instruments have continued and new algorithms based on e.g. various cluster and decision tree algorithms have been used to quantify the uncertainty and variation in single particle “type” depolarization ratio. These laboratory and airborne data have been summarized and published (Nichman et al., 2016).
This will be used as a template with which to compare the performance of the BCPD in ongoing future data analysis from different cloud environments as part of new airborne campaigns. The BCPD has been shown to successfully discriminate between water and non-spherical particles, but more work is needed to assess response within the non-spherical classes for a wider range of ice crystal habits. This information is available from the FAAM aircraft and is being analysed as part of PhD student projects. The single particle information provided by BCPD has also been shown to be useful in improving data quality control.
In summary the goal of delivering a compact BCP with depolarisation has been successfully completed and its performance shown to be superior to the current BCP and consistent with research polarization particle spectrometers. This progress is summarized in D5.7.
Task 5.4: Development and certification of a particle extinction measurement instrument based on the cavity attenuated phase shift principle
In the framework of IGAS, the commercially available instrument CAPS PMex was characterized in a laboratory setting and adapted for airborne use. The tests showed that the instrument physical principle of cavity attenuated phase shift was operative for pressure levels down to 200 hPa. There is no dependence of the instrument noise on the operational pressure.
The response of the instrument, when operating with particles at both ambient and low pressure levels is in accordance with what would be expected from Mie Theory. In summary, the tests reported less than 10% deviation between the CAPS PMex instrument response and calculated extinction coefficients over the investigated pressure range. The modified flow system operates properly at low pressure, and the limit of detection is still within the range of extinction coefficient values suitable for the targeted atmospheric layer.
The prototype of the modified CAPS PMex was then certified for operation aboard a research aircraft. The prototype instrument P2e was first deployed on board the research aircraft Polar 6 of type Basler BT-67, operated by the Alfred Wegener Institute. The flight tests were conducted in the framework of the field project BALTEX, which took place in August 2015 over the Baltic Sea. In total 20 flight hours were conducted. The Polar 6 aircraft operated out of the island of Bornholm, Denmark.
Two particular flights of BALTEX 2015 were of interest for the validation of P2e.
(1) During the transfer flight from Bremerhaven to Bornholm on 25 August 2015, clean air was sampled in the free troposphere at pressure levels down to 700 hPa. This flight sequence was used to validate the instrument’s limit of detection of detection (LOD), which was found to be sufficiently below typical aerosol extinction coefficients observed in the upper troposphere and tropopause region, which are of the order of 1 Mm-1. This LOD can be achieved when averaging signals over a period of 60s.
(2) During the transfer flight from Bornholm to Bremerhaven on 30 August 2015, vertical profiles of the free troposphere and the planetary boundary layer aerosol extinction coefficients were measured over the Meteorological Observatory of the German Weather Service at Lindenberg in parallel to lidar observations of the aerosol profiles. These profiles showed a high concentration of aerosol particles and light extinction across the free troposphere and the planetary boundary layer. Intercomparison between extinction profiles deduced from lidar and obtained directly from the IAGOS prototype P2e instrument showed excellent agreement, with deviations of less than 1 Mm-1 after correcting for ambient humidity effects on aerosol scattering.
Summarising, the prototype instrument P2e performed well during the research flights conducted during BALTEX 2015. The modifications to the original instrument’s set-up concerning flow stabilization and temperature control turned out to be successful and lowered the LOD of the instrument sufficiently to make it suitable for the planned application. Finally, optical closure between ground-based lidar and in-situ measurements of the aerosol extinction coefficient by IAGOS prototype P2e instruments was achieved. More details can be found in the D5.8 deliverable report.
Task 5.5: Development and evaluation of a reduced size instrument measuring water vapour and total water
This task aimed to further develop the Hilase-Hygro photo-acoustic instrument measuring water vapour and total water for autonomous use on board aircraft.
The targeted weight and size reduction of the prototype of the Hilase-Hygro instrument was successfully achieved by using much smaller and lighter electronics which inherited all the features of the old electronics. These new electronics provided a more user-friendly interface than the previous version.
In the past, stainless steel cells had to be used in order to ensure sufficiently fast response of the instrument to sudden concentration variations. However due to a special coating which is applied in the inner walls of the cells, it is possible now to make the cells of aluminium. In this way the cells are significantly lighter without sacrificing the fast response of the instrument.
Various self-checking algorithms, including laser wavelength locking and a simple method for sensitivity verification (either during flight or on the ground), are implemented into the instrument, which ensure its long-term reliable operation.
Flight tests were successfully completed, confirming the ability of the Hilase-Hygro instrument to simultaneously measure atmospheric water vapour and total water. Further details can be found in D5.9.
Potential Impact:
The final outcomes from the IGAS project entail:
• The provision of a reliable interface between IAGOS and the Copernicus Atmosphere Service in WP2 and the installation of a system for real time data transmission (WP3)
• The development of tools to facilitate the use of IAGOS data for satellite validation and its application in the Copernicus Atmosphere Service (WP2)
• The establishment of procedures for routine evaluation and documentation of data quality (WP4)
• The development of novel instrumentation for the next generation of operational airborne in-situ measurements (WP5)
The work performed in IGAS ensures that the unique data stream produced by IAGOS can be more fully exploited by the scientific community. The provision of the profile data in near-real-time (WP3) ensures that the profile measurements can be integrated by the Copernicus Atmosphere Monitoring Service, and potentially other operational meteorological services as well. These data are expected to contribute in particular to the reduction of bias in the assimilation of satellite data, which in turn will improve the forecasts and analysis of atmospheric composition and air quality. At the end of the project, the operational transfer of the profile data by the GSM network has been established, which ensures that the profile measurements are made available to CAMS within about six hours of their collection on average (shorter for descent profiles, longer for ascent profiles). The groundwork has been laid for the installation of the RTTU as well, which although not possible within the lifetime of IGAS, will enable the provision of all profile measurements to CAMS within the targeted time span. Improvements in these forecasts have clear socio-economic benefits, in addition to their inherent scientific value.
The IAGOS data, with the help of the column-completion and collocation tools developed in IGAS WP2, can also be used directly by the satellite community to improve retrieval algorithms and reduce biases in their products. This has been demonstrated within the IGAS project for the following instruments (species): SCIAMACHY (NO2, CH4, CO2), OMI (CO, O3), IASI (CO, O3), and GOSAT/TANSO-FTS (CO2, CH4), and these tools can potentially be extended to new sensors and species. Measurement of atmospheric composition by satellite is a growing field, and improvement to these datasets can influence a range of scientific fields. In particular, a clear application is seen for the upcoming launch of Sentinel-5P, which will be measuring CO, O3, NO2, and CH4, all of which are measured by IAGOS. Other potential options for validation include the planned Chinese CO2-monitoring mission TanSat (expected to launch later this year) and NASA’s OCO-2.
The interoperable IAGOS database services IAGOS, JOIN and HALO-DB allowing single login access to the IAGOS data is expected to increase data use by the scientific community. The newly established IAGOS web portal provides user services including search by variable, time, or region, and quicklook data plotting for all observations. The integration with JOIN will also enable online data exploration and comparison with modeled fields from CAMS.
One of the instruments developed in WP5, the CAPS-PMex measuring particle light extinction, was specifically requested by GISC in order to better constrain modelled and remote sensing aerosol optical depth (AOD) measurements with a comparable in situ measurement, and thus has clear applications and impacts. GISC further requested the development of the BCPOL (now called BCPD), which measures not only the presence or absence of clouds, but is partially able to discriminate between water, ice, and other aerosol types such as volcanic ash and mineral dust. Although commercial aircraft are not expected to fly when volcanic ash is likely, being able to determine the absence of volcanic ash is also important, and can improve the forecasting of volcanic ash plumes, which have been very disruptive to aviation, resulting in significant socio-economic repercussions. Another of the developed instruments, the PTR-MS-2 for measuring VOCs, was likewise requested by MACC-II, as there is a dearth of measurements to validate some species of importance for the understanding and modelling of atmospheric chemistry.
IAGOS has been established as a long-term infrastructure, and the work performed in IGAS to establish a framework for evaluation, harmonization, and documentation of the quality of these data is crucial to the use of this data record, both now and in the future. Thorough documentation of procedures and quality checks is critical when dealing with long-term records, especially when assessing climate trends on a decadal scale. The tropopause is expected to be particularly sensitive to changes in climate, and IAGOS is uniquely positioned to monitor any changes. It is essential that the uncertainties in the measurements are known, and that variations over time are not simply spurious results due to changes in calibration procedures, for example. The procedures for evaluation and harmonization of the data quality developed within IGAS document the quality of the observations for the data user, and are thus of long-term benefit in terms of better understanding climate change, which has broad societal implications.
Dissemination
The dissemination activities resulting from this project have largely focused on the scientific community, as these are the direct users of the work enabled by IGAS. Outreach beyond this community is rather left to IAGOS-AISBL, as this is the longer lasting infrastructure that IGAS is supporting, and which comes into contact with a broader audience through its partnership with civil aviation.
Of the scientific dissemination activities to date, 14 papers have been published in peer-reviewed journals, and further articles are in the review process. Of these, the majority are related to instrument development (WP5). There have also been over 37 presentations related to IGAS on the national, European, and international level. Participation in the major international conferences of the European Geophysical Union and the American Geophysical Union have ensured that the developments have met as wide an audience as possible. The topics of the presentations have covered all aspects of the IGAS work, including project overviews.
Exploitation/sustainability of results
Because of its position between the research infrastructure of IAGOS and the operational service CAMS, most of the developments realized within the IGAS project will find an application through one or both of these.
More concretely, the database developments of WP2 and the value-added products (such as the representativeness error) have been integrated into the new IAGOS data portal, currently in beta form at iagos.sedoo.fr to be moved to iagos.fr in September 2016. The maintenance of this data portal will continue under the auspices of IAGOS-AISBL into the future.
The tools for satellite validation developed at KNMI is hosted there, and the tool developed at UPS is hosted in Toulouse. Both link to the IAGOS Data Centre, but it is unrealistic to host the tool directly on the server due to the need to access massive amounts of satellite data. The tool for GHG satellite validation developed at MPG-BGC is currently not hosted in an operational capacity, as it is reliant on the presence of the delayed IAGOS-GHG data stream. However the development is in place, and MPG-BGC is committed to providing this tool in a publicly accessible way once the data come online.
The development of the observation operator and diagnostics for GHG and aerosol data were carried out by ECMWF, and are fully integrated in the operational system of CAMS, which will continue well beyond the length of the project.
The first version of near-real-time data transfer that was established in IGAS, via the GSM network upon landing, is fully operational, and being continued within IAGOS. The certification of the RTTU that took place within the project will enable the installation in an aircraft soon. Unfortunately this installation was not possible to realize within the lifetime of the project as planned, but contingency plans have been drawn up to ensure that the installation will take place, drawing upon alternate sources of funding. This is discussed further in the second periodic report. The software development to select representative levels and compress it into BUFR format will also be applied to the eventual installation of the RTTU, thus implementing this development in the operational network of IAGOS.
The quality assurance/quality control and data harmonization activities established in WP4 were intended to set the standard which IAGOS will follow going forward. The traceability and documentation of the measurement procedures is invaluable for the interpretation of a long-term data record. IAGOS-AISBL recognizes this, and is taking steps to overtake responsibility for the continuation of this work. The matter will be discussed at the next IAGOS-AISBL Executive Board meeting in September, 2016, during which an official decision on this issue is expected.
Each developed instrument has its own unique plan for exploitation and sustainability after the end of IGAS. For the PTR-MS-2, certification has already been attained for the installation in the IAGOS-CARIBIC container, and this improved version is planned to replace the previously operating PTR-MS going forward. It will be integrated into the IAGOS-CARIBIC container in October 2016, and first flights are foreseen in January 2017.
For the BCPD, initial tests in the laboratory and on aircraft have indicated that it outperforms the current BCP deployed within IAGOS, measuring over a broader size range and adding information about the shape (and thus speciation) of the particles through the measurement of the polarization of the backscattered light. A decision will be made by IAGOS-AISBL in 2017 whether the BCPD will be installed on the IAGOS CORE packages. This decision is partly financial, as it would require replacing the BCPs currently deployed within IAGOS (>12). The certification may also be problematic: while it may be possible to use a new “version” type certification, building off the existing certification, the fact that the instrument is slightly larger than the current BCP might require new certification for the installation in IAGOS aircraft.
The prototype instrument P2e, a modified CAPS PMex developed in Task 5.4 has been tested in the laboratory and in research aircraft, and found to be suitable for the planned application. The next steps will be to investigate the certification of the instrument for an eventual inclusion in the IAGOS-CORE fleet, in addition to further use on a campaign basis by the team in JUELICH.
The improved prototype of the Hilase-Hygro instrument, developed in Task 5.5 is now appropriate for autonomous, long-term operation. Flight and laboratory tests have shown good performance, and the instrument is considered ready for use in research aircraft. Discussions are underway between Hilase and national funding agencies in Hungary to support the installation in further commercial aircraft.
List of Websites:
Public website: http://igas-project.org
Newly-developed IAGOS data portal: to be moved to http://iagos.fr in September 2016
Coordinator contact details:
Dr. habil. Christoph Gerbig
Phone: + 49 3641 576373
fax: + 49 3641 577300
email: cgerbig@bgc-jena.mpg.de
This project aimed to facilitate the use of the data stream provided by the European Research Infrastructure IAGOS (In-service Aircraft for a Global Observing System). IAGOS is establishing a distributed infrastructure for long-term observations of atmospheric composition on a global scale from an initial fleet of 10-20 long-range in-service aircraft of internationally operating airlines, providing accurate in-situ observations of greenhouse gases (GHGs), reactive gases, aerosols, and cloud particles at high spatial resolution.
IGAS served as a bridge between this data stream and the Copernicus Atmosphere Monitoring Service (CAMS), previously called the GMES Atmosphere Service, as in the acronym IGAS: IAGOS for the GMES Atmosphere Service. This work included implementing near-real-time data transfer of profile measurements upon landing, and setting the stage for the real-time transfer of measurements from the air via the E-AMDAR network. Work was undertaken to ensure that the emerging greenhouse gas (GHG) and aerosol measurements could be immediately integrated in the CAMS monitoring/forecasting system.
To facilitate the exploitation of these data by the broader scientific community, the existing IAGOS database was improved, enhancing the search and retrieval capabilities, and linking the database with the HALO database hosted by DLR and the JOIN interface to the CAMS database in Jülich. Furthermore, tools were developed to provide value-added products such as the assessment of the representation error for measurements made near airports. Flexible programs were developed to enable satellite validation based on the measurements of several different species, including collocation, profile completion, and the application of averaging kernels. This work is particularly relevant given the upcoming launch of Sentinel-5P.
As the IAGOS data stream is intended for long-term climate monitoring, able to detect trends over decadal scales, it is critical that the quality of the measurements is maintained and fully documented. To that end, IGAS established procedures for regular evaluation of the measurements, following the guidelines of the World Meteorological Organization (WMO) and drawing upon the guidance of external experts. IGAS also introduced automated comparison of IAGOS measurements with similar collocated measurements to provide rapid feedback on the quality of the data.
Finally, IGAS also fostered instrument development, targeting specific quantities that were requested by CAMS and its predecessors. These newly-developed instruments will be deployed in the future within IAGOS. Thus measurements of the IAGOS fleet remain at the cutting edge of atmospheric science, filling a defined need in the operational monitoring framework. This ensures that the IAGOS measurements will be taken up by the operational community, allowing maximal exploitation of these data.
Through these measures, IGAS has brought the measurements of IAGOS to a broader community, making the data more accessible to both operational users and the scientific community at large. At the same time, the IGAS has ensured the quality, consistency, and relevance of IAGOS measurements into the future.
Project Context and Objectives:
Faced with the specter of climate change, monitoring of atmospheric composition grows increasingly important. Not only in terms of tracking the concentration and distribution of greenhouse gases, the main drivers of global warming, but also in order to monitor atmospheric chemistry under a variable climate. Through the European Research Infrastructure IAGOS (In-service Aircraft for a Global Observing System), this is being done via the deployment of instrumentation on long-range civil aircraft operated by cooperating airlines. These measurements provide both profiles (during takeoff and landing) and long transects through the climatically relevant tropopause region. The species measured within IAGOS cover the Essential Climate Variables related to atmospheric composition, including GHGs, reactive gases, aerosols, and cloud particles, all measured continuously and autonomously along the flight track.
The overarching objective of IGAS was to improve the accessibility and exploitation of IAGOS measurements, facilitating their use in the scientific community. The Copernicus Atmosphere Monitoring Service (CAMS) in particular was targeted as an operational end user.
This work was divided into four main goals within the project:
1) Optimally linking IAGOS products to CAMS and satellite validation (WP2)
2) Implementing the near-real-time data stream (WP3)
3) Evaluating and harmonizing data quality in routine aircraft observations (WP4)
4) Enhancing the IAGOS measurement capabilities to detect key species that are important for CAMS (WP5)
Each of these goals was realized through the tasks of a given work package, as outlined below.
WP2: Enhancing the IAGOS-CAMS link, by
• Making IAGOS airborne in-situ data quickly and easily available for direct use within CAMS
• Developing the IAGOS database into a system with full semantic interoperability, leading to enhanced search and retrieval capabilities, and inclusion of the complete IAGOS-CORE and a major part of the IAGOS-CARIBIC data
• Linking the three different IGAS data services (the IAGOS database in Toulouse, the HALO database at DLR, and the JOIN interface to the MACC database in Jülich) in an interoperable manner via a web portal (data portal at iagos.sedoo.fr)
• Enhancing the use of airborne in-situ data for satellite validation activities by provision of standardised completed atmospheric profiles, co-location tools, and representativeness assessment of IAGOS data
• Developing the use of IAGOS GHG and aerosol data in the CAMS monitoring/forecasting system
WP3: Implementation of operational data transfer via
• Completing the certification of and implementing near-real-time (NRT) data transfer from IAGOS aircraft via SATCOM (SATellite COMmunication) and E-AMDAR (EUMETNET (The network of European Meteorological Services) Aircraft Meteorological Data Relay)
• Providing IAGOS data for direct use within the NRT operation of CAMS by distributing the IAGOS NRT reports over the real-time network of the WMO Information System (WIS)
WP4: Evaluation and harmonization of data quality in routine aircraft observations, by
• Establishing procedures for regular evaluation and documentation of the quality of the IAGOS measurements, following WMO (World Meteorological Organization) guidelines
• Comparing IAGOS data within the IAGOS aircraft fleet and with other existing observing systems
WP5: Enhancing the IAGOS observational capabilities, namely through
• Improving IAGOS-CORE aerosol and cloud detection capabilities by advancing instruments measuring total water, cloud droplets, ice crystals, and particle extinction as explicitly requested by GISC (GMES in-situ Coordination)
• Enhancing IAGOS-CARIBIC detection capabilities for VOCs as requested by MACC-II
• Assessing future IAGOS-CORE instrument configurations with increased flexibility
The work on each of these goals was necessary to improve the accessibility and use of the IAGOS data. The database development ensures that the data can be more easily found by users, even if they are searching from a different data portal. The tools for satellite validation open up a range of applications, here demonstrated for four species and sensors. The preparatory work to ingest the new IAGOS data streams into CAMS ensures operational uptake of these measurements.
Critical to this operational application is the automated transfer of the data in near-real-time, with CAMS targeting access within three hours of the measurement being made. The IGAS efforts to implement near-real-time data transfer are twofold: the transfer upon landing via the GSM network results in half the profiles being transmitted within this time window, and most within six hours. The use of the RTTU would ensure that all the profile measurements from IAGOS would be available to operational users in near-real-time.
The work to harmonize and evaluate the data quality within IAGOS is key for longer-term applications in climate monitoring. In order to detect small trends over time, a consistent level of data quality is required, and a full description of any changes in the standard operating procedure or calibration of the measurements is critical for their interpretation. Being able to trace the measurements to internationally recognized standards is key for the appropriate interpretation of the data, and will encourage their uptake within the community.
Finally, the instrument development contributes to the continued relevance of the European Research Infrastructure IAGOS. The developments come in direct response to requests from operational users (CAMS and its predecessors), fulfilling a recognized need. As the instrument development is taking place within the context of IAGOS, an operational application of the technical advances is clearly available.
Project Results:
WP2: Enhancing the IAGOS-CAMS link
The first task of this goal focused on database developments to improve the accessibility of the data to both CAMS and other users. For this, the metadata of the IAGOS measurements were redefined to ensure that they were compliant with CF naming conventions. Semantic information was also added to the metadata to improve the search capabilities, and the harmonized uncertainties on each measurement, as established in WP4, were incorporated into the records. Through this effort the IAGOS-CARIBIC, historic MOZAIC, and IAGOS-CORE data were integrated into the IAGOS Data Centre in Toulouse in a common format.
To facilitate the application of IAGOS and other airborne data with the fields generated from CAMS, interoperability was established between the IAGOS Database, the HALO Database at DLR, and the JOIN interface to the CAMS database in Jülich, all of which were linked with a web portal. The DLR HALO database was extended to allow machine-to-machine interoperability with the IAGOS database and the JOIN interface. The interactive visualization across servers that has been realized in JOIN was a major step forward, with the JOIN common data model (CDM) allowing flexible access to aircraft and model data, whereby datasets with many different geometries can be accessed through a common API.
The IAGOS user interface, hosted in Toulouse, France, serves as the web portal linking the three IAGOS data centres. A shared authorization between the IAGOS data centres using API-keys was established such that users do not have to log in multiple times when using data from different sources. The IAGOS user interface was also improved through pagination, a new database manager (MongoDB) and a description page for each flight. Quicklook plots have been added for browsing data, and flight tracks are plotted both in real time and when making data selections. Details are reported in deliverables D2.2 (“Interoperable service for IAGOS-core, IAGOS-CARIBIC, campaign, and model data”) and D2.3 (“Web portal linking the different individual IAGOS data services”).
Another focus of this work package was the application of the measurements for the purposes of satellite validation. Satellite validation tools using IAGOS profile and flight track data have been developed and demonstrated for a range of satellite instruments (IASI, SCIAMACHY, OMI, GOSAT), and species (CO, O3, NO¬2, CO2, and CH4). To this end, species-specific tools were developed to collocate both profile and flight track data (for CO) with satellite measurements. For profile measurements, the measured profile was then extended to cover the whole atmosphere (including the unsampled stratosphere) using another data source, such as model fields, satellite prior profiles, or independent satellite profiles. In one case (the GHG satellite validation tool) the user is allowed to choose which data source to use for column completion from a range of options, allowing for sensitivity testing. Finally, averaging kernels were applied, and statistical comparisons were carried out between the satellite measurements and collocated IAGOS measurements. Thus tools were created to enable the use of IAGOS measurements for satellite validation across species and platforms. These tools are described in detail in the deliverable D2.5 report.
Another tool developed dealt with the representation error of the carbon monoxide profile measurements in close proximity to airports. This is intended to aid in the interpretation of the measurements and comparison with modeled data at different spatial resolutions. The output from this tool is stored along with the profile measurements on the IAGOS database. For details, the reader is referred to the deliverable D2.6 report.
The final component of the task to enhance the IAGOS-CAMS link focused on the implementation of the forthcoming IAGOS-GHG data stream in the CAMS monitoring/forecasting system. Here an observation operator was developed in order to ingest GHG profiles in near-real time, and implemented in the CAMS forecast and analysis system (see deliverable reports D2.7 and D2.8).
Due to delays in the provision of the IAGOS-GHG data, the operator was tested on profile data provided by the Japanese CONTRAIL program (Comprehensive Observation Network for TRace gases by AIrLiner). The application of this observation operator for aircraft CO2 profiles allow CAMS to monitor forecast and analysis departures in near-real-time. This allows for the routine production of CO2 profile monitoring statistics and diagnostics within the CAMS system.
WP3: Implementation of operational data transfer
This Work Package was dedicated to the implementation of operational data transfer, such that the IAGOS measurements are available in near-real-time for CAMS and other operational users.
As a first step, data transfer was established over the mobile phone network upon landing (fully described in deliverable report D3.1). IAGOS Package One (P1) near-real-time (NRT) data profiles (atmospheric ozone mixing ratio and carbon monoxide) and coordinates are collected on-board in real time and transmitted to the IAGOS Data Centre once the aircraft has arrived at its parking stand, using a GPRS (General Packet Radio System). These profiles are then made available in BUFR format about 6 hours after the observation time on average (less for landing profiles and more for takeoff).
The profiles are averaged on a 150 m vertical resolution from ground level to an altitude of about 12 km (cruise altitude) during the ascent and descent profiles. This vertical resolution was chosen as it is still higher than that of most global models. These data profiles, unvalidated and uncalibrated, are automatically transferred to the IAGOS data server at CNRS-LA (eole.aero.obs- mip.fr). From there they are downloaded by the CAMS as soon as a new file becomes available. The CAMS system checks this server every 10 minutes and receives the new files through a secure shell data exchange.
In order to ensure that all the IAGOS profile measurements (ascent and descent) are available for CAMS within the targeted 3-hour near-real-time window, work was undertaken to implement real-time data transmission through the air via the RTTU (real-time transmission unit). Software was developed and installed into Package 1 (P1) to reduce the vertical resolution of the profile to 40 representative levels, in order to keep data transmission costs in check (see detailed description in the deliverable D3.2 report). The data are then transferred from P1 to the RTTU, which transmits them via satellite to the ground-AMDAR centre. From there they are inserted as BUFR data into the WMO Information System (WIS). This would ensure that profile measurements are transferred within 30 minutes of collection for takeoff and within 3 minutes of collection for landing.
The IGAS project saw the development, testing, and implementation of the P1 software for real-time applications. Furthermore, both the software and the hardware (the first RTTU) were certified for installation on board a participating aircraft. Unfortunately due to scheduling difficulties with the maintenance schedule of the aircraft, the installation of the RTTU had to be postponed until after the end of the project. However, partner CNRM confirmed its commitment to install the RTTU after the end of the IGAS project, if necessary at its own expenses.
WP4: Evaluation and harmonisation of data quality in routine aircraft observations
In this work package, a conceptual model for the QA/QC (quality assurance/quality control) of IAGOS data was established. The concept is described in deliverable D4.1 report, and its implementation is reported in D4.2. For a long-term data record, this work is seen as critical in order to ensure that any trends seen over time are not the result in changes in measurement procedures or calibrations.
The first task in this concept was the preparation and evaluation of standard operating procedures (SOPs) and factsheet for each IAGOS instrument. Furthermore, guidelines were developed for the harmonized storage of the measurements in the IAGOS database including their uncertainties and a data flagging index describing the status of the measurements.
The implementation of this QA/QC concept and its structure were tested and evaluated by external experts at workshops held at the end of the second and third years of the project, and their feedback was incorporated (D4.2). The experience obtained in this two round evaluation process demonstrated that external experts are essential in order to achieve an independent and objective assessment of the IAGOS-QA/QC concept in operation. The QA/QC documentation (including all QA/QC-protocols) has been identified as an essential part of the archived flight data, and thus has been archived as metadata at the IAGOS database.
Because the same quantity is measured by different instruments within the IAGOS fleet, one of the pillars of the QA/QC procedure is the evaluation of the data consistency internal to the IAGOS network. This can involve either two of the same instruments flying on two different aircraft or measurements made by two different kinds of instruments flying on the same aircraft.
This is done by means of an automatic application of detection of 4D coincidences within IAGOS fleet, implemented in the IAGOS central data base (details in report D4.4). Criteria of matching in time and space have been clearly defined for every compound (O3, CO, H2O, NOx, NOy, aerosols, CO2, CH4, cloud particles) based on input from the PIs of the different packages.
This procedure is currently operational for O3, CO and H2O only, as the other packages are not flying yet on board equipped aircraft. Statistical analysis for the years 2006 and 2012 shows excellent agreements between different CO instruments. The defined score is a bit lower for ozone, but the criteria are more severe and difficult to meet given that variability/reactivity of ozone is higher. Details can be found in the deliverable report D4.4. The methodology has been successfully applied to (profiles and cruise) data over the year 2013 and the results have been published in Nédélec et al., 2015 (MOZAIC-IAGOS special issue in Tellus-B).
The tool has been further developed in order to calculate and plot the statistics of all detected matching cases (i.e. coincidences in time and space). The tool has been made more more flexible such that one can specify criteria for the strictness of the match in horizontal distance, vertical distance, and difference in time. In addition, the tool is capable of finding instances where the same air mass was sampled twice even if there is a difference in time or space between the two sampling events.
In addition to these internal consistency checks, Lagrangian trajectories were used to detect measurements of common air masses from IAGOS-CORE or IAGOS-CARIBIC aircrafts and external platforms like CONTRAIL, NOAA ESRL/GMD or DLR research aircraft.
The experiences gained during the software development and first tests has been summarised in the Milestone MS45 report submitted in July 2014. It also contains a detailed description of the coincidence detection algorithm and examples for coincidences found using flights in 2006.
Testing revealed that standard IAGOS-CORE 20-day backward trajectories stored with 24 hours time-steps are not suitable for this kind of analysis, and new 7-day trajectories with hourly resolution were calculated.
The output of the coincidence detection software was discussed with the IAGOS instrument PIs. To facilitate their work, the output now includes a tabular list of all matching data points including the measured trace gas concentrations for the air mass at both ends of the Lagrangian trajectories. The results are summarized in D4.5.
In order to collect the QA/QC documentation for each IAGOS instrument during a given deployment period, a template was created jointly with the IAGOS instrument PIs. The template provides for reporting of pre- and post-deployment QA/QC as well as QA/QC information during the deployment period, but also for reporting the results of internal consistency tests (using the comparison of observations of the same quantity by different IAGOS instruments) and external consistency tests (involving instruments from outside of IAGOS). Details can be found in the deliverable report D4.3. The templates were revised based on feedback from both instrument PIs (from both IAGOS-CORE and -CARIBIC) and the external reviewers.
WP5: Enhancing the IAGOS observational capabilities
The tasks of this work package operated essentially independently of one another, targeting specific instrumental development to improve the capabilities and flexibility of the IAGOS fleet.
Task 5.1 Investigation of possible change in configuration of IAGOS setup in order to allow more flexibility in the installation of different combinations of instruments in the future
The suite of instruments operated on IAGOS-CORE aircraft is based on a modular design. Package 1 (P1) operates on all IAGOS-CORE aircraft, and measures ozone, water vapour, carbon monoxide, and cloud particle number concentration. The concept of the second unit (Package 2, or P2) was introduced during the transition from MOZAIC to IAGOS, and includes compact and highly specialized instruments for measuring specific species and properties such as nitrogen-containing compounds (P2a: NOy, P2b: NOx), aerosol particle properties (P2c) or greenhouse gases (P2d: CO2, CH4, CO, water vapour). In the final configuration of the infrastructure, one option of this second unit (Package 2, option a-d) will be installed on each aircraft.
This task considered the possible combinations of different instruments within the existing IAGOS-CORE installation. The requirements for a stand- alone data acquisition system (S-DAS), the minimum requirements of the different instruments regarding the need for gases and/or liquids, and the certification needs for changing the scientific instrumentation in the housing of P1 have been assessed.
It was found that key to any re-configuration is the development of a stand-alone data acquisition system (S-DAS). The required components of an S-DAS have been determined and in total four different options for the installation of an S-DAS have been identified. The advantages and disadvantages of each option, including certification issues, time planning and costs, were identified. Based on this assessment, reconfiguration of the IAGOS-CORE rack is not planned by IAGOS-AISBL in the near future, not least because of the costs involved. Details are reported in D5.5.
Task 5.2: Development and construction of PTR-MS
A new and improved Proton Transfer Reactor – Mass Spectrometer (named PTR-MS-2) for the detection of volatile organic compounds (VOCs) was developed to replace the presently installed PTR-MS-1 (in use since 2005). PTR-MS-2 is has been adapted from a prototype instrument that was developed (until 2015) for use on board research aircraft, with the aim of operating an identical instrument for IAGOS-CARIBIC. Owing to the reconstruction of the vacuum recipient made of nickel coated aluminium alloy and the newly-built control unit, the weight of PTR-MS-2 is 50 kg (without rack), which is much lighter than PTR-MS-1 (~100 kg). The new instrument control in PTR-MS-2 is based on a compact V25 control computer (MPI-C Mainz) and several control boards. One great advantage of PTR-MS-2 lies in its robustness and ease of operation. For instance broken boards can easily be exchanged on the fly which is an important issue when performing regular flight series each month with strict deadlines. This allows for all housekeeping data to be logged and reviewed after flights to locate technical problems.
Furthermore, the design of the inlet system of PTR-MS-2 was improved and offers a much better regulation of the drift tube pressure (PDT) compared to PTR-MS-1, i.e. 2.3(±0.02%) hPa versus 2.3±(0.3%) hPa. As the instrumental sensitivity depends quadratically on PDT, the PTR-MS-2 offers a much better precision. Besides, lab measurements point to the fact that part of the chemical background in PTR-MS-1 is caused by a Viton®-sealed proportional valve for pressure control, which (owing to the redesign of the inlet system) is not used in the PTR-MS-2. Therefore we expect a considerably lower chemical background in PTR-MS-2.
PTR-MS-2 will also expand the list of detectable VOCs. While PTR-MS-1 is only able to measure tropospheric mixing ratios of acetone, acetonitrile and (to a minor extent) methanol, PTR-MS-2 can in addition measure benzene (limit of detection (LOD): ~20 pptV), methanol (LOD: ~500 pptV) and formaldehyde (LOD: ~4 ppbV) as demonstrated in D5.6.
Recently PTR-MS-1 became available as normal lab instrument, so modified components can be tested and integrated into PTR-MS-2 gradually after successful lab tests. Among these is an improved drift tube which uses static dissipative SEMITRON® spacers as isolating elements to eliminate charging effects and a modified ion source drift region to reduce the water vapour concentration in the drift tube, improving, for instance, the detection limit of formaldehyde. In the long run, the operation of an ion funnel in the last segments of the drift tube is expected to improve ion transmission and therefore the sensitivity.
Task 5.3: Development and evaluation of a miniature cloud spectrometer with particle depolarisation detection capability able to discriminate between water, ice and other particle type suitable for future replacement of the IAGOS BCP instrument
A new improved backscatter cloud probe has been developed and tested both in the laboratory and on a number of aircraft. The new probe, (called backscatter cloud probe with depolarisation, BCPD), includes an additional channel to measure linear backscatter polarisation with the aim of providing additional information to derive better statistical representations of water-spherical and ice and ash non-spherical particles in mixed phase cloud and dust-ash environments.
The instrument was certified for flight on the FAAM Bae 146 (UK), Halo and Falcon aircraft (Germany), Cessna Citation (Univ. Wyoming) and has also been flown on a Cessna research aircraft in Japan.
A number of improvements to the optical arrangement and detector electronics provide single particle information to allow a fraction of the sample particles to be assessed using various backscatter scattering ratios of polarisation and size to assess the shape of the particles. The instrument is also able to detect particles down to significantly smaller sizes than the traditional BCP, (< 2 μm compared to 5 μm). Data acquisition on the Bae146 initially used LabView-based software but this has now been replaced by an option in the DMT PADS data acquisition software, allowing the probe to be integrated with other DMT cloud probes and data acquisition systems used on a number of different aircraft. Data analysis software has been produced allowing routine retrieval of particle size distribution data along with the core data files from the Bae146 aircraft. The instrument’s sample area has also been characterised using a new scanning droplet gun generator to improve uncertainty in sample volume calculation. The instrument has been tested in Arctic stratocumulus and convective clouds in the south of England using the BAe146 and validated against a range of research cloud spectrometers. All data have been archived at the UK CEDA and BADC web sites. Preliminary analysis of retrieved particle size distribution measurements from BCPD showed good agreement under most of the sampled mixed phase cloud conditions with the larger research instruments. A technical publication is being prepared and posters have already been presented at conferences. It was noted that the BCPD provides better overlap with the aerosol spectrometers particle size distributions under certain conditions compared with e.g. CDP. Based on a typical uncertainty analysis for spectrometers of this kind the retrieved size distributions are deemed good.
The instrument was also recently flown through a volcanic plume and water cloud downwind of the Sakurajima volcano in Japan and was able to identify the presence of ash particles within the water cloud. The UK BCPD was also deployed at a laboratory experiment at DMT to assess response to different mineral/dust articles and data are being analysed.
Laboratory experiments with BCPD and analogue instruments have continued and new algorithms based on e.g. various cluster and decision tree algorithms have been used to quantify the uncertainty and variation in single particle “type” depolarization ratio. These laboratory and airborne data have been summarized and published (Nichman et al., 2016).
This will be used as a template with which to compare the performance of the BCPD in ongoing future data analysis from different cloud environments as part of new airborne campaigns. The BCPD has been shown to successfully discriminate between water and non-spherical particles, but more work is needed to assess response within the non-spherical classes for a wider range of ice crystal habits. This information is available from the FAAM aircraft and is being analysed as part of PhD student projects. The single particle information provided by BCPD has also been shown to be useful in improving data quality control.
In summary the goal of delivering a compact BCP with depolarisation has been successfully completed and its performance shown to be superior to the current BCP and consistent with research polarization particle spectrometers. This progress is summarized in D5.7.
Task 5.4: Development and certification of a particle extinction measurement instrument based on the cavity attenuated phase shift principle
In the framework of IGAS, the commercially available instrument CAPS PMex was characterized in a laboratory setting and adapted for airborne use. The tests showed that the instrument physical principle of cavity attenuated phase shift was operative for pressure levels down to 200 hPa. There is no dependence of the instrument noise on the operational pressure.
The response of the instrument, when operating with particles at both ambient and low pressure levels is in accordance with what would be expected from Mie Theory. In summary, the tests reported less than 10% deviation between the CAPS PMex instrument response and calculated extinction coefficients over the investigated pressure range. The modified flow system operates properly at low pressure, and the limit of detection is still within the range of extinction coefficient values suitable for the targeted atmospheric layer.
The prototype of the modified CAPS PMex was then certified for operation aboard a research aircraft. The prototype instrument P2e was first deployed on board the research aircraft Polar 6 of type Basler BT-67, operated by the Alfred Wegener Institute. The flight tests were conducted in the framework of the field project BALTEX, which took place in August 2015 over the Baltic Sea. In total 20 flight hours were conducted. The Polar 6 aircraft operated out of the island of Bornholm, Denmark.
Two particular flights of BALTEX 2015 were of interest for the validation of P2e.
(1) During the transfer flight from Bremerhaven to Bornholm on 25 August 2015, clean air was sampled in the free troposphere at pressure levels down to 700 hPa. This flight sequence was used to validate the instrument’s limit of detection of detection (LOD), which was found to be sufficiently below typical aerosol extinction coefficients observed in the upper troposphere and tropopause region, which are of the order of 1 Mm-1. This LOD can be achieved when averaging signals over a period of 60s.
(2) During the transfer flight from Bornholm to Bremerhaven on 30 August 2015, vertical profiles of the free troposphere and the planetary boundary layer aerosol extinction coefficients were measured over the Meteorological Observatory of the German Weather Service at Lindenberg in parallel to lidar observations of the aerosol profiles. These profiles showed a high concentration of aerosol particles and light extinction across the free troposphere and the planetary boundary layer. Intercomparison between extinction profiles deduced from lidar and obtained directly from the IAGOS prototype P2e instrument showed excellent agreement, with deviations of less than 1 Mm-1 after correcting for ambient humidity effects on aerosol scattering.
Summarising, the prototype instrument P2e performed well during the research flights conducted during BALTEX 2015. The modifications to the original instrument’s set-up concerning flow stabilization and temperature control turned out to be successful and lowered the LOD of the instrument sufficiently to make it suitable for the planned application. Finally, optical closure between ground-based lidar and in-situ measurements of the aerosol extinction coefficient by IAGOS prototype P2e instruments was achieved. More details can be found in the D5.8 deliverable report.
Task 5.5: Development and evaluation of a reduced size instrument measuring water vapour and total water
This task aimed to further develop the Hilase-Hygro photo-acoustic instrument measuring water vapour and total water for autonomous use on board aircraft.
The targeted weight and size reduction of the prototype of the Hilase-Hygro instrument was successfully achieved by using much smaller and lighter electronics which inherited all the features of the old electronics. These new electronics provided a more user-friendly interface than the previous version.
In the past, stainless steel cells had to be used in order to ensure sufficiently fast response of the instrument to sudden concentration variations. However due to a special coating which is applied in the inner walls of the cells, it is possible now to make the cells of aluminium. In this way the cells are significantly lighter without sacrificing the fast response of the instrument.
Various self-checking algorithms, including laser wavelength locking and a simple method for sensitivity verification (either during flight or on the ground), are implemented into the instrument, which ensure its long-term reliable operation.
Flight tests were successfully completed, confirming the ability of the Hilase-Hygro instrument to simultaneously measure atmospheric water vapour and total water. Further details can be found in D5.9.
Potential Impact:
The final outcomes from the IGAS project entail:
• The provision of a reliable interface between IAGOS and the Copernicus Atmosphere Service in WP2 and the installation of a system for real time data transmission (WP3)
• The development of tools to facilitate the use of IAGOS data for satellite validation and its application in the Copernicus Atmosphere Service (WP2)
• The establishment of procedures for routine evaluation and documentation of data quality (WP4)
• The development of novel instrumentation for the next generation of operational airborne in-situ measurements (WP5)
The work performed in IGAS ensures that the unique data stream produced by IAGOS can be more fully exploited by the scientific community. The provision of the profile data in near-real-time (WP3) ensures that the profile measurements can be integrated by the Copernicus Atmosphere Monitoring Service, and potentially other operational meteorological services as well. These data are expected to contribute in particular to the reduction of bias in the assimilation of satellite data, which in turn will improve the forecasts and analysis of atmospheric composition and air quality. At the end of the project, the operational transfer of the profile data by the GSM network has been established, which ensures that the profile measurements are made available to CAMS within about six hours of their collection on average (shorter for descent profiles, longer for ascent profiles). The groundwork has been laid for the installation of the RTTU as well, which although not possible within the lifetime of IGAS, will enable the provision of all profile measurements to CAMS within the targeted time span. Improvements in these forecasts have clear socio-economic benefits, in addition to their inherent scientific value.
The IAGOS data, with the help of the column-completion and collocation tools developed in IGAS WP2, can also be used directly by the satellite community to improve retrieval algorithms and reduce biases in their products. This has been demonstrated within the IGAS project for the following instruments (species): SCIAMACHY (NO2, CH4, CO2), OMI (CO, O3), IASI (CO, O3), and GOSAT/TANSO-FTS (CO2, CH4), and these tools can potentially be extended to new sensors and species. Measurement of atmospheric composition by satellite is a growing field, and improvement to these datasets can influence a range of scientific fields. In particular, a clear application is seen for the upcoming launch of Sentinel-5P, which will be measuring CO, O3, NO2, and CH4, all of which are measured by IAGOS. Other potential options for validation include the planned Chinese CO2-monitoring mission TanSat (expected to launch later this year) and NASA’s OCO-2.
The interoperable IAGOS database services IAGOS, JOIN and HALO-DB allowing single login access to the IAGOS data is expected to increase data use by the scientific community. The newly established IAGOS web portal provides user services including search by variable, time, or region, and quicklook data plotting for all observations. The integration with JOIN will also enable online data exploration and comparison with modeled fields from CAMS.
One of the instruments developed in WP5, the CAPS-PMex measuring particle light extinction, was specifically requested by GISC in order to better constrain modelled and remote sensing aerosol optical depth (AOD) measurements with a comparable in situ measurement, and thus has clear applications and impacts. GISC further requested the development of the BCPOL (now called BCPD), which measures not only the presence or absence of clouds, but is partially able to discriminate between water, ice, and other aerosol types such as volcanic ash and mineral dust. Although commercial aircraft are not expected to fly when volcanic ash is likely, being able to determine the absence of volcanic ash is also important, and can improve the forecasting of volcanic ash plumes, which have been very disruptive to aviation, resulting in significant socio-economic repercussions. Another of the developed instruments, the PTR-MS-2 for measuring VOCs, was likewise requested by MACC-II, as there is a dearth of measurements to validate some species of importance for the understanding and modelling of atmospheric chemistry.
IAGOS has been established as a long-term infrastructure, and the work performed in IGAS to establish a framework for evaluation, harmonization, and documentation of the quality of these data is crucial to the use of this data record, both now and in the future. Thorough documentation of procedures and quality checks is critical when dealing with long-term records, especially when assessing climate trends on a decadal scale. The tropopause is expected to be particularly sensitive to changes in climate, and IAGOS is uniquely positioned to monitor any changes. It is essential that the uncertainties in the measurements are known, and that variations over time are not simply spurious results due to changes in calibration procedures, for example. The procedures for evaluation and harmonization of the data quality developed within IGAS document the quality of the observations for the data user, and are thus of long-term benefit in terms of better understanding climate change, which has broad societal implications.
Dissemination
The dissemination activities resulting from this project have largely focused on the scientific community, as these are the direct users of the work enabled by IGAS. Outreach beyond this community is rather left to IAGOS-AISBL, as this is the longer lasting infrastructure that IGAS is supporting, and which comes into contact with a broader audience through its partnership with civil aviation.
Of the scientific dissemination activities to date, 14 papers have been published in peer-reviewed journals, and further articles are in the review process. Of these, the majority are related to instrument development (WP5). There have also been over 37 presentations related to IGAS on the national, European, and international level. Participation in the major international conferences of the European Geophysical Union and the American Geophysical Union have ensured that the developments have met as wide an audience as possible. The topics of the presentations have covered all aspects of the IGAS work, including project overviews.
Exploitation/sustainability of results
Because of its position between the research infrastructure of IAGOS and the operational service CAMS, most of the developments realized within the IGAS project will find an application through one or both of these.
More concretely, the database developments of WP2 and the value-added products (such as the representativeness error) have been integrated into the new IAGOS data portal, currently in beta form at iagos.sedoo.fr to be moved to iagos.fr in September 2016. The maintenance of this data portal will continue under the auspices of IAGOS-AISBL into the future.
The tools for satellite validation developed at KNMI is hosted there, and the tool developed at UPS is hosted in Toulouse. Both link to the IAGOS Data Centre, but it is unrealistic to host the tool directly on the server due to the need to access massive amounts of satellite data. The tool for GHG satellite validation developed at MPG-BGC is currently not hosted in an operational capacity, as it is reliant on the presence of the delayed IAGOS-GHG data stream. However the development is in place, and MPG-BGC is committed to providing this tool in a publicly accessible way once the data come online.
The development of the observation operator and diagnostics for GHG and aerosol data were carried out by ECMWF, and are fully integrated in the operational system of CAMS, which will continue well beyond the length of the project.
The first version of near-real-time data transfer that was established in IGAS, via the GSM network upon landing, is fully operational, and being continued within IAGOS. The certification of the RTTU that took place within the project will enable the installation in an aircraft soon. Unfortunately this installation was not possible to realize within the lifetime of the project as planned, but contingency plans have been drawn up to ensure that the installation will take place, drawing upon alternate sources of funding. This is discussed further in the second periodic report. The software development to select representative levels and compress it into BUFR format will also be applied to the eventual installation of the RTTU, thus implementing this development in the operational network of IAGOS.
The quality assurance/quality control and data harmonization activities established in WP4 were intended to set the standard which IAGOS will follow going forward. The traceability and documentation of the measurement procedures is invaluable for the interpretation of a long-term data record. IAGOS-AISBL recognizes this, and is taking steps to overtake responsibility for the continuation of this work. The matter will be discussed at the next IAGOS-AISBL Executive Board meeting in September, 2016, during which an official decision on this issue is expected.
Each developed instrument has its own unique plan for exploitation and sustainability after the end of IGAS. For the PTR-MS-2, certification has already been attained for the installation in the IAGOS-CARIBIC container, and this improved version is planned to replace the previously operating PTR-MS going forward. It will be integrated into the IAGOS-CARIBIC container in October 2016, and first flights are foreseen in January 2017.
For the BCPD, initial tests in the laboratory and on aircraft have indicated that it outperforms the current BCP deployed within IAGOS, measuring over a broader size range and adding information about the shape (and thus speciation) of the particles through the measurement of the polarization of the backscattered light. A decision will be made by IAGOS-AISBL in 2017 whether the BCPD will be installed on the IAGOS CORE packages. This decision is partly financial, as it would require replacing the BCPs currently deployed within IAGOS (>12). The certification may also be problematic: while it may be possible to use a new “version” type certification, building off the existing certification, the fact that the instrument is slightly larger than the current BCP might require new certification for the installation in IAGOS aircraft.
The prototype instrument P2e, a modified CAPS PMex developed in Task 5.4 has been tested in the laboratory and in research aircraft, and found to be suitable for the planned application. The next steps will be to investigate the certification of the instrument for an eventual inclusion in the IAGOS-CORE fleet, in addition to further use on a campaign basis by the team in JUELICH.
The improved prototype of the Hilase-Hygro instrument, developed in Task 5.5 is now appropriate for autonomous, long-term operation. Flight and laboratory tests have shown good performance, and the instrument is considered ready for use in research aircraft. Discussions are underway between Hilase and national funding agencies in Hungary to support the installation in further commercial aircraft.
List of Websites:
Public website: http://igas-project.org
Newly-developed IAGOS data portal: to be moved to http://iagos.fr in September 2016
Coordinator contact details:
Dr. habil. Christoph Gerbig
Phone: + 49 3641 576373
fax: + 49 3641 577300
email: cgerbig@bgc-jena.mpg.de