Final Report Summary - EARLINET-ASOS (European aerosol research Lidar network: advanced sustainable observation system)
The main objective of the EARLINET-ASOS was to improve the EARLINET infrastructure resulting in a better spatial and temporal coverage of the observations, continuous quality control for the complete observation system, and fast availability of standardised data products.
The present knowledge of the aerosol distribution in the atmosphere is far from sufficient to properly estimate the role of aerosols in changes of the global and regional environmental conditions and climate. The information on the vertical distribution is particularly lacking and Lidar remote sensing is the most appropriate tool to close this observational gap. In particular, Lidar networks are fundamental to study aerosol on large spatial scale and to investigate transport and modification phenomena. These are the motivations for the establishment in 2000 of Earlinet, the European aerosol research Lidar network, as a research program funded by the European Commission in the frame of the Fifth Framework Programme (FP5).
The overall objectives were:
- to extend the development of the European aerosol research Lidar network as a world-leading instrument for the observation of the four-dimensional spatio-temporal distribution of aerosols on a continental scale, resulting in accurate, well-defined, and easily accessible data products for use in science and environmental services;
- to enhance the operation of this instrument to foster aerosol-related process studies, validation of satellite sensors, model development and validation, assimilation of aerosol data into operational models, and to build a comprehensive climatology of the aerosol distribution;
- to play a leading role in the development of a global observation network for the aerosol vertical distribution as a major innovative element of GEOSS, by setting the standards for instruments, methodology, and organisation in this specific area.
These objectives have been reached by strengthening the cooperation among the partners with several networking activities: exchange of expertise with the main goal of defining and disseminating best practice and knowledge; quality assurance program for both algorithms and instruments for assessing and assuring common high quality standards; optimisation of instruments for achieving a better temporal coverage and standardisation of performance; optimisation of data processing with the goal of establishing an automatic processing from raw data to final products, establishing a database provided with an user interface for dissemination of data.
The project was carried out through the following Networking activities (NA):
NA2: Exchange of expertise
The main objective of the exchange of expertise NA2 in EARLINET-ASOS was to provide effective means to improve the EARLINET infrastructure through a comprehensive and organised flow of information both between the participants and between the participants and the external scientific and general community.
NA2 has been a transversal and instrumental networking activity enabling the other focused activities in EARLINET-ASOS to fulfil the project overall objectives of extending the development of EARLINET as a world-leading instrument for the observation of the space-time distribution of aerosols on a continental scale, enhancing the operation of this instrument, and playing a leading role in the development of a global observation network for the aerosol vertical distribution.
The exchange of expertise both inside EARLINET and with the external scientific community has been successfully achieved through a combination of regular activities (workshops, training courses), contributions to journals and conferences, outreaching planned activities such as the setting and keeping up of a web site and the final symposium, and spontaneous exchanges involving EARLINET participants and partner institutions, as well as the external community.
This activity has resulted in an effective improvement of the EARLINET infrastructure (see the part of NA4 in this report summary) and has helped in defining technologies, techniques and procedures that will significantly contribute to the Galion worldwide aerosol Lidar network and to the new European aerosol, clouds and trace gases research infrastructure (ACTRIS) starting in 2011 under a European Commission's Seventh Framework Programme (FP7) infrastructures project.
NA3: Quality assurance
In the frame of EARLINET-ASOS the activity NA3 had been established to develop tools for testing the accuracy and the temporal stability of the quality of the Lidar systems (NA3.1 Quality assurance of the hardware) and to develop tools for testing of the accuracy of the analysis software (NA3.2 Quality assurance of the algorithms).
The results of the inter-comparison of the derived products between the Single calculus chain (SCC) retrieval and the synthetic input data was excellent both in terms of shape and of statistical error. Concerning the comparison using experimental data, the retrievals of backscatter and extinction obtained with the SCC and the same products obtained with the data analysis performed with the algorithms of the single groups, already quality assured previously, also in this case it gives good results both in terms of shape and of statistical error. The result is very important, because it demonstrates that the common algorithms developed in the frame of the activity NA5 and implemented in SCC have been assessed to produce quality controlled data. The SCC will be the calculus tool that will provide derived products to the EARLINET database starting from raw Lidar signals. This will assure that the data in the Earlinet database will be quality controlled.
NA4: Optimisation of instruments
The objective of this activity was to improve the instruments of the observation system by selecting the optimal approach from the various solutions existing at individual stations, for the components and subsystems as well as for system integration. In system integration, emphasis is on automation of operation in order to reduce the necessary manpower, to improve temporal coverage, and to make the performance independent from operator characteristics.
The EARLINET-ASOS implementation of the handbook is currently the most comprehensive handbook for Lidar instruments in the world. Representatives from other Lidar networks (Galion, NDACC) have expressed interest in adopting the approach of the EARLINET handbook of instruments.
In the development of the network we also observe the adoption of new techniques for aerosol profiling. In particular, new technologies have enabled the use of pure-rotational Raman channels. Using this technique, usually better daytime performance is gained, which is important for combined use of aerosol Lidar data with passive (sun-light dependent) measurements, e.g. sunphotometers. At the same time, two pure rotational Raman channel bands give access to temperature measurements.
Alternative techniques were also adopted. For one station, the high-spectral resolution Lidar (HSRL) technique for aerosol profiling has been implemented.
At a more detailed level, the handbook of instruments provides information on further standardisation in sub-systems which has taken place. In particular, we see that the types of digitisers, detectors, optical components, imaging systems and overlap techniques developed in one or a few stations, have been adopted by other stations, where possible.
In-house developments, as well as close collaboration with SME's have enabled the development of the network.
NA5: Optimisation of data processing
The optimisation of Lidar data processing, activity 5, represents an important and innovative part of the project. The main objective of this activity has been to provide all partners with the possibility to use a common calculus system (SCC) for the evaluation of their data, from raw signals to final products, so providing the EARLINET database with homogeneous results of well characterised quality results in nearly real time.
The main difficulty consisted in handling raw data coming from different and not standardised instruments used by individual groups. Differences exist for the wavelengths used, acquisition mode (analog and / or photoncounting), space resolution, detection systems. Moreover, the implementation of the SCC analysis algorithms had to match all the analysis algorithms developed by each group.
All the goals planned in the activity were reached and results of high interest were achieved. The activity provided the unique opportunity to share the expertise in data analysis within EARLINET and to build a common analysis calculus chain able to process data in a standard and automatic way, suitable for all the systems. This will produce a great benefit for the end users of the EARLINET database because it guarantees quality and homogeneity of data and the possibility to have data availability in near real time. Moreover, the implementation of the SCC had a positive impact on scientific community also because it demonstrates how it is possible to establish a methodology to develop a common data analysis system within networks equipped with different experimental setups.
On the other hand, the development of modules for microphysical analysis derived from the sharing of the expertise in this field allowed to focus and solve several technical problems related to the retrieval of aerosol microphysical properties starting from Lidar data and to implement almost unsupervised retrieval procedures. This aspect was also completed with the implementation of a code to combine Lidar and sun-photometer data for the retrieval of aerosol microphysical properties. Also the achieved results in this subject will have a great impact on the scientific community, because they provide a big contribution in the critic problem of the retrieval of aerosol microphysical properties, but also because this is the first attempt to implement automatic procedures for the inversion using Lidar measurements.
The carried out research and the achieved results enhanced the EARLINET infrastructure providing it with data analysis calculus systems able to characterise aerosol in terms of both optical and microphysical properties and to provide quality checked, standardised and homogeneous results to the database in an automatic way.
NA6: Data base construction and operation
It was one of the central objectives of the EARLINET-ASOS project to provide all internal and external users with a single location where they could access all data from all stations of EARLINET-ASOS (and EARLINET as a broader community) in a homogeneous way. This was accomplished by establishing the EARLINET data centre at the Max-Planck-Institute in Hamburg; this data centre is responsible for collection of data from all EARLINET stations. These data are checked for technical correctness before they are made accessible to the public.
To keep a maximum of data available, all data from measurements performed before the EARLINET-ASOS period (2006 - 2011) have been reformatted, checked and added to the database. Additionally measurements from the preceding -Deutsches Lidarnetz, a nationwide German Lidar network (1998 - 2000), found their way into the EARLINET data set.
The EARLINET database has been operated for several years very successful. The EARLINET members have accepted it as a central repository for their aerosol Lidar measurements. The public user interface is operational since April 2010. It is intensely used by the public community. In this year more than 200 000 files have been downloaded through this web portal.
The EARLINET database with its well established formats and methods will be a very important component of the integrated database of the European Aerosol, clouds and trace gases research infrastructure (ACTRIS), an infrastructure project in frame of FP7.
The present knowledge of the aerosol distribution in the atmosphere is far from sufficient to properly estimate the role of aerosols in changes of the global and regional environmental conditions and climate. The information on the vertical distribution is particularly lacking and Lidar remote sensing is the most appropriate tool to close this observational gap. In particular, Lidar networks are fundamental to study aerosol on large spatial scale and to investigate transport and modification phenomena. These are the motivations for the establishment in 2000 of Earlinet, the European aerosol research Lidar network, as a research program funded by the European Commission in the frame of the Fifth Framework Programme (FP5).
The overall objectives were:
- to extend the development of the European aerosol research Lidar network as a world-leading instrument for the observation of the four-dimensional spatio-temporal distribution of aerosols on a continental scale, resulting in accurate, well-defined, and easily accessible data products for use in science and environmental services;
- to enhance the operation of this instrument to foster aerosol-related process studies, validation of satellite sensors, model development and validation, assimilation of aerosol data into operational models, and to build a comprehensive climatology of the aerosol distribution;
- to play a leading role in the development of a global observation network for the aerosol vertical distribution as a major innovative element of GEOSS, by setting the standards for instruments, methodology, and organisation in this specific area.
These objectives have been reached by strengthening the cooperation among the partners with several networking activities: exchange of expertise with the main goal of defining and disseminating best practice and knowledge; quality assurance program for both algorithms and instruments for assessing and assuring common high quality standards; optimisation of instruments for achieving a better temporal coverage and standardisation of performance; optimisation of data processing with the goal of establishing an automatic processing from raw data to final products, establishing a database provided with an user interface for dissemination of data.
The project was carried out through the following Networking activities (NA):
NA2: Exchange of expertise
The main objective of the exchange of expertise NA2 in EARLINET-ASOS was to provide effective means to improve the EARLINET infrastructure through a comprehensive and organised flow of information both between the participants and between the participants and the external scientific and general community.
NA2 has been a transversal and instrumental networking activity enabling the other focused activities in EARLINET-ASOS to fulfil the project overall objectives of extending the development of EARLINET as a world-leading instrument for the observation of the space-time distribution of aerosols on a continental scale, enhancing the operation of this instrument, and playing a leading role in the development of a global observation network for the aerosol vertical distribution.
The exchange of expertise both inside EARLINET and with the external scientific community has been successfully achieved through a combination of regular activities (workshops, training courses), contributions to journals and conferences, outreaching planned activities such as the setting and keeping up of a web site and the final symposium, and spontaneous exchanges involving EARLINET participants and partner institutions, as well as the external community.
This activity has resulted in an effective improvement of the EARLINET infrastructure (see the part of NA4 in this report summary) and has helped in defining technologies, techniques and procedures that will significantly contribute to the Galion worldwide aerosol Lidar network and to the new European aerosol, clouds and trace gases research infrastructure (ACTRIS) starting in 2011 under a European Commission's Seventh Framework Programme (FP7) infrastructures project.
NA3: Quality assurance
In the frame of EARLINET-ASOS the activity NA3 had been established to develop tools for testing the accuracy and the temporal stability of the quality of the Lidar systems (NA3.1 Quality assurance of the hardware) and to develop tools for testing of the accuracy of the analysis software (NA3.2 Quality assurance of the algorithms).
The results of the inter-comparison of the derived products between the Single calculus chain (SCC) retrieval and the synthetic input data was excellent both in terms of shape and of statistical error. Concerning the comparison using experimental data, the retrievals of backscatter and extinction obtained with the SCC and the same products obtained with the data analysis performed with the algorithms of the single groups, already quality assured previously, also in this case it gives good results both in terms of shape and of statistical error. The result is very important, because it demonstrates that the common algorithms developed in the frame of the activity NA5 and implemented in SCC have been assessed to produce quality controlled data. The SCC will be the calculus tool that will provide derived products to the EARLINET database starting from raw Lidar signals. This will assure that the data in the Earlinet database will be quality controlled.
NA4: Optimisation of instruments
The objective of this activity was to improve the instruments of the observation system by selecting the optimal approach from the various solutions existing at individual stations, for the components and subsystems as well as for system integration. In system integration, emphasis is on automation of operation in order to reduce the necessary manpower, to improve temporal coverage, and to make the performance independent from operator characteristics.
The EARLINET-ASOS implementation of the handbook is currently the most comprehensive handbook for Lidar instruments in the world. Representatives from other Lidar networks (Galion, NDACC) have expressed interest in adopting the approach of the EARLINET handbook of instruments.
In the development of the network we also observe the adoption of new techniques for aerosol profiling. In particular, new technologies have enabled the use of pure-rotational Raman channels. Using this technique, usually better daytime performance is gained, which is important for combined use of aerosol Lidar data with passive (sun-light dependent) measurements, e.g. sunphotometers. At the same time, two pure rotational Raman channel bands give access to temperature measurements.
Alternative techniques were also adopted. For one station, the high-spectral resolution Lidar (HSRL) technique for aerosol profiling has been implemented.
At a more detailed level, the handbook of instruments provides information on further standardisation in sub-systems which has taken place. In particular, we see that the types of digitisers, detectors, optical components, imaging systems and overlap techniques developed in one or a few stations, have been adopted by other stations, where possible.
In-house developments, as well as close collaboration with SME's have enabled the development of the network.
NA5: Optimisation of data processing
The optimisation of Lidar data processing, activity 5, represents an important and innovative part of the project. The main objective of this activity has been to provide all partners with the possibility to use a common calculus system (SCC) for the evaluation of their data, from raw signals to final products, so providing the EARLINET database with homogeneous results of well characterised quality results in nearly real time.
The main difficulty consisted in handling raw data coming from different and not standardised instruments used by individual groups. Differences exist for the wavelengths used, acquisition mode (analog and / or photoncounting), space resolution, detection systems. Moreover, the implementation of the SCC analysis algorithms had to match all the analysis algorithms developed by each group.
All the goals planned in the activity were reached and results of high interest were achieved. The activity provided the unique opportunity to share the expertise in data analysis within EARLINET and to build a common analysis calculus chain able to process data in a standard and automatic way, suitable for all the systems. This will produce a great benefit for the end users of the EARLINET database because it guarantees quality and homogeneity of data and the possibility to have data availability in near real time. Moreover, the implementation of the SCC had a positive impact on scientific community also because it demonstrates how it is possible to establish a methodology to develop a common data analysis system within networks equipped with different experimental setups.
On the other hand, the development of modules for microphysical analysis derived from the sharing of the expertise in this field allowed to focus and solve several technical problems related to the retrieval of aerosol microphysical properties starting from Lidar data and to implement almost unsupervised retrieval procedures. This aspect was also completed with the implementation of a code to combine Lidar and sun-photometer data for the retrieval of aerosol microphysical properties. Also the achieved results in this subject will have a great impact on the scientific community, because they provide a big contribution in the critic problem of the retrieval of aerosol microphysical properties, but also because this is the first attempt to implement automatic procedures for the inversion using Lidar measurements.
The carried out research and the achieved results enhanced the EARLINET infrastructure providing it with data analysis calculus systems able to characterise aerosol in terms of both optical and microphysical properties and to provide quality checked, standardised and homogeneous results to the database in an automatic way.
NA6: Data base construction and operation
It was one of the central objectives of the EARLINET-ASOS project to provide all internal and external users with a single location where they could access all data from all stations of EARLINET-ASOS (and EARLINET as a broader community) in a homogeneous way. This was accomplished by establishing the EARLINET data centre at the Max-Planck-Institute in Hamburg; this data centre is responsible for collection of data from all EARLINET stations. These data are checked for technical correctness before they are made accessible to the public.
To keep a maximum of data available, all data from measurements performed before the EARLINET-ASOS period (2006 - 2011) have been reformatted, checked and added to the database. Additionally measurements from the preceding -Deutsches Lidarnetz, a nationwide German Lidar network (1998 - 2000), found their way into the EARLINET data set.
The EARLINET database has been operated for several years very successful. The EARLINET members have accepted it as a central repository for their aerosol Lidar measurements. The public user interface is operational since April 2010. It is intensely used by the public community. In this year more than 200 000 files have been downloaded through this web portal.
The EARLINET database with its well established formats and methods will be a very important component of the integrated database of the European Aerosol, clouds and trace gases research infrastructure (ACTRIS), an infrastructure project in frame of FP7.
