Final Report Summary - UERRA (Uncertainties in Ensembles of Regional Reanalyses)
Executive Summary:
UERRA built on from its predecessor EURO4M project and both included observational gridding data sets and NWP model based Regional RAs together with extensive validation efforts against independent data sets.
The Data Rescue of historical observational data and their quality control and development has continued in UERRA but focussed on sub-daily data and more after the 1950s than before. Over 8 Million data values were digitised from journals on paper or in scanned form. Other openly available data have been assembled (112 Million) and provided to international data centres and to RAs.
The 35+ and 55+ years of European Reanalyses have been produced with the Met Office Unified Model 4D-VAR ensemble system and the HARMONIE ALADIN system at SMHI, respectively, and with 70 or 65 levels. A five year period has been assimilated with 20 ensemble members with the COSMO system at University of Bonn, all at 11 or 12 km horizontal resolution (except the Met Office ensembles at 36 km). A high resolution 5 km 2 m temperature and humidity and precipitation and soil analysis was produced at Météo-France coupled to the SMHI RA and with an ensemble for 5 of the years.
An extensive evaluation activity with a common software package was carried out by DWD, Meteo-Swiss and MET Norway and KNMI for climate indices. Also other RAs were compared like the ERA-Interim and the COSMO 5 km at DWD. The quality of the reanalyses was generally good and good benefits could be seen from the high resolution. Over the Alpine and Scandinavian mountains there were quite big differences in precipitation and mostly overestimations compared with high resolution gridded data sets. Moreover were the different models and the ensemble members not enough spread to fall within the catchment estimates from high resolution data at Meteo-Swiss even though the spatial patterns were good. This was also seen at NMA-Romania where also precipitation data have been digitised in the Project.
The Regional Reanalysis have produced large data amounts, almost 1 Petabyte which has been archived in a common UERRA archive in MARS at ECMWF. The data have been defined and converted to WMO GRIB2 with several new features like new variables and levels at fixed heights and in the ground. All the data are in MARS and common access methods can be employed and multiple tools for handling the data are available and to the whole community. A subset of the data has also been stored on an ESGF node as a demonstration for the general climate community.
Project Context and Objectives:
There is a wide demand for gridded high quality gridded data sets of Essential Climate Variables (ECVs) and additional derived environmental variables on the same grid with a high resolution in time and covering climatic multi-decadal time scales.
Data sets may be constructed from observations only using statistically based interpolators to regular grid points in space. On the global scale the resolution is relatively coarse, usually about 50 km and on the European scale about 25 km. Sub-regional or national observational data sets have been constructed at much higher resolutions, between 10 and 2 or even 1 km. The data are the observed surface variables of temperature, humidity and precipitation. These data sets are limited to certain regions and there is no general pan-European data set at very high resolution. The national or sub-regional data sets serve important national needs in climatology for one own national responsibility but no general one for Europe. Other observational data are constructed from satellite and other remote sensing data and may then include radiation related variable including clouds and surface properties.
The other type of data sets are the Numerical Weather Prediction (NWP) model based reanalyses of observations also on a regular horizontal grid but also adding the vertical dimension with many levels in the atmosphere and with a comprehensive set of model variables that are internally consistent. The Reanalyses (RAs) use as wide selection of observations as possible and combines them with the model evolution in space and time in an optimal way.
Global RAs have earned a high reputation and become widely used for many applications. National and regional applications often demand a locally higher resolution than the global ones and downscaling from about 50 km to 20-10 km or better may be done for these purposes.
The requirements for regional European RAs were identified by the European Commission and the Scientific community and in the FP7 SPACE programme 5 pre-operational Copernicus projects were launched and one of the was for Ensembles of European Reanalyses which then materialised as the UERRA project. Uncertainty estimation was also an important requirement and therefore a multitude of methods and models were employed.
UERRA was set up to answer to the need of this pan-European approach of high resolution climate quality reanalyses for many general purposes over the whole domain as well as over owns own territory.
UERRA built on from its predecessor EURO4M project and both included observational gridding data sets and NWP model based Regional RAs together with extensive validation efforts against independent data sets. The RRAs have a more dominant role in UERRA and most of the resources were allocated to the development and production of three different ensemble model based 3 and 4-dimensional ensemble reanalyses (with time being the 4th) and also an ensemble surface RA. Ensemble means that a multitude of RRAs were produced that were slightly different due to inherent uncertainties in the observations and models and the chaotic nature. Also the observation gridding uncertainties have been estimated with newly developed methods in the Project.
The time period in UERRA is long enough to cover climatological needs (55+ years or 35+ years for the two main RAs in the Project) and there is a data services part of the Project so that access to the data is easily provided in a uniform way to any user.
A new aspect in UERRA is the requirement of providing not only the best estimate of the Essential Climate Variables but also the associated uncertainties. Evaluation of RAs has been done before and has also been done in UERRA and this provides a part of uncertainty but far from the full answer. The uncertainties cannot be universally defined but depend on the scales in time and space for the applications. Then the reference data to be compared with also have their uncertainty and the data are limited to certain time and space scales, like how often in time and how dense the observations were made.
Also the model ensemble based estimates have limitations in that they provide spread using all the same model and all members have certain inherent common errors that those ensembles cannot measure. Therefore it was deemed important in UERRA to employ several different model and data assimilation systems and even observation gridding system to provide more spread due to the different models used (multi-model approach).
To answer the need of very detailed near surface analyses of certain ECVs and provide products which are closer to observations than the full NWP model RAs, UERRA has also a 2-dimensional downscaling reanalysis at higher resolution (5 km) than the 3-dimensional upper air RRAs. This is akin to many national observation interpolation products employed in the NMSs but is pan-European and gives a high resolution 2m temperature, humidity and (24 hour) precipitation analysis. In contrast to the pure observation gridded products, the UERRA surface analyses still uses NWP based RRA as an input, but it is not restricted by the 3-dimensional consistency of the full upper air RRA. The limitation of the product is the density of observations which is variable over the continent due to data polices or data rescue or transfer of data.
From the DoW (Annex I, Part B), the following objectives for the project are defined.
The overriding objective is to produce long-term high-resolution climate quality datasets over Europe complete with estimations of their quality and uncertainty (WP1, 2, 3 and 4)
To produce these through 3 and 4-dimensional reanalyses (RA) and 2-dimensional downscaling RA and extended observation gridded datasets (WP2 and WP1)
To estimate the uncertainty of the individual RA through ensemble data assimilation for Europe and produce a high-resolution ensemble RA for as long multi-decadal time period (WP2)
To provide additional observations to be used for these RAs, other projects and for the community at large (WP1)
To make the RA data available to a large number of users (WP4)
To provide data services and visualisation portals for a large number of RA fields (WP4)
To quantify uncertainties and establish knowledge of the quality of the different RA in many different ways, between datasets and with respect to observation gridded sets and satellite-based datasets and river discharge data (WP3)
To get a consistent knowledge for Europe with a common evaluation procedure for ECVs, climate indicators, extremes and scales of variability in space and time and distributions (WP3).
To document how well extremes and special climate features are reproduced in the RA (WP3)
To show how the data can be exploited for user-oriented products (WP8 and WP3)
To provide a unique and useful datasets for a wide range of downstream applications (WP4, WP8)
To support Climate change services and climate adaptation (WP7)
To support and aid policy development and monitoring of climate for European wide and European national applications (WP7)
To establish good user contacts and get early feedback on the user products (WP8)
To have a long lasting impact also after the end of the project (WP1, 2, 3, 4, 6, and 7)
Project Results:
1 Data rescue and development
The work carried out under UERRA’s WP1 in its three different tasks was going well and as planned except for some small delays. The last year (4) of the Project saw all the tasks completed and Deliverables on time or almost on time.
The main tasks were gathering and data digitisation (T1.1 by URV and NMA-RO) of the recovered data and metadata; quality-control (T1.2 by URV and UEA) and the enhancement of methodologies to reduce uncertainties in gridded products, such as E-OBS (T1.3 by KNMI, UEA and EDI).
The achievements under WP1’s Task1.1 have been remarkable, since this task has not only kept on track, but has also exceeded the digitisation targets as they were planned. A larger number of observations at the synoptic scale than planned have been digitised by URV. This rescue effort includes hourly observations of air pressure (SLP), temperature (TMP), wind speed (WS) and direction (WD), temperature dew point (TDP) and relative humidity (RH), in addition to snow-depth (SD), snowfall (FS) and precipitation (RR) observations at the daily scale.
The spatial coverage was mainly from the Balkans (Slovenia, Bosnia-Herzegovina, Serbian Republic, Turkey, Germany, Mediterranean (Egypt, Algeria, Tunisia, Morocco). There are many areas to choose from and many more data remain to be digitised, but it is also question of access to the original paper data or scanned version of them. More will happen in the C3S project (DS311a_Lot1) which will provide software for users and more co-ordinated plans at global scale.
The quality control of the digitised 8.63 Million (M) synoptic observations was completed and the Deliverable 1.5 was submitted early autumn 2016. URV have extracted 112 M already digitised data values from Sweden, Norway and Catalonia. Also the 221 thousand (k) Romanian precipitation data values were collected.
The work committed under T1.2 on high-quality synoptic-scale data development has progressed as planned, since a wide number of quality control (QC) tests (e.g. outliers, bivariate outliers, big jumps, sharp spikes, inter-variables inconsistencies checks) have been implemented and applied to the 8M of recovered and digitised observations. In this regard, a battery of QC tests have been designed and programmed by URV to QC the data at the hourly and daily scale. The Universal Quality Control procedure (UQC) has been implemented to QC hourly TMP, WS, WD and RH observations, while the Sea Level Pressure Quality Control procedure (SLPQC) developed to QC SLP observations. RClimDex extraQC has been used to QC RR data at the daily scale and a Simple Snow QC (SSQ) implemented to QC SD and FS observations at the daily scale. From the 8.2M of observations digitised, about 31K observations have been labelled as suspicious and are currently being verified by URV to validate or reject them and substitute them by true observations when possible. This is a very time-consuming task, since it requires human intervention to crosscheck the digitised data against the original values in the data sources.
In addition to the six station for which the team of NMA-RO has rescued the 6-hourly precipitation data and digitised them, they have also performed a QC of their data using an automatic procedure applied in the database to compare the 6-hours amounts with 12-hours sums and with the precipitation in 24 hours. A list of errors has been provided to the digitising operator at NMA-RO who checked again in the original document and made the corrections. The team of NMA-RO has also rescued and digitised hail diameter data and submitted a paper to Monthly Weather Review about the climatology of hail in Romania.
The digitised and quality-controlled data have also been tested for spatial consistency and Deliverable 1.6 was completed early in 2017. The data were delivered, first, to WP2 partners (Deliverable 1.7) and, second, to the MARS Archive (in the future, first going through the ECA&D (Deliverable 1.8). Other global databanks and data repositories, such as ISPD, ZENODO, HadISDH and ISTI, also received all the recovered data produced in UERRA. In addition, the developed data have been made available through the World Meteorological Organization Secretary General to the Permanent Representatives of each beneficiary NMSs, in order to foster the culture of knowledge and data sharing. Deliverables 1.7 and 1.8 suffered from some small delays since there were be staff changes. The Romanian data have also been delivered (some 221 k).
There were several lesson learned from the Data Rescue effort:
- Exploring digitised data already present in global and regional data centres is not trivial – it is time consuming due to the need of cataloguing all the input data to support reanalyses and gridded products.
- DARE coordination is required to avoid duplication.
- Examining the quality of the data sources is necessary before and after digitisation and to report them.
- The digitisation methods should be organised and chosen depending on the sources.
- Use templates to mimic the source's layout to reduce mistakes.
- There are a few more NMSs with open data polices but for most NMSs data policies still preclude open data exchange.
2 Gridded observational datasets
A gamma-transform technique for improving the gridding of precipitation data, particularly in mountainous areas, has been developed, refined and tested by KNMI and UEA against the high-resolution gridded datasets produced by various National Meteorological Services across Europe, as well as against the E-OBS constructed using the existing techniques (Figure 1). To further improve the gridding in the E-OBS dataset a new technique has been developed which provides a much better interpolation of all variables. This technique (regression-kriging) is applied to the monthly resolution data, which serve to constrain the daily values, and provides a more stable spline over time which is less vulnerable to the changing station data used for the gridding, which is an inherent problem with the E-OBS dataset (see D1.10).
This new technique also allows for the incorporation of additional topographically-derived parameters to be incorporated into the gridding procedure, which allows for a better interpolation of the station data. Furthermore, this technique forms the basis for the production of an E-OBS dataset consisting of multiple-realizations, which is reported in D1.14. To allow for this development of the E-OBS dataset a new package written in the R computing language has been developed, which also contains extensive user documentation. Furthermore, the aforementioned development of the E-OBS dataset using the new R package has been conducted on the ECMWF high-performance computing cluster (see D1.12). This is a significantly higher specification system than has hitherto been used to produce E-OBS and will allow for more computationally demanding techniques to be developed in the future, after the conclusion of the UERRA project. Operational production of the E-OBS dataset has moved over to this system.
KNMI has led the development of the E-OBS dataset (Deliverables D1.10 to D1.14). This work has consisted of an overhaul to the method used to construct the dataset, with a particular emphasis on improving the estimates of uncertainty through the generation of a 100-member ensemble of daily realizations for each variable (Cornes et al. 2017). The code has been written in R, and is version controlled on a private GitHub repository. Further refinements to E-OBS are being undertaken as part of the C3S project (D311a_Lot4), which will also make the ensemble dataset operational. Several experiments were conducted to determine the best method of generating the ensemble. The initial planned conditional simulation method was simply not possible with the time and computing resources available, and hence a different method was developed using Bayesian reasoning, and which is consistent with the standard error estimates used in earlier versions of E-OBS.
Half-yearly full updates and monthly preliminary updates of the original version of E-OBS have been continued in this period.
KNMI made the E-OBS update (D1.13) with their version 14.0 and with data covering 1950 to August 2016. Apart from including the most recent data at the time, it has also corrections to a problem with the minimum and maximum temperatures found in the UK sources..
EDI has developed a new method to quantify uncertainties in observation-based spatial analyses (grid datasets) of precipitation. The procedure represents analyses in terms of an ensemble of equiprobable realizations, the spread of which informs users about inherent analysis uncertainties, related, for example, to the limited observation density or short-scale variance of precipitation. The method builds on statistical (conditional) simulation and extends it to rectify for several shortcomings. Most notably, the present development takes account of the uncertainty in the statistical parameters. A detailed verification in the Alpine region demonstrates the reliability and consistency of results. The technique has been employed to derive a new multi-year ensemble dataset of daily precipitation over the entire Alpine region using high-resolution rain gauge. The new dataset formed as a reference for the evaluation of regional re-analyses in WP3. The findings of this work are also relevant for the development of ensemble gridding techniques with other grid datasets.
These ensemble analyses over the Alps offer users to trace analysis uncertainty into their application and they avoid several known difficulties of single best estimates, including spatial smoothing, biases in the statistics of extremes and ambiguities in effective resolution.
The methodological developments made during UERRA build on and extend conditional stochastic simulation techniques previously proposed for ensemble gridding. Notably, the technique adopts a more flexible stochastic model (Trans-Gaussian Random Fields) to account for skewness and heteroscedasticity of precipitation, and it employs Bayesian statistical inference to incorporate uncertainty of parameters into the final ensemble.
The ensemble dataset of Alpine daily precipitation is represented as area-mean precipitation on a four-level hierarchical subdivision of the domain into hydrological units (the European river catchments dataset of EEA). The ensemble encompasses 50 members. Once all computations are completed, the analysis will extend over 38 years (1971-2008); results for 15 years were available for comparisons and evaluation in WP3.
In-depth analysis of the ensemble dataset demonstrates plausible dependencies of ensemble spread (i.e. analysis uncertainty) upon weather situation (e.g. convective versus stratiform precipitation), density of nearby rain-gauge observations, and the sizes of the catchments. Our evaluation suggests that the dataset has a good statistical reliability, enabling users to confidently use the ensemble members as equally likely potential realizations of the true (unknown) precipitation amounts over the catchments.
An attractive feature of ensemble datasets is that uncertainties can easily be deduced for results of further data processing. Figure 3 depicts the ensemble median and spread for two frequently used precipitation indices, the frequency of wet days and the 95% (all-day) quantile. The ensemble median reveals known pattern of the precipitation climate in the Alps, and the ensemble spread shows plausible variations of uncertainties between areas (larger in data sparse regions), catchment sizes (larger in small area units) and indices (larger for extremes).
A paper describing the methodological developments and the dataset is in preparation (Frei & Isotta 2017). The findings of this work have partly been included in the development of an ensemble grid dataset based on E-OBS. Moreover, the probabilistic Alpine dataset has been used in the evaluation and uncertainty assessment of re-analyses in WP3.
3 Ensemble Data Assimilation Regional Reanalyses
In UERRA there were three different 3-dimensional upper air regional reanalyses, all covering the EU-CORDEX domain and at about 11 km horizontal resolution. This in itself gives a multi-model variation ensemble due to differences in the model and data assimilation formulations and some differences in observational inputs. With the Met Office Unified Model (UM) system there was also an internal ensemble of 20 perturbed members and also with the University of Bonn COSMO based nudging assimilation system. SMHI ran two model versions for 5 years but then one version for over 55 years, from 1961. The Met Office ran from 1979 (to 2015 incl.) but the Univ. of Bonn just for 5 years. In addition there was a dedicated surface downscaling reanalysis at 5.5 km using more in situ observations, of temperature, humidity and precipitation. This also had an internal ensemble for 5 years.
The cloud fraction reanalysis has overcome several scientific problems to produce a tuned system that can cope with horizontal inhomogeneities. It was discovered that CM SAF are making available an improved quality METEOSAT cloud product for this period that should result in a better quality reanalysis. Given other delays already, it is planned to wait for the new product before processing the early part of the period. The later part (2004-2008) could be run using already available MSG SEVIRI cloud products.
A significant circumstance is that all the regional reanalyses are set up in quite different configurations compared with each partner's operational data assimilation and forecasting systems. There was a sizeable research and development component of WP2 before the production phase, but it has extended more than expected. One cannot lean on and rely on operational developments at the institutes to any high degree.
As recognition of the hard work by the Project partners the European Regional Reanalyses have been continued with the Copernicus Climate Change Service (C3S). URRA was a pre-operational Copernicus project laying the ground for the operational Service. Some of the partners, and some additional ones, are Contractors to carry out both a European-Atlantic and an Arctic reanalysis service. (C3S_322 lot1 and lot2).
3.1 The Met Office Ensemble 4D-VAR Reanalysis
The Met Office reanalysis is an ensemble of independent runs, each assimilating observations in a 4DVar analysis system. The spread of the ensemble will come from several sources of perturbations. The observations are perturbed with random noise according to the expected observation error. The Sea Surface Temperatures (SST) come from an ensemble of analyses, HadISST2 (Titchner and Rayner 2014). Model error is accounted for by the method of Piccolo and Cullen (2015) where the model forecast is perturbed by previous analysis increments, taken to be representative of model error.
It was intended also to drive the ensemble with an ensemble of lateral boundary conditions from the ERA5 global reanalysis. Start of ERA5 production is later than was planned and the full ERA5 dataset is not expected until the end of 2017, too late for this project. Therefore we intend to use the deterministic ERA-Interim to provide boundary conditions.
A suite has been assembled to run on the ECMWF HPC implementing the method above. It includes several developments required for a long-period reanalysis. Satellite radiances are bias-corrected with a variational scheme VarBC, following Dee and Uppala (2009). Soil moisture is adjusted according to differences between forecast and observed screen level temperature and humidity, analysed through an Extended Kalman Filter. This is an adaptation of the scheme used for Met Office global NWP. Observation reject lists will be updated monthly based on O-B (observation minus background) differences from the previous month. Figure 5 shows verification of the ensemble mean for January 2007 against 850hPa radiosonde temperatures, and also the ensemble spread. The ensemble spread is the 'uncertainty'. For a perfect system, the uncertainty should be close to the RMSE of the ensemble mean. It can be seen here that the spread is smaller than the RMSE. This is a common feature of real-world ensembles. It is partly due to the difficulty in representing every source of uncertainty in the ensemble. It is also partly due to errors in the observations. Observation and representivity error (the observation is seeing detail that a 12km model cannot) both contribute to inflating the RMSE.
The principal contributions of the Met Office to the Uncertainties in Regional Reanalysis (UERRA)
project, [Unden et al., 2014], are a deterministic atmospheric reanalysis at 12km and a 20 mem-
ber ensemble atmospheric reanalysis at 36km over Europe for the satellite era (1979-2016).
Production started in January 2017 and completed in January 2018. The develop-
ment of the project is documented in deliverable reports, which are publicly available through
the UERRA website, http://uerra.eu. In D2.1 [Jermey et al., 2015], the reanalysis systems are
documented, together with initial test results and a demonstration of ensemble products. D2.2
[Mahmood et al., 2016], describes use of observations and other data used as boundary condi-
tions for the reanalyses. Initial diagnostics from the ensemble system are presented in D2.3
[Jermey et al., 2016], which demonstrates the validity and quality of the ensemble production
data, as well as reporting on trials of the satellite bias correction method, land surface anal-
ysis scheme and ensemble size. The deterministic reanalysis uses a hybrid approach to data
assimilation, exploiting the ensemble information to improve the reanalysis, as documented in
D2.4 [Jermey et al., 2017b]. D2.4 also contains tuning and initial trial results for the deter-
ministic reanalysis system. An assessment of the quality of the ensemble reanalysis is given in
D2.14 [Jermey et al., 2017a], which includes comparison with the 20th century ensemble reanal-
ysis (CERA-20C), [Laloyaux et al., 2016] from the European Centre for Medium Range Weather
Forecasts (ECMWF) and the University of Bonn/Deutscher Wetterdienst ensemble reanalysis
for UERRA, [Bach et al., 2016].
The reanalyses are driven by advanced data assimilation techniques which produce an estimate
of the atmospheric state, by drawing a background atmospheric state towards a set of observa-
tions. These reanalyses are produced every six hours, with an atmospheric model forecasting
to the next reanalysis time. This (re)forecast is then the background atmospheric state for the
next reanalysis. The reanalyses cover the EUR-11 domain of the Coordinated Regional climate
Downscaling Experiment (CORDEX, [Jacob et al., 2014.
Both reanalyses cover the period 1979-2016 and are free cycling, but are dependent on ERA-
Interim for lateral boundary conditions (LBCs). Each production stream also requires an initial
background to start the cycling process. This data also comes from ERA-Interim. Each stream
has a month (ensemble) or a week (deterministic) of spin-up time from this initial background
before production data is processed and archived. As an example, the ensemble stream of
1997-1999 is spun-up from ERA-Interim data on 1st December 1996 and begins producing and
archiving data on the 1st January 1997.
The observations assimilated by the reanalyses are mostly supplied by European Centre for
Medium Range Weather Forecasts (ECMWF), [Dee et al., 2011]. Prior to 2003 these observa-
tions come from ECMWF’s reanalysis archives and 2003-2016 from operational archives. These
observations include SYNOP stations, ships, buoys, sondes, aircraft and satellite measurements
using advanced infared sounders (AIRS), (Advanced) Television Infrared Observation Satellite
Operational Vertical Sounder ((A)TOVS), scatterometer winds and Infrared Atmospheric Sound-
ing Interferometer (IASI). These are complemented with atmospheric motion vectors (AMV)
from the Met Office and reprocessed global positioning system zenith total delays (GroundGPS),
Further satellite data using GPS radio occulation (GPSRO), Spinning Enhanced Visible
and InfraRed Imager clear sky (SEVIRIclear) and Special Sensor Microwave Imager/Sounder
(SSMIS) was not attempted due to time and resource limitations. A guide to the number of
assimilated observations is given in figure 2. This figure shows that there is a substantial increase
in surface, aircraft and satellite observation volume between 1979 and 2016, but sonde volume
remains relatively steady.
The new Monitoring and Updating Station Lists (MUSLi) system is used to reject and correct
observations based on monthly departure statistics. Corrections are made for surface observa-
tions of pressure and for temperature observations from aircraft and upper air data, [Davie, 2017].
Variational bias correction is applied to the satellite radiances, [Lorenc, 2012, Dee and Uppala, 2009]
This allows for bias corrections to vary with time so as to fit drifts in instrument bias. The
method has been previously well tested for global models and is used here for the first time in a
regional model. High peaking channels are rejected, see [Jermey et al., 2015].
For most of the period, the lower sea boundary is given by version 2 of the Hadley Centre Sea
Ice and Sea Surface Temperature (HadISST2), [Titchner and Rayner, 2014]. This contains a
ten member analysis of sea surface temperatures at 0.25 º consistent with a single deterministic
analysis of sea ice fraction at 1.0 º . This dataset does not cover 2011-2016, so from Septem-
ber 2011, the Met Office Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA),
[Donlon et al., 2012], is used, reduced to the resolution of HadISST2. For this later period, an
ensemble dataset is produced by adding HadISST2 from a random year to the reduced resolution
OSTIA fields. During both periods, the ten member ensemble of sea boundary data is shared
so that each sea boundary member is used by two reanalysis ensemble members.
The land boundary is provided by the Met Office Land Surface Data Assimilation System
(SURF), [Candy, 2014], used in a regional context for the first time. As with the atmospheric
analysis, each member of the ensemble performs its own land surface analysis from a different
realisation of in-situ observations.
In order for the ensemble of reanalyses to represent uncertainty in the reanalysis system, the
sources of uncertainty are perturbed so that each member receives a different realisation. In
addition to different realisations of the boundaries, different realisations of the model and ob-
servations are also employed. The model is perturbed by adding random analysis increments
as tendencies across the forecast. A random analysis increment may be thought of as a proxy
for model error. Each observation is also perturbed by adding a random amount bounded by
the estimated observation error. This follows a system for generating global ensembles used at
the Met Office for research into model error, [Piccolo and Cullen, 2016]. Each ensemble member
is isolated as there is no re-centring of the ensemble at analysis time. An unperturbed control
member is run separately from the ensemble in order to monitor and update observation black-
lists and calculate satellite bias correction at the ensemble resolution.
The ensemble reanalysis was originally intended to use four-dimensional variational assimilation
(4DVAR), [Rawlins et al., 2007]. However this proved to be prohibitively expensive and so three-
dimensional variational assimilation (3DVAR), [Lorenc et al., 2000] is used instead. 3DVAR suf-
fers from spin-up issues for precipitation so a consequence of this change is that the ensemble
precipitation values are far too large. The spread of the precipitation is, however, still useful for
estimating reanalysis uncertainty, [Jermey et al., 2017a].
Variational assimilation (3DVAR, 4DVAR and hybrid 4DVAR) draws the model close to the
observation data, which in the case of the ensemble is perturbed, by minimising a cost function
which penalises distance from observations, distance from the background and high frequency
behaviour. The weights to the observation and background terms are inverses of the estimates of
the error covariances. It produces an increment to the background which estimates the optimum
state of the atmosphere, given the background and observations across a six hour assimilation
window (T-3 to T+3), taking into account the three-dimensional position of each observation.
As is common practice, the assimilation is carried out at half the horizontal resolution of the
model.
The deterministic reanalysis uses hybrid 4DVAR, [Clayton et al., 2013], to assimilate obser-
vations. As with 3DVAR, this is carried out by minimising a cost function, which penalises
distance from observations, distance from the background and high frequency behaviour. (Hy-
brid) 4DVAR also estimates the optimum state of the atmosphere, given the background and
observations across a six hour assimilation window (T-3 to T+3), taking into account the three-
dimensional position of each observation, but also taking into account the validity of each obser-
vation within the window. The use of the fourth dimension leads to analysis increments which
are more in balance with the model than 3DVAR and therefore it does not suffer from precipi-
tation production which is initially too large.
Traditionally the background error covariance is estimated by a smooth parameterised approx-
imation to climatology tuned by forecast differences over a long period. This is sufficiently
accurate to consistently produce an analysis which is closer to the true atmospheric state than
the background. However, this estimation does not vary from cycle to cycle and regions of ac-
tivity in the background are given the same weight as regions of stability.
Hybrid 4DVAR uses a combination of the climatological background error covariance (used for
the ensemble) and a error covariance derived from the ensemble. This combined covariance cre-
ates a low-noise estimation that is dependent on synoptic pattern. This assimilation method has
been used for background error covariance estimation in operational global forecasting at the
Met Office since 2011, [Clayton et al., 2013]. It is used here for the first time in a regional con-
text. A diagram of this is shown in figure 3 and the hybrid settings are detailed in table 3, which
were determined by a combination of experimental tuning, [Jermey et al., 2017b], and in-house
experience. To avoid spuriously large covariances near the top of the atmosphere, climatology
only is used for this region. A middle region features a linear ramp from hybrid covariances to
climatological covariance. Use of the ensemble data ensures that the background error covariance
is synoptically dependent in the deterministic reanalysis.
The same model is used by both the ensemble and the deterministic reanalyses. This is the
Unified Model (UM), [Davies et al., 2005], using the dynamical core currently used for opera-
tional forecasting at the Met Office (Even Newer Dynamics for General atmospheric modelling
of the environment, ENDGame, [Wood et al., 2014]). The reanalyses both use six hour cycling,
following the Met Office operational global forecasting system. A (re)forecast is carried out from
a combination of the analysis increment, provided by the data assimilation, and the previous
forecast (background) from T-3 to at least T+9. The reanalysis fields are output at T+0 (the
centre of the six hour assimilation window) and the background for the next cycle between T+3
and T+9. The model is computed over 63 levels using Charney-Phillips staggering between 10m
above orography to the model top - a fixed radius from the centre of the Earth.
Figure 6 in the Met Office Final Report shows the RMSE and mean error of 6h forecasts from the
reanalysis and ERA-Interim against 2m temperature observations for March 1979, 1988, 1997,
2006 and 2016. This demonstrates that the quality of the representation of 2m temperature in
UERRA-MO is an improvement on ERA-Interim throughout the reanalysis period. This bene-
fit increases with increasing time/observation volume, see table 6, which details the percentage
change in RMSE from ERA-Interim to UERRA-MO. This suggests that higher resolution as-
similation is of greatest benefit when a dense observation network is available. The quality of
both reanalyses improves with time and both reanalysis have low mean error (bias) throughout,
with UERRA-MO slightly warmer than ERA-Interim.
Figure 7 shows the RMSE and mean error of 6h forecasts from the UERRA-MO deterministic
reanalysis and ERA-Interim against 10m wind vector observations for the same periods as figure
6. As with 2m temperature, both reanalyses improve with time/observation volume. Again,
UERRA-MO is an improvement on ERA-Interim throughout the reanalysis period and the im-
provement increases with increasing time/observation volume, see table 6. Both reanalyses show
a small bias, with UERRA-MO consistently less biased than ERA-Interim.
Ensemble spread is intended to represent the uncertainty in the ensemble mean. Figures 14 and
15 show the spread of each ensemble at observation positions for UERRA-MO and CERA-20C.
CERA-20C is a global ten member ensemble reanalysis at 125km and assimilating only surface
pressure, marine winds and ocean profiles, [Laloyaux et al., 2016]. The observations are selected
daily mean temperatures from the ECA&D dataset, [Klein Tank et al., 2002], which are inde-
pendent of the reanalyses. The models feature smoothed orography which may differ greatly from
individual station height. The model temperatures are first corrected to the height of the observa-
tions by applying a lapse rate of 0.0065Km −1 , [International Organization for Standardization,
1975].
In each case the spread is inflated using a multiplicative factor to match an assumed observation
representivity error variance of 0.96 as described in table 7, following [Saetra et al., 2004]. Jan-
uary to March (JFM) is shown for 1987, 1997 and 2007 for both ensemble reanalyses, with 2016
also shown for UERRA-MO (since CERA-20C has no data beyond 2010). These figures show
that, after inflating the ensemble to take account of observation representivity error, the magni-
tude of the spread in UERRA-MO is similar to that of its mean RMSE. However, the same is not
true of that of CERA-20C. The Pearson’s rank correlation coefficient (labelled ‘Prcc’), between
the time series of the spread and the RMSE of the mean, is also higher in UERRA-MO than in
CERA-20C. This indicates that the ensemble of UERRA-MO is able to represent uncertainties
in 2m temperature at least as well as CERA-20C, but at much higher resolution.
Figure 14 also shows that with increasing decade/observation volume, the RMSE of the mean,
the spread and the raw spread are all reduced. The correlation of the spread to the ensemble
mean is also reduced. This may indicate that the RMSE of the mean is becoming dominated
by errors in converting from model space to observation space, e.g. height difference and local
effects, or that its variability is becoming dominated by sub-grid scales that are not well captured
by simple inflation.
The UERRA-MO reanalyses cover 1979-2016 over Europe and the Mediterranean. These include
a lower resolution (36km) 20 member ensemble reanalysis, driven by an ensemble of 3DVAR as-
similations, and a higher resolution (12km) deterministic reanalysis, driven by hybrid 4DVAR
assimilations. The reanalyses assimilate a wide range of conventional and satellite data and
a wide range of consistent atmospheric variables is produced from the surface to a height of
approximately 40km.
Both reanalyses make use of technical innovations. The ensemble uses a random draw of analysis
increments to represent model error. The deterministic reanalysis uses a hybrid 4DVAR which
uses ensemble information to estimate the background error covariances. Both are used for the
first time in a regional context.
The parent model for the UERRA-MO reanalyses is ERA-Interim. The deterministic reanalysis
shows an improvement of representation of most atmospheric variables over ERA-Interim for
this region. This improvement increases with observation volume, indicating that the higher res-
olution hybrid 4DVAR assimilation system makes better use of a dense network. As expected,
larger scale variables are better represented in the global reanalysis.
Due to cost limitations of running a 40 year ensemble reanalysis, 3DVAR instead of 4DVAR
is used to calculate the reanalyses. This restriction has lead to spuriously large precipitation
in the initial hours of the (re)forecast. With 3DVAR it is usual to allow a period of spin-up
before taking production assimilations. This was not done for UERRA-MO, since extending
the (re)forecast period would have increased the cost. Although the subsequent magnitudes of
precipitation in the ensemble reanalysis are very large, the ensemble is still able to capture the
PDFs of observed values with a similar accuracy to CERA-20C, indicating that there is useful
uncertainty information in these fields.
For other fields, the ensemble reanalysis members have realistic representation and are able to
capture uncertainties well, demonstrating some improvement on CERA-20C for capturing the
uncertainty in the ensemble mean with its spread.
Satellite data has a beneficial impact on the ensemble reanalysis, with later periods where more
data is available reaping greater benefits. The effect of the increased volume of satellite data
can be clearly seen. The greatest improvement due to satellite data between 1987 and 2007 is
seen in the temperature and wind speed, especially in the mid and upper troposphere. This
would seem to reflect benefits particularly from the satellite sounding instruments and AMVs.
For 10m wind, the verifying observations are mostly over land, while the scatterometer data is
from over the oceans and so the impact is harder to measure.
These reanalyses provide the community with hourly data across a wide range of atmospheric
variables for 1979-2016. High resolution fields are available from the deterministic reanalysis.
The lower resolution ensemble reanalysis provides users with twenty realisations of the entire
period whose spread is a useful measure of uncertainty in the mean.
3.2 The SMHI ALADIN HARMONIE 3D-VAR Reanalysis
The HARMONIE-ALADIN re-analysis was produced and archived from January 1961 to December 2015 (now also 2016) with a horizontal resolution of 11 km and the ALADIN physics scheme. Both upper air as well as surface data assimilation was included. To introduce large scale information from the global reanalyses a large scale constraint has been added to the cost function.
For the upper air data assimilation only the so called conventional observations are included. This means observations from SYNOP stations, ships, drifting buoys, aircraft observations and temperature soundings. For the surface assimilation temperature and relative humidity at two meters as well as snow water equivalent, all from SYNOP stations are included. In the observation monitoring shown here only the upper air observations are shown.
An observation monitoring system has been partly developed within UERRA. Figure 2 shows the total number of observations used together with the amount of observations from each observation type. Shown are the monthly averages from 1961 to 2015.
As expected the number of observations increases during the re-analysis period. Aircraft observations are not available until 1980 and after that it is constantly increasing, especially at high altitude, i.e. cruising level (not shown). In the 1980s and 1990s all of the aircraft observations were reported manually as AIREP (AIRcraft REPorts) but later more and more are automatic AMDAR (Aircraft Meteorological DAta Relay). The latter together with the increase in air traffic is noticeable not only in the number of observations but also in the distribution of the observations both horizontally and vertically.
There are a number of interesting features in Figure 2 that need further investigation but most strikingly is the dramatic reduction of the number of aircraft observations during 2015. This is due to a change in the BUFR templates in the aircraft reports from December 2014. This has now been taken into account and the last year has been re-run in order to have a complete data set with all available observations.
Another example of the importance of observation monitoring is to check if the assimilation is working properly. This can be done by comparing the first guess (background) and analysis departure, i.e. how much the observations differ from the first guess and from the resulting analysis. If everything is working well the analysis departure should be smaller than the first guess departure. This means that the model has adjusted to the observations. How big this adjustment is will depend on both the background and the observation error. Examples can be found in D2.7 and they show that the model behaves as we would expect.
The results of the HARMONIE-ALADIN re-analysis have been compared results from ERA40 and ERA-Interim. For most parameters the results are very similar although the difference in resolution is obvious giving more details in HARMONIE-ALADIN, especially in areas of steep topography. For precipitation, however, there are larger differences between UERRA compared to ERA-Interim. In HARMONIE-ALADIN there are rather large amounts of precipitation in mountainous areas that is not seen in the corresponding ERA reanalyses. What the reason for this is and which is more correct need to be studied further but it is known from another study, presented in UERRA deliverable D3.6 that UERRA produces exaggerated precipitation amounts in the Alpine region. It has also been seen that the ALADIN scheme in general, produces too much precipitation.
The HARMONIE verification system WebGraF has been used to verify the forecasts for the 55-year period. Due to the amount of data, the number of parameters that were verified had to be reduced. For the surface, wind speed at 10 meters altitude, temperature and dew point at 2 meters, mean sea level pressure, cloud cover and 12 hour accumulated precipitation have been verified. For upper air: geopotential, temperature, wind speed and relative humidity. The forecasts are verified against the same observations that were used for the initial analysis.
In order to summarise the verification results a scorecard has been constructed where a few parameters are subjectively evaluated for each decade and compared with the corresponding verification for ERA40 and ERA-Interim. The scorecard is presented in Figure 3 and it can be seen that for the surface parameters (mean sea level pressure (PMSL), two metre temperature (T2m), wind at ten meters (U10m) and cloud cover (CC)), except the relative humidity (Rh2m), HARMONIE-ALADIN re-analysis performs better (green triangles) or as good as (circles) the ERA re-analyses compared to observations when it comes to standard deviation (STDV). For the profiles of temperature (Temp), wind speed (WS) and geopotential height (Geop), i.e. higher altitudes the results are more mixed except for the relative humidity (RH) where ERA is better for all periods.
As users have started to use the HARMONIE-ALADIN data a few erroneous fields has been discovered. Some we are able to correct and re-archive while some we are not able to correct without having to rerun the entire re-analysis. The re-archiving wouldl take a very long time since retrieving the data is a rather slow process so in the mean time, and for the non-correctable data, a document is provided giving instructions to the users of what data to be careful with.
3.3 The Météo-France downscaling reanalysis
The configuration of the ensemble system MESCAN-SURFEX-ENS is based on 8 members obtained with several combinations of temperature analysis, precipitation analysis and two types of radiative fluxes and wind speed from the ALARO and ALADIN physics. The production of the 8 members for the period 2006-2010 took more time than estimated due to the hourly output and the large number of SURFEX variables especially because of the 14 soil layers. Finally, the MESCAN-SURFEX-ENS production was available on the MARS system (test) in two steps: first the MESCAN 8 members was available in March 2017 and used by the WP3 for evaluation. Then, in a second step, the SURFEX output such as snow cover, soil temperature and soil moisture (14 layers) were available on MARS by the end of June 2017. One of the main difficulties was to transform the native output SURFEX NetCdf files into GRIB2 and to define the new soil layers and variables in GRIB2 format from SURFEX. This last point required a lot of exchange with ECMWF and the validation of the surface and soil data in GRIB2 took a lot of time. All those problems partly explain the reason for a MESCAN-SURFEX-ENS delayed report (D2.9) in addition to a tragic unexpected event in the MF team. Finally, the deliverable D2.9 has been published in June 2017.
Some preliminary evaluation done by WP3 on the precipitation analysis has shown (GA in November 2016) a tendency to underestimate the number of “dry days” in the MESCAN precipitation analysis. After investigations and many tests it was decided to use a varying σο, instead of a constant one, to better fit the observations in case of weak rainfall :
σο=0.001mm when RR_obs=0.mm ; σο=0.7 + RR_obs * 0.1 when RR_obs < 50mm and σο=5,7mm for RR_obs > 50mm
The “final” 55 years production began in January 2017 with 4 parallel streams: 1961-1965, 1965-1970, 1981-1985 and 2005-2010, the last two periods were available in March 2017 for the WP3 and the evaluation. Unfortunately for those periods no additional surface data such as 2m-temperature or 24h-precipitation recovered by WP1 have been used in the MESCAN analysis. The MESCAN surface analysis is now finished and is available on the MARS archive (under prod) at the end of November 2017. Figure 4 shows the 2m mean temperature (blue line) over Scandinavia (top) and Pyrénées (bottom), the green lines are all the members from the MESCAN-SURFEX-ENS which was only available for 2006-2010.
For the SURFEX-TRIP part (driven by the MESCAN analysis) it was not possible to run several parallel periods due to the long-time spin-up of the deep soil moisture in some parts of Europe. Consequently, it was decided to run continuously SURFEX-TRIP from 1961 to 2015. The main drawback of this decision is the delay of the complete final production such as MESCAN and SURFEX.. The end of the SURFEX-TRIP is planned for middle December, nevertheless we have already started to archive the SURFEX output in MARS (under prod). The period 1961-2015 is now available.
The precipitation analysis requires more verification than the 2m-temperature to avoid some spurious features due to “bad” observations not rejected by the automatic quality control (Fig 5 a, left), the only way to solve this problem is a manual correction and then re-run the MESCAN analysis. The observations used for the precipitation analysis come from several database: MARS, ECA&D, French, Swedish and Norwegian national databases. Very often, the “bad” observations come from the MARS archive, it is not really surprising because the precipitation observations are not used (as an input) by ECMWF in the re-analysis and in the operational system, and therefore the quality control is probably not done.
As shown for temperature in Figure 4, Figure 5 (below) shows the annual accumulated rainfall over the Alps. The spread of the ensemble is rather large for this region, which makes sense due to the uncertainties of rain and snow fall observations in mountainous area.
Precipitation is one of the most essential variables, together with temperature, to drive a “hydrological model” for water resource management. Figure 6 shows an example of the river discharge computed by MESCAN-SURFEX-TRIP (blue line) for the Danube river, since 1961 up to 1976. The main evolution of the river discharge is well captured by the MESCAN-SURFEX-TRIP system compared to the observation (red line).
3.4 Cloud Fraction reanalysis (SMHI)
The objective of this task is to provide a 2D optimal interpolation (OI) analysis of hourly cloud fraction for 30 years (1982-2013) at 5.5 km resolution. The idea is to combine, on a common grid, good quality CM-SAF datasets from geostationary METEOSAT and AVHRR polar platforms and to use NWP reanalysis data for gap-filling.
During the start up of this task it became clear that the CM-SAF is about to start processing a new homogeneous cloud fractional cover product for the entire METEOSAT period (1982 onwards) and that the whole 30+ year period was finalized only by the end of 2016. This should probably result in a significantly improved cloud product to what is now available for the pre SEVIRI (MSG) era. Because of this it was decided to wait for this dataset to become available and focus on the SEVIRI era, where high quality data is readily available, to begin with.
The OI analysis needs estimates of the first guess and observation error. In this case both of these are spatially correlated and the method by Desrozier et. al was used to estimate the error matrices based on forecast difference statistics from the HIRLAM EURO4M cloud cover. The method converges nicely to realistic estimates and by carrying out the OI analysis in Fourier space, where the error matrices are diagonal, the analysis can be done in an efficient way.
In areas with missing satellite data EURO4M HIRLAM data is used as a gap filler since complete fields are needed in order to use the Fourier transform and get diagonal error matrices. This can result in discontinuities in areas with missing data where HIRLAM and the satellites disagree. To reduce these effects we tried to add a weight matrix to the OI formulation and pay less attention to observations in areas where these are actually just NWP data (low weights for NWP). Unfortunately the minimization problem then became much more demanding (non-diagonal matrices). As a remedy a fast adaptive filtering approach is now used where information from neighbouring observations and NWP forecasts is combined according in order to get a smooth transition from the edge to the centre of a missing data region.
3.5 Ensemble Nudging Data Assimilation Reanalysis (University of Bonn)
UB's task as part of WP2 in UERRA is to provide a regional ensemble reanalysis system as well as a proof of concept high-resolution data set for Europe. In D2.11 a technique called ensemble nudging to perturb observations in order to account for observational uncertainty in an ensemble has been introduced. This had been planned to be part of a hybrid technique in combination with a local ensemble transform Kalman filter (LETKF) that is newly developed at Deutscher Wetterdienst. However, due to a couple of reasons that are detailed in D2.12 as well as technical issues that were solved later, but still left inherent problems, the hybrid combination turned out to be of limited usefulness for the production of a comprehensive dataset as an ensemble reanalysis. Moreover, the absence of the PI of the UB work due to parental leave from 2016/01 to 2016/12 led to an agreement with the UERRA management team that the originally intended hybrid system would be replaced by the ensemble nudging component as data assimilation system for the UB reanalysis. The usefulness of this technique is comprehensively shown in D2.12.
The reanalysis has been carefully monitored for observation statistics and departures against the data. The reanalysis was seen to be stable throughout the period and the results have been documented in the diagnostics report D2.13 delivered at the very end of 2017. Moreover, the reanalysis and particularly the ensemble performance has been compared and documented in the RA uncertainty evaluation by the Met Office (Peter Jermey et al.) (D2.13 in November 2017, see also below).
Concerning the diagnostics of uncertainty estimation capabilities that are under task T2.5 a range of knowledge and codes have been assembled during the course of the project. Some of these are shown in D2.12 and Bach et. al, 2016. These are also useful for task T2.6 a comparative study of the UERRA ensemble reanalyses.
3.6 Reanalysis Cross Evaluation
This work was done once production data is available from all the reanalyses started to become available. A comprehensive work and report was made by Peter Jermey (Met Office) with participation from the other RRA producers (D 2.14). Particularly the ensemble quality has been validated and there is considerable information in the spread and particularly the COSMO nudging ensemble shows good ensemble properties e.g. in terms of Rank histograms (see the report).
4 Assessment of uncertainties by evaluation against independent observational datasets
In collaboration with WP8, we managed to maintain a clear user focus in WP3, in line with the WP3 objective ‘common evaluation procedure for ECVs, derived climate indicators, extremes and scales of variability that are of particular interest to users’.
The work was achieved under the lead of DWD and with the contribution of all the partners (MET Norway, Meteo-Swiss, KNMI and UEA) which acted very flexibly, although only little time was left due to the delay in the RA production. In hindsight, it was most helpful that activities were at full speed already because of the preliminary analysis (D3.5) where input was prepared, scripts and programming was set up, user interest appropriately catered for, and first experiences could be gained. The experience from the preliminary exercise resulted also in a realistic estimate of archive extraction and data handling time, and a sharpened focus for the actual scientific evaluation (D3.6). Development of new methods were pursued independently of the delay (e.g. evaluation methods for the independent probabilistic Alpine dataset).
In line with planned resources, the work focussed on the assessment of uncertainties over the European domain and on sub-regions of particular interest. Satellite data (radiation products) by CM SAF were pre-processed in order to be used for further ongoing work. Gridded data sets were prepared, with the focus of improving the representation of precipitation extremes. In addition, uncertainty estimates have been developed based on the spread using an ensemble of gridded reconstructions (rather than the single uncertainty estimate).
In T1.3 (WP1), EDI has developed a new method to quantify uncertainties in observation-based gridded datasets of precipitation. The results have been verified in detail in the Alpine region demonstrating their reliability and consistency. The method was employed to derive a multi-year ensemble dataset of daily precipitation over the entire Alpine region using high-resolution rain gauge data
A first release of the Nordic Gridded Climate Dataset (NGCD) has been established. It is a high-resolution gridded dataset for daily mean temperature and daily accumulated precipitation based on measurements from weather stations covering Fennoscandia. It was used as a reference dataset for reanalysis evaluation in that region
Speedy production of results in the final months of the project was enabled by common evaluation procedures that had been agreed early on (see D3.2). These cover derived climate indicators, extremes and scales of variability that are of particular interest to users. Appropriate considerations of users was ensured by the interaction with WP8. It proved useful that the two main participants of WP3 were also heavily involved in WP8 and WP7, naturally leading to that evaluation was kept relevant to user interest.
The focus of the WP3 evaluation has been on: precipitation (EDI, MI, KNMI), drought (NMA-RO), wind speed (DWD), radiation (DWD), temperature (KNMI) and climate indices (KNMI), at the hourly, daily, monthly and inter-annual scale.
All data sets produced in WP2 had been included in the evaluation, and all showed good results, the differences depend on region, parameter, and evaluation score. The evaluation was mainly performed against station data (by DWD, KNMI), and gridded station data (by KNMI, EDI, MI, NMA-RO). Further, evaluation with probabilistic ensembles of gridded data (by EDI, MI) and with satellite data sets from EUMETSAT’s CM SAF (by DWD) has been performed. MO contributed with know-how on NWP evaluation and URV and also EDI with methodological advice on evaluation scores.
One lesson of the WP3 work was that evaluation results vary strongly with region. Thus it was fortunate there were, in addition to the whole of Europe, three sub-regions (Fennoscandia, Alps, and Germany) which were analysed in more detail. It highlighted also for the users that results cannot be generalized and performance of the various models could depend on the region they were optimized for.
The results of WP3 were always communicated without time delay to the WP2 producers, firstly to inform them about any irregularities found, and second, for scientific discussion on the causes of the evaluation results. This proved beneficial to both WP3 and WP2.Particularly comparing the results of D3.6 and D2.14 which were done independently, allowed a check on our results and enhanced interpretation which could be relayed to the users.
However, for task T3.2 a number of satellite products were originally envisaged to contribute to the evaluation, but all were dismissed as they were not appropriate in terms of quality and resolution to serve as “truth” for the regional reanalysis, as the latter have higher spatial and temporal resolution as well as higher accuracy in the parameter space. Even for radiation, estimation from the regional reanalysis were found superior (with respect to locally and annually varying bias) to the satellite data, as was shown by DWD with a third, high quality reference data set (HelioMont) over the Alpine region.
In close collaboration with MI, EDI compared several datasets produced in WP1 and WP2 as well as several existing gridded products well known in the climate community. EDI focussed on daily precipitation (06-06UTC) for the Alpine Region using APGD (Alpine Precipitation Gridded Dataset) as reference. The reference datasets were available in two versions: a deterministic one, developed in the FP7 EURO4M project, and a probabilistic version, described in this document from the contributions of EDI for WP1.
For the Alpine region the following datasets were evaluated: COSMO reanalyses (two datasets, a deterministic and an ensemble reanalysis, produced by DWD and UniBonn), HARMONIE (with a second version over a short time period with modified model physics, SHMI), MESCAN (six additional versions over a short time period with modified model physics and input data, Météo-France), Met Office reanalyses (two datasets, a deterministic and an ensemble reanalysis, Met Office) and additionally ERA-Interim (ECMWF), ERA20C (ECMWF), E-OBS and MESAN (EURO4M dataset).
The dataset preparation was time demanding (download from MARS Archive, conversion from GRIB2 to NetCDF, conversion between coordinate systems and aggregation to daily 06h-06h precipitation). The evaluation was mainly performed with data from the relatively short period 2006-2008, which corresponds to the time interval when all datasets of interest where available. Two grids were defined and the datasets accordingly rescaled: a coarse-resolution grid (0.25 degrees regular grid) and a fine-resolution grid of 5 Km.
In the evaluation, we have examined climate indices (such as mean annual precipitation, wet-day frequency and the 95% quantile), performance scores (such as the Root Mean Squared Error and Brier Skill Score), and the annual cycle and distribution function at various scales (such as the scale of grid pixels and hydrological catchments). Figure 9 shows, as an example, the mean annual precipitation of nine different datasets.
Some emphasis in the evaluation was placed on uncertainties in regional reanalyses and their dependence on the space scale. The scale dependency was derived by comparing the performance for different size classes of a hierarchical subdivision into river catchments. Despite the short period available for comparison, our study could identify some major differences between the datasets. Even though the statistical significance and level of detail may be somewhat limited, particularly so for extremes, this analyses points to features that are of interest to the developers of the UERRA regional reanalyses and relevant to users of these datasets. The results of our analyses are described in detail in deliverable 3.6 (report).
KNMI has compared regional reanalyses from WP2 with the E-OBS dataset from WP1. Due to the delayed archiving in MARS and the slow performance of retrieving the data from MARS and transferring it to KNMI limited the evaluation to only a few common years. For the ensemble reanalyses only 1 year could be used due to the limited amount of time between the archiving in MARS and the deadline for Deliverable D3.6. Climate indices based on daily minimum and maximum temperature were used for the evaluation of the reanalyses and E-OBS. A general observation is that the similarity between reanalysis and observations is good. While there are marked differences between reanalysis, these variations are generally rather small. We did find systematic biases of one reanalysis over parts of eastern Europe but this was ‘compensated’ by a spot-on resemblance with observations over other regions.
A recurring problem (also for the indices work in WP4) is the interaction with the MARS archive. Extracting data from MARS is often tedious and slow. A considerable amount of time was wasted on data recovery from MARS which turned out to incomplete, and therefore useless for further processing (like aggregation). Another issue which slowed progress was the lack of a standard conversion between GRIB and NetCDF. While some software was shared between institutes, the correct configuration of this software was not always clear.
While the MARS archive is valuable, a more efficient and direct data exchange between the producers of reanalyses and the limited amount of users (within the project) would have facilitated this work. Furthermore, putting the obligation to provide software to transfer data from GRIB to NetCDF (or the provision of data in both formats) to the data producers rather than the data users would increase efficiency. An example script to extract e.g. hourly (minimum) temperature values would have been useful as well.
The workshop on the synthesis of evaluation results was held 30 November – 1 December 2017 together with the concluding user workshop (from WP8). The user friendly synthesis report will be written after these workshops.
NMA Romania has assessed UERRA RRA data over Romania for precipitation against the ROCADA gridded data set. A good agreement in spatial structure was found but the Met Office RRA had a clear overestimation of the amounts. It also correlates well on the temporal scale, daily as well as monthly variations.
5 Facilitating downstram services (data, derived products and outreach)
The work on UERRA data portal and all related tools for data processing and archiving in MARS at ECMWF has continued and required a lot of effort but is now completed. See the UERRA web and the links to ECMWF wiki for detailed descriptions.
Important lesson learnt from the project beginning is how much time and effort cost to define all data standards and exact project requirements before any actual archiving work could start. With so many varying and heterogeneous re-analysis systems participating in the project it took first two years to identify fully all details specific to each datasets and to prepare the final list of all UERRA parameters including all necessary metadata (output steps, levels etc). The parallel work on defining common standards, based on WMO compliant GRIB2 format, meant to introduce some new parameters and concepts (e.g. new soil level type) not existing before. The work on common data standards is crucial for similar projects’ success as it allows unified access to the data and its inter-comparison.
Another longer then expected step was to prepare full data samples from each provider fully compliant with agreed requirements. Sometimes only during this step new challenges were identified impacting the decisions taken before. Once finally all full samples were gathered only then the final MARS design could be finalized. The test and following production archiving could be considered as rather simpler final step, still requiring a lot of effort and time due to the processed data volume (almost one petabyte).
An ESGF node of the type “data-node only” was installed at KNMI (https://esgf.knmi.nl/thredds). A dataset with the deterministic forecast analysis for 2-meter temperatures from the SMHI model was created for the period 2005-2010, by extracting the GRIB data from the ECMWF MARS archive and converting the GRIB data files to a NetCDF dataset with FIMEX (http://fimex.met.no). Then metadata (according to the ClipC DRS) was added to the NetCDF dataset and the resulting dataset was published on the ESGF node.
For the evaluation work from WP3, KNMI already calculated a few temperature indices on specific years of the available reanalyses datasets and E-OBS. Work is ongoing to run the indices on a larger number of years, but the retrieval from MARS and transfer to KNMI is slow (see also the part under WP3), partly due to the large size of the datasets. The focus is on 2006-2010 for all datasets, but there might be constraints due to the time available to the end of the year and disc space needed to store the raw data and derived products. Unfortunately, trend calculations do not seem realistic with the limited time period that is available for the reanalyses. Also part of the indices cannot be determined since these will need a 30 year reference period for each dataset which is not available in the archive.
SMHI has resumed its hydrological evaluation of the Regional reanalyses for as long time periods as available. See D 4.7.
Potential Impact:
1 Outreach activities
UERRA has had contacts and exchange with other SPACE and H2020 Projects for and at the Uncertainty Workshop in September 2016 (see above under Goals WP9). There have been general outreach activities via the web site, conferences and national contacts with users of climate and climate change data.
Two Newsletters have been published on the web and linked information has been sent to contacts.
Information / Project Profile leaflets have been prepared with two Magazines targeting policy makers and the EU commission and bodies. Parliament and Adjacent Government. The first was published in March (see https://www.theparliamentmagazine.eu/printpdf/3943 https://www.theparliamentmagazine.eu/articles/magazines/issue-429-07-march-2016)
The second one is in http://www.adjacentgovernment.co.uk/wp-content/uploads/2016/08/AG11-web-smaller-final.pdf#page=210&zoom=auto,-12,365
A third Newsletter was been published on the web and linked information has been sent to contacts.
.
http://uerra.eu/component/dpattachments/?task=attachment.download&id=176 and was quite nice and received attention judging at least from phone calls. The second will be published in August.The Project has been presented at several conferences by scientists in the project, Work package leaders and the Coordinator.
In particular at the European Meteorological Society/European Conference on applied Climatology/Meteorology in Sofia, September 2015, where two talks about UERRA in the context of Climate Services and of Reanalysis were given by the Coordinator jointly with the UERRA WP leaders. The were then the following European Meteorological Society/European Conferences on applied Climatology/Meteorology in Trieste in October 2016 and Dublin in October 2017. There where two talks about UERRA in the context of Climate Services and of Reanalysis were given by the Coordinator jointly with the UERRA WP leaders plus a few other UERRA or UERRA related talks.
See also a list of Conference presentations below, after the References (NB a selected list and it cannot claim to be complete as more UERRA scientists have given presentations in other forums and with some UERRA material included.
DWD contributed to the WP7 objectives by developing training material, particular to enable new users to start handling the reanalysis fields and extract areas of interest, to start plotting and to calculate means.
The KNMI contribution to a workshop (organized outside UERRA) in Africa for GFCS outreach was delayed due to the lack of a suitable workshop. An abstract has been accepted for the Climate
change in Africa: Evidence, mechanisms and Impacts, Past and Present workshop in Marrakesh, Morocco on 6-11 November 2017 and presented at that Workshop.
As part of Workpackage 8 (User Feedback) of UERRA two user workshops are planned: one half way, and one at the end of the project. Goals and set-up of both workshops differ, since at the time of the first workshop the UERRA data products will not yet be delivered, while at the second the final products and services will be presented to the users.
The 1st workshop which was held after the 3rd General Assembly of the project in Toulouse, France.
For the workshop 48 participants from 12 countries (18 from France) were registered. Of this group, 19 were working on applications of re-analyses data and 10 were not related to UERRA.
Participants came from a wide range of sectors. Applications (potentially) using re-analysis data that were presented at the workshop were: Energy (wind, solar, demand), Insurance, Transport, Agriculture, Defense, Hydrology, Climate Impacts, Model evaluation, and Atmospheric Physics.
The participants were asked to very briefly present their interest in re-analyses, examples of use, requirements, etcetera. This was done in the first part of the meeting.
A significant amount of time was allocated to discuss user requirements for data as well as for scientific and technological support. Although a wide range of sectors were represented in the meeting, it was a clear advantage that all users either already worked with re-analyses data, or were interested to do so in the near future. About half of the participating users worked with re-analyses data before. Several examples were shown of the use of ERA-40, ERA-Interim, or analyses made for operational weather forecasts. Because of the need for very detailed information (often related to extreme situations) several participants used some form of statistical downscaling to get to the scale relevant for their specific applications. Although it is clear that not all requirements can be met with the UERRA products, the discussion on the future data products did not lead to significant changes in the list of products to be produced by UERRA. Items not on this list, but considered useful by some participants were wind gusts at 100m height and CAPE (Convective Available Potential Energy) .
A short summary of the findings of the user presentations are listed in a table in the Workshop Report (D8.2).
A second workshop involving external climate service providers was organised by KNMI and other UERRA partners together with the evaluation workshop back-to-back with the UERRA general assembly (30 November 2017 – 1 December 2017). A user guidance was written using the results from the workshop.
DWD contributed to WP8 with substantial material for D8.1. DWD co-organised both user workshops and wrote the summary for the user requirements from the first workshop, ensuring consideration of it in WP3.
2 The multidude of Reanalyses
The multitude of RA datasets at high resolution will be a reason for extensive use of the datasets in climate change research. One important user group is the climate modeling community. The development of increasingly sophisticated climate models has reinforced the need for basic observations. Reference data for the present and recent past climate (including climate trends) during the last 30-50 years is an established requirement for all climate model evaluation exercises. The UERRA reanalyses also form the necessary baseline for future projections of climate change.
In theory, the analysis state at a particular time in any of the RAs is really only one realization of all possible atmospheric states in a probability distribution an ensemble of multiple analyses should be used. Moreover, this one realization might be affected by NWP model bias to some extent. Reducing Regional Climate Model (RCM) bias against one dataset is sub-optimal (see the ongoing FP7 EMBRACE project). Therefore, regional climate modellers want to compare their present-day climate simulations against as many different observational datasets as possible. At present, most of these are from satellites or gridded station observations at relatively coarse resolution. If UERRA is approved, there will be a multitude of new datasets at the end of the project that fit the needs of the RCM community much better than anything else available today.
For the full 50 years there are two different datasets (MESCAN and HARMONIE). For the satellite era from 1978 there will be two similar high resolution datasets produced with quite different modelling and analysis systems (HARMONIE and Unified Model). Additionally, there is a third alternative, the 5 km downscaling RA with MESCAN, again different to the first two, and even more different due to the different methods and observations (as explained in 1.2).
A 4th resp. 3rd (for 30 or 50 years) additional dataset will be provided by DWD and HErZ to the project to aid in the uncertainty estimation, and when the quality of the COSMO RA is evaluated in WP3, the RCM community will be recommended to also use this RA.
The additional RAs developed in the project (with different physics, satellites or not, and the KFENDA from UB) are probably too short to be of major interest for this community. However, the UB ensemble RA is developed with the proposed support from UERRA and is expected and planned by the HErZ and UB to continue this work to a multi-decadal RA as well, providing an additional deterministic control (central member in the ensemble).
3 Uncertainty estimates
A novel expected impact will come from the delivered uncertainties. Since we usually do not have uncertainty measures associated with most of today's observational datasets, it is somewhat speculative to predict how much the user community will make use of them. In general though, the notion of uncertainties and probabilities has won a lot of ground in hydrological and meteorological forecasting through ensemble techniques. The probabilistic approach requires some years to become accepted by users. For use in applications the probabilistic approach requires that ECV probabilities are translated to used impact parameters. For research users in the RCM community, this process will be easier than for the downstream users of RAs. The ensemble spread based RA uncertainties are expected to be quite widely used for model evaluation. They provide an indication for the relationship between model errors and the errors of the verifying RA dataset. In fact all model validation against any gridded data will become more stringent when the probability distribution function (PDF) of the gridded dataset is taken into account (in addition to the PDF of the model data). Since uncertainties from the ensemble RA will be produced for all modeled variables (both surface and upper air) and at all model time steps (as often as archived, for RA usually every 6 hours) the error bars are complete for all the model variables of the RCM undergoing the test. Uncertainty estimates are a definite advantage to the modelling scientist. However, also most downstream user products will benefit from uncertainty information, e.g. a customer or a journalist may want to know the reliability of analyzed trends in temperature, cirrus clouds, or in the occurrence of condensation trace from aircraft.
For a selected set of ECVs of particular user interest, to be established early in WP3, the most qualified uncertainty estimation will be made against several quite different datasets in WP3. The verifying observation datasets are state-of-the-art (including E-OBS and CRU which are enhanced with their error bars in WP1) and used to validate the climate quality of the RA datasets. Also the satellite derived datasets of e.g. cloudiness selected for RA evaluation are the best available for EUROPE (developed as part of CM-SAF). The uncertainties for climate time scales and a number of space scales will add credential to the Climate Indicators that will be derived from the RA and provided to downstream users and policy makers. The uncertainties will be as comprehensive as is possible for anyone to derive for a long time to come. To be able to answer the questions from policy makers about the reliability of the RA and the derived indicators this information is extremely important and will widen the use of the RA data.
UERRA must interface directly with the full range of intermediate- and end-user requirements, including disaster prevention, health, energy, water resources, ecosystems, forestry agriculture, transport, tourism and biodiversity. Developing user friendly data services is the most important step to making the long-term high resolution climate datasets and derived products available to the different user communities. Building on existing expertise, UERRA will develop these services by implementing existing techniques and extending these for RA data and products. We will start from the services built for the MARS archives in the TIGGE/LAM project at ECMWF and extend these to include the EURO4M data and link to the services for the THREDDS archives of climate model data developed at KNMI. The concepts are proven but there will be still significant work to apply them and enhance the services further to include RA data. These data services will for a start be used between the UERRA partners; WP2 will archive the data and WP4 will provide the data services for the evaluation of uncertainties in WP3. Subsequently, the RA data will be made available to outside users. It is anticipated that the consortium in CLIPC will develop data services for observational data coming out of UERRA (and its predecessor EURO4M) too. Therefore, the synergy effects of the two projects will be exploited. Data services for EURO4M data will facilitate the developments in UERRA WP3, since they are likely to be available before the main UERRA RA have been produced.
The observational datasets will substantially support (in combination with climate model predictions) climate change impact and adaptation action assessments, policy development and policy monitoring for European and national users. UERRA will complement the national gridded datasets, where they exist, in terms of being European wide and based on beyond state-of-the-art RA systems. For non-experts in geophysics and for policy makers the ECVs will be translated into climate indicator information adapted to user needs. As an important source of reliable information about the state of the climate in Europe, the suggested collaborative project is an important building block for Copernicus. The RAs will provide basic information for climate services that deliver consistent products and support downstream applications. We will assess whether the ensemble of reanalyses developed in this project is fit for generating climate change products that meet the user needs. The goal is to explore how the reanalysis data (and associated uncertainty information) are best exploited to develop user oriented products such as derived climate indicators and how best to utilize this additional information for “understanding past climates and climate change in Europe”. This activity will assure that the RA datasets and products are useful for policy-makers, planners and the citizens and their organizations. It will provide scientific input for policy implementation.
National decision makers and local authorities will be able to utilise the state–of–the–art UERRA data products and services for their country or region as input to climate change assessments, and the formulation of adaptation and mitigation strategies. It is the longer multi-decadal time scale addressed in UERRA that is needed for governments to minimize and adapt to the societal and environmental impacts of climate variability and change. European countries can directly use the results of the proposed project for their “national communications on climate change policies” which are a written requirement for the Conference of the Parties of the UNFCCC and include national GCOS implementation activities.
The processing systems set up in UERRA for reanalysis, post-processing of ensembles, data services and for products will be built up to a much higher level than available now. The demands for efficient production of the heavy computational runs, archiving of large amounts of data and user friendly and efficient data dissemination are high. It will be necessary to enhance the existing infrastructure for particular aspects of RA and data dissemination. This means that the Consortium will be in a good position to continue in real time the RA after UERRA finishes, provided that subsequent funding can be organized. Most likely there will be a need for re-doing the RA once NWP models and RA systems have developed further and/or additional input data have become available. Also, more computer power will enable higher resolution and/or longer time periods in future. As such, UERRA has the potential to evolve into a future Copernicus service on climate change monitoring that is fully complimentary and supporting the existing operational services.
List of Websites:
http://www.uerra.eu/
Per Undén
SMHI
60123 Norrköping
SWEDED
Per.Unden@smhi.se Tel. +46-11-4958449 or 4958000
UERRA built on from its predecessor EURO4M project and both included observational gridding data sets and NWP model based Regional RAs together with extensive validation efforts against independent data sets.
The Data Rescue of historical observational data and their quality control and development has continued in UERRA but focussed on sub-daily data and more after the 1950s than before. Over 8 Million data values were digitised from journals on paper or in scanned form. Other openly available data have been assembled (112 Million) and provided to international data centres and to RAs.
The 35+ and 55+ years of European Reanalyses have been produced with the Met Office Unified Model 4D-VAR ensemble system and the HARMONIE ALADIN system at SMHI, respectively, and with 70 or 65 levels. A five year period has been assimilated with 20 ensemble members with the COSMO system at University of Bonn, all at 11 or 12 km horizontal resolution (except the Met Office ensembles at 36 km). A high resolution 5 km 2 m temperature and humidity and precipitation and soil analysis was produced at Météo-France coupled to the SMHI RA and with an ensemble for 5 of the years.
An extensive evaluation activity with a common software package was carried out by DWD, Meteo-Swiss and MET Norway and KNMI for climate indices. Also other RAs were compared like the ERA-Interim and the COSMO 5 km at DWD. The quality of the reanalyses was generally good and good benefits could be seen from the high resolution. Over the Alpine and Scandinavian mountains there were quite big differences in precipitation and mostly overestimations compared with high resolution gridded data sets. Moreover were the different models and the ensemble members not enough spread to fall within the catchment estimates from high resolution data at Meteo-Swiss even though the spatial patterns were good. This was also seen at NMA-Romania where also precipitation data have been digitised in the Project.
The Regional Reanalysis have produced large data amounts, almost 1 Petabyte which has been archived in a common UERRA archive in MARS at ECMWF. The data have been defined and converted to WMO GRIB2 with several new features like new variables and levels at fixed heights and in the ground. All the data are in MARS and common access methods can be employed and multiple tools for handling the data are available and to the whole community. A subset of the data has also been stored on an ESGF node as a demonstration for the general climate community.
Project Context and Objectives:
There is a wide demand for gridded high quality gridded data sets of Essential Climate Variables (ECVs) and additional derived environmental variables on the same grid with a high resolution in time and covering climatic multi-decadal time scales.
Data sets may be constructed from observations only using statistically based interpolators to regular grid points in space. On the global scale the resolution is relatively coarse, usually about 50 km and on the European scale about 25 km. Sub-regional or national observational data sets have been constructed at much higher resolutions, between 10 and 2 or even 1 km. The data are the observed surface variables of temperature, humidity and precipitation. These data sets are limited to certain regions and there is no general pan-European data set at very high resolution. The national or sub-regional data sets serve important national needs in climatology for one own national responsibility but no general one for Europe. Other observational data are constructed from satellite and other remote sensing data and may then include radiation related variable including clouds and surface properties.
The other type of data sets are the Numerical Weather Prediction (NWP) model based reanalyses of observations also on a regular horizontal grid but also adding the vertical dimension with many levels in the atmosphere and with a comprehensive set of model variables that are internally consistent. The Reanalyses (RAs) use as wide selection of observations as possible and combines them with the model evolution in space and time in an optimal way.
Global RAs have earned a high reputation and become widely used for many applications. National and regional applications often demand a locally higher resolution than the global ones and downscaling from about 50 km to 20-10 km or better may be done for these purposes.
The requirements for regional European RAs were identified by the European Commission and the Scientific community and in the FP7 SPACE programme 5 pre-operational Copernicus projects were launched and one of the was for Ensembles of European Reanalyses which then materialised as the UERRA project. Uncertainty estimation was also an important requirement and therefore a multitude of methods and models were employed.
UERRA was set up to answer to the need of this pan-European approach of high resolution climate quality reanalyses for many general purposes over the whole domain as well as over owns own territory.
UERRA built on from its predecessor EURO4M project and both included observational gridding data sets and NWP model based Regional RAs together with extensive validation efforts against independent data sets. The RRAs have a more dominant role in UERRA and most of the resources were allocated to the development and production of three different ensemble model based 3 and 4-dimensional ensemble reanalyses (with time being the 4th) and also an ensemble surface RA. Ensemble means that a multitude of RRAs were produced that were slightly different due to inherent uncertainties in the observations and models and the chaotic nature. Also the observation gridding uncertainties have been estimated with newly developed methods in the Project.
The time period in UERRA is long enough to cover climatological needs (55+ years or 35+ years for the two main RAs in the Project) and there is a data services part of the Project so that access to the data is easily provided in a uniform way to any user.
A new aspect in UERRA is the requirement of providing not only the best estimate of the Essential Climate Variables but also the associated uncertainties. Evaluation of RAs has been done before and has also been done in UERRA and this provides a part of uncertainty but far from the full answer. The uncertainties cannot be universally defined but depend on the scales in time and space for the applications. Then the reference data to be compared with also have their uncertainty and the data are limited to certain time and space scales, like how often in time and how dense the observations were made.
Also the model ensemble based estimates have limitations in that they provide spread using all the same model and all members have certain inherent common errors that those ensembles cannot measure. Therefore it was deemed important in UERRA to employ several different model and data assimilation systems and even observation gridding system to provide more spread due to the different models used (multi-model approach).
To answer the need of very detailed near surface analyses of certain ECVs and provide products which are closer to observations than the full NWP model RAs, UERRA has also a 2-dimensional downscaling reanalysis at higher resolution (5 km) than the 3-dimensional upper air RRAs. This is akin to many national observation interpolation products employed in the NMSs but is pan-European and gives a high resolution 2m temperature, humidity and (24 hour) precipitation analysis. In contrast to the pure observation gridded products, the UERRA surface analyses still uses NWP based RRA as an input, but it is not restricted by the 3-dimensional consistency of the full upper air RRA. The limitation of the product is the density of observations which is variable over the continent due to data polices or data rescue or transfer of data.
From the DoW (Annex I, Part B), the following objectives for the project are defined.
The overriding objective is to produce long-term high-resolution climate quality datasets over Europe complete with estimations of their quality and uncertainty (WP1, 2, 3 and 4)
To produce these through 3 and 4-dimensional reanalyses (RA) and 2-dimensional downscaling RA and extended observation gridded datasets (WP2 and WP1)
To estimate the uncertainty of the individual RA through ensemble data assimilation for Europe and produce a high-resolution ensemble RA for as long multi-decadal time period (WP2)
To provide additional observations to be used for these RAs, other projects and for the community at large (WP1)
To make the RA data available to a large number of users (WP4)
To provide data services and visualisation portals for a large number of RA fields (WP4)
To quantify uncertainties and establish knowledge of the quality of the different RA in many different ways, between datasets and with respect to observation gridded sets and satellite-based datasets and river discharge data (WP3)
To get a consistent knowledge for Europe with a common evaluation procedure for ECVs, climate indicators, extremes and scales of variability in space and time and distributions (WP3).
To document how well extremes and special climate features are reproduced in the RA (WP3)
To show how the data can be exploited for user-oriented products (WP8 and WP3)
To provide a unique and useful datasets for a wide range of downstream applications (WP4, WP8)
To support Climate change services and climate adaptation (WP7)
To support and aid policy development and monitoring of climate for European wide and European national applications (WP7)
To establish good user contacts and get early feedback on the user products (WP8)
To have a long lasting impact also after the end of the project (WP1, 2, 3, 4, 6, and 7)
Project Results:
1 Data rescue and development
The work carried out under UERRA’s WP1 in its three different tasks was going well and as planned except for some small delays. The last year (4) of the Project saw all the tasks completed and Deliverables on time or almost on time.
The main tasks were gathering and data digitisation (T1.1 by URV and NMA-RO) of the recovered data and metadata; quality-control (T1.2 by URV and UEA) and the enhancement of methodologies to reduce uncertainties in gridded products, such as E-OBS (T1.3 by KNMI, UEA and EDI).
The achievements under WP1’s Task1.1 have been remarkable, since this task has not only kept on track, but has also exceeded the digitisation targets as they were planned. A larger number of observations at the synoptic scale than planned have been digitised by URV. This rescue effort includes hourly observations of air pressure (SLP), temperature (TMP), wind speed (WS) and direction (WD), temperature dew point (TDP) and relative humidity (RH), in addition to snow-depth (SD), snowfall (FS) and precipitation (RR) observations at the daily scale.
The spatial coverage was mainly from the Balkans (Slovenia, Bosnia-Herzegovina, Serbian Republic, Turkey, Germany, Mediterranean (Egypt, Algeria, Tunisia, Morocco). There are many areas to choose from and many more data remain to be digitised, but it is also question of access to the original paper data or scanned version of them. More will happen in the C3S project (DS311a_Lot1) which will provide software for users and more co-ordinated plans at global scale.
The quality control of the digitised 8.63 Million (M) synoptic observations was completed and the Deliverable 1.5 was submitted early autumn 2016. URV have extracted 112 M already digitised data values from Sweden, Norway and Catalonia. Also the 221 thousand (k) Romanian precipitation data values were collected.
The work committed under T1.2 on high-quality synoptic-scale data development has progressed as planned, since a wide number of quality control (QC) tests (e.g. outliers, bivariate outliers, big jumps, sharp spikes, inter-variables inconsistencies checks) have been implemented and applied to the 8M of recovered and digitised observations. In this regard, a battery of QC tests have been designed and programmed by URV to QC the data at the hourly and daily scale. The Universal Quality Control procedure (UQC) has been implemented to QC hourly TMP, WS, WD and RH observations, while the Sea Level Pressure Quality Control procedure (SLPQC) developed to QC SLP observations. RClimDex extraQC has been used to QC RR data at the daily scale and a Simple Snow QC (SSQ) implemented to QC SD and FS observations at the daily scale. From the 8.2M of observations digitised, about 31K observations have been labelled as suspicious and are currently being verified by URV to validate or reject them and substitute them by true observations when possible. This is a very time-consuming task, since it requires human intervention to crosscheck the digitised data against the original values in the data sources.
In addition to the six station for which the team of NMA-RO has rescued the 6-hourly precipitation data and digitised them, they have also performed a QC of their data using an automatic procedure applied in the database to compare the 6-hours amounts with 12-hours sums and with the precipitation in 24 hours. A list of errors has been provided to the digitising operator at NMA-RO who checked again in the original document and made the corrections. The team of NMA-RO has also rescued and digitised hail diameter data and submitted a paper to Monthly Weather Review about the climatology of hail in Romania.
The digitised and quality-controlled data have also been tested for spatial consistency and Deliverable 1.6 was completed early in 2017. The data were delivered, first, to WP2 partners (Deliverable 1.7) and, second, to the MARS Archive (in the future, first going through the ECA&D (Deliverable 1.8). Other global databanks and data repositories, such as ISPD, ZENODO, HadISDH and ISTI, also received all the recovered data produced in UERRA. In addition, the developed data have been made available through the World Meteorological Organization Secretary General to the Permanent Representatives of each beneficiary NMSs, in order to foster the culture of knowledge and data sharing. Deliverables 1.7 and 1.8 suffered from some small delays since there were be staff changes. The Romanian data have also been delivered (some 221 k).
There were several lesson learned from the Data Rescue effort:
- Exploring digitised data already present in global and regional data centres is not trivial – it is time consuming due to the need of cataloguing all the input data to support reanalyses and gridded products.
- DARE coordination is required to avoid duplication.
- Examining the quality of the data sources is necessary before and after digitisation and to report them.
- The digitisation methods should be organised and chosen depending on the sources.
- Use templates to mimic the source's layout to reduce mistakes.
- There are a few more NMSs with open data polices but for most NMSs data policies still preclude open data exchange.
2 Gridded observational datasets
A gamma-transform technique for improving the gridding of precipitation data, particularly in mountainous areas, has been developed, refined and tested by KNMI and UEA against the high-resolution gridded datasets produced by various National Meteorological Services across Europe, as well as against the E-OBS constructed using the existing techniques (Figure 1). To further improve the gridding in the E-OBS dataset a new technique has been developed which provides a much better interpolation of all variables. This technique (regression-kriging) is applied to the monthly resolution data, which serve to constrain the daily values, and provides a more stable spline over time which is less vulnerable to the changing station data used for the gridding, which is an inherent problem with the E-OBS dataset (see D1.10).
This new technique also allows for the incorporation of additional topographically-derived parameters to be incorporated into the gridding procedure, which allows for a better interpolation of the station data. Furthermore, this technique forms the basis for the production of an E-OBS dataset consisting of multiple-realizations, which is reported in D1.14. To allow for this development of the E-OBS dataset a new package written in the R computing language has been developed, which also contains extensive user documentation. Furthermore, the aforementioned development of the E-OBS dataset using the new R package has been conducted on the ECMWF high-performance computing cluster (see D1.12). This is a significantly higher specification system than has hitherto been used to produce E-OBS and will allow for more computationally demanding techniques to be developed in the future, after the conclusion of the UERRA project. Operational production of the E-OBS dataset has moved over to this system.
KNMI has led the development of the E-OBS dataset (Deliverables D1.10 to D1.14). This work has consisted of an overhaul to the method used to construct the dataset, with a particular emphasis on improving the estimates of uncertainty through the generation of a 100-member ensemble of daily realizations for each variable (Cornes et al. 2017). The code has been written in R, and is version controlled on a private GitHub repository. Further refinements to E-OBS are being undertaken as part of the C3S project (D311a_Lot4), which will also make the ensemble dataset operational. Several experiments were conducted to determine the best method of generating the ensemble. The initial planned conditional simulation method was simply not possible with the time and computing resources available, and hence a different method was developed using Bayesian reasoning, and which is consistent with the standard error estimates used in earlier versions of E-OBS.
Half-yearly full updates and monthly preliminary updates of the original version of E-OBS have been continued in this period.
KNMI made the E-OBS update (D1.13) with their version 14.0 and with data covering 1950 to August 2016. Apart from including the most recent data at the time, it has also corrections to a problem with the minimum and maximum temperatures found in the UK sources..
EDI has developed a new method to quantify uncertainties in observation-based spatial analyses (grid datasets) of precipitation. The procedure represents analyses in terms of an ensemble of equiprobable realizations, the spread of which informs users about inherent analysis uncertainties, related, for example, to the limited observation density or short-scale variance of precipitation. The method builds on statistical (conditional) simulation and extends it to rectify for several shortcomings. Most notably, the present development takes account of the uncertainty in the statistical parameters. A detailed verification in the Alpine region demonstrates the reliability and consistency of results. The technique has been employed to derive a new multi-year ensemble dataset of daily precipitation over the entire Alpine region using high-resolution rain gauge. The new dataset formed as a reference for the evaluation of regional re-analyses in WP3. The findings of this work are also relevant for the development of ensemble gridding techniques with other grid datasets.
These ensemble analyses over the Alps offer users to trace analysis uncertainty into their application and they avoid several known difficulties of single best estimates, including spatial smoothing, biases in the statistics of extremes and ambiguities in effective resolution.
The methodological developments made during UERRA build on and extend conditional stochastic simulation techniques previously proposed for ensemble gridding. Notably, the technique adopts a more flexible stochastic model (Trans-Gaussian Random Fields) to account for skewness and heteroscedasticity of precipitation, and it employs Bayesian statistical inference to incorporate uncertainty of parameters into the final ensemble.
The ensemble dataset of Alpine daily precipitation is represented as area-mean precipitation on a four-level hierarchical subdivision of the domain into hydrological units (the European river catchments dataset of EEA). The ensemble encompasses 50 members. Once all computations are completed, the analysis will extend over 38 years (1971-2008); results for 15 years were available for comparisons and evaluation in WP3.
In-depth analysis of the ensemble dataset demonstrates plausible dependencies of ensemble spread (i.e. analysis uncertainty) upon weather situation (e.g. convective versus stratiform precipitation), density of nearby rain-gauge observations, and the sizes of the catchments. Our evaluation suggests that the dataset has a good statistical reliability, enabling users to confidently use the ensemble members as equally likely potential realizations of the true (unknown) precipitation amounts over the catchments.
An attractive feature of ensemble datasets is that uncertainties can easily be deduced for results of further data processing. Figure 3 depicts the ensemble median and spread for two frequently used precipitation indices, the frequency of wet days and the 95% (all-day) quantile. The ensemble median reveals known pattern of the precipitation climate in the Alps, and the ensemble spread shows plausible variations of uncertainties between areas (larger in data sparse regions), catchment sizes (larger in small area units) and indices (larger for extremes).
A paper describing the methodological developments and the dataset is in preparation (Frei & Isotta 2017). The findings of this work have partly been included in the development of an ensemble grid dataset based on E-OBS. Moreover, the probabilistic Alpine dataset has been used in the evaluation and uncertainty assessment of re-analyses in WP3.
3 Ensemble Data Assimilation Regional Reanalyses
In UERRA there were three different 3-dimensional upper air regional reanalyses, all covering the EU-CORDEX domain and at about 11 km horizontal resolution. This in itself gives a multi-model variation ensemble due to differences in the model and data assimilation formulations and some differences in observational inputs. With the Met Office Unified Model (UM) system there was also an internal ensemble of 20 perturbed members and also with the University of Bonn COSMO based nudging assimilation system. SMHI ran two model versions for 5 years but then one version for over 55 years, from 1961. The Met Office ran from 1979 (to 2015 incl.) but the Univ. of Bonn just for 5 years. In addition there was a dedicated surface downscaling reanalysis at 5.5 km using more in situ observations, of temperature, humidity and precipitation. This also had an internal ensemble for 5 years.
The cloud fraction reanalysis has overcome several scientific problems to produce a tuned system that can cope with horizontal inhomogeneities. It was discovered that CM SAF are making available an improved quality METEOSAT cloud product for this period that should result in a better quality reanalysis. Given other delays already, it is planned to wait for the new product before processing the early part of the period. The later part (2004-2008) could be run using already available MSG SEVIRI cloud products.
A significant circumstance is that all the regional reanalyses are set up in quite different configurations compared with each partner's operational data assimilation and forecasting systems. There was a sizeable research and development component of WP2 before the production phase, but it has extended more than expected. One cannot lean on and rely on operational developments at the institutes to any high degree.
As recognition of the hard work by the Project partners the European Regional Reanalyses have been continued with the Copernicus Climate Change Service (C3S). URRA was a pre-operational Copernicus project laying the ground for the operational Service. Some of the partners, and some additional ones, are Contractors to carry out both a European-Atlantic and an Arctic reanalysis service. (C3S_322 lot1 and lot2).
3.1 The Met Office Ensemble 4D-VAR Reanalysis
The Met Office reanalysis is an ensemble of independent runs, each assimilating observations in a 4DVar analysis system. The spread of the ensemble will come from several sources of perturbations. The observations are perturbed with random noise according to the expected observation error. The Sea Surface Temperatures (SST) come from an ensemble of analyses, HadISST2 (Titchner and Rayner 2014). Model error is accounted for by the method of Piccolo and Cullen (2015) where the model forecast is perturbed by previous analysis increments, taken to be representative of model error.
It was intended also to drive the ensemble with an ensemble of lateral boundary conditions from the ERA5 global reanalysis. Start of ERA5 production is later than was planned and the full ERA5 dataset is not expected until the end of 2017, too late for this project. Therefore we intend to use the deterministic ERA-Interim to provide boundary conditions.
A suite has been assembled to run on the ECMWF HPC implementing the method above. It includes several developments required for a long-period reanalysis. Satellite radiances are bias-corrected with a variational scheme VarBC, following Dee and Uppala (2009). Soil moisture is adjusted according to differences between forecast and observed screen level temperature and humidity, analysed through an Extended Kalman Filter. This is an adaptation of the scheme used for Met Office global NWP. Observation reject lists will be updated monthly based on O-B (observation minus background) differences from the previous month. Figure 5 shows verification of the ensemble mean for January 2007 against 850hPa radiosonde temperatures, and also the ensemble spread. The ensemble spread is the 'uncertainty'. For a perfect system, the uncertainty should be close to the RMSE of the ensemble mean. It can be seen here that the spread is smaller than the RMSE. This is a common feature of real-world ensembles. It is partly due to the difficulty in representing every source of uncertainty in the ensemble. It is also partly due to errors in the observations. Observation and representivity error (the observation is seeing detail that a 12km model cannot) both contribute to inflating the RMSE.
The principal contributions of the Met Office to the Uncertainties in Regional Reanalysis (UERRA)
project, [Unden et al., 2014], are a deterministic atmospheric reanalysis at 12km and a 20 mem-
ber ensemble atmospheric reanalysis at 36km over Europe for the satellite era (1979-2016).
Production started in January 2017 and completed in January 2018. The develop-
ment of the project is documented in deliverable reports, which are publicly available through
the UERRA website, http://uerra.eu. In D2.1 [Jermey et al., 2015], the reanalysis systems are
documented, together with initial test results and a demonstration of ensemble products. D2.2
[Mahmood et al., 2016], describes use of observations and other data used as boundary condi-
tions for the reanalyses. Initial diagnostics from the ensemble system are presented in D2.3
[Jermey et al., 2016], which demonstrates the validity and quality of the ensemble production
data, as well as reporting on trials of the satellite bias correction method, land surface anal-
ysis scheme and ensemble size. The deterministic reanalysis uses a hybrid approach to data
assimilation, exploiting the ensemble information to improve the reanalysis, as documented in
D2.4 [Jermey et al., 2017b]. D2.4 also contains tuning and initial trial results for the deter-
ministic reanalysis system. An assessment of the quality of the ensemble reanalysis is given in
D2.14 [Jermey et al., 2017a], which includes comparison with the 20th century ensemble reanal-
ysis (CERA-20C), [Laloyaux et al., 2016] from the European Centre for Medium Range Weather
Forecasts (ECMWF) and the University of Bonn/Deutscher Wetterdienst ensemble reanalysis
for UERRA, [Bach et al., 2016].
The reanalyses are driven by advanced data assimilation techniques which produce an estimate
of the atmospheric state, by drawing a background atmospheric state towards a set of observa-
tions. These reanalyses are produced every six hours, with an atmospheric model forecasting
to the next reanalysis time. This (re)forecast is then the background atmospheric state for the
next reanalysis. The reanalyses cover the EUR-11 domain of the Coordinated Regional climate
Downscaling Experiment (CORDEX, [Jacob et al., 2014.
Both reanalyses cover the period 1979-2016 and are free cycling, but are dependent on ERA-
Interim for lateral boundary conditions (LBCs). Each production stream also requires an initial
background to start the cycling process. This data also comes from ERA-Interim. Each stream
has a month (ensemble) or a week (deterministic) of spin-up time from this initial background
before production data is processed and archived. As an example, the ensemble stream of
1997-1999 is spun-up from ERA-Interim data on 1st December 1996 and begins producing and
archiving data on the 1st January 1997.
The observations assimilated by the reanalyses are mostly supplied by European Centre for
Medium Range Weather Forecasts (ECMWF), [Dee et al., 2011]. Prior to 2003 these observa-
tions come from ECMWF’s reanalysis archives and 2003-2016 from operational archives. These
observations include SYNOP stations, ships, buoys, sondes, aircraft and satellite measurements
using advanced infared sounders (AIRS), (Advanced) Television Infrared Observation Satellite
Operational Vertical Sounder ((A)TOVS), scatterometer winds and Infrared Atmospheric Sound-
ing Interferometer (IASI). These are complemented with atmospheric motion vectors (AMV)
from the Met Office and reprocessed global positioning system zenith total delays (GroundGPS),
Further satellite data using GPS radio occulation (GPSRO), Spinning Enhanced Visible
and InfraRed Imager clear sky (SEVIRIclear) and Special Sensor Microwave Imager/Sounder
(SSMIS) was not attempted due to time and resource limitations. A guide to the number of
assimilated observations is given in figure 2. This figure shows that there is a substantial increase
in surface, aircraft and satellite observation volume between 1979 and 2016, but sonde volume
remains relatively steady.
The new Monitoring and Updating Station Lists (MUSLi) system is used to reject and correct
observations based on monthly departure statistics. Corrections are made for surface observa-
tions of pressure and for temperature observations from aircraft and upper air data, [Davie, 2017].
Variational bias correction is applied to the satellite radiances, [Lorenc, 2012, Dee and Uppala, 2009]
This allows for bias corrections to vary with time so as to fit drifts in instrument bias. The
method has been previously well tested for global models and is used here for the first time in a
regional model. High peaking channels are rejected, see [Jermey et al., 2015].
For most of the period, the lower sea boundary is given by version 2 of the Hadley Centre Sea
Ice and Sea Surface Temperature (HadISST2), [Titchner and Rayner, 2014]. This contains a
ten member analysis of sea surface temperatures at 0.25 º consistent with a single deterministic
analysis of sea ice fraction at 1.0 º . This dataset does not cover 2011-2016, so from Septem-
ber 2011, the Met Office Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA),
[Donlon et al., 2012], is used, reduced to the resolution of HadISST2. For this later period, an
ensemble dataset is produced by adding HadISST2 from a random year to the reduced resolution
OSTIA fields. During both periods, the ten member ensemble of sea boundary data is shared
so that each sea boundary member is used by two reanalysis ensemble members.
The land boundary is provided by the Met Office Land Surface Data Assimilation System
(SURF), [Candy, 2014], used in a regional context for the first time. As with the atmospheric
analysis, each member of the ensemble performs its own land surface analysis from a different
realisation of in-situ observations.
In order for the ensemble of reanalyses to represent uncertainty in the reanalysis system, the
sources of uncertainty are perturbed so that each member receives a different realisation. In
addition to different realisations of the boundaries, different realisations of the model and ob-
servations are also employed. The model is perturbed by adding random analysis increments
as tendencies across the forecast. A random analysis increment may be thought of as a proxy
for model error. Each observation is also perturbed by adding a random amount bounded by
the estimated observation error. This follows a system for generating global ensembles used at
the Met Office for research into model error, [Piccolo and Cullen, 2016]. Each ensemble member
is isolated as there is no re-centring of the ensemble at analysis time. An unperturbed control
member is run separately from the ensemble in order to monitor and update observation black-
lists and calculate satellite bias correction at the ensemble resolution.
The ensemble reanalysis was originally intended to use four-dimensional variational assimilation
(4DVAR), [Rawlins et al., 2007]. However this proved to be prohibitively expensive and so three-
dimensional variational assimilation (3DVAR), [Lorenc et al., 2000] is used instead. 3DVAR suf-
fers from spin-up issues for precipitation so a consequence of this change is that the ensemble
precipitation values are far too large. The spread of the precipitation is, however, still useful for
estimating reanalysis uncertainty, [Jermey et al., 2017a].
Variational assimilation (3DVAR, 4DVAR and hybrid 4DVAR) draws the model close to the
observation data, which in the case of the ensemble is perturbed, by minimising a cost function
which penalises distance from observations, distance from the background and high frequency
behaviour. The weights to the observation and background terms are inverses of the estimates of
the error covariances. It produces an increment to the background which estimates the optimum
state of the atmosphere, given the background and observations across a six hour assimilation
window (T-3 to T+3), taking into account the three-dimensional position of each observation.
As is common practice, the assimilation is carried out at half the horizontal resolution of the
model.
The deterministic reanalysis uses hybrid 4DVAR, [Clayton et al., 2013], to assimilate obser-
vations. As with 3DVAR, this is carried out by minimising a cost function, which penalises
distance from observations, distance from the background and high frequency behaviour. (Hy-
brid) 4DVAR also estimates the optimum state of the atmosphere, given the background and
observations across a six hour assimilation window (T-3 to T+3), taking into account the three-
dimensional position of each observation, but also taking into account the validity of each obser-
vation within the window. The use of the fourth dimension leads to analysis increments which
are more in balance with the model than 3DVAR and therefore it does not suffer from precipi-
tation production which is initially too large.
Traditionally the background error covariance is estimated by a smooth parameterised approx-
imation to climatology tuned by forecast differences over a long period. This is sufficiently
accurate to consistently produce an analysis which is closer to the true atmospheric state than
the background. However, this estimation does not vary from cycle to cycle and regions of ac-
tivity in the background are given the same weight as regions of stability.
Hybrid 4DVAR uses a combination of the climatological background error covariance (used for
the ensemble) and a error covariance derived from the ensemble. This combined covariance cre-
ates a low-noise estimation that is dependent on synoptic pattern. This assimilation method has
been used for background error covariance estimation in operational global forecasting at the
Met Office since 2011, [Clayton et al., 2013]. It is used here for the first time in a regional con-
text. A diagram of this is shown in figure 3 and the hybrid settings are detailed in table 3, which
were determined by a combination of experimental tuning, [Jermey et al., 2017b], and in-house
experience. To avoid spuriously large covariances near the top of the atmosphere, climatology
only is used for this region. A middle region features a linear ramp from hybrid covariances to
climatological covariance. Use of the ensemble data ensures that the background error covariance
is synoptically dependent in the deterministic reanalysis.
The same model is used by both the ensemble and the deterministic reanalyses. This is the
Unified Model (UM), [Davies et al., 2005], using the dynamical core currently used for opera-
tional forecasting at the Met Office (Even Newer Dynamics for General atmospheric modelling
of the environment, ENDGame, [Wood et al., 2014]). The reanalyses both use six hour cycling,
following the Met Office operational global forecasting system. A (re)forecast is carried out from
a combination of the analysis increment, provided by the data assimilation, and the previous
forecast (background) from T-3 to at least T+9. The reanalysis fields are output at T+0 (the
centre of the six hour assimilation window) and the background for the next cycle between T+3
and T+9. The model is computed over 63 levels using Charney-Phillips staggering between 10m
above orography to the model top - a fixed radius from the centre of the Earth.
Figure 6 in the Met Office Final Report shows the RMSE and mean error of 6h forecasts from the
reanalysis and ERA-Interim against 2m temperature observations for March 1979, 1988, 1997,
2006 and 2016. This demonstrates that the quality of the representation of 2m temperature in
UERRA-MO is an improvement on ERA-Interim throughout the reanalysis period. This bene-
fit increases with increasing time/observation volume, see table 6, which details the percentage
change in RMSE from ERA-Interim to UERRA-MO. This suggests that higher resolution as-
similation is of greatest benefit when a dense observation network is available. The quality of
both reanalyses improves with time and both reanalysis have low mean error (bias) throughout,
with UERRA-MO slightly warmer than ERA-Interim.
Figure 7 shows the RMSE and mean error of 6h forecasts from the UERRA-MO deterministic
reanalysis and ERA-Interim against 10m wind vector observations for the same periods as figure
6. As with 2m temperature, both reanalyses improve with time/observation volume. Again,
UERRA-MO is an improvement on ERA-Interim throughout the reanalysis period and the im-
provement increases with increasing time/observation volume, see table 6. Both reanalyses show
a small bias, with UERRA-MO consistently less biased than ERA-Interim.
Ensemble spread is intended to represent the uncertainty in the ensemble mean. Figures 14 and
15 show the spread of each ensemble at observation positions for UERRA-MO and CERA-20C.
CERA-20C is a global ten member ensemble reanalysis at 125km and assimilating only surface
pressure, marine winds and ocean profiles, [Laloyaux et al., 2016]. The observations are selected
daily mean temperatures from the ECA&D dataset, [Klein Tank et al., 2002], which are inde-
pendent of the reanalyses. The models feature smoothed orography which may differ greatly from
individual station height. The model temperatures are first corrected to the height of the observa-
tions by applying a lapse rate of 0.0065Km −1 , [International Organization for Standardization,
1975].
In each case the spread is inflated using a multiplicative factor to match an assumed observation
representivity error variance of 0.96 as described in table 7, following [Saetra et al., 2004]. Jan-
uary to March (JFM) is shown for 1987, 1997 and 2007 for both ensemble reanalyses, with 2016
also shown for UERRA-MO (since CERA-20C has no data beyond 2010). These figures show
that, after inflating the ensemble to take account of observation representivity error, the magni-
tude of the spread in UERRA-MO is similar to that of its mean RMSE. However, the same is not
true of that of CERA-20C. The Pearson’s rank correlation coefficient (labelled ‘Prcc’), between
the time series of the spread and the RMSE of the mean, is also higher in UERRA-MO than in
CERA-20C. This indicates that the ensemble of UERRA-MO is able to represent uncertainties
in 2m temperature at least as well as CERA-20C, but at much higher resolution.
Figure 14 also shows that with increasing decade/observation volume, the RMSE of the mean,
the spread and the raw spread are all reduced. The correlation of the spread to the ensemble
mean is also reduced. This may indicate that the RMSE of the mean is becoming dominated
by errors in converting from model space to observation space, e.g. height difference and local
effects, or that its variability is becoming dominated by sub-grid scales that are not well captured
by simple inflation.
The UERRA-MO reanalyses cover 1979-2016 over Europe and the Mediterranean. These include
a lower resolution (36km) 20 member ensemble reanalysis, driven by an ensemble of 3DVAR as-
similations, and a higher resolution (12km) deterministic reanalysis, driven by hybrid 4DVAR
assimilations. The reanalyses assimilate a wide range of conventional and satellite data and
a wide range of consistent atmospheric variables is produced from the surface to a height of
approximately 40km.
Both reanalyses make use of technical innovations. The ensemble uses a random draw of analysis
increments to represent model error. The deterministic reanalysis uses a hybrid 4DVAR which
uses ensemble information to estimate the background error covariances. Both are used for the
first time in a regional context.
The parent model for the UERRA-MO reanalyses is ERA-Interim. The deterministic reanalysis
shows an improvement of representation of most atmospheric variables over ERA-Interim for
this region. This improvement increases with observation volume, indicating that the higher res-
olution hybrid 4DVAR assimilation system makes better use of a dense network. As expected,
larger scale variables are better represented in the global reanalysis.
Due to cost limitations of running a 40 year ensemble reanalysis, 3DVAR instead of 4DVAR
is used to calculate the reanalyses. This restriction has lead to spuriously large precipitation
in the initial hours of the (re)forecast. With 3DVAR it is usual to allow a period of spin-up
before taking production assimilations. This was not done for UERRA-MO, since extending
the (re)forecast period would have increased the cost. Although the subsequent magnitudes of
precipitation in the ensemble reanalysis are very large, the ensemble is still able to capture the
PDFs of observed values with a similar accuracy to CERA-20C, indicating that there is useful
uncertainty information in these fields.
For other fields, the ensemble reanalysis members have realistic representation and are able to
capture uncertainties well, demonstrating some improvement on CERA-20C for capturing the
uncertainty in the ensemble mean with its spread.
Satellite data has a beneficial impact on the ensemble reanalysis, with later periods where more
data is available reaping greater benefits. The effect of the increased volume of satellite data
can be clearly seen. The greatest improvement due to satellite data between 1987 and 2007 is
seen in the temperature and wind speed, especially in the mid and upper troposphere. This
would seem to reflect benefits particularly from the satellite sounding instruments and AMVs.
For 10m wind, the verifying observations are mostly over land, while the scatterometer data is
from over the oceans and so the impact is harder to measure.
These reanalyses provide the community with hourly data across a wide range of atmospheric
variables for 1979-2016. High resolution fields are available from the deterministic reanalysis.
The lower resolution ensemble reanalysis provides users with twenty realisations of the entire
period whose spread is a useful measure of uncertainty in the mean.
3.2 The SMHI ALADIN HARMONIE 3D-VAR Reanalysis
The HARMONIE-ALADIN re-analysis was produced and archived from January 1961 to December 2015 (now also 2016) with a horizontal resolution of 11 km and the ALADIN physics scheme. Both upper air as well as surface data assimilation was included. To introduce large scale information from the global reanalyses a large scale constraint has been added to the cost function.
For the upper air data assimilation only the so called conventional observations are included. This means observations from SYNOP stations, ships, drifting buoys, aircraft observations and temperature soundings. For the surface assimilation temperature and relative humidity at two meters as well as snow water equivalent, all from SYNOP stations are included. In the observation monitoring shown here only the upper air observations are shown.
An observation monitoring system has been partly developed within UERRA. Figure 2 shows the total number of observations used together with the amount of observations from each observation type. Shown are the monthly averages from 1961 to 2015.
As expected the number of observations increases during the re-analysis period. Aircraft observations are not available until 1980 and after that it is constantly increasing, especially at high altitude, i.e. cruising level (not shown). In the 1980s and 1990s all of the aircraft observations were reported manually as AIREP (AIRcraft REPorts) but later more and more are automatic AMDAR (Aircraft Meteorological DAta Relay). The latter together with the increase in air traffic is noticeable not only in the number of observations but also in the distribution of the observations both horizontally and vertically.
There are a number of interesting features in Figure 2 that need further investigation but most strikingly is the dramatic reduction of the number of aircraft observations during 2015. This is due to a change in the BUFR templates in the aircraft reports from December 2014. This has now been taken into account and the last year has been re-run in order to have a complete data set with all available observations.
Another example of the importance of observation monitoring is to check if the assimilation is working properly. This can be done by comparing the first guess (background) and analysis departure, i.e. how much the observations differ from the first guess and from the resulting analysis. If everything is working well the analysis departure should be smaller than the first guess departure. This means that the model has adjusted to the observations. How big this adjustment is will depend on both the background and the observation error. Examples can be found in D2.7 and they show that the model behaves as we would expect.
The results of the HARMONIE-ALADIN re-analysis have been compared results from ERA40 and ERA-Interim. For most parameters the results are very similar although the difference in resolution is obvious giving more details in HARMONIE-ALADIN, especially in areas of steep topography. For precipitation, however, there are larger differences between UERRA compared to ERA-Interim. In HARMONIE-ALADIN there are rather large amounts of precipitation in mountainous areas that is not seen in the corresponding ERA reanalyses. What the reason for this is and which is more correct need to be studied further but it is known from another study, presented in UERRA deliverable D3.6 that UERRA produces exaggerated precipitation amounts in the Alpine region. It has also been seen that the ALADIN scheme in general, produces too much precipitation.
The HARMONIE verification system WebGraF has been used to verify the forecasts for the 55-year period. Due to the amount of data, the number of parameters that were verified had to be reduced. For the surface, wind speed at 10 meters altitude, temperature and dew point at 2 meters, mean sea level pressure, cloud cover and 12 hour accumulated precipitation have been verified. For upper air: geopotential, temperature, wind speed and relative humidity. The forecasts are verified against the same observations that were used for the initial analysis.
In order to summarise the verification results a scorecard has been constructed where a few parameters are subjectively evaluated for each decade and compared with the corresponding verification for ERA40 and ERA-Interim. The scorecard is presented in Figure 3 and it can be seen that for the surface parameters (mean sea level pressure (PMSL), two metre temperature (T2m), wind at ten meters (U10m) and cloud cover (CC)), except the relative humidity (Rh2m), HARMONIE-ALADIN re-analysis performs better (green triangles) or as good as (circles) the ERA re-analyses compared to observations when it comes to standard deviation (STDV). For the profiles of temperature (Temp), wind speed (WS) and geopotential height (Geop), i.e. higher altitudes the results are more mixed except for the relative humidity (RH) where ERA is better for all periods.
As users have started to use the HARMONIE-ALADIN data a few erroneous fields has been discovered. Some we are able to correct and re-archive while some we are not able to correct without having to rerun the entire re-analysis. The re-archiving wouldl take a very long time since retrieving the data is a rather slow process so in the mean time, and for the non-correctable data, a document is provided giving instructions to the users of what data to be careful with.
3.3 The Météo-France downscaling reanalysis
The configuration of the ensemble system MESCAN-SURFEX-ENS is based on 8 members obtained with several combinations of temperature analysis, precipitation analysis and two types of radiative fluxes and wind speed from the ALARO and ALADIN physics. The production of the 8 members for the period 2006-2010 took more time than estimated due to the hourly output and the large number of SURFEX variables especially because of the 14 soil layers. Finally, the MESCAN-SURFEX-ENS production was available on the MARS system (test) in two steps: first the MESCAN 8 members was available in March 2017 and used by the WP3 for evaluation. Then, in a second step, the SURFEX output such as snow cover, soil temperature and soil moisture (14 layers) were available on MARS by the end of June 2017. One of the main difficulties was to transform the native output SURFEX NetCdf files into GRIB2 and to define the new soil layers and variables in GRIB2 format from SURFEX. This last point required a lot of exchange with ECMWF and the validation of the surface and soil data in GRIB2 took a lot of time. All those problems partly explain the reason for a MESCAN-SURFEX-ENS delayed report (D2.9) in addition to a tragic unexpected event in the MF team. Finally, the deliverable D2.9 has been published in June 2017.
Some preliminary evaluation done by WP3 on the precipitation analysis has shown (GA in November 2016) a tendency to underestimate the number of “dry days” in the MESCAN precipitation analysis. After investigations and many tests it was decided to use a varying σο, instead of a constant one, to better fit the observations in case of weak rainfall :
σο=0.001mm when RR_obs=0.mm ; σο=0.7 + RR_obs * 0.1 when RR_obs < 50mm and σο=5,7mm for RR_obs > 50mm
The “final” 55 years production began in January 2017 with 4 parallel streams: 1961-1965, 1965-1970, 1981-1985 and 2005-2010, the last two periods were available in March 2017 for the WP3 and the evaluation. Unfortunately for those periods no additional surface data such as 2m-temperature or 24h-precipitation recovered by WP1 have been used in the MESCAN analysis. The MESCAN surface analysis is now finished and is available on the MARS archive (under prod) at the end of November 2017. Figure 4 shows the 2m mean temperature (blue line) over Scandinavia (top) and Pyrénées (bottom), the green lines are all the members from the MESCAN-SURFEX-ENS which was only available for 2006-2010.
For the SURFEX-TRIP part (driven by the MESCAN analysis) it was not possible to run several parallel periods due to the long-time spin-up of the deep soil moisture in some parts of Europe. Consequently, it was decided to run continuously SURFEX-TRIP from 1961 to 2015. The main drawback of this decision is the delay of the complete final production such as MESCAN and SURFEX.. The end of the SURFEX-TRIP is planned for middle December, nevertheless we have already started to archive the SURFEX output in MARS (under prod). The period 1961-2015 is now available.
The precipitation analysis requires more verification than the 2m-temperature to avoid some spurious features due to “bad” observations not rejected by the automatic quality control (Fig 5 a, left), the only way to solve this problem is a manual correction and then re-run the MESCAN analysis. The observations used for the precipitation analysis come from several database: MARS, ECA&D, French, Swedish and Norwegian national databases. Very often, the “bad” observations come from the MARS archive, it is not really surprising because the precipitation observations are not used (as an input) by ECMWF in the re-analysis and in the operational system, and therefore the quality control is probably not done.
As shown for temperature in Figure 4, Figure 5 (below) shows the annual accumulated rainfall over the Alps. The spread of the ensemble is rather large for this region, which makes sense due to the uncertainties of rain and snow fall observations in mountainous area.
Precipitation is one of the most essential variables, together with temperature, to drive a “hydrological model” for water resource management. Figure 6 shows an example of the river discharge computed by MESCAN-SURFEX-TRIP (blue line) for the Danube river, since 1961 up to 1976. The main evolution of the river discharge is well captured by the MESCAN-SURFEX-TRIP system compared to the observation (red line).
3.4 Cloud Fraction reanalysis (SMHI)
The objective of this task is to provide a 2D optimal interpolation (OI) analysis of hourly cloud fraction for 30 years (1982-2013) at 5.5 km resolution. The idea is to combine, on a common grid, good quality CM-SAF datasets from geostationary METEOSAT and AVHRR polar platforms and to use NWP reanalysis data for gap-filling.
During the start up of this task it became clear that the CM-SAF is about to start processing a new homogeneous cloud fractional cover product for the entire METEOSAT period (1982 onwards) and that the whole 30+ year period was finalized only by the end of 2016. This should probably result in a significantly improved cloud product to what is now available for the pre SEVIRI (MSG) era. Because of this it was decided to wait for this dataset to become available and focus on the SEVIRI era, where high quality data is readily available, to begin with.
The OI analysis needs estimates of the first guess and observation error. In this case both of these are spatially correlated and the method by Desrozier et. al was used to estimate the error matrices based on forecast difference statistics from the HIRLAM EURO4M cloud cover. The method converges nicely to realistic estimates and by carrying out the OI analysis in Fourier space, where the error matrices are diagonal, the analysis can be done in an efficient way.
In areas with missing satellite data EURO4M HIRLAM data is used as a gap filler since complete fields are needed in order to use the Fourier transform and get diagonal error matrices. This can result in discontinuities in areas with missing data where HIRLAM and the satellites disagree. To reduce these effects we tried to add a weight matrix to the OI formulation and pay less attention to observations in areas where these are actually just NWP data (low weights for NWP). Unfortunately the minimization problem then became much more demanding (non-diagonal matrices). As a remedy a fast adaptive filtering approach is now used where information from neighbouring observations and NWP forecasts is combined according in order to get a smooth transition from the edge to the centre of a missing data region.
3.5 Ensemble Nudging Data Assimilation Reanalysis (University of Bonn)
UB's task as part of WP2 in UERRA is to provide a regional ensemble reanalysis system as well as a proof of concept high-resolution data set for Europe. In D2.11 a technique called ensemble nudging to perturb observations in order to account for observational uncertainty in an ensemble has been introduced. This had been planned to be part of a hybrid technique in combination with a local ensemble transform Kalman filter (LETKF) that is newly developed at Deutscher Wetterdienst. However, due to a couple of reasons that are detailed in D2.12 as well as technical issues that were solved later, but still left inherent problems, the hybrid combination turned out to be of limited usefulness for the production of a comprehensive dataset as an ensemble reanalysis. Moreover, the absence of the PI of the UB work due to parental leave from 2016/01 to 2016/12 led to an agreement with the UERRA management team that the originally intended hybrid system would be replaced by the ensemble nudging component as data assimilation system for the UB reanalysis. The usefulness of this technique is comprehensively shown in D2.12.
The reanalysis has been carefully monitored for observation statistics and departures against the data. The reanalysis was seen to be stable throughout the period and the results have been documented in the diagnostics report D2.13 delivered at the very end of 2017. Moreover, the reanalysis and particularly the ensemble performance has been compared and documented in the RA uncertainty evaluation by the Met Office (Peter Jermey et al.) (D2.13 in November 2017, see also below).
Concerning the diagnostics of uncertainty estimation capabilities that are under task T2.5 a range of knowledge and codes have been assembled during the course of the project. Some of these are shown in D2.12 and Bach et. al, 2016. These are also useful for task T2.6 a comparative study of the UERRA ensemble reanalyses.
3.6 Reanalysis Cross Evaluation
This work was done once production data is available from all the reanalyses started to become available. A comprehensive work and report was made by Peter Jermey (Met Office) with participation from the other RRA producers (D 2.14). Particularly the ensemble quality has been validated and there is considerable information in the spread and particularly the COSMO nudging ensemble shows good ensemble properties e.g. in terms of Rank histograms (see the report).
4 Assessment of uncertainties by evaluation against independent observational datasets
In collaboration with WP8, we managed to maintain a clear user focus in WP3, in line with the WP3 objective ‘common evaluation procedure for ECVs, derived climate indicators, extremes and scales of variability that are of particular interest to users’.
The work was achieved under the lead of DWD and with the contribution of all the partners (MET Norway, Meteo-Swiss, KNMI and UEA) which acted very flexibly, although only little time was left due to the delay in the RA production. In hindsight, it was most helpful that activities were at full speed already because of the preliminary analysis (D3.5) where input was prepared, scripts and programming was set up, user interest appropriately catered for, and first experiences could be gained. The experience from the preliminary exercise resulted also in a realistic estimate of archive extraction and data handling time, and a sharpened focus for the actual scientific evaluation (D3.6). Development of new methods were pursued independently of the delay (e.g. evaluation methods for the independent probabilistic Alpine dataset).
In line with planned resources, the work focussed on the assessment of uncertainties over the European domain and on sub-regions of particular interest. Satellite data (radiation products) by CM SAF were pre-processed in order to be used for further ongoing work. Gridded data sets were prepared, with the focus of improving the representation of precipitation extremes. In addition, uncertainty estimates have been developed based on the spread using an ensemble of gridded reconstructions (rather than the single uncertainty estimate).
In T1.3 (WP1), EDI has developed a new method to quantify uncertainties in observation-based gridded datasets of precipitation. The results have been verified in detail in the Alpine region demonstrating their reliability and consistency. The method was employed to derive a multi-year ensemble dataset of daily precipitation over the entire Alpine region using high-resolution rain gauge data
A first release of the Nordic Gridded Climate Dataset (NGCD) has been established. It is a high-resolution gridded dataset for daily mean temperature and daily accumulated precipitation based on measurements from weather stations covering Fennoscandia. It was used as a reference dataset for reanalysis evaluation in that region
Speedy production of results in the final months of the project was enabled by common evaluation procedures that had been agreed early on (see D3.2). These cover derived climate indicators, extremes and scales of variability that are of particular interest to users. Appropriate considerations of users was ensured by the interaction with WP8. It proved useful that the two main participants of WP3 were also heavily involved in WP8 and WP7, naturally leading to that evaluation was kept relevant to user interest.
The focus of the WP3 evaluation has been on: precipitation (EDI, MI, KNMI), drought (NMA-RO), wind speed (DWD), radiation (DWD), temperature (KNMI) and climate indices (KNMI), at the hourly, daily, monthly and inter-annual scale.
All data sets produced in WP2 had been included in the evaluation, and all showed good results, the differences depend on region, parameter, and evaluation score. The evaluation was mainly performed against station data (by DWD, KNMI), and gridded station data (by KNMI, EDI, MI, NMA-RO). Further, evaluation with probabilistic ensembles of gridded data (by EDI, MI) and with satellite data sets from EUMETSAT’s CM SAF (by DWD) has been performed. MO contributed with know-how on NWP evaluation and URV and also EDI with methodological advice on evaluation scores.
One lesson of the WP3 work was that evaluation results vary strongly with region. Thus it was fortunate there were, in addition to the whole of Europe, three sub-regions (Fennoscandia, Alps, and Germany) which were analysed in more detail. It highlighted also for the users that results cannot be generalized and performance of the various models could depend on the region they were optimized for.
The results of WP3 were always communicated without time delay to the WP2 producers, firstly to inform them about any irregularities found, and second, for scientific discussion on the causes of the evaluation results. This proved beneficial to both WP3 and WP2.Particularly comparing the results of D3.6 and D2.14 which were done independently, allowed a check on our results and enhanced interpretation which could be relayed to the users.
However, for task T3.2 a number of satellite products were originally envisaged to contribute to the evaluation, but all were dismissed as they were not appropriate in terms of quality and resolution to serve as “truth” for the regional reanalysis, as the latter have higher spatial and temporal resolution as well as higher accuracy in the parameter space. Even for radiation, estimation from the regional reanalysis were found superior (with respect to locally and annually varying bias) to the satellite data, as was shown by DWD with a third, high quality reference data set (HelioMont) over the Alpine region.
In close collaboration with MI, EDI compared several datasets produced in WP1 and WP2 as well as several existing gridded products well known in the climate community. EDI focussed on daily precipitation (06-06UTC) for the Alpine Region using APGD (Alpine Precipitation Gridded Dataset) as reference. The reference datasets were available in two versions: a deterministic one, developed in the FP7 EURO4M project, and a probabilistic version, described in this document from the contributions of EDI for WP1.
For the Alpine region the following datasets were evaluated: COSMO reanalyses (two datasets, a deterministic and an ensemble reanalysis, produced by DWD and UniBonn), HARMONIE (with a second version over a short time period with modified model physics, SHMI), MESCAN (six additional versions over a short time period with modified model physics and input data, Météo-France), Met Office reanalyses (two datasets, a deterministic and an ensemble reanalysis, Met Office) and additionally ERA-Interim (ECMWF), ERA20C (ECMWF), E-OBS and MESAN (EURO4M dataset).
The dataset preparation was time demanding (download from MARS Archive, conversion from GRIB2 to NetCDF, conversion between coordinate systems and aggregation to daily 06h-06h precipitation). The evaluation was mainly performed with data from the relatively short period 2006-2008, which corresponds to the time interval when all datasets of interest where available. Two grids were defined and the datasets accordingly rescaled: a coarse-resolution grid (0.25 degrees regular grid) and a fine-resolution grid of 5 Km.
In the evaluation, we have examined climate indices (such as mean annual precipitation, wet-day frequency and the 95% quantile), performance scores (such as the Root Mean Squared Error and Brier Skill Score), and the annual cycle and distribution function at various scales (such as the scale of grid pixels and hydrological catchments). Figure 9 shows, as an example, the mean annual precipitation of nine different datasets.
Some emphasis in the evaluation was placed on uncertainties in regional reanalyses and their dependence on the space scale. The scale dependency was derived by comparing the performance for different size classes of a hierarchical subdivision into river catchments. Despite the short period available for comparison, our study could identify some major differences between the datasets. Even though the statistical significance and level of detail may be somewhat limited, particularly so for extremes, this analyses points to features that are of interest to the developers of the UERRA regional reanalyses and relevant to users of these datasets. The results of our analyses are described in detail in deliverable 3.6 (report).
KNMI has compared regional reanalyses from WP2 with the E-OBS dataset from WP1. Due to the delayed archiving in MARS and the slow performance of retrieving the data from MARS and transferring it to KNMI limited the evaluation to only a few common years. For the ensemble reanalyses only 1 year could be used due to the limited amount of time between the archiving in MARS and the deadline for Deliverable D3.6. Climate indices based on daily minimum and maximum temperature were used for the evaluation of the reanalyses and E-OBS. A general observation is that the similarity between reanalysis and observations is good. While there are marked differences between reanalysis, these variations are generally rather small. We did find systematic biases of one reanalysis over parts of eastern Europe but this was ‘compensated’ by a spot-on resemblance with observations over other regions.
A recurring problem (also for the indices work in WP4) is the interaction with the MARS archive. Extracting data from MARS is often tedious and slow. A considerable amount of time was wasted on data recovery from MARS which turned out to incomplete, and therefore useless for further processing (like aggregation). Another issue which slowed progress was the lack of a standard conversion between GRIB and NetCDF. While some software was shared between institutes, the correct configuration of this software was not always clear.
While the MARS archive is valuable, a more efficient and direct data exchange between the producers of reanalyses and the limited amount of users (within the project) would have facilitated this work. Furthermore, putting the obligation to provide software to transfer data from GRIB to NetCDF (or the provision of data in both formats) to the data producers rather than the data users would increase efficiency. An example script to extract e.g. hourly (minimum) temperature values would have been useful as well.
The workshop on the synthesis of evaluation results was held 30 November – 1 December 2017 together with the concluding user workshop (from WP8). The user friendly synthesis report will be written after these workshops.
NMA Romania has assessed UERRA RRA data over Romania for precipitation against the ROCADA gridded data set. A good agreement in spatial structure was found but the Met Office RRA had a clear overestimation of the amounts. It also correlates well on the temporal scale, daily as well as monthly variations.
5 Facilitating downstram services (data, derived products and outreach)
The work on UERRA data portal and all related tools for data processing and archiving in MARS at ECMWF has continued and required a lot of effort but is now completed. See the UERRA web and the links to ECMWF wiki for detailed descriptions.
Important lesson learnt from the project beginning is how much time and effort cost to define all data standards and exact project requirements before any actual archiving work could start. With so many varying and heterogeneous re-analysis systems participating in the project it took first two years to identify fully all details specific to each datasets and to prepare the final list of all UERRA parameters including all necessary metadata (output steps, levels etc). The parallel work on defining common standards, based on WMO compliant GRIB2 format, meant to introduce some new parameters and concepts (e.g. new soil level type) not existing before. The work on common data standards is crucial for similar projects’ success as it allows unified access to the data and its inter-comparison.
Another longer then expected step was to prepare full data samples from each provider fully compliant with agreed requirements. Sometimes only during this step new challenges were identified impacting the decisions taken before. Once finally all full samples were gathered only then the final MARS design could be finalized. The test and following production archiving could be considered as rather simpler final step, still requiring a lot of effort and time due to the processed data volume (almost one petabyte).
An ESGF node of the type “data-node only” was installed at KNMI (https://esgf.knmi.nl/thredds). A dataset with the deterministic forecast analysis for 2-meter temperatures from the SMHI model was created for the period 2005-2010, by extracting the GRIB data from the ECMWF MARS archive and converting the GRIB data files to a NetCDF dataset with FIMEX (http://fimex.met.no). Then metadata (according to the ClipC DRS) was added to the NetCDF dataset and the resulting dataset was published on the ESGF node.
For the evaluation work from WP3, KNMI already calculated a few temperature indices on specific years of the available reanalyses datasets and E-OBS. Work is ongoing to run the indices on a larger number of years, but the retrieval from MARS and transfer to KNMI is slow (see also the part under WP3), partly due to the large size of the datasets. The focus is on 2006-2010 for all datasets, but there might be constraints due to the time available to the end of the year and disc space needed to store the raw data and derived products. Unfortunately, trend calculations do not seem realistic with the limited time period that is available for the reanalyses. Also part of the indices cannot be determined since these will need a 30 year reference period for each dataset which is not available in the archive.
SMHI has resumed its hydrological evaluation of the Regional reanalyses for as long time periods as available. See D 4.7.
Potential Impact:
1 Outreach activities
UERRA has had contacts and exchange with other SPACE and H2020 Projects for and at the Uncertainty Workshop in September 2016 (see above under Goals WP9). There have been general outreach activities via the web site, conferences and national contacts with users of climate and climate change data.
Two Newsletters have been published on the web and linked information has been sent to contacts.
Information / Project Profile leaflets have been prepared with two Magazines targeting policy makers and the EU commission and bodies. Parliament and Adjacent Government. The first was published in March (see https://www.theparliamentmagazine.eu/printpdf/3943 https://www.theparliamentmagazine.eu/articles/magazines/issue-429-07-march-2016)
The second one is in http://www.adjacentgovernment.co.uk/wp-content/uploads/2016/08/AG11-web-smaller-final.pdf#page=210&zoom=auto,-12,365
A third Newsletter was been published on the web and linked information has been sent to contacts.
.
http://uerra.eu/component/dpattachments/?task=attachment.download&id=176 and was quite nice and received attention judging at least from phone calls. The second will be published in August.The Project has been presented at several conferences by scientists in the project, Work package leaders and the Coordinator.
In particular at the European Meteorological Society/European Conference on applied Climatology/Meteorology in Sofia, September 2015, where two talks about UERRA in the context of Climate Services and of Reanalysis were given by the Coordinator jointly with the UERRA WP leaders. The were then the following European Meteorological Society/European Conferences on applied Climatology/Meteorology in Trieste in October 2016 and Dublin in October 2017. There where two talks about UERRA in the context of Climate Services and of Reanalysis were given by the Coordinator jointly with the UERRA WP leaders plus a few other UERRA or UERRA related talks.
See also a list of Conference presentations below, after the References (NB a selected list and it cannot claim to be complete as more UERRA scientists have given presentations in other forums and with some UERRA material included.
DWD contributed to the WP7 objectives by developing training material, particular to enable new users to start handling the reanalysis fields and extract areas of interest, to start plotting and to calculate means.
The KNMI contribution to a workshop (organized outside UERRA) in Africa for GFCS outreach was delayed due to the lack of a suitable workshop. An abstract has been accepted for the Climate
change in Africa: Evidence, mechanisms and Impacts, Past and Present workshop in Marrakesh, Morocco on 6-11 November 2017 and presented at that Workshop.
As part of Workpackage 8 (User Feedback) of UERRA two user workshops are planned: one half way, and one at the end of the project. Goals and set-up of both workshops differ, since at the time of the first workshop the UERRA data products will not yet be delivered, while at the second the final products and services will be presented to the users.
The 1st workshop which was held after the 3rd General Assembly of the project in Toulouse, France.
For the workshop 48 participants from 12 countries (18 from France) were registered. Of this group, 19 were working on applications of re-analyses data and 10 were not related to UERRA.
Participants came from a wide range of sectors. Applications (potentially) using re-analysis data that were presented at the workshop were: Energy (wind, solar, demand), Insurance, Transport, Agriculture, Defense, Hydrology, Climate Impacts, Model evaluation, and Atmospheric Physics.
The participants were asked to very briefly present their interest in re-analyses, examples of use, requirements, etcetera. This was done in the first part of the meeting.
A significant amount of time was allocated to discuss user requirements for data as well as for scientific and technological support. Although a wide range of sectors were represented in the meeting, it was a clear advantage that all users either already worked with re-analyses data, or were interested to do so in the near future. About half of the participating users worked with re-analyses data before. Several examples were shown of the use of ERA-40, ERA-Interim, or analyses made for operational weather forecasts. Because of the need for very detailed information (often related to extreme situations) several participants used some form of statistical downscaling to get to the scale relevant for their specific applications. Although it is clear that not all requirements can be met with the UERRA products, the discussion on the future data products did not lead to significant changes in the list of products to be produced by UERRA. Items not on this list, but considered useful by some participants were wind gusts at 100m height and CAPE (Convective Available Potential Energy) .
A short summary of the findings of the user presentations are listed in a table in the Workshop Report (D8.2).
A second workshop involving external climate service providers was organised by KNMI and other UERRA partners together with the evaluation workshop back-to-back with the UERRA general assembly (30 November 2017 – 1 December 2017). A user guidance was written using the results from the workshop.
DWD contributed to WP8 with substantial material for D8.1. DWD co-organised both user workshops and wrote the summary for the user requirements from the first workshop, ensuring consideration of it in WP3.
2 The multidude of Reanalyses
The multitude of RA datasets at high resolution will be a reason for extensive use of the datasets in climate change research. One important user group is the climate modeling community. The development of increasingly sophisticated climate models has reinforced the need for basic observations. Reference data for the present and recent past climate (including climate trends) during the last 30-50 years is an established requirement for all climate model evaluation exercises. The UERRA reanalyses also form the necessary baseline for future projections of climate change.
In theory, the analysis state at a particular time in any of the RAs is really only one realization of all possible atmospheric states in a probability distribution an ensemble of multiple analyses should be used. Moreover, this one realization might be affected by NWP model bias to some extent. Reducing Regional Climate Model (RCM) bias against one dataset is sub-optimal (see the ongoing FP7 EMBRACE project). Therefore, regional climate modellers want to compare their present-day climate simulations against as many different observational datasets as possible. At present, most of these are from satellites or gridded station observations at relatively coarse resolution. If UERRA is approved, there will be a multitude of new datasets at the end of the project that fit the needs of the RCM community much better than anything else available today.
For the full 50 years there are two different datasets (MESCAN and HARMONIE). For the satellite era from 1978 there will be two similar high resolution datasets produced with quite different modelling and analysis systems (HARMONIE and Unified Model). Additionally, there is a third alternative, the 5 km downscaling RA with MESCAN, again different to the first two, and even more different due to the different methods and observations (as explained in 1.2).
A 4th resp. 3rd (for 30 or 50 years) additional dataset will be provided by DWD and HErZ to the project to aid in the uncertainty estimation, and when the quality of the COSMO RA is evaluated in WP3, the RCM community will be recommended to also use this RA.
The additional RAs developed in the project (with different physics, satellites or not, and the KFENDA from UB) are probably too short to be of major interest for this community. However, the UB ensemble RA is developed with the proposed support from UERRA and is expected and planned by the HErZ and UB to continue this work to a multi-decadal RA as well, providing an additional deterministic control (central member in the ensemble).
3 Uncertainty estimates
A novel expected impact will come from the delivered uncertainties. Since we usually do not have uncertainty measures associated with most of today's observational datasets, it is somewhat speculative to predict how much the user community will make use of them. In general though, the notion of uncertainties and probabilities has won a lot of ground in hydrological and meteorological forecasting through ensemble techniques. The probabilistic approach requires some years to become accepted by users. For use in applications the probabilistic approach requires that ECV probabilities are translated to used impact parameters. For research users in the RCM community, this process will be easier than for the downstream users of RAs. The ensemble spread based RA uncertainties are expected to be quite widely used for model evaluation. They provide an indication for the relationship between model errors and the errors of the verifying RA dataset. In fact all model validation against any gridded data will become more stringent when the probability distribution function (PDF) of the gridded dataset is taken into account (in addition to the PDF of the model data). Since uncertainties from the ensemble RA will be produced for all modeled variables (both surface and upper air) and at all model time steps (as often as archived, for RA usually every 6 hours) the error bars are complete for all the model variables of the RCM undergoing the test. Uncertainty estimates are a definite advantage to the modelling scientist. However, also most downstream user products will benefit from uncertainty information, e.g. a customer or a journalist may want to know the reliability of analyzed trends in temperature, cirrus clouds, or in the occurrence of condensation trace from aircraft.
For a selected set of ECVs of particular user interest, to be established early in WP3, the most qualified uncertainty estimation will be made against several quite different datasets in WP3. The verifying observation datasets are state-of-the-art (including E-OBS and CRU which are enhanced with their error bars in WP1) and used to validate the climate quality of the RA datasets. Also the satellite derived datasets of e.g. cloudiness selected for RA evaluation are the best available for EUROPE (developed as part of CM-SAF). The uncertainties for climate time scales and a number of space scales will add credential to the Climate Indicators that will be derived from the RA and provided to downstream users and policy makers. The uncertainties will be as comprehensive as is possible for anyone to derive for a long time to come. To be able to answer the questions from policy makers about the reliability of the RA and the derived indicators this information is extremely important and will widen the use of the RA data.
UERRA must interface directly with the full range of intermediate- and end-user requirements, including disaster prevention, health, energy, water resources, ecosystems, forestry agriculture, transport, tourism and biodiversity. Developing user friendly data services is the most important step to making the long-term high resolution climate datasets and derived products available to the different user communities. Building on existing expertise, UERRA will develop these services by implementing existing techniques and extending these for RA data and products. We will start from the services built for the MARS archives in the TIGGE/LAM project at ECMWF and extend these to include the EURO4M data and link to the services for the THREDDS archives of climate model data developed at KNMI. The concepts are proven but there will be still significant work to apply them and enhance the services further to include RA data. These data services will for a start be used between the UERRA partners; WP2 will archive the data and WP4 will provide the data services for the evaluation of uncertainties in WP3. Subsequently, the RA data will be made available to outside users. It is anticipated that the consortium in CLIPC will develop data services for observational data coming out of UERRA (and its predecessor EURO4M) too. Therefore, the synergy effects of the two projects will be exploited. Data services for EURO4M data will facilitate the developments in UERRA WP3, since they are likely to be available before the main UERRA RA have been produced.
The observational datasets will substantially support (in combination with climate model predictions) climate change impact and adaptation action assessments, policy development and policy monitoring for European and national users. UERRA will complement the national gridded datasets, where they exist, in terms of being European wide and based on beyond state-of-the-art RA systems. For non-experts in geophysics and for policy makers the ECVs will be translated into climate indicator information adapted to user needs. As an important source of reliable information about the state of the climate in Europe, the suggested collaborative project is an important building block for Copernicus. The RAs will provide basic information for climate services that deliver consistent products and support downstream applications. We will assess whether the ensemble of reanalyses developed in this project is fit for generating climate change products that meet the user needs. The goal is to explore how the reanalysis data (and associated uncertainty information) are best exploited to develop user oriented products such as derived climate indicators and how best to utilize this additional information for “understanding past climates and climate change in Europe”. This activity will assure that the RA datasets and products are useful for policy-makers, planners and the citizens and their organizations. It will provide scientific input for policy implementation.
National decision makers and local authorities will be able to utilise the state–of–the–art UERRA data products and services for their country or region as input to climate change assessments, and the formulation of adaptation and mitigation strategies. It is the longer multi-decadal time scale addressed in UERRA that is needed for governments to minimize and adapt to the societal and environmental impacts of climate variability and change. European countries can directly use the results of the proposed project for their “national communications on climate change policies” which are a written requirement for the Conference of the Parties of the UNFCCC and include national GCOS implementation activities.
The processing systems set up in UERRA for reanalysis, post-processing of ensembles, data services and for products will be built up to a much higher level than available now. The demands for efficient production of the heavy computational runs, archiving of large amounts of data and user friendly and efficient data dissemination are high. It will be necessary to enhance the existing infrastructure for particular aspects of RA and data dissemination. This means that the Consortium will be in a good position to continue in real time the RA after UERRA finishes, provided that subsequent funding can be organized. Most likely there will be a need for re-doing the RA once NWP models and RA systems have developed further and/or additional input data have become available. Also, more computer power will enable higher resolution and/or longer time periods in future. As such, UERRA has the potential to evolve into a future Copernicus service on climate change monitoring that is fully complimentary and supporting the existing operational services.
List of Websites:
http://www.uerra.eu/
Per Undén
SMHI
60123 Norrköping
SWEDED
Per.Unden@smhi.se Tel. +46-11-4958449 or 4958000