InfraStructure for the European Network for Earth System Modelling

Informacje na temat projektu

IS-ENES

Identyfikator umowy o grant: 228203

Projekt został zamknięty

Data rozpoczęcia 1 Marca 2009

Data zakończenia 28 Lutego 2013

Finansowanie w ramach

• Specific Programme "Capacities": Research infrastructures

Koszt całkowity € 10 666 284,16

Wkład UE € 7 591 850,55

Koordynowany przez

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE CNRS
France

Final Report Summary - IS-ENES (InfraStructure for the European Network for Earth System Modelling)

Executive Summary:

IS-ENES is the distributed e-infrastructure of models, model data and metadata of the European Network for Earth System Modelling (ENES). This network gathers together the European modelling community working on understanding and predicting climate variability and change. The main objectives of IS-ENES are to further integrate the European climate modelling community, stimulate common developments of software for models and their environments, foster the execution and exploitation of high-end simulations and support the dissemination of model results to the climate research and impact communities. This infrastructure supports European contributions to international experiments used in assessments of the Intergovernmental Panel on Climate Change and provides the predictions on which EU mitigation and adaptation policies are built.

Integrate the European Earth’s climate system modelling community

IS-ENES has elaborated the ENES infrastructure strategy for 2012-2022, involving not only IS-ENES partners but more largely European climate modelling groups. It provides a long-term view on models, model data, high performance computing and organisation. This strategy is timely for the start of JPI Climate, ECRA and preparation of Horizon 2020. A first European training school on climate models has been launched. The ENES Portal has also been developed as a virtual resource centre (http://enes.org) providing access not only to IS-ENES services but also to information relevant for the community.

Stimulate common developments of Earth System Models for the understanding of climate change

IS-ENES has started a service on models including access to the ocean platform NEMO, the OASIS coupler and the post-processing tool CDO. For the first time it has gathered documentation of the European climate models. A Portal for model evaluation has been developed providing access to observational datasets commonly used for model evaluation as well as some diagnostic tools.

Foster high-end simulations enabling to better understand and predict future climate change

IS-ENES has stimulated the integration of the community around high-performance computing (HPC) with the establishment of an HPC task force, the realisation of two international workshops, the elaboration of a common strategy for HPC and collaboration with PRACE. Performance and portability have been tested on a range of platforms and developments have improved bottlenecks for the use of massively parallel computers such as the management of input/output data flow and coupling software.

Support the dissemination of Earth system model data to the climate and impact research communities

IS-ENES has supported the deployment of the European contribution to the international experiment CMIP5 and started to support a similar approach for the regional coordinated experiments CORDEX. IS-ENES has contributed to the development of software for the international distributed database, Earth System Grid Federation. A prototype of a portal for the impact community has been developed that provides access to documented use cases on methodology aspects and enables first developments of common tools and web services.

Project Context and Objectives:

IS-ENES is the infrastructure of the European Network for Earth System modelling (ENES). ENES gathers together the European community developing and exploiting climate models of the Earth system. This community aims to better understand present and past observed climates and predict future variability and changes under given boundary conditions of anthropogenic and external natural forcing. This community is strongly involved in the assessments of the Intergovernmental Panel on Climate Change (IPCC) and provides the predictions on which EU mitigation and adaptation policies are built. Since 2001, ENES continues to foster collaboration among the modelling groups to speed-up the development of models of the complex Earth’s climate system, called Earth System models, and facilitates common strategies and sharing of research infrastructures, including models, data repositories and high-performance computing facilities.

Challenges in the field of climate modelling

IS-ENES has been developed in order to contribute to the challenges facing the climate modelling community:

• To improve our understanding and prediction of future climate changes requires the analysis of the full complexity of the Earth system, i.e. the physical, biological and chemical dimensions coupled together.
• To improve our understanding and prediction of climate change impacts in all their socio-economic dimensions requires to better account for climate change on regional scales and to enhance interactions with the climate change impact community. This will be particularly required to prepare for adaptation to climate change.

In order to improve European competitiveness and expertise, there is also a need to:

• Better integrate countries new to the subject that want to be involved in the study of climate change. Indeed, with the increasing threat of climate change, these countries want to develop their own expertise on climate change prediction to prepare for adaptation.
• Perform the most up-to-date and accurate climate simulations. This requires sophisticated models, world-class high-performance computing and data storage systems, and state-of-the-art software infrastructures to make efficient use of the models and the hardware.

International and European background

IS-ENES has been developed in the context of the preparation of the 5th IPCC Assessment report to be published in 2013. In order to support IPCC AR5, the World Climate Research Program has organised an unprecedented set of coordinated international experiments with climate models, called the Coupled Model Intercomparison Project Phase 5, CMIP5. In Europe, seven models have contributed, from the UK, France (2), Germany, Italy, Norway and a consortium of countries EC-Earth lead by Netherlands. These simulations aim to support model evaluation, model process studies and climate projections which are defined by the WCRP Working Group on Coupled Models (WGCM) with the support of PCMDI in the USA. After the start of IS-ENES, coordinated experiments at the regional scale, called the Coordinated Regional Downscaling Experiments, CORDEX, have also been defined. Although this was not in the original plans, IS-ENES decided to contribute to CORDEX, too, due to it’s relevance for the community, and society at large.

Beyond coordinated experiments, the strategic framework of WCRP (http://www.wcrp-climate.org/) and the conclusions of the 2008 World Modelling Summit for Climate Prediction , both emphasized the need to move towards a common climate and Earth system modelling infrastructure. “This will help centres develop their own models, exchange model components, perform multi-model experiments and generally improve collaboration and efficiency.” (WCRP Strategy). This conclusion is fully in agreement with the recommendations of the first European foresight elaborated in 1998 within the concerted action “Euroclivar” (http://www.knmi.nl/euroclivar/frsum.html) “a better integration of the European modelling effort with respect to human potential, hardware and software”. More specifically, Euroclivar recommended to develop collaboration, to establish a European climate computing facility, and to enhance the exchange of software and model results. These recommendations led to the establishment of ENES, as the European Climate Modelling Group advocated by Euroclivar. ENES set up the FP5 infrastructure project “Program for Integrated Earth System Modelling” (PRISM, http://prism.enes.org/). PRISM carried out a successful first step towards the Euroclivar recommendations, establishing a network of expertise around ESM software environments and promoting a standard technical coupling interface now used world-wide, the OASIS coupler. This continued effort has helped to build the IS-ENES project. It was complemented by an FP7 e-Infrastructure project, METAFOR, “Common Metadata for Climate Modelling Digital repositories” (2007-2011 which developed a common information model to describe in a standard way climate model experiments for use in CMIP5. These metadata standards have been disseminated and implemented in IS-ENES and will be expanded in IS-ENES2.

Main objectives of IS-ENES

IS-ENES aimed at developing a common climate and Earth system modelling distributed research infrastructure in Europe, following the general strategy of the World Climate Modelling Program, in order to facilitate the development and exploitation of climate models and better fulfil the societal needs with regards to climate change issues.

IS-ENES has been organized around four main objectives: to further integrate the European climate modelling community, to stimulate common software development for models and their environments, to foster the execution and exploitation of high-end simulations and to support the dissemination of model results to the climate research and impact communities.

The four objectives were described as follow in the Description of Work:

Foster the integration of the European climate and Earth system modelling community

• Further integrate the European ESM community, through networking activities focusing on the development of the future ENES strategy, the exchange of expertise and the development of training activities (NA1 and NA3)
• Develop a virtual Earth System Modelling Resource Centre (v.E.R.C.) using ICT technologies to integrate the different distributed facilities currently existing or developed during this project (NA2)

Foster the development of Earth System Models for the understanding of climate change

• Increase the services around ESMs, by enhancing model documentation and developing a service on common tools and model components (NA3 and SA1)
• Foster the joint development and common evaluation of the European ESMs through networking activities and joint research activities on ESM software environment (i.e. the tools to prepare, run, store, evaluate and exploit model simulations) and ESM components (NA2, JRA1 and JRA3)

Foster high-end simulations enabling to better understand and predict future climate change

• Ensure an efficient access and execution of ESMs on high-performance computing facilities, by developing a common strategy, by enhancing the interface with and access to the EU large infrastructures DEISA2 and PRACE, by improving model performance on different computer architectures (NA1 and JRA2)

Foster the application of Earth system model simulations to better predict and understand future climate change impacts

• Enhance the dissemination of model results, by enhancing the service around model results following the INSPIRE EU directive and developing more efficient tools to access data (SA2 and JRA4)
• Enhance the interaction with decision makers and user communities, mainly concerned by climate change impact studies, through service activity and joint research development on data access as well as more adapted indicators. This will help Europe prepare for adaptation as recommended by the 2007 EU Green paper “Adapting to climate change in Europe” (NA1 and JRA5)

These four objectives are fulfilled through networking, service and joint research activities, serving the climate modelling community (ENES) as well as the impact research community.

Project Results:

1. Foster the integration of the European climate and Earth system modelling community

IS-ENES has elaborated the ENES infrastructure strategy for 2012-2022, involving not only IS-ENES partners but more largely European climate modelling groups. It provides a long-term view on models, model data, high performance computing and organisation. A first European training school on climate models has been launched in order to better integrate the young generation of climate modellers. The ENES Portal has also been developed as a virtual resource centre (http://enes.org) providing access not only to IS-ENES services but also to information relevant for the community.

1.1. ENES infrastructure strategy (2012-2022)

IS-ENES has worked with the ENES community to elaborate its infrastructure strategy for the next 10 years, now available on http://is.enes.org. The most demanding goal is to ensure that, by the end of the decade, global climate models will resolve kilometer scale convective processes with the objective to obtain more reliable regional climate predictions for the next few decades. This in turn will require having access to exascale computers, i.e. being able to perform 1018 operations per second. There will also be a need to prepare new generations of climate models able to run successfully on such massively parallel computers and able to improve their representation of physical and biogeochemical processes. This represents a major challenge for the European and worldwide community that will require stronger collaborations to reduce the scientific and technical burden of such developments. It will also be an opportunity to assess the extent of the scientific diversity of models that is required for maximum scientific knowledge generation, and to organize the European model diversity in order to better address uncertainties associated with models. Main recommendations are:

• Provide a blend of high-performance computing facilities ranging from national machines to a world-class computing facility suitable for climate applications, which, given the workload anticipated, may well have to be dedicated to climate simulations.
• Accelerate the preparation for exascale computing, e.g. by establishing closer links to PRACE and by developing new algorithms for massively parallel many-core computing
• Ensure data from climate simulations are easily available, well documented and quality controlled, especially for the climate impacts community
• Build a physical network connecting national archives with transfer capacities exceeding Tbits/sec.
• Strengthen the European expertise in climate science and computing to enable the long term vision to be realized

The ENES strategy also emphasizes several directions of work that have been used to prepare the second phase of IS-ENES:

• Strengthen the collaboration among European climate modeling groups; develop the European network of science experts and software engineers for enhancing future common developments, with a focus on bottom-up approaches which have proved to be very efficient; enhance community building through training;
• Strengthen collaboration with PRACE and prepare the next generation of climate models able to run on exascale computers; strengthen collaboration with computing and IT communities,
• Strengthen the integration of distributed databases and their documentation for a diversity of users; prepare tools able to manage very large volumes of data and develop interoperability with observations (satellite and in-situ) and re-analyses;
• Develop interface with the climate impact community; provide access to both global and regional model results, as well as tools and guidance on uncertainties adapted for these communities,
• Strengthen the ENES organisation and governance to increase visibility by the community and by national research institutions supporting climate modelling, and to enhance the international contribution of European scientists in international climate research infrastructure developments.

D2.1 - Infrastructure strategy for the European Earth System Modelling Community 2012-2022

D2.5 - Science Policy delivery of “The ENES Strategy and its implementation Plan”

1.2. First European training school on climate modelling (June 2012)

In 2012, ENES carried out it‘s first „European Earth System and Climate Modelling School“ (E2SCMS) supported by IS-ENES. The school took place on the Greek island of Kos from June 1 to 11 2012. The school educated 32 young researchers from all over Europe in Earth system modelling by providing a series of lectures on the Earth system and models for it, and, concurrently, giving them introductions and hands-on experiences in running up-to-date ESMs. A totally new property of the school was the fact that two comprehensive Coupled Earth System Models - the UK NCAS model and the MPI-ESM - were used for the school. The school was viewed to be very successful by most participants. It is planned in IS-ENES2 to have schools every other year.

D2.3 – "ENES First Prototype Summer School on Earth System Modelling"

1.3. ENES Portal

The ENES portal (http://enes.org) initially named the virtual Earth System Modelling Resource Centre (v.E.R.C.) was designed to be the single central portal, which acts as a common entrance point to the ESM infrastructure built within IS-ENES. The objective is to collect, integrate and link (IS-ENES) services and information in a coherent fashion, providing an overview of community resources and to offer a communication platform for the ESM community.

The main structure of the portal is organized along the main topics of ESM – models, data, computing and community. The service section gives direct access to the IS-ENES services. Additionally, all support issues are collected, a glossary is built up and announcements of community relevant news, events and vacancies are regularly updated. The content has been elaborated with IS-ENES partners and the display improved by a professional designer.

The portal is increasingly being noticed and used by the community, as reflected by 3800 visitors per month (average 2012) as well as support, contribution and support requests send via the contact form of the portal. The portal will be sustained as an information and collaboration platform for ENES.

D3.1 – Report on v.E.R.C. Portal Setup and Status
D3.7 – Final and comprehensive report on virtual ESM Resource Center

2. Foster the development of Earth System Models for the understanding of climate change

In order to foster the development of Earth System models in Europe, IS-ENES has focused on establishing standard documentation for European climate models, on implementing a service and supporting network activities on common tools and one model component, and on developing an evaluation portal giving access to datasets used to evaluate ESMs as well as some diagnostic tools.

2.1. Increasing the visibility of European climate model through access to model documentation

When preparing IS-ENES, we diagnosed a need to increase the visibility of European climate models and to provide information on the ENES Portal for European models used in CMIP5. This is important for both the users of model results to better understand the characteristics of each ESM and for model developers to know what are the different ESM component models available. Figure 1 shows the different model components used in ESMs and Table 1 highlights shared components of European ESMs. It was established for the Foresight based on model documentation provided by IS-ENES.

For all European ESMs, standard pages have been created on the ENES Portal, providing links to detailed description of each component. In a first step a standardized assembling guide for each ESM was elaborated (D4.1). This documentation covers the 6 European partners of IS-ENES, C-ESM from CMCC (Italy), CNRM-CM5 from Météo-France and CERFACS (France), MPI-ESM from MPG (Germany), EC-Earth from its consortium, HadGEM2-ES from the MetOffice (UK) and IPSL-ESM from CNRS-IPSL (France). It was then complemented with a 7th ESM, Norclim from Norway, also participating to CMIP5. A scientific contact was also identified for each ESM. This will help establishing extended services on models in IS-ENES2.

During CMIP5, an important effort was devoted to provide standard ESM metadata following the Common Information Model (CIM) developed in the ENES METAFOR EU project and adopted at the international level. IS-ENES supported the modeling groups for filling out the related questionnaire and helped ensuring consistency with data management protocols (D10.1). The ENES Portal now provides access to the 7 CIM model documentations. The CIM definition suffered from some delays, which led to combine the two deliverables D4.2 and D4.6

D4.1 - ESMs current documentation, including a description of associated tools and specific “assembling guide”
D4.6 combined with D4.2 – ESM revised documentation following the full CIM metadata.

2.2. Establishing services and networking around model environment tools and components

One objective of IS-ENES was to start services on models and their environment. It was decided to focus on the common software tools: OASIS, the coupler developed at CERFACS widely used to couple atmosphere and ocean models and sometimes land surface and biogeochemistry components (see Figure 1), and CDO, developed at MPG, which gathers a large range of operators to analyse and post-process model output data. On the model side, in a first step, it was decided to focus on the ocean modelling platform NEMO, used in many European ESMs and developed jointly by a consortium led by CNRS-IPSL and including France, UK and recently Italy.
Networking activities in WP4/NA3 have accompanied these services to further develop web sites, including documentation, user guides, tutorials, FAQs and user forums. For OASIS, networking also helped providing support to a wide range of users, both IS-ENES partners and other European modelling groups. For NEMO, a configuration database was established to help users select the best configuration for their use. An assessment of the services provided is provided below.

2.2.1. OASIS coupler

• Networking on OASIS

The OASIS coupler is a software allowing synchronized exchanges of coupling information between numerical codes representing different components of the Earth system and is used by about 35 climate modeling groups around the world. A comprehensive web site, completely integrated in the ENES Portal has been set up for OASIS. This web site (https://verc.enes.org/oasis) includes news and events, source download, documentation, tutorial, technical information, FAQ and forum, and dissemination information.

Within IS-ENES, WP4/NA3 proposed to provide OASIS Dedicated User Support (ODUS) to implement new coupled models or improve existing configurations. Calls for applicants have been opened at the beginning of each year of the project. A total of 3 person-months were available each year. In total, 31 applications, from 8 different countries, were assessed by an internal Selection Committee and 12 pm of ODUS were distributed in 7 different laboratories, including 4 non IS-ENES partners (AWI, ETHZ, BTU and UBO). Table 2 shows that only 2 national climate modelling centres were granted (Rossby and Hadley centres, associated to SMHI and Met Office Meteorological agencies) while other laboratory activities focus on ocean (AWI, LOCEAN, UBO), land surfaces (ETHZ) or atmosphere (BTU).

This support resulted in the implementation of HPC compliant interfaces in ECHAM for NEMO and FEOM, interfaces for regional modelling (COSMO-CLM-Parflow), an interface with already integrated Earth-system (COSMO-CESM), an interface for 3D regional/global two way nesting (COSMO-ECHAM) and an interface for global model with zoom (NEMO/ERNA-ARPEGE). OASIS4 and OASIS3-MCT behaviour and performance on HPC configurations were also evaluated. It helped to optimise climate model performances on supercomputers (Ec-Earth, COSMO-CLM), developing a specific tool for performance measurement (“lucia”). The ODUS program provided bug reports and coupler enhancement suggestions to CERFACS.

The ODUS program has been very much appreciated and was complemented by 5 general OASIS training sessions held in CERFACS. However, this support is very demanding in man power; general training session will be continued in IS-ENES2 but ODUS will only be possible through specific contracting with CERFACS.

D4.3 – OASIS complete web site accessible via the v.E.R.C. portal
D4.7 – OASIS Dedicated User Support (original title: General coupled model assembling guide based on practical experience gained when providing dedicated user support

• Service on OASIS

Services for OASIS started through the existing wiki at https://oasistrac.cerfacs.fr and continued through the new OASIS web site and Redmine project management site accessible from there (https://verc.enes.org/models/software-tools/oasis). Through this wiki and now through the Redmine site, users can browse OASIS sources and documentation registered in the Subversion server at CERFACS and consult tickets describing developments. Service activity for OASIS is summarized in Table 3.

M 5.7 – Report of v.E.R.C. services on OASIS, NEMO and CDO

2.2.2. CDO Climate Data Operators

• Networking on CDO

CDO is a set about more than 600 operators that can be used for operations on climate model output using standard data formats used in the community. In IS-ENES WP4/NA3, a new version of the CDO web platform has been implemented in March 2010 (https://code.zmaw.de/projects/cdo). It provides basic and advanced features of a sophisticated open source project management service. It contains a complete documentation on every operator of CDO, a detailed installation guide for multiple platforms, a list of solutions for frequently arising problems and a tutorial. Main parts of the new CDO web page are directly accessible from the ENES portal (https://verc.enes.org/models/software-tools/cdo) namely the homepage, the user guide and the download area.

Users have the opportunity to get in contact with each other on a single platform using dedicated forums. At the end of IS-ENES there are more than 1400 contributions to the different forums on the website. This helps other users solving their problems in a faster way. Forums are also utilized for discussions about new features between users and developers. These forums highly improve the acceptance of CDO and represent a great support to the development team. It marks the entry point of a user driven development.

The issue tracking system takes this aspect a step further. Focusing on certain topics, it ensures the transparency of the current work: Responsibilities, design decisions and implementation are available to any user. This facilitates to precisely implement the features discussed in the forums influenced by the user community. It is an extensively used feature of the new web service.

D4.4 – CDO complete web site accessible via the v.E.R.C. portal

• Service on CDO

The service on CDO is accessed through the project page. The community of users (~about 1000 users) consists not only of scientific users, but also of people from industry. Users can download CDOs from the web site and documentation. The CDO developers at MPG answer requests from users, collect errors and maintain discussion forums. During the third period a new version has been released CDO 1.5.4.

The three period reporting periods exhibit a large and growing user community of the CDOs (Table 4). By means of the new website that was setup in the IS-ENES project, up to 16 experts were able to respond to hundreds of user requests and thousands of downloads were registered. At the same time the webpage offers communication space to allow the users to help each other and the vast amount 1400 contributions over the website reflect the acceptance of these services by the community.

M 5.7 – Report of v.E.R.C. services on OASIS, NEMO and CDO

2.2.3. NEMO ocean platform

• Networking on NEMO

The NEMO modelling platform (Nucleus for European Modelling of the Ocean) is widely used by climate modellers in Europe. In order to help users, IS-ENES has supported the definition of key diagnostics, the setting-up of tools and a web database in order to facilitate the documentation/illustration of the diversity of the simulations that can be achieved with NEMO, helping the users to understand the processes at stake in ocean modelling and to select the most appropriate configuration for their ESM.

In a first step, the definition of an experimental protocol, the implementation of extensive diagnostics and a first example was implemented in the database (D4.5). The experimental protocol is based on 2 kinds of experiments: a 2000-year long simulation forced with the perpetual normal year (dedicated long term equilibrium and thermohaline circulation) and a 180-year long simulation made of 3 time 60 years of the inter-annual forcing (dedicated to validate the configuration over recent years). 34 diagnostics have been implemented and specifically written to fit the specificities of the global grid used in NEMO. They produce more than 50 plots and allow either a comparison with observed climatologies, or a comparison between 2 experiments. Time-averaged diagnostics produce maps (in surface or at depth, pole projections…), vertical sections (global or basin mean, equatorial sections…), stream functions, integrated heat transport… and spatially-averaged diagnostics display global mean times series and time-depth diagrams.

In a second step, the diagnostics have been applied by the NEMO system team (CNRS-IPSL) to validate NEMO v3.4.1 which was released early March 2013. The database and its diagnostics was also extensively used in a study on the impact of the sea-ice model on ocean at global scale in order to illustrate the usefulness of the database and encourage others to fill in the database with their results (D4.8) (Figure 2).

D4.5 – web NEMO configurations database filled up by NEMO team
D4.8 – web NEMO configurations database filled up by NEMO team and partners

• Service on NEMO

NEMO services are organized around the existing web page http://www.nemo-ocean.eu Through this web page, users have access to detailed information such as reference manuals, user guides, publications, forums, meetings and news. In addition NEMO is registered in the Subversion server http://forge.ipsl.jussieu.fr/nemo/ allowing a complete access of the code (including its full history) and a ticketing system.

During the IS-ENES project this material has been maintained and extended e.g. by a forum dedicated to the use of NEMO in coupled models to share information related to the preparation of next IPCC simulations. Support services have been provided during all three reporting periods (RP) by distributing NEMO to the community, processing user requests by fixing tickets received from users (93 for RP3; 214 for RP2; 64 for RP1) leading to 464 commits in RP3, 1251 in RP2 and 206 in RP1, direct user support via mails (~100 emails sent during RP3, ~400 during RP2 and ~ 300 during RP1).

M 5.7 – Report of v.E.R.C. services on OASIS, NEMO and CDO

2.3. Evaluation Portal

In order to facilitate the scientific evaluation of complex ESMs, IS-ENES has developed an a portal devoted to model evaluation (WP9/JRA3). The aim of this infrastructure was to standardize, harmonize and simplify - where possible - the tools and methodologies used in existing or past model inter-comparison projects to evaluate the basic quality of an ESM model. It aimed to gather and in a “one-stop-shop” all the necessary information and data.

The evaluation infrastructure (D9.2) provides links to observational datasets used by European climate models to evaluate different variables concerning: aerosols, carbon cycle, chemistry, clouds, ocean, precipitation, radiation and surface fluxes. In order to obtain this information, a questionnaire was sent to the modelling groups participating in IS-ENES (D9.1). Gathering this information is completely innovative and shows the diversity of datasets used for model evaluation. It is a slight deviation from the DOW, which initially planned to use results from model inter-comparison projects. This was decided because it appeared more practical and efficient to survey directly the major European ESM groups and more adapted for a use by European ESMs.

In the evaluation portal, the user can access either different layers of information: model components (aerosols, carbon cycle…), properties evaluated (variables), observational datasets used for model evaluation. For aerosols, 15 different datasets are used by 5 ESMs, for the carbon cycle 26 (5 ESMs), for clouds 16 datasets (5 ESMs), for ocean 41 datasets (3 ESMs), for precipitation 9 datasets (5 ESMs), for radiation 13 datasets (5 ESMs), for surface fluxes 24 datasets (5 ESMs). The different sources of data are displayed in Figure 3.

The Evaluation Portal was also enriched with a section that included new evaluation toolkits (D9.3). Four new toolkits were produced, at least partially, in the framework of IS-ENES: AEROCOM tool for aerosols, CCMVal-Diag tool for chemistry, HOAPS simulator for ocean atmosphere parameters and fluxes, and MCMS algorithm for storm tracks. Those new toolkits facilitate the comparison of the results of the ESMs to observational data, access to which is also provided. In addition the comparison with observations provided by the users is supported as well as the inter-comparison between different models. All the necessary information for the potential user of each tool is documented in a common homogenized format. These tools have been tested with one ESM, the one from CNRS-IPSL, in order to ensure their use by external users (D9.4). This evaluation was not exhaustive but allowed to show the usability of these diagnostic tools and provide some recommendations for further improvements.

IS-ENES has contributed to the development of some aspects related to the infrastructure needed for model evaluation. The evaluation portal gathers a lot of interesting information but also shows the limit of this approach, in particular the difficulty to maintain up to date information and provide a large range of toolkits required for model evaluation. Moreover, model evaluation still requires a lot of research development, some being tackled in the ENES EMBRACE project. In IS-ENES2 it was decided to elaborate a roadmap on infrastructure for model evaluation and what has been produced in IS-ENES can provide an interesting input to this foresight exercise. This issue is not only raised at the European level and requires an international organisation around model evaluation.

D9.1 – Report on evaluation datasets, methods, and problems for a survey of European ESM groups
D9.2 – Evaluation portal: a support structure to facilitate model evaluation
D9.3 – Report on creation and provision of access to new evaluation tools and the corresponding observational datasets
D9.4 – Report on the pilot study on the implementation of evaluation toolkits on CNRS-IPSL ESM output

3. Foster high-end simulations enabling to better understand and predict future climate change

High-performance computing is a very critical component of the climate modeling infrastructure. If the provision of computing access is beyond the scope of IS-ENES, the project has worked in several directions to improve and ease access to HPC in Europe: on the organizational and networking levels including the collaboration with European facilities (WP2/NA1), on workflows and integration of the European HPC ecosystem (WP3/NA2), on model performance (WP8/JRA2) and removing two important simulation bottlenecks: data flow (Input/output) and coupler (WP7/JRA1).

3.1. Organizational and networking activities related to HPC

An ENES High-Performance Computing (HPC) Task Force was established within IS-ENES to help providing concerted responses to different HPC issues in Europe. Several members contributed to the EESI FP7 roadmap on exascale in 2011 as well as to the PRACE scientific case update in 2012. The ENES HPC TF also efficiently prepared the HPC strategy part of the ENES infrastructure strategy and the ENES letter of intent regarding PRACE programme access (M2.2). IS-ENES also eased discussion to prepare the European responses to the G8 call on exascale computing in the field of environment which has led to the success of 2 projects: ExArch on data and ICOMEX on dynamical cores.

3.1.1. Collaboration with DEISA2 and PRACE

IS-ENES interacted with the European facilities PRACE and DEISA-2, in order to facilitate their use and prepare for the future challenges posed by highly massively parallel architectures.

During the first period, ENES was recognised by DEISA2 as a “virtual community”. ENES was granted 2 Mhours of computing time in 2009 on the HECToR UK National Supercomputing facility used by the community for code testing. Although this allocation was too limited to run numerical experiments, it proved very useful to test the scalability of the Metoffice model and yield to a large Tier-0 project on PRACE machines (UPSCALE project).

Two meetings were organised with DEISA2 and PRACE, one in May 2010 in Barcelona and one in December 2010 in Paris. They helped establish collaborations with PRACE. The Paris meeting launched the interaction with the PRACE first implementation project (1IP). The ENES community chose to devote this collaboration to preparing the EC-Earth model for PRACE Tier 0 and Tier 1 machines and to sharing outcomes with other modelling groups. This work helped, in particular, to increase NEMO ocean model efficiency. The final report was presented at the final IS-ENES General Assembly in Toulouse.

During the September 2011 meeting in Paris, the HPC Task-Force elaborated with members of the community a letter of intent for PRACE programme access and decided on the main climate modeling topical priorities for the PRACE 2IP (WP8) collaboration: coupler, new dynamical cores, model Input/output and fault tolerance. Results were also discussed at the 3rd General Assembly. Within this project, collaboration was mainly developed with computing centers. Some fruitful contributions on our priorities could be achieved, but more limited than expected since it required a convergence of interest between the computing center and the climate modelling community.

The PRACE Board of directors invited ENES to discuss on how to better fulfill climate modelling needs within PRACE on March 20th, 2013. This is the first user community meeting ever organized by PRACE. It was an opportunity for ENES to present its infrastructure strategy (D2.4) and requirements as emphasized in the Letter of Intent for programme access (M2.2). More specifically, it was an opportunity to emphasize the strong demand of the ENES community to access world-class computers, such as those provided by PRACE, but also to present our specificities limiting an efficient use of the current PRACE systems, such as the need for large data storage, multi-year access and stable environments, and in most countries the use of national dedicated centres not part of PRACE. It was decided to investigate the possibility of a special action to better integrate climate modelling in PRACE infrastructure, in particular how PRACE could be used for some high-end experiments within the next IPCC experiments.

3.1.2. Networking on HPC

Two international workshops on HPC were organized in order to foster the exchange on the state of the art of HPC for climate models. A first workshop was organized, mainly by CMCC, and held in Lecce in December 2011. It was an opportunity to share views on on-going developments in Europe with USA and Japan, to investigate the issues related with future generations of HPC and the need to revisit model dynamical cores.

A 2nd IS-ENES workshop on “HPC for climate models”, organized by CERFACS, CNRS-IPSL and CMCC, was held in Toulouse in January 2013 with invited speakers from Europe and USA. The workshop emphasized the importance of accessing world-class computers adapted to climate needs and was an opportunity to exchange on model performance, on the new dynamical cores (showing their importance to enhance model scalability), on the need to include performance information in metadata, and on the usefulness of establishing a series of seminars on performance. Discussions on future architectures also emphasized that the issue of dealing with large data sets (“exabytes”) is probably even more critical than the issue of reaching exaflop performance. A summary paper was submitted to BAMS for publication.

D2.4 – Foresight position paper on the “ENES strategy and its implementation plan”

M2.2 – First list of requirements for PRACE (covered by the ENES letter of intent to the ENES Programme access)

3.2. Improving model computing performance

3.2.1. Performance of European ESMs

Analyzing and improving model performance is very important in order to use computers with best efficiency. In IS-ENES, tests on portability and analysis of model performance have been performed and shared for different European ESMs. It was the first time that the community organized a comparison of technical model performance results.

In a first step, an “evaluation suite” has been defined (D8.1) gathering information about the ESMs provided to the project by the project partners. D8.1 gave details of model configuration, example input file configuration, code and library dependencies and basic performance information for a machine on which the model has been executed. It formed the basis for determining work on performance and helped to disseminate knowledge and experiences with current ESMs throughout the project partners, and beyond.

A first set of performance analyses have been performed in 2010 (D8.2) on four of the 6 ESMs in the IS-ENES suite plus work on the NEMO ocean model, using a number of different machines. A collaborative effort between application owners and computer specialists has led to the identification and investigation of a number of issues, further investigated in D8.4. This latter work included a larger set of models and HPC architectures, in particular targeting PRACE Tier0 and Tier 1 machines often under PRACE preparatory access grants, using model configurations with a higher spatial resolution, to better emphasize the current status for high-end experiments. These analyses included a specific focus on experiences porting the models, enabling consolidated feedback to be provided to PRACE. These experiments showed that the provision of standard environments on new machines improves the porting of models and that running a mixture of MPI and MPI/OpenMP models raises difficulties. It also demonstrated the benefit of using state-of-the-art performance tools, such as PARAVER at BSC, and the benefit of working closely with performance experts. Finally, specific scalability limitations in several ESMs were identified, and improvements made. Figure 4 gathers results on model performance for models at about 30 km resolution. The graph shows that most European models, at 25-30 km resolution, currently scale to around 1000-3000 cores with performance in the range 0.5 to 2 simulated years per day. Although this allows to use reasonably well current Tier-0 machines, European models tend to be less fast and more limited in their use of a large number of processors, calling for changes in dynamical cores as experienced in the US. Moreover, for all models, scalability well beyond tens of thousands of cores will be required for the Exascale computers envisaged to be in production towards the end of the current decade and require further developments.

D8.1 – Report on the Definition of the Evaluation Suite
D8.2 – Report on the Description of the Evaluation Suite and Base-case Results
D8.4 – Final Report on Portability and Performance in IS-ENES ESMs

3.2.2. Improving computing performance for the coupler

As shown in Figure 1, the coupler is required to exchange fields between the atmosphere and ocean and even between some other components for some ESMs. One initial objective of IS-ENES was to deliver a fully parallelised and optimized version of, the OASIS4 coupler.
As described in details on D7.2 a lot of effort was devoted to the development of OASIS4 to improve its fully parallel calculation of the re-gridding weights and addresses. A beta version was provided and used in few coupled configurations e.g. for 3D coupling between atmosphere and atmospheric chemistry components at ECMWF, KNMI and Météo-France in the framework of the EU GEMS project, at BoM (Australia) for ocean-atmosphere limited area coupled model and at SMHI (Sweden) for regional ocean-atmosphere coupling applied to the Arctic region. But different problems were also highlighted that could not be solved with available human resources. Furthermore, an analysis conducted within IS-ENES showed that the OASIS4 code base was not a stable basis to build on as it had reached a point where it was too complex to evolve easily to answer current and future climate modeling coupling needs, in particular the support of unstructured grids. The development of OASIS4 was abandoned in July 2011.
At the time it was however expected that even if OASIS3 still addressed the needs of coupled climate models, its limited parallelism would soon become a bottleneck in coupled simulations running on more than O(1000) cores.

In order to solve this issue, the possibility of interfacing OASIS3 to the "Model Coupling Toolkit" (MCT), a communication library developed at the US Argonne National Laboratory, has been investigated. Initial testing indicates that models using OASIS3 could easily use the new parallel OASIS3-MCT and would benefit from significant improvement in scalability. Therefore it was decided to further develop OASIS3-MCT, instead of OASIS4, to fulfil the D7.4 deliverable. The OASIS3-MCT_2.0 was released in April 2013. It offers today a fully parallel implementation of coupling field re-gridding and exchange, while supporting most of OASIS3.3 previous functions. The scalability tests done with OASIS3-MCT_2.0 at high number of cores (Figure 5) and the fact that it supports unstructured grids allow us to conclude that this coupler offers today a fully parallel and efficient coupling solution answering the short and mid term needs of the European climate modelling community.

D7.2 – Fully parallelised and optimised version of OASIS4 answering needs of current coupled climate models
D7.4 – OASIS3-MCT parallel coupler (Original title in DoW: OASIS4 including high priority developments as identified in the User Survey)

3.2.3. Flexible construction of ESMs

IS-ENES has supported research into future coupling technologies through the further development of the Bespoke Framework Generation version 2 (BFG2). BFG seeks to provide flexibility in the construction and deployment of future, community-based, ESMs using appropriate underlying coupling systems, such as OASIS and ESMF. The proposal is that developers of Earth System Modelling scientific software adopt a standard, relatively low-level interface for specifying the details of couplings etc. that separates science code from the specific details of the specific software infrastructure selected to implement a coupled model in code. This approach frees the model from being limited to using a particular software infrastructure and also isolates the model from future changes in infrastructure code. Isolating science code from details of specific infrastructure technologies provides flexibility in the use of that model, both in terms of coupling a model to other models and in terms of deploying models onto a variety of computing systems. For example, it allows the implementation of the infrastructure to change with no impact on the science code. This promotes, among other things, the sharing of models between groups in the community by allowing one model to be used in the context of a coupled model implemented explicitly using another technology. For example, such a model may be 'targeted' to use ESMF or OASIS as the coupling technology as required.

In IS-ENES WP8/JRA2, research has been undertaken to extend the functionality of the BFG to address the emerging needs of the Earth System Modelling community. (A companion EU project, ERMITAGE (265170), has also looked at addressing the requirements of the Integrated Assessment modelling community). In the first two year phase of IS-ENES (resulting in D8.3) in response to the outcome of the first IS-ENES-sponsored « Workshop on Coupling Technologies for ESM », organized by CERFACS in Toulouse in 2011, BFG was extended to support the coupling of models that are already coded as separate programs. This resulted in the definition of a program-compliant interface for BFG and demonstrations, using toy models, were developed to illustrate the use of both component- and program- compliant models (and their joint use in a coupled model). An initial BFG portal was also developed to provide on-line access to information about BFG, and to act as a repository for examples of BFG metadata, providing the ability to run BFG on user-supplied metadata and also to run other BFG utilities that were developed in the project. In the final year of IS-ENES, the main undertaking was to research the provision of full support for the coupling of parallel models in BFG (D8.5). The work however had to be re-targeted following the departure of one of the main developers of BFG from Manchester to STFC. The development therefore targeted the use of the emerging quasi-uniform grids used in new dynamical cores to improve scalability. This system was demonstrated on toy models, and future work will result in a BFG generation system that targets the coupling of parallel models (using ESMF and/or OASIS-MCT, for example). Further work was also undertaken to extend the functionality of the BFG portal, such as viewers for composition metadata, examples of coupled models and editing facilities for user-uploaded metadata.

D8.3 – Towards Flexible Construction of ESMs using BFG
D8.5 – Towards Flexible Construction of ESMs using BFG, Final Report

3.2.4. Improving parallel I/O

Climate simulation and weather prediction models have to write out field data for later analysis regularly. Depending on the write-out frequency and the grid resolution of the model the amount of data written to hard disk can be significant. The common method for writing this data was (and partially still is) to collect all data on a single process and write it to hard disk from there in serial. For today’s climate models this is a huge bottleneck. Instead of collecting the data on a single process, it is possible to let all processes write out their data in parallel, which is known as parallel I/O. However, this has its own problems. A better solution is to dedicate a set of processes to do the writing. Meanwhile, the computing process can continue with the simulation.

Work on IO within WP7/JRA1 of the IS-ENES project envisaged a review of a range of existing IO technology following a workshop organised by IS-ENES in February 2012 (D7.3). Two main approaches have been considered under the umbrella of IS-ENES both including parallel IO and IO server but responding to different needs. On one hand, the further development of an IO server system, named XIOS, was done by CNRS-IPSL, using parallel IO functionality of the common format used for climate models NetCDF and external configuration files of model output. XIOS was tested and released with the latest version of NEMO. On the other hand, a review and trialling of existing parallel IO packages at DKRZ (D7.1) has resulted in extending the existing CDI serial IO library for parallel IO with IO servers (CDI-pio). CDI-pio works on files using the GRIB format commonly used in weather prediction and climate models.

To improve flexibility, XIOS greatly simplifies the method defining outputs from a model by describing in an XML file, parsed at runtime. By implementing concepts of inheritance hierarchy within the syntax of the XML, definitions are much more compact and non-redundant. To improve performance, a proportion of the processors used to run the model are set up to run as “IO servers” exclusively dedicated to file operations (Figure 6). Data are transferred from client processes (models) to server processes through asynchronous communications, allowing concurrency of computing, data transfer and write access to the file system. In addition, through the sequence of layers NETCDF4/HDF5/MPI-IO libraries, XIOS exploits the parallelism of the computer file system. Results on NEMO demonstrated the possibility to write a much larger volume of data for the same period of time.

CDI (Climate Data Interface) is an I/O library that is being developed at MPG. It supports serial writing using various data formats. Based on this, CDI-pio extends the CDI library with parallel I/O server capabilities. In order to reduce the impact of writing on the overall run time as much as possible, CDI-pio uses a one-sided communication scheme to transfer the data to the I/O processes. For this, the model copies the data that needs to be written into a special buffer, which supports remote direct memory access. The I/O processes can then collect the data using MPI operations. Depending on the MPI implementation and underlying hardware these operations can be implemented in a way that the transfer can be done without interfering with the computing resource of the compute processes. When combined with compression, this format typically results in a 1/6th reduction in storage when compared to similar NetCDF output files and is therefore suited for archival purposes.

D7.1 – Documentation and tutorial on parallel I/O and I/O servers and their benefits
D7.3 – Reference implementations of Parallel I/O and of I/O Server
D7.5 – Reference implementations of IO server combined with Parallel I/O

3.3. Integrating the HPC ecosystem in Europe: workflows, unified HPC environment and prototyping ESM grid environment

Users as well as developers of ESMs rely on an infrastructure, consisting of high-end computing, data storage and network resources to perform complex and demanding simulations. In the past, mainly local resources and infrastructures were used. However, the increasing requirements on computing capability and capacity as well as on data storage facilities often exceed the possibilities of single centres. Today, a typical workflow in ES modelling consists of many individual steps, from source code archiving, model documentation, data pre and post-processing, that are performed on different machines, potentially at different sites, and often by different scientists. In Europe, there is the need to deploy and, where needed, to develop technologies in order to provide climate scientists with virtual proximity to distributed data and distributed compute resources.

3.3.1. ESM workflows

Different typical ESM workflows used by modelling groups in Europe have been analysed using a questionnaire developed in IS-ENES, focusing in particular on bottlenecks, restrictions and requirements for IT resources. Results from the questionnaire have been complemented by the user survey done in WP7/JRA1 (M7.1) and the metadata CIM questionnaire filled for CMIP5.

The analysis of the survey yielded basic ‘reference workflows’, i.e. generalized workflow templates for basic ESM applications. These workflows are presented in the ENES portal to the science community in a proper, comprehensible and understandable way using different adapted software (Figure 7). D3.5 focuses on the overarching and comprehensive “CMIP5 workflow” that contains other ‘reference workflows’ as sub-workflows, typically “data creation” (runs), “data archiving”, “data dissemination” and “data evaluation”. Each sub-workflow can itself be described by a number of sub-workflows It is hoped that scientists will benefit from this representation not only by getting a structured view of the complex underlying processing pipelines, but also by re-using sub-workflows and by referred to the effective usage of tools and services. The overall intention is to simplify composition and enhancement of user-specific workflows, which will result in an effective execution and optimal performance.

D3.5 combined with D3.2 – The suite of reference workflows

3.3.2. Unified HPC environment

In recent years we see a trend to answer the growing demand of resources with a provision of a gridded network of services all over Europe. However, using different computers raises hard problems to Earth System modelers due to the difference in the environments found in different supercomputing sites. Whereas there exist some – although not sufficient - standards in terms of operating, queuing, file and archive systems, there is no standard at all for Earth System models. So whenever a modeler or a model has to move to a new site, everything needs to be readjusted to the new environment.

Following the work on workflows, D3.6 analyses answers to questionnaires and practice from different modeling center in order to investigate the possibility for unification and standardization on different computing centers. Results show a demand to unify the modeling environment and infrastructure to enable successful collaboration and intercomparison of model results. Models or code parts have to be shared as well as data and results. From D3.6 analyses, unification potential can be found at every step of the workflows. Within CMIP5, first steps have even been made to standardize and unify data access (ESGF) and model experiment description (CIM metadata), which is a big step towards easier interaction and collaboration. However, there is still a big gap in defining minimum standards in software development, e.g. defining interfaces for model components, standard I/O and others. Moreover the workflows from retrieving code and environment to archiving the results are far away from standardized or uniform. The structures have historical reasons on the one hand, and are resulting from restrictions of the target platforms. To go beyond present state will only be possible if the scientific community agrees upon in certain standards and is willing to accept changes in their traditional workflows. One important conclusion is that an environment provided by HPC centers (directory structure, libraries, tolls, compiler versions…) has to be as stable as possible during a long as possible time span, at least for the lifespan of a project (which typically is in the order of many months to a couple of years!). The reason is that the overall time needed for porting a code, physically tuning the code, optimizing the performance for a special architecture / machine, optimizing the code for a special physical set up, is so time consuming and special for every target platform, that it only makes sense if one can rely on this customer tailored configuration for a sufficiently long time. Also data processing, dissemination and archiving require a stable HPC environment since also these processes are also automatized like the modelling workflow itself.

D3.6 combined with D3.3 – Remarks to a Unified HPC Environment

3.3.3. Prototype grid environment

The WP3/NA2 aimed at setting up and deploying an e-infrastructure providing climate scientists with the necessary virtual proximity to distributed data and distributed compute resources using the ENES portal as an interface. D3.4 reports on the deployment of a prototype used for to test a distributed environment for running ensembles of multi-model experiments. Considering existing grid infrastructures and services, the design of this grid prototype has been lead by the necessity to build a framework that will leverage the external services offered within the European HPC ecosystem, e.g. DEISA2 and PRACE (now merged).
The ESM Grid Environment prototype allows exploiting grid services already available grid services, namely GRB services and Globus middleware. A web service based system allows the brokering and scheduling of jobs in a distributed environment, supporting different Grid middleware. The prototype has been deployed involving the three sites composed of CMCC, DKRZ and BSC. Information of the job status has been enriched with details on an ensemble experiment, knowing, at run time, the progress of each single ensemble member, involved resources, etc. Finally a case study related to a global coupled ocean-atmosphere general circulation model (AOGCM) developed by CMCC, has been considered and preliminary tests carried out on CMCC and DKRZ sites have been reported.

Following the development of the Grid Prototype, a series of tutorials has been organised in several climate centres in order to increase the interest and knowledge related to grid technologies; familiarizing users with the Grid Environment and its features through an active hands-on approach; introducing a user-friendly tool for executing and monitoring ensemble experiments through the ENES portal. Seven tutorials have been organized among the main climate centres in Europe. In particular, the tutorials have been organized at: CMCC, BSC, CNRS-IPSL, DKRZ and INGV (Italy). The Grid infrastructure used for the tutorial exploited the three computing nodes available. The tutorials were targeted mainly to users with a medium skill level on climate modelling and basic knowledge on grid computing and shell programming. Fruitful feedback has been acquired from the audience for improving the Grid Prototype. The overall evaluation of the grid environment can be considered positive. Nevertheless the tools are still lacking on several key aspects. In the next phase, the infrastructure will be improved with the deployment of several other climate coupled models. The idea is to create different Virtual Organizations in which the users of the same group share the same models. The infrastructure will also grant the access for guest users that can execute a toy model for demo purposes.

D3.4 - Report on design and deployment of Grid portal and development environment
D3.8 - Report on Training Sessions

4. Foster the application of Earth system model simulations to better predict and understand future climate change impacts

A lot of effort has been devoted to the deployment of the European contribution to CMIP5. Thanks to the service on data, European nodes have been the first to publish data, the first to meet agreed data access standards and the first to complete the second level of the 3 level quality control process. Two of the three core data nodes in CMIP5 ESGF (DKRZ and BADC) are located in Europe. Through IS-ENES, Europe has contributed to the development of software, both directly through design and delivery of packages and through participation in testing and evaluation of the infrastructure of the Earth System grid Federation (ESGF). A prototype of a portal for the impact community has also been developed that provides access to documented use cases and download and visualisation tools. IS-ENES has also developed collaboration with the FP7 EUDAT project on European Data bases (http://www.eudat.eu/) aiming at providing a pan-European solution to the challenge of data proliferation in Europe's scientific and research communities. ENES operates as one the five core communities in EUDAT.

IS-ENES has been mainly devoted to the CMIP5 international experiments. However, as the CORDEX experiments have developed, IS-ENES has decided to use some unused funding to support the use of ESGF by CORDEX with the objective to establish a common distributed database include both global and regional climate model results. This will strongly ease common analyses and was a strong request from users from the climate impact community (D2.2).

Setting up ESGF has been more complex than originally planned with delays and changes occurring during the course of IS-ENES. The organization of Europe within IS-ENES has strongly helped the community adapt to changes and contribute to key developments and evolution such as the significant redesign of the distributed archive system implemented in 2012 in the Earth System Grid Federation, named as the “Peer-to-peer” system (P2P). IS-ENES partners, in collaboration with PCMDI in charge of this upgrade, have helped to prepare a smooth transition, which has resulted in significant performance improvements, improved reliability and reduced system maintenance requirements. Basic outcome of this collaborative process is that the ESGF data infrastructure has been established as the globally accepted data infrastructure for climate research.

The IS-ENES project has provided a clear management structure for the European components of the global system, but managing priorities with partners outside Europe has been problematic, despite the clear shared priorities in delivering the CMIP5 archive. Efforts to establish a clear global governance structure for the emerging archive system are being pushed forward by IS-ENES partners and will be further supported by IS-ENES2.

4.1. Service on ESM data

The main objective of the IS-ENES service on model data is to ensure unified public access to the distributed European climate change simulation data archives. The service has started with access to model results from the previous IPCC Report (AR4) based on existing service of the World Data Centre Climate (WDCC run by DKRZ) and the British Atmospheric Data Centre (BADC run by STFC). When results from CMIP5 experiments started to become available during the second report period, the service has been extended to model results for IPCC AR5, assisting both data users and providers of model results by supplying tools for an efficient dissemination of climate data and their metadata. Full data service was available on CMIP5 during the third reporting period with important downloads in 2012 for the analyses of model results to be published for IPCC AR5 Working group 1 report. WDCC and BADC as member of ENES are responsible to establish the IPCC AR5 reference data archive (IPCC DDC).

For CMIP5 a distributed data infrastructure has been chosen to ease distribution of the expected 2 PB of data. Each model publishes their results on a “datanode”. In a first phase, datanodes were planned to be accessed through data gateways using Earth System Grid software. IS-ENES implemented two gateways STFC/BADC and at MPG with DKRZ. However, in 2011 when the flow of data arrived this system appeared unable to efficiently search for large amount of datasets. A new Peer-to-Peer technology has then been installed allowing much more flexibility and forming the Earth System Grid Federation. This infrastructure now provides data access through newly deployed ESGF index nodes (replacing the old ENES ESG gateways) (Figure 8). The new index nodes were put into production in 2012 and provide a much improved end-user experience.

Worldwide, 27 modelling groups have participated to CMIP5 (6 USA, 1 Canada, 7 Europe, 1 Russia, 5 China, 1 Korea, 4 Japan, 2 Australia). IS-ENES supports the European contribution to ESGF with 7 European data nodes - hosting model data from the 7 European climate models participating to CMIP5 at STFC (BADC), CMCC, CNRM with CERFACS, MPG with DKRZ, CNRS-IPSL, Irish Computing Center (ICHEC) and Norwegian Computing Center (NCC) – and 4 index nodes hosted by STFC (BADC), CMCC, MPG with DKRZ and CNRS-IPSL by means of which users are able to search and access data and their metadata. These index nodes are visible also from the ENES portal (Figure 9). To increase the data availability to European users substantial effort was put into the replication of non-European CMIP5 data to the data centers at STFC/BADC and MPG with DKRZ. Data replication also contributes to long term archival of data in the IPCC Data Distribution Center and thus to long term data accessibility for users.

ESGF hosts about 2 PB of data from CMIP5 with IS-ENES for Europe and USA hosting the largest parts (Figure 10). These data are replicated in the three core data nodes (2 in Europe and 1 in US).

4.2. Developing the ESM data infrastructure

In order to help the development of the data infrastructure, WP10/JRA4 has supported the management and deployment of the data infrastructure as well as some software developments needed to facilitate the provision of transparent and efficient data services. Activities have allowed to ensure consistency with METAFOR developments (D10.1) reviews of terms of use and software required for the infrastructure (D10.2) developed software and deployed the service (D10.3) and developed user services to help the use of ESGF (D10.4). Developments in the parallel project METAFOR have been done in close cooperation with IS-ENES, The searchable experiment ad model description which has been initiated for CMIP5 is basically accepted in climate research and will be operationally transferred to IS-ENES2.

4.2.1. Data policy

The CMIP5 data policy, which has been adopted by WCRP, foresees two access categories, "academic only" or "unrestricted". BADC AND WDCC together with PCMDI encourages the modeling groups to follow the second option, because data generated with public money should be publically available. Comparable data policy constrains are suggested to CORDEX by IS-ENES. A white paper on principals and procedures for coordinating management of a distributed archive has been drawn up.

4.2.2. Operational service

The ENES data archive is now supported by stable software infrastructure, the ESGF Peer-to-Peer system (P2P). Development of P2P was led by PCMDI and put into place for the global CMIP5 archive, but IS-ENES partners contributed key components of the system and played a major role in requirements specification, software development, early deployment and testing. The final system meets the projects goals of a distributed archive infrastructure allowing data at multiple sites to be accessed through a central interface with a range of advanced data services. Access to data is controlled by security certificates, allowing usage to be monitored and reported. Checksums are provided along with tools that verify correct data transfers for users, ensuring safe transport of large data volumes. Access to subsets of data is enabled through the widely used OPeNDAP standard, allowing user applications to extract small subsets of data where complete files are not needed. The system has been tested in extremis, distributing the peta-scale CMIP5 archive, and is running smoothly. Although the development of the software for the CMIP5 archive is led by PCMDI, IS-ENES partners were the first to distribute CMIP5 data with the full implementation of version control, checksums and scriptable downloads services which are essential for of large data collections. The demonstration of these services by the IS-ENES project partners in 2011 contributed to their gradual adoption across the global federation in 2012. IS-ENES also developed the ESGF data replication software stack which enables the data replication between ESGF data nodes. Although the planned METAFOR discovery service was not realised, the detailed metadata collected from CMIP5 data providers during the METAFOR project is available through the ESGF portals of the ENES data archive.

4.2.3. User service developments

A range of tools and services have been developed to support access to the archive. The complexity and volume of the data collection is such that users could loose much time in a maze of options. Flexible and robust tools allow users to quickly select a well constrained sub-set of the data, avoiding lengthy searches in a browser interface or the overhead of downloading more data than required. “Synchro-data” developed by CNRS-IPSL is a package which supports users wanting to identify the data they need, download it from the distributed archive, verify checksums and keep their collection up to date as new versions of the data are published. The command line approach can, for users managing large data collections, be much more efficient than reliance on the browser interface. Other software have been developed and are described with the foreground (such as Esgf-pyclient, ESGF Download Script). Visualisation is also important for users. The Live Access Server (LAS) developed by US has been integrated into the ESGF software stack. This has been deployed at the CNRS-IPSL data node for evaluation. LAS provides quick-looks and some sampling and analysis capabilities Visualisation as also been developed in the climate4impact portal (see below).

D10.1 – Report on CIM dependencies
D10.2 – Data policy and software review
D10.3 – Operational service package
D10.4 – User service package

4.3. Bridging climate research data and the needs of the impact community: climate4impact portal

The results of climate projections represent a wealthy source of data for a wide range of applications. Even though most of the data are easily available through specialized databases, their use can be misleading without enough background. Indeed, it is difficult for a user to find its way in the diversity of approaches and portals or to understand how to best use these data for its own application. A central objective of WP11/JRA5 was to provide a prototype for a web service interface, called the e-impact portal or now the climate4impact portal, to bridge the gap between the climate modelling community, the climate impact community and decision makers (the users or stakeholders thereafter) for developing adaptation and mitigation policies. For that purpose a number of selected and representative national Use Cases for climate data have been selected and described (D11.1). A detailed analysis of these national Use Cases has been performed in D11.2 and serves as the basis to design and build the e-impact portal. A workshop has been organized with WP2/NA1 to investigate user needs. D11.3 describes and analyzes the e-impact portal software requirements and architectural design and provides the basis for the detailed design and the implementation of the e-impact portal whereas D11.4 describes the different modules of the climate4impact portal and their coherence and configuration and D11.5 is a journal paper presenting the key characteristics of the e-impact portal.

4.3.1. IS-ENES Use cases

In order to provide guidance on how to use climate data for a range of applications as well as define needs for an e-impact portal, 17 national and representative “use cases for climate data” have been selected and analyzed. They have been gathered from the experiences of eight research institutes (CERFACS, CMCC, INHGA, CNRS-IPSL, KNMI, Météo-France, SMHI, WUR) in five European countries (France, Italy, Netherlands, Romania, Sweden). Their analysis - from the definition of the request by users to the delivery of final products - provides a good overview of European practices.

The use cases address a wide range of questions related to impact and vulnerability, decision-making, adaptation strategies or mitigation (Figure XX). The geographical domains covered by the use cases extend from very local zones like sites with specific ecological properties, rivers and cities to very large regions like continents (Europe and Mediterranean regions or world scale). Most requests refer to European rivers (Danube, Somme, Seine, Loire, Adour, Gironde) and countries (Netherland, Sweden, Italy and France). The most frequent requests are for regional data (7-50 km) (46%), but we record also a large score for global data (100-300 km) (42%). Requests for local data (< 1 km) are still minor (12%) presumably because of the difficulty to compute and process such data, along with significantly higher uncertainties. Different types of data are requested from raw data to indices or elaborated products depending on categories of users (Figure 11). Use cases have been used to analyse the different methods and tools used for adapting climate data and products to user needs allowing to identify the workflow required to design a dedicated portal for the impact community (below).

D11.1 – Final description of selected Use Cases including user requirements specifications
D11.2 – Baseline documents on e-resources/tools and transverse themes

4.3.2. User needs

A workshop has been organized with the ERANET Circle2 and the European Environment Agency in order to better emphasize user needs and drive both the ENES strategy and the development of the climate4impact portal (D2.2).
This workshop led to a number of key requirements from the climate impact community, namely: access to both global and regional climate change simulations in the same portal; provision of processing tools and processed data; need for guidance on uncertainties and how to use climate model results; eventually would be good to provide different sources of information in one linked system (virtual portal); need to improve access to data and training.
Developments in IS-ENES follow most of these requirements with the eventual provision of both CMIP5 and CORDEX, the development of guidance including description of use cases and analyses of uncertainties, the provision of visualisation tools.

D2.2 Policy position paper on “Climate data needs in support of the EU Climate Adaptation Strategy””

4.3.3. The climate4impact portal

From use cases, a set of user requirements has been specified and a conceptual model of the e-impact, also named climate4impact portal, to cover the user’s needs (D11.3). Five main components have been identified: discovery services, view service, download service, transformation service and a documentation component, all complying with the EU INSPIRE directives, indicating that data must be described, discoverable, viewable and downloadable according to INSPIRE standards. The e-impact portal (Figure 12) aims to have a role in data distribution and processing, but also in being the information broker for the impact community to handle specific requests that need expert knowledge. The portal does not aim at replacing national data portals, but rather proposes a structure, standard workflows and metadata, common tools, documentation, support system, etc. to access decentralized federated data repositories using standardized technologies, enhancing interoperability for future data repositories using the proposed standards.

A prototype portal version of the ENES portal interface for climate impact communities has been released and is available at www.climate4impact.eu. The portal is connected to all Earth System Grid Federation (ESGF) nodes containing global climate model data (GCM data) from CMIP5 and later CORDEX. This global network of all major climate model data centers offers services for data description, discovery and download. The climate4impact portal connects to ESGF services and offers an easier and more ergonomic user interface for searching, visualizing and downloading global climate model data, as well as pre-computed downscaled bias-corrected climate projections and more. Some interactive examples of workflows have been implemented, such as averaging and generation of uncertainties graphs. The visualization services are based on standards OGC web services, such as Web Processing Services (WPS) and Web Map Services (WMS) which are very efficient to provide easy and fast access to maps and graphs, displaying data products which can be the result of on-the-fly data sub-setting or processing. A challenging task was to describe the available model data and how it can be used in an appropriate way. The portal tries to inform users about possible caveats when using model data, and later with dynamic guidance based on user interactions. All impact use cases are described in the documentation section, using highlighted keywords pointing to detailed information in the glossary. It is built to explore state-of-art technologies to provide improved access to climate model data. The prototype will be evaluated and is the basis for development of an operational service to be deployed in IS-ENES2. It will then be part of the ENES Portal. The portal and services provided will be sustained and supported during the development of these operational services in IS-ENES2.

D11.3 – The e-impact portal Software Requirements/Architectural Design/IO Specification
D11.4 – Software Code and e-impact-portal full documentation
D11.5 – Report and manuscript journal paper on key characteristics for a prototype of an e-based pan-European climate data service network

5. Conclusions and perspectives

All main objectives of IS-ENES have been achieved. The project has even gone beyond, for example with regards to the support to CORDEX, which was not planned initially. In some aspects, however, time schedule had to be revised in particular due to delays in the international development of CMIP5. For some other aspects, ambitions had to be slightly revised without threatening crucial objectives, e.g. on workflows. Details are given in the three detailed period reports (D1.1 D1.2 & D1.3).

As detailed in the part dedicated to impacts, IS-ENES has strengthened the integration of the European ESM community, particularly with regards to high-performance computing, and contributed to the objectives of JPI Climate. It has enhanced cooperation between researchers and engineers, between young scientists, as well as with ICT. It has enhanced the European visibility in the international development of ESGF and has strongly helped Europe face the development of data nodes and contribute to software (see foreground), contributing to knowledge for IPCC AR5. It serves as a basis for the development of an integrated information platform for society within Copernicus services for climate.

IS-ENES in its second phase, IS-ENES2, will further integrate the European climate modelling community, stimulate common developments of software for models and their environments, foster the execution and exploitation of high-end simulations and support the dissemination of model results to the climate research and impact communities. IS-ENES2 will implement the ENES strategy published in 2012 by: extending its services on data from global to regional climate models, supporting metadata developments based on the FP7 METAFOR project, easing access to climate projections for studies on climate impact and preparing common high-resolution modeling experiments for the large European computing facilities. IS-ENES2 will also underpin the community’s efforts to prepare for the challenge of future exascale architectures. Following progress done in IS-ENES it will pave the way to install a long-term sustained research infrastructure for climate modeling in Europe.

Potential Impact:
IS-ENES has had an important impact on the climate modeling community in Europe. It has strengthened its internal collaboration as well as collaboration with other communities such as the ICT community. IS-ENES has also strongly reinforced the visibility of ENES in Europe, with regards to HPC and ICT technologies, but also internationally with a key contribution to the Earth System Grid Federation. IS-ENES has also had impacts on policy through support for IPCC AR5 and on societal innovation as a basis for climate services.

A. Impacts

1. Impacts on European research in climate modelling

1.1. Strengthening the integration of the ENES community

Several achievements of IS-ENES have strengthened the European integration of the ENES community.

The ENES Portal has, for the first time, established a comprehensive e-infrastructure, providing an easy-to-use and centralized access to the different (possibly distributed) resources needed for Earth system modelling, including information on the community, on Earth system models and their associated software tools, on High-Performance Computing (HPC) platforms, and on ESM data archives.

The common ENES infrastructure strategy for 2012-2022 published in 2012, was the first foresight focusing on ENES infrastructure since 1998. It has already been used to prepare the second phase of IS-ENES and serves as a basis in the discussions on HPC in Europe with PRACE (see below). It is a first step towards preparing a long-term European research infrastructure for climate modelling. It serves as input for the emerging JPI Climate strategy for the next generation of Earth System models.

Training of young researchers provided through the IS-ENES work programme also strengthens collaboration and cooperation in the European Research Area. It is too early to analyse its impact on integration, but this activity and its successors in IS-ENES2 should help further integrate the community of Earth system modellers.

IS-ENES has been launched at a time when countries new to climate modelling were starting activities in this field, especially through the EC-EARTH consortium. IS-ENES has accelerated the integration of this consortium into the overall community.

1.2. Enhancing climate model developments and analyses

A new integrated service on models and tools has been implemented. Services on OASIS, CDO and NEMO have increased the number of users, assisted users and the interactions with and between users. OASIS and NEMO help the development of coupled models, CDO is important to ease post-processing of model output. Workshops on coupler helped to emphasize needs for users and developments.

IS-ENES has also developed the evaluation portal and made available some key diagnostics. The evaluation portal provides a better view of all datasets used by the community and can help other groups in their choice of observational datasets. Diagnostics are hoped to help the development of the aerosol and chemistry component developments as well as provide some key diagnostics for evaporation and extra-tropical storms.

IS-ENES has supported the development of the European contribution to the CMIP5 database through the Earth System Grid Federation. Thanks to IS-ENES, European climate modelling groups were among the first to publish their data on ESGF and have shifted very efficiently to the new peer-to-peer system deployed in 2012. It has contributed to ease access to model results and to increase the number of submitted papers on CMIP5 model results.

1.3. Facilitating the access to and optimised use of HPC resources

Access to world-class HPC is a critical requirement for Earth System modelling. IS-ENES has worked on model performance and supported developments of computing efficiency for I/O and the coupler. Two international workshops have allowed a better view on state of the art development of new dynamical cores and model performance.

IS-ENES has also stimulated collaboration with DEISA2 and PRACE. A common strategy has been elaborated and implemented to collaborate with PRACE 1IP and 2IP. Meetings have been organised with PRACE to enhance interaction and collaboration. IS-ENES has increased collaboration and exchange of expertise among the modelling groups. It has also clearly enhanced the visibility of the ENES with regards to HPC in Europe, as demonstrates the meeting with PRACE board of directors (March 20th 2013) emphasizing for the first time community needs with regards to PRACE. ENES is also invited to a HPC strategy meeting organised by DG Cnect on April 30th 2013.

1.4. Increasing collaboration and exchange among researchers and engineers

IS-ENES, through networking, service and joint research activities, has allowed stronger interactions among software engineers but also between Earth system model developers and engineers. One illustration was given by KNMI for which IS-ENES initiated collaboration between the ICT and climate modelling teams. Developments on parallel I/O and performance analyses have also triggered exchange between engineering teams of different institutions. IS-ENES2 plans to go a step further by supporting workshops on key software issues.

1.5. Providing a sustained and stable infrastructure

The work of IS-ENES will be continued in a follow-up project. IS-ENES2 is designed to continue the work and stabilize the integration of the ENES community. It will strongly build upon and exploit the work done in IS-ENES. For example a central part of IS-ENES2 is a large multi model experiment, which will make use of the coupling and I/O solutions evaluated and developed in IS-ENES. The portal that was developed in IS-ENES will be used as a central communication tool by the ENES scientific officer, introduced by IS-ENES2. The large number of leading institutions having participated in IS-ENES and continuing participation in IS-ENES2 shows that there the community is supporting the infrastructure and will be interested in enhancing and sustaining it.

2. Impacts on other research communities

2.1. Helping the regional climate modelling community

IS-ENES was mainly focusing initially on global climate models. However, several IS-ENES activities have had impacts on regional climate modelling. Through IS-ENES and ESGF, the climate modelling community has been able to get model data to force regional climate models. IS-ENES has also supported the use of ESGF by CORDEX that will help the regional climate modelling community access all model data in a common distributed database. Beyond these data aspects, IS-ENES has also had an impact on the development of coupled regional climate models as several of the OASIS dedicated user support have been provided to this community and was very much appreciated (e.g. by the COSMO community)

2.2. Easing access to model data for the climate impact communities

IS-ENES has developed an interface to ease access to climate model data through the climate4impact portal. It also has set the beginning of the definition of common standards for describing the climate downscaling community data that will permit a better interoperability, as well as easing the future development of common tools that can be used within other specialized or national portals. The workshop on user needs involved many different impact sectors (agriculture, water, ecosystems…) and clearly emphasized the requirements of the diverse impact communities and users. The climate4impact portal is still a prototype but will be operational during the first year of IS-ENES2.

2.3. Fostering the interaction between ESM and e-technologies communities

IS-ENES facilitates the integration between climate scientists and e-technology scientists. Through its HPC Task Force and developments on model performance, ENES has contributed to the European Exascale Software Initiative (EESI1 and 2, http://www.eesi-project.eu/) to brainstorm on demands for exascale computing. ENES is also one of the five core communities recognised in the European Data Infrastructure, EUDAT (http://www.eudat.eu/) where our community shares its expertise in issues such as data replication, data management and metadata. As mentioned above, IS-ENES also collaborates with PRACE Implementation projects also allowing collaboration with ICT.

3. Impacts at the international level

IS-ENES has increased the visibility and efficiency of the European contribution to the Earth System Grid Federation. ESGF is led by PCMDI (USA), but strong European organisation provided by IS-ENES led to a substantial European contribution to ESGF on several levels, and paves the way for a significant European role in ESGF governance. Europe contributes through the development of software (e.g. dashboard and some ESGF software, see foreground), through the implementation of data nodes, data index nodes and core data nodes. PCMDI, which is the main developer of ESGF, relies heavily on collaboration with STFC (BADC) and WDCC (DKRZ) for quality control and replicas.

Although not foreseen initially, IS-ENES has adopted in its main objectives the promotion and support of the development of an international distributed archive for CORDEX data products following the same standards as CMIP5. Coordination and planning has been supported and test archive services configured and deployed. Further support and deployment of an operational archive is part of IS-ENES2, including support to Africa CORDEX.

The ENES community has played a crucial role in the development of the metadata Common Information Model within the METAFOR project. After the end of METAFOR, IS-ENES has supported networking with University of Reading and the US. The follow-on activity of METAFOR is included in IS-ENES2.

4. Policy impacts

CMIP5 and CORDEX have been organised in order to support the IPCC 5th Assessment Report. Results of CMIP5 will be used extensively in the WGI report on the physical basis of climate change. By supporting the European contribution to ESGF, IS-ENES contributes to this international scientific effort to strengthen the knowledge of climate change. Beyond this contribution, the synchro-data software developed by CNRS-IPSL within IS-ENES has been very successful to enable access and maintenance of a large range of model results from ESGF. This was in particular used at ETH Zürich where a local database has been used to derive all the graphics on model projections.

5. Societal innovation: contribution to develop climate services

Adaptation to climate change raises the challenge to provide reliable information on climate change to society. This challenge is beyond the scope of IS-ENES but IS-ENES contributes to it by developing the data infrastructure on climate model results and also by participating in the Climate Service Partnership workshops and networking activities. IS-ENES indeed serves as a basis of the Copernicus (formerly GMES) project under negotiation CLIPC (climate information portal – formerly GMES-CLIP) that aims to deliver information on climate and climate impacts for society. CLIPC will rely on IS-ENES for the distribution of model results and will complement IS-ENES by adding access to both observations and a range of climate indicators. The development of sustained climate services will require a long-term sustained research infrastructure on climate models.

B. Dissemination activities

Information on IS-ENES is available at http://is.enes.org.

Several large audience publications have been published on IS-ENES, complemented by flyers and newsletters. 9 publications present results obtained within IS-ENES? Including the ENES infrastructure strategy. 32 talks have been given at diverse international conferences and European meetings complemented by 16 posters that have been displayed at different conferences (e.g. EGU, AGU, 2011 WCRP open science conference). A complete description of publications, talks, posters and workshops is available in D2.6

Publications on IS-ENES

Joussaume S., The European Network for Earth System Modelling, Public Service Review: Science and Technology 4, September 2009.
Joussaume S., Innovation through integration, International Innovation, October 2010.
Joussaume S., Encouraging connections in climate change research, International Innovation, October 2010
Joussaume S., Earth System Modelling, Pan European Networks: Science & Technology 6, April 2013
ENES Newsletter #1, 29 June 2011
ENES Newsletter #2, 13 February 2012
COMBINE Quarterly Newsletter 7, 6 February 2012

Flyers

Factsheet presenting the project

Flyer for E2SCMS

GEOSS for Climate brochure, 2010

Publications associated with work developed in IS-ENES

• Mitchell J, Budich R, Joussaume S, Lawrence B and Marotzke J (2012), Infrastructure strategy for the European Earth System Modelling community 2012-2022, ENES Report Series 1, 33 pp
• Gettelman, A., Eyring, V., Fischer, C., Shiona, H., Cionni, I., Neish, M., Morgenstern, O., Wood, S. W., and Li, Z., A community diagnostic tool for chemistry climate model validation, Geosci. Model Dev., 5, 1061-1073, doi:10.5194/gmd-5-1061-2012 2012
• Valcke S., 2013: The OASIS3 coupler: a European climate modelling community software, Geosci. Model Dev., 5, 2139-2178, doi:10.5194/gmdd-5-2139-2012
• Valcke S., V. Balaji, A. Craig, C. Deluca, R. Dunlap, R. Ford, R. Jacob, J. Larson, R. O'Kuinghttons, G. Riley, M. Vertenstein, Coupling technologies for Earth System Modelling, Geosci. Model Dev., 5, 1589-1596, doi:10.5194/gmd-5-1589-2012
• Mares C. I. Mares, A. Stanciu, M. Mihailescu, 2012: North Atlantic Oscillation influence on the Danube lower Basin. Online publication in Proceedings of V th International Scientific Conference on Water, Climate and Environment, Balwois 2012, 28 May, 2 June 2012.
• Koffi, B., and 23 co authors Application of the CALIOP layer product to evaluate the vertical distribution of aerosols estimated by global models: AeroCom phase I results, J. Geophys. Res., 117, D10201, doi: 10.1029/2011JD016858. (Funded in part by IS-ENES and in part by Centre National d'Études Spatiales (CNES).)
• Valcke S. et al, One book chapter of Brief on “Earth System Modelling – Volume 4 – “Coupling software and strategies”, R. Budich and R. Redler Eds, Springer, 75 pp.
• Joussaume S. and R. Budich, The Infrastructure Project of the European Network for Earth System Modelling: IS-ENES Brief on “Earth System Modelling – Volume 1, Recent developments and projects, K. Puri, R. Budich, R. Redler (eds), Springer, 2013?, pages 4-8
• André J.C. G. Aloisio, J. Biercamp, R. Budich, S. Joussaume, B. Lawrence and S. Valcke, The 2nd IS-ENES Workshop on "High-Performance Computing for Climate Models" Toulouse, Jan.30th-Feb.1st 2013", BAMS, 2013, submitted.

Talks about IS-ENES

• Valcke S., IS-ENES, the European Infrastructure project for Earth System Modelling and the OASIS4 coupler. Computing in Atmospheric Sciences Workshop organised by NCAR Computational and Information Systems Laboratory, Annecy, 2009
• Joussaume S., IS-ENES: Infrastructure for the European Network for Earth System Modelling, EGU 2010, session ESSI5, Vienna, Austria, 7th May 2010.
• Kindermann S., E-infrastructure components to support the earth system modeling community in Europe, EGU 2010, session ESSI5, Vienna, Austria, 7th May 2010
• Som de Cerff W. and M. Plieger, Enabling the use of climate model data in the Dutch climate effect community, EGU 2010, session ESSI5, Vienna, Austria, 7th May 2010
• Som de Cerff W. and M. Plieger, Enabling the use of climate model data in the Dutch climate effect community, EGU 2010, session ESSI5, Vienna, Austria, 7th May 2010
• Tselioudis G., IS-ENES Evaluation Portal during the EU Cloud Intercomparison Process Study and Evaluation Project, EUCLIPSE Kick-off meeting, Utrecht (NL), 27-28 September 2010
• Joussaume S., Presentation of IS-ENES, IS-ENES/CIRCLE/EEA Workshop on Bridging Climate Research Data and the Needs of the Impact Community, - EEA, Copenhagen, Denmark, 11-12 January 2011
• Déandréis C., Lessons learned from IS-ENES use cases - towards the implementation of an e-impact portal, IS-ENES/CIRCLE/EEA Workshop on Bridging Climate Research Data and the Needs of the Impact Community, - EEA, Copenhagen, Denmark, 11-12 January 2011
• Joussaume S., Climate scenarios with global climate models: dealing with uncertainties, EEA Scientific Committee seminar on Climate scenarios, impacts and uncertainties, European Environment Agency, March 1st 2011
• André J.C. with the HPC Task Force, On the HPC strategy for climate, Adhoc-Meeting on European HPC landscape, Munich, Germany, 18th April 2011
• Joussaume S., e-infrastructure to address frontiers in climate modeling: the IS-ENES initiative, Conference on the Role of e-infrastructures for Climate Change Research, Italy, 18 May 2011
• Joussaume S., The European infrastructure for Earth System Modelling: the IS-ENES project COMBINE, General Assembly, Met Office Hadley Center, Exeter, UK, 24-27th May 2011
• Joussaume S., Climate Modelling in Europe, GMES and Climate Change, Helsinki, 16-17th June 2011
• Guilyardi E. for S. Joussaume, IS-ENES: Infrastructure for ENES, ESFRI Research Infrastructures and Joint Programming Initiatives in the field of Environmental Sciences, Workshop, Brussels, 17 October 2011
• Swart R., Outcome of IS-ENES Workshop on Bridging Climate Research data and the Needs of the Impact Community, CIRCLE2/JPI Climate Workshop on: Processes for improving the interface between climate research and its application: sharing experiences on climate services development in Europe, Bologna, 8-9 November 2011
• Swart, R., J., Pagé, C., Defining climate modeling user needs: which data are actually required to support impact analysis and adaptation policy development? 2010 AGU Fall Meeting, San Francisco, USA, 13-17 December 2010
• Joussaume S., Infrastructure Strategy for the European Earth System Modelling Community 2012-2022, JPI Governing Board meeting, Amsterdam, May 10th 2012
• Joussaume S., InfraStructure for the European Network for Earth System modelling, Coordination meeting on climate modelling and on science for climate services (organized by EC), Brussels, 7-8 June 2012
• Guilyardi E., ENES and JPI Climate, ECRA meeting, Brussels, Belgium, June 2012
• Juckes M., The data deluge: the case of climate modelling, Australia – European Union Research Infrastructure Second Workshop, Environment working group, Brussels, 26-27 June 2012
• Schulz M. and B. Koffi, What CALIPSO data tell us about vertical distribution in modeling, 11th AeroCom workshop, Seattle, September 10-13, 2012
• Lawrence B. for IS-ENES, Exploiting (high volume) Climate Simulations and Observations, ESA Climate Change Initiative, Frascati (Italy), September 24th, 2012
• Joussaume S., InfraStructure for the European Network for Earth System modelling, PCMDI, Livermore (USA) October 1rst, 2012
• Joussaume S., The European InfraStructure for Earth System modeling, NCAR (National Center for Atmospheric Research), Climate and Global Dynamics division seminar, Boulder (Colorado, USA) October 9, 2012
• Joussaume S., Modelling the Earth’s climate system: data and computing challenges, Supercomputing conference 2012, Salt Lake City (USA), 14 November 2012
• Joussaume S., IS-ENES: Infrastructure for ENES, EC meeting: Towards an integrated atmospheric observing system in Europe, Brussels, December 3rd 2012
• Aloisio G. (for ENES), The efficient use of high-end computing resources for climate: towards a common methodological approach, PRACE 2IP WP8 meeting, Lugano, March 7th, 2013

Joussaume S., B. Lawrence, A. Navarra, J. Biercamp and the ENES HPC Task Force, ENES – PRACE Board of Directors meeting, Brussels, March 20th, 2013

Plieger M., climate4impact Search, access and work with CMIP5 data EC-Earth workshop, de Bilt, March 21, 2013

Joussaume S., InfraStructure for the European Network for Earth System modelling, EGU 2013, NH1.8 session “ICT-based hydrometeorology science and natural disaster societal impact assessment”, April 8th, 2013 http://meetingorganizer.copernicus.org/EGU2013/EGU2013-10026.pdf

Som de Cerff W., S. Joussaume, M. Juckes, C. Pagé, Climate4impact: Bridging model data to the impact communities, EGU 2013, Splinter Session 1.27 talk and demo, April 11th 2013

Workshops organised or co-organised by IS-ENES with external participants to IS-ENES project

• IS-ENES Scoping Meeting, Montvillargennes, France, 29-31 March 2010
• EGU 2010, Townhall meeting (TM3), “The European Network for Earth System Modelling (ENES) Strategy and Resource Centre”, convener M. Jukes, 6 May 2010
• IS-ENES/PRACE meeting, Paris 30 November - 1 December 2010
• Workshop on "Coupling Technologies for Earth System Modelling: Today and Tomorrow" - CERFACS, Toulouse (France) - 15t-17th December 2010
• IS-ENES/CIRCLE/EEA User Needs workshop, EEA, Copenhagen, Sweden 11-12 January 2011
• IS-ENES Foresight meeting, Hamburg, Germany 2-4 February 2011
• Workshop on dynamical cores for climate models, Lecce, Italy, 15-16 December 2011
• Workshop: Scalable IO in climate models, Hamburg, Germany, 27-28 February 2012
• ENES Summer School E2SCMS, Kos Greece, June 2012
• IS-ENES workshop on Statistical downscaling of climate scenarios for the impact communities. A CMIP5 perspective, Paris, 16-17th October 2012
• 2nd IS-ENES Workshop on HPC for climate models, Toulouse, France, 30 January - 1st February 2013
• Second Workshop on Coupling Technologies for Earth System Models (CW2013), Boulder, USA, 20-22 February 2013
• Climate Data Access: Ways Forward, Wageningen University Research Centre, Wageningen, Netherlands, 11-12 February 2013
• Climate4impact: Bridging Climate Model data to the impacts communities (public), EGU 2013 Splinter Meeting, SPM1.27 convener W. Som de Cerff et al. 11 April 2013

C. Exploitable foreground

Standard documents

• CMIP5 Data Reference Syntax (DRS) (standard document developed jointly with PCMDI and DKRZ)

The DRS includes vocabularies and specifications for file formats in CMIP5. It provides a uniform file structure which is robust and transparent to facilitate management of the archive and exploitation by users. The CMIP5 DRS is in use as a key element of the operational ESGF archive distributing the data. The DRS is an essential component of petascale climate data archives.

• CORDEX Facet Mappings: (standard document developed by STFC)

The document gives an outline of how file level meta-data in the CORDEX archive will be mapped onto data selection facets shown in the archive search interface. It ensures that the transformation from user requirements in terms of a clear search interface to file meta-data requirements is well understood. It will provide a reference for the configuration of publication software when publication of CORDEX data in the ESGF archive starts in 2013.

Software related to ESGF

• drslib python library (software developed by STFC)

The software supports the publication of CMIP5 data in ESGF archive by implementing the version control system and organising the data in the recommended directory structure. The software is part of the standard distribution of ESGF software and can be used by archive node managers as part of their data publication workflow. It is a key component of the ENES distributed archive service infrastructure.

• ESGF Python client library (developed by STFC, open source licence)

ESGF Pyclient allows users to search ESGF data holding programmatically through the Python programming language and can be used as a component of other tools and services needing to interface with ESGF services. ESGF Pyclient can be used to build search and download systems for ESGF. It is already used by the UK MetOffice as part of their operational data product service to their scientists. It is also used by a data search and download tool developed for the ExArch project. It is expected to be a central component in future ESGF software.

• ESGF data download script (open source software stack owned by DKRZ and ESGF software team)

The ESGF download script facility enables secure, reliable data download from ESGF P2P data nodes and is integral part of the ESGF index node. It provides: download and storage of security credentials, secure data node access with credentials, multi-file download with checksumming, retry possibility in case of transfer problems. The software is released as part of the ESGF open source software stack under the ESGF / BSD open source license. It has been used by more than 900 users of the ENES federation in 2012 to download more than 600 Tbyte of Data and is also used by non-european users worldwide.

• Synchro-data (open source licence developed by CNRS-IPSL)

Synchro-data is a Python program managing discovery / authentication / certificate / download processes from CMIP5 archive in an easy way. It eases access and keeps precise track of downloaded files and versions from the ESGF system. As an ESGF download scheduler synchro-data has been used by several a large number of users (> 100) so as to keep a significant subset of the ESGF content in sync with the distributed archive. Interested parties are all parties using ESGF data (CMIP5, CORDEX, obs4MIPs...). For example ETH, GFDL, NOAA, IPSL made extensive use of synchro-data serving more than 100 researchers and scientifics.

• MyProxyClient python library and script (software developed by STFC)

The software allows users to download ESGF security certificates to their local computer. This will allow more flexible access to the data held within the ENES data archives. It provides a functional implementation of the security interface in the widely used python language.

• EnesGetCert shell script (software developed by STFC)

The software allows users to download ESGF security certificates to their local computer. EnesGetCert provides a simple but robust foundation on which to build further tools. Its role is as a component of a larger system.

• ESGF Dashboard (developed by CMCC, to be released under an open source license)

The dashboard provides monitoring capabilities of the IS-ENES infrastructure in terms of Network topology (peer-groups composition), Node type (host/services mapping), System metrics (service availability, RTT, etc.). It can be exploited by any institution deploying a P2P ESGF node to perform monitoring activities and is available since the end of 2011.

Other software

• CDO (Developed by MPG/MPI für Meteorologie)

CDO is a collection of command line Operators to manipulate and analyse Climate and NWP model Data. Supported data formats are GRIB 1/2, netCDF 3/4, and others. There are more than 600 operators available.The software is used by more than 250 institutions world-wide. It is a quasi-standard in climate science. Easier understanding of the data at hand and faster time to solution than with self-developed software is granted.

• Web NEMO configuration database (developed by CNRS-IPSL, freely available)

The configuration database is a source code of more than 50 diagnostics. It is meant to illustrate the different aspects and impacts of the different physical packages available in NEMO. The source code can be downloaded on NEMO source code repository. Users have then to follow the documentation available.

• The OASIS3-MCT parallel coupling software (owner: CERFACS, released under an LGPL license. in August 2012)

OASIS3-MCT is a software implemented as a parallel coupling library that performs the interpolation of the coupling fields as a parallel matrix-vector product and a parallel redistribution of the coupling fields directly from the source to the target component processes. It is expected that most climate modelling groups currently using the OASIS3 coupler will shortly migrate to the new parallel version OASIS3-MCT to benefit from its increased performance on massively-parallel platforms. The development of OASIS3-MCT will be pursued during the IS-ENES2 project to add few functions although no major evolutions are foreseen.

• Bespoke Framework Generator (BFG) software (Currently free non-commercial license. Agreement reached to release under LGPL in near future)

The BFG supports the rapid construction of coupled models. BFG takes as input a description of the models to be coupled, what data they need to exchange and how the models are to be mapped to programs for execution and generates the required 'wrapper code' required to build the coupled model using a specified exiting coupling technology, such as OASIS-MCT. BFG is freely available to coupled model developers and is available from the ENES portal.

• cdi-pio Developed by MPG/MPI für Meteorologie
CDI-pio is an internal client/server infrastructure to allow for running cdi asynchronous on dedicated hardware. This is implementing one possible low-level architecture for parallel output of climate and NWP models. It could be used for analysis tools as well. The software is an abstraction layer on top of grib1/2 and netcdf 3/4 and allows transparent write and read of this formats. It provides a common I/O interface for models and analysis tools like cdo. It allows for at least one more HPC computer architecture generation to scale (maybe till 2018).

• XiOS - Highly configurable parallel out system with client-server mode (software developed by CNRS-IPSL, open source)

Embedded in Earth system models, the software allows to ouput model fields to disks during a climate simulation. It uses a client-server system with parallel communications and parallel file systems for performance. It uses a versatile XML configuration file. The performance of the software will fully benefit to the global performance of Earth system models on parallel super computers. The XML configuration system will speed-up the set up phase of future climate simulations.

• Prototype of grid infrastructure (developed by CMCC, freely available)

The main goal is to provide a high level environment for climate simulations for running and monitoring ensemble experiments through web interfaces in a grid environment. The grid environment prototype exploits already available grid services, namely GRB services and Globus middleware. Web based interfaces have been developed for accessing the infrastructure. The prototype allows the submission and monitoring of ensemble climate experiments.

• Extensions to the GRelC service to support CIM metadata resources (developed by CMCC, to be released under an open source license)

The GRelC service provides access, indexing and querying functionalities related to CIM metadata. They could be used to get access, query and index CIM metadata (and more in general XML documents) in a grid environment. Unfortunately the GRelC service and the IS-ENES infrastructure uses two different security paradigms at the user level (GSI vs OpenID). This security mismatch is a barrier for a stronger adoption of this service to manage CIM metadata documents.

• CORDEX Writer (open source license, developed by SMHI)

The CORDEX writer has been developed for converting output from regional climate models to netcdf files accordingly to the CORDEX specifications (variable names, units, attributes etc.). The CORDEX writer can be exploited by any Regional Climate Modelling Centre for post-processing regional climate model output. The CORDEX writer is available from February 2013 and can be used as whole or by parts. Limited support can be provided.

• Chemistry-Climate Model Validation Diagnostic (CCMVal-Diag) (software developed by NCAR and DLR, released in July 2012, open source license)

The CCMVal-Diag tool is a flexible and extensible open source package that facilitates the complex evaluation of global models. Models can be compared to other models, ensemble members (simulations with the same model), and/or many types of observations. The initial construction and application is to coupled chemistry-climate models (CCMs) participating in CCMVal, but the evaluation of climate models that submitted output to the Coupled Model Intercomparison Project (CMIP) is also possible.

• HOAPS satellite simulator tool for climate model evaluation (MPG/MPI für Meteorologie)

The HOAPS tool is a simulator package for SSM/I satellite retrievals that allows generating bias corrections of turbulent surface fluxes in climate model output under consideration of uneven satellite sampling. The HOAPS satellite simulator tool can be used by any researcher in need for evaluating climate model output of turbulent surface fluxes through the link provided on the services page of the ENES portal.

• Aerosol model evaluation tool and web interface (developed by Met.No CNRS-IPSL, MPG, NASA, Goddard, freely available)

The AeroCom tools and associated web interface allow for the consistent evaluation of global aerosol models. Models can be compared to other models and/or many types of observations. Model versions can be inspected to see model progress. Communication in the AeroCom model comparison initiative is facilitated by the open access to the results. Aerosol models shall be improved to reduce uncertainty in aerosol radiative forcing.

• MCMS, Algorithm for data mining and selected results (developed by NASA-GISS with AA, available on line)

MCMS provides an algorithm for the objective recognition and tracking of mid-latitude low-pressure systems (storms) as well as for the recognition of the area attributed to the storm and for the objective calculation of parameters concerning the storms. The input consists in gridded sea surface pressure time-series. The algorithm may be used with input from re-analysis datasets or from climate model simulations. Selected results generated using re-analysis input are given.

Portals

• ENES portal (http://enes.portal.org) (developed by DKRZ)

The ENES portal was designed to be the single central portal, which acts not only as a common entrance point to the ESM infrastructure built within IS-ENES, but also as a presentation and virtual meeting point of the ESM community. The objective was to collect, integrate and link (IS-ENES) services and information in a coherent fashion, providing an overview of community resources and to offer a communication platform for the ESM community.

• Evaluation portal (Database and website for the dissemination of the data)

It is a web portal featuring a collection of information on ways, data and methods used for the development of climate models, more specifically for the part of evaluating the performance of models. It can be used as a one-shop-stop for modellers as well as for anyone interested in the evaluation process of climate models. The portal is available through the ENES Portal.

• Climate4impact (web portal) (http://climate4impact.eu) (developed by KNMI)

The climate4impact portal is oriented towards climate change impact modellers, impact and adaptation consultants, as well as other experts using climate change data. It offers access to data and quick looks of global climate models scenarios, as well as some regional climate models and downscaled higher resolution climate data. The portal provides data transformation tooling for tailoring data to your needs and mapping & plotting capabilities. Guidance on how to use climate scenarios, documentation on the climate system, frequently asked questions (FAQ) and examples in several impact and adaptation themes (Use Cases) are presented and described. The current portal is a prototype, which will be made operational in the IS-ENES2 project.

• BFG portal (http://bfg.cs.man.ac.uk) (developed by U. Manchester)

The BFG portal is a portal supporting the use of the Bespoke Framework Generator, a flexible coupling technology. The BFG portal makes available information and examples of the BFG flexible coupling system. Users may also upload coupling descriptions (metadata) and run BFG to produce coupling source code and various other BFG utilities (validation and visualisation of metadata descriptions, for example.

List of Websites:

http://is.enes.org

Project website

http://enes.org

enes information portal

final1-is-enes-final-publishable-summary.pdf

Powiązane dokumenty

• final1-is-enes-final-publishable-summary.pdf