Periodic Reporting for period 2 - COEGSS (Center of Excellence for Global Systems Science)
Periodo di rendicontazione: 2017-04-01 al 2018-09-30
CoeGSS is a European consortium that kick-started the use of high-performance computing (HPC) for decision-support in the face of global challenges, e.g. the risks of pandemics, those of climate change, and more. The consortium produced software and modeling tools for studying global challenges with the help of HPC. In Europe, the work of the consortium ushered in a successor Center of Excellence, coordinated by ATOS, one of the CoeGSS consortium members. Globally, it led to a cooperation that the Global Climate Forum, another consortium member, has set up with the Konrad Zuse Institute in Berlin – partner of the North-German HPC network –, and universities in Europe, China, and the US.
The work of CoeGSS proceeded in three steps:
1) assessment of needs and available tools. This included the test and evaluation of existing HPC platforms for multi-agent models that may be used to study global systems and the challenges they engender. It also included the benchmarking of different microchip architectures when used to run HPC software relevant for global systems science.
2) development of a toolbox of modules that allow bringing the resources of HPC to the analysis of global challenges and of options to tackle them. The modules deal with data collection and management, with model design and code optimisation, a platform for HPC-based multi-agent models of global systems, programming tools for building such models, advanced visualization techniques, and more. These lines of work were connected by the common reference to three "pilots", i.e. specific examples of global challenges: the epidemic character of the deadly habit of smoking, the disruption of the global car market through electric vehicles, and the challenges of urban planning. The toolbox is now available for the scientific community and practitioners to use as they think fit.
3) Models for the three pilots were finalized and put to use for concrete simulations. The interface to stakeholders and potential customers was implemented in the format of a web-based portal, again available for future use.
The project advanced technologies for data analytics, in particular for highly unstructured and uncorrelated data. This encompasses pre- as well as post- processing steps for the actual execution of an application, but also novel approaches like on-site data analytics. Consequently, a high potential for innovation and enhancements of the current state is given, also beyond the scope of CoeGSS:
1) There are just a handful of standards available for agent-based modelling and simulation, so that there was a clear potential for enhancing the current state of the art. Furthermore, not only the standards are limited, also the available tools are tailored to specific use cases, which prevent their application for generic problem statements. The project recognised this issue timely and consequently worked towards integrated workflows for agent-based modelling and simulation.
3) CoeGSS advanced the state of the art by capturing the requirements for universal Synthetic Populations and analysing commonalities as well as possibilities for substitutions. Within the scope of the project, a synthetic population based model has been implemented and benchmarked on manifold HPC systems as well as innovative architectures.
3) Building up Synthetic Information Systems including Synthetic Populations and Synthetic Networks is one of the key breakthroughs of CoeGSS. During year three, the ideas of year two have been transformed into a proof-of-concept implementation that creates real-world representations of social networks. In particular, a HPC-ready implementation has been developed, which can be executed in a parallel fashion on the HPC infrastructures.
1) assessment of needs and available tools. This included the test and evaluation of existing HPC platforms for multi-agent models that may be used to study global systems and the challenges they engender. It also included the benchmarking of different microchip architectures when used to run HPC software relevant for global systems science.
2) development of a toolbox of modules that allow bringing the resources of HPC to the analysis of global challenges and of options to tackle them. The modules deal with data collection and management, with model design and code optimisation, a platform for HPC-based multi-agent models of global systems, programming tools for building such models, advanced visualization techniques, and more. These lines of work were connected by the common reference to three "pilots", i.e. specific examples of global challenges: the epidemic character of the deadly habit of smoking, the disruption of the global car market through electric vehicles, and the challenges of urban planning. The toolbox is now available for the scientific community and practitioners to use as they think fit.
3) Models for the three pilots were finalized and put to use for concrete simulations. The interface to stakeholders and potential customers was implemented in the format of a web-based portal, again available for future use.
The project advanced technologies for data analytics, in particular for highly unstructured and uncorrelated data. This encompasses pre- as well as post- processing steps for the actual execution of an application, but also novel approaches like on-site data analytics. Consequently, a high potential for innovation and enhancements of the current state is given, also beyond the scope of CoeGSS:
1) There are just a handful of standards available for agent-based modelling and simulation, so that there was a clear potential for enhancing the current state of the art. Furthermore, not only the standards are limited, also the available tools are tailored to specific use cases, which prevent their application for generic problem statements. The project recognised this issue timely and consequently worked towards integrated workflows for agent-based modelling and simulation.
3) CoeGSS advanced the state of the art by capturing the requirements for universal Synthetic Populations and analysing commonalities as well as possibilities for substitutions. Within the scope of the project, a synthetic population based model has been implemented and benchmarked on manifold HPC systems as well as innovative architectures.
3) Building up Synthetic Information Systems including Synthetic Populations and Synthetic Networks is one of the key breakthroughs of CoeGSS. During year three, the ideas of year two have been transformed into a proof-of-concept implementation that creates real-world representations of social networks. In particular, a HPC-ready implementation has been developed, which can be executed in a parallel fashion on the HPC infrastructures.
A key result of the overall work by the consortium is the adaptation of the methodology of synthetic populations to the needs of global systems research. Another result is a new understanding of the difference between two kinds of architectures for high-performance computing (HPC) software. On one hand there is the architecture of parallelized software used to model large numbers of entitites linked by proximity relations, with fluid dynamics as the paradigmatic domain. Most existing software for HPC is of this kind and exploits the proximity relations between large numbers of hardware components. On the other hand there is the architecture of parallelized software used to model large numbers of entitites linked not only by proximity relations but also by direct long-distance links. Advances in the development of such software are among the main results of CoeGSS.
Together with the already available Synthetic Population generator and the introduced Synthetic Network generator, a workflow for creating entire Synthetic Information Systems has been established. This workflow is HPC capable and supports parallel execution by making use of the state-of-the-art Message Passing Interface. Furthermore, efficient mechanisms for managing specific workflows have been developed, based on four Domain Specific languages (DSLs) that simplify workflow composition, but also data manipulation and interchange.
Next, the examples of global challenges chosen by CoeGSS – green growth ,health habits, and urbanisation – have been addressed with specific models built by members of the consortium. It has been possible to perform related parametric studies in a reasonable amount of time, enabled by HPC simulation, High Performance Data Analytics and in-situ visualisation.
Last not least, a Portal for CoeGSS has been built, relying on a completed Single-Sign-On mechanism that offers a new level of transparency to its users. Through the application of a testing and verification methodology, potential issues were detected and eliminated so that a well-balanced Portal is available to the Pilots and CoeGSS stakeholders. Finally, the Portal offers the ability to submit HPC batch jobs to HPC clusters.
A rich set of publications during the duration of the project established the mandatory impact. In conjunction with two well-organised Conferences in Lucca / Italy and a series of stakeholder interactions, the awareness for the potential of HPC has been raised among experts for global challenges, while HPC specialists have begun to perceive the importanceof those challenges for their work.
In Europe and beyond, the work of the consortium ushered in a series of initiatives bringing HPC to the new frontier of global challenges. In Europe, this includes the successor Center of Excellence, HiDALGO – HPC and Big Data Technologies for Global Challenges –, coordinated by ATOS, one of the CoeGSS consortium members. Globally, it includes the cooperation that the Global Climate Forum, another consortium member, has set up on the basis of CoeGSS with the Konrad Zuse Institute in Berlin – partner of the North-German HPC network –, with Leuphana University in Lüneburg, Germany, as well as with Arizona State University in the U.S. and with the Beijing Institute for Big Data Research in China.
Together with the already available Synthetic Population generator and the introduced Synthetic Network generator, a workflow for creating entire Synthetic Information Systems has been established. This workflow is HPC capable and supports parallel execution by making use of the state-of-the-art Message Passing Interface. Furthermore, efficient mechanisms for managing specific workflows have been developed, based on four Domain Specific languages (DSLs) that simplify workflow composition, but also data manipulation and interchange.
Next, the examples of global challenges chosen by CoeGSS – green growth ,health habits, and urbanisation – have been addressed with specific models built by members of the consortium. It has been possible to perform related parametric studies in a reasonable amount of time, enabled by HPC simulation, High Performance Data Analytics and in-situ visualisation.
Last not least, a Portal for CoeGSS has been built, relying on a completed Single-Sign-On mechanism that offers a new level of transparency to its users. Through the application of a testing and verification methodology, potential issues were detected and eliminated so that a well-balanced Portal is available to the Pilots and CoeGSS stakeholders. Finally, the Portal offers the ability to submit HPC batch jobs to HPC clusters.
A rich set of publications during the duration of the project established the mandatory impact. In conjunction with two well-organised Conferences in Lucca / Italy and a series of stakeholder interactions, the awareness for the potential of HPC has been raised among experts for global challenges, while HPC specialists have begun to perceive the importanceof those challenges for their work.
In Europe and beyond, the work of the consortium ushered in a series of initiatives bringing HPC to the new frontier of global challenges. In Europe, this includes the successor Center of Excellence, HiDALGO – HPC and Big Data Technologies for Global Challenges –, coordinated by ATOS, one of the CoeGSS consortium members. Globally, it includes the cooperation that the Global Climate Forum, another consortium member, has set up on the basis of CoeGSS with the Konrad Zuse Institute in Berlin – partner of the North-German HPC network –, with Leuphana University in Lüneburg, Germany, as well as with Arizona State University in the U.S. and with the Beijing Institute for Big Data Research in China.