Skip to main content

European Spatio-TEmporal Data Infrastructure for high-performance computing

Objective

The project will establish a European standard for the storage and retrieval of multidimensional high-performance computing (HPC) data. It addresses a main technical obstacle, the delivery bottleneck of large HPC results to the users, by augmenting high volume data generators with a flexible data management and extraction tool for spatio-temporal data. The multidimensional database system RasDaMan developed in FP4 will be enhanced with intelligent mass storage handling and optimised towards HPC. A critical mass of large European HPC centres participates; they all will operate the common platform, evaluating it in different HPC fields (University of Surrey: turbulent and combusting flows; CLRC, DKRZ: climate modelling; Cineca: cosmology; DLR: satellite image archives; IHPC&DB: gene expression simulation; CSCS: flow modelling of chemical reactors; Numeca: CFD post processing); the RasDaMan developers provide database skills. The outcome will be a field-tested open prototype platform with flexible, standard based retrieval on multi-Terabyte data in heterogeneous networks. The result will be produced by two SME partners, Active Knowledge GmbH and Numeca International S.A.

Objectives:
The overall objective is to develop a standard architecture and best practice knowledge for the efficient combination of HPC data processing and management, including fast & flexible retrieval, which is accepted as a broad consensus among European HPC centres. The participating HPC centres will act as multipliers to further centres and the European industry using HPC. The architecture will be implemented in various environments and for the most important HPC applications, forming a common pilot platform thoroughly evaluated by end users. Each application field is assigned to a partner with particular experience in this area. Individual objectives of the partners are: - For the vendor partners Active Knowledge GmbH and Numeca International S.A. the outcome significantly enhances their products, increases product visibility, and opens up large new markets. - For the HPC institutions, goal is to strengthen their research position and to offer improved services to their users and clients. - For the database researchers, goal is to strengthen their internationally acknowledged position in multidimensional database technology.

Work description:
Based on the multidimensional DBMS RasDaMan provided by the partner Active Knowledge GmbH, work is divided as follows: Each HPC partner will couple their HPC data generator and visualisation/post-processing front ends to RasDaMan as the common data management platform. The resulting systems will be operated under real life conditions with end users involved.

Each application area will be assigned to a particular partner with acknowledged experience:
- University of Surrey: simulation and experimentation turbulent and combusting flows;
- CLRC and DKRZ: climate modelling and research;
- Cineca: cosmology;
- DLR: satellite image retrieval and information extraction by post processing;
- IHPC&DB: simulation of the dynamics of gene expression;
- ETHZ flow modelling of chemical reactors

Numeca International S.A. will adapt its software to the DBMS to get into new markets. Active Knowledge GmbH, together with the research lab FORWISS, will extend the DBMS to optimally cope with the multi-Terabyte data sets by adding tertiary storage support and parallelism to the DBMS. Further, these partners will give necessary support to the other partners concerning database organization and operation. ERCOFTAC (European Research Community on Flow, Turbulenc and Combustion) will organise and lead the User Interest Group to raise awareness among further institutions and, particularly, among industry.

The milestones in the project are: M1 (9 months): analysis and specification (report) M2 (18 months): Demonstration of pilot platforms at each HPC lab: pilot platform 1st version; M3 (24 months): pilot platform second version; M4 (30 months): pilot platform 3rd version; user assessment; M5 (36 months): pilot platform final version and applications.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

BAYERISCHES FORSCHUNGSZENTRUM FUER WISSENSBASIERTE SYSTEME
Address
Boltzmannstrasse 3
85747 Garching Bei Muenchen
Germany

Participants (9)

ACTIVE KNOWLEDGE GMBH
Germany
Address
Kirchenstrasse 88
81675 Muenchen
CENTRE OF ADVANCED STUDIES
Russia
Address
Polytechnicheskaya, 29
195251 St. Petersburg
CONSORZIO INTERUNIVERSITARIO PER LA GESTIONE DEL CENTRO DI CALCOLO ELETTRONICO DELL'ITALIA NORD-ORIENTALE
Italy
Address
Via Magnanelli 6/3
40033 Casalecchio Di Reno (Bologna)
COUNCIL FOR THE CENTRAL LABORATORY OF THE RESEARCH COUNCILS
United Kingdom
Address
Rutherford Appleton Laboratory
OX11 0QX Chilton, Didcot
DEUTSCHES ZENTRUM FUER LUFT - UND R
Germany
Address
Linder Hohe
51147 Koln
EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH
Switzerland
Address
Raemistrasse 101
Zuerich
MAX-PLANCK INSTITUT FUER METEOROLOGIE
Germany
Address
Bundesstrasse 55
20146 Hamburg
NUMERICAL MECHANICS APPLICATIONS INTERNATIONAL S.A
Belgium
Address
Avenue Franklin Roosevelt 5
1050 Brussel
THE UNIVERSITY OF SURREY
United Kingdom
Address

GU2 7XH Guildford