Skip to main content

Middleware for memory and data-awareness in workflows

Objective

Maestro will build a data-aware and memory-aware middleware framework that addresses ubiquitous problems of data movement in complex memory hierarchies and at many levels of the HPC software stack.

Though HPC and HPDA applications pose a broad variety of efficiency challenges, it would be fair to say that the performance of both has become dominated by data movement through the memory and storage systems, as opposed to floating point computational capability. Despite this shift, current software technologies remain severely limited in their ability to optimise data movement. The Maestro project addresses what it sees as the two major impediments of modern HPC software:

1. Moving data through memory was not always the bottleneck. The software stack that HPC relies upon was built through decades of a different situation – when the cost of performing floating point operations (FLOPS) was paramount. Several decades of technical evolution built a software stack and programming models highly fit for optimising floating point operations but lacking in basic data handling functionality. We characterise the set of technical issues at missing data-awareness.

2. Software rightfully insulates users from hardware details, especially as we move higher up the software stack. But HPC applications, programming environments and systems software cannot make key data movement decisions without some understanding of the hardware, especially the increasingly complex memory hierarchy. With the exception of runtimes, which treat memory in a domain-specific manner, software typically must make hardware-neutral decisions which can often leave performance on the table . We characterise this issue as missing memory-awareness.

Maestro proposes a middleware framework that enables memory- and data-awareness.

Call for proposal

H2020-FETHPC-2016-2017

See other projects for this call

Sub call

H2020-FETHPC-2017

Coordinator

FORSCHUNGSZENTRUM JULICH GMBH
Net EU contribution
€ 629 156,25
Address
Wilhelm Johnen Strasse
52428 Julich
Germany

See on map

Region
Nordrhein-Westfalen Köln Düren
Activity type
Research Organisations
Non-EU contribution
€ 0,00

Participants (7)

COMMISSARIAT A L ENERGIE ATOMIQUE ET AUX ENERGIES ALTERNATIVES
France
Net EU contribution
€ 575 375,00
Address
Rue Leblanc 25
75015 Paris 15

See on map

Region
Ile-de-France Ile-de-France Paris
Activity type
Research Organisations
Non-EU contribution
€ 0,00
APPENTRA SOLUTIONS SL
Spain
Net EU contribution
€ 269 150,00
Address
Lugar Campus De Elvina S/n Edificio De Servicios
15071 A Coruna

See on map

SME

The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.

Yes
Region
Galicia Ourense
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Non-EU contribution
€ 0,00
EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH
Switzerland
Net EU contribution
€ 567 955,00
Address
Raemistrasse 101
8092 Zuerich

See on map

Region
Schweiz/Suisse/Svizzera Zürich Zürich
Activity type
Higher or Secondary Education Establishments
Non-EU contribution
€ 0,00
EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS
United Kingdom
Net EU contribution
€ 655 375,00
Address
Shinfield Park
RG2 9AX Reading

See on map

Region
South East (England) Berkshire, Buckinghamshire and Oxfordshire Berkshire
Activity type
Research Organisations
Non-EU contribution
€ 0,00
SEAGATE SYSTEMS UK LIMITED
United Kingdom
Net EU contribution
€ 595 280,00
Address
Langstone Road
PO9 1SA Havant

See on map

Region
South East (England) Hampshire and Isle of Wight South Hampshire
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Non-EU contribution
€ 0,00
CRAY COMPUTER GMBH

Participation ended

Switzerland
Net EU contribution
€ 273 050,11
Address
Hochbergerstrasse 60C
4057 Basel

See on map

Region
Schweiz/Suisse/Svizzera Nordwestschweiz Basel-Stadt
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Non-EU contribution
€ 0,00
HEWLETT-PACKARD (SCHWEIZ) GMBH
Switzerland
Net EU contribution
€ 424 149,89
Address
Neue Winterthurerstrasse 99
8304 Wallisellen

See on map

Region
Schweiz/Suisse/Svizzera Zürich Zürich
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Non-EU contribution
€ 0,00