Skip to main content

Software Defined Storage for Big Data

Objective

The main objective is to create IOStack: a Software Defined Storage toolkit for Big Data on top of the OpenStack platform. IOStack will enable efficient execution of virtualized analytics applications over virtualized storage resources thanks to flexible, automated, and low cost data management models based on software defined storage (SDS). Major challenges are:

1) Storage and compute disaggregation and virtualization.
Virtualizing data analytics to reduce costs implies disaggregation of existing hardware resources.
This requires the creation a virtual model for compute, storage and networking that allows orchestration
tools to manage resources in an efficient manner. We will provide policy-based provisioning tools so that the provisioning of virtual components for the analytics platform is made according to the set of QoS policies.

2) SDS Services for Analytics.
The objective is to define, design, and build a stack of SDS data services enabling virtualized analytics services with improved performance and usability. Among these services we include native object store analytics that will allow running analytics close to the data without taxing initial migration, data reduction services, specialized persistent caching mechanisms, advanced prefetching, and data placement.

3) Orchestration and deployment of big data analytics services.
The objective is to design and build efficient deployment strategies for virtualized analytic-as-a-service instances (both ephemeral and permanent). In particular, the focus of this work is on data-intensive systems such as Apache Hadoop and Apache Spark, which enable users to define both batch and latency-sensitive analytics. This objective includes the design of scalable algorithms that strive at optimizing a service-wide objective function (e.g., optimize performance, minimize cost) under different workloads.

Finally, we will create a SDS toolkit for Big Data on top of the OpenStack projects Sahara, Cinder, Nova and Swift.

Field of science

  • /natural sciences/computer and information sciences/data science/big data
  • /natural sciences/computer and information sciences/software/software development/software architecture

Call for proposal

H2020-ICT-2014-1
See other projects for this call

Funding Scheme

RIA - Research and Innovation action

Coordinator

UNIVERSITAT ROVIRA I VIRGILI
Address
Carrer De Escorxador
43003 Tarragona
Spain
Activity type
Higher or Secondary Education Establishments
EU contribution
€ 551 125

Participants (7)

IBM ISRAEL - SCIENCE AND TECHNOLOGY LTD
Israel
EU contribution
€ 788 351,50
Address
94 Derech Em-hamoshavot
49527 Petach Tikva
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
EURECOM
France
EU contribution
€ 496 847,50
Address
Route Des Chappes 450 Campus Sophiatech
06410 Biot
Activity type
Higher or Secondary Education Establishments
BARCELONA SUPERCOMPUTING CENTER - CENTRO NACIONAL DE SUPERCOMPUTACION
Spain
EU contribution
€ 397 250
Address
Calle Jordi Girona 31
08034 Barcelona
Activity type
Research Organisations
MPSTOR Limited
Ireland
EU contribution
€ 620 625
Address
Rubicon Centre
Bishopstown Cork
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
IDIADA AUTOMOTIVE TECHNOLOGY SA
Spain
EU contribution
€ 113 300
Address
L Albornar
43710 Santa Oliva
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
GRIDPOCKET SAS
France
EU contribution
€ 205 750
Address
Route De Cretes 300
06560 Valbonne Sophia Antipolis
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
ARCTUR RACUNALNISKI INZENIRING DOO
Slovenia
EU contribution
€ 145 375
Address
Industrijska Cesta 1A
5000 Nova Gorica
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)