Skip to main content
European Commission logo
français français
CORDIS - Résultats de la recherche de l’UE
CORDIS

Serverless Data Analytics Platform

Livrables

Final dissemination, exploitation, and adoption report

Thorough description of all dissemination activities and specifically our impact on major open source projects It will also describe all exploitation activities in the different industrial partners and future plans for each product The adoption report will contain all main contributions of the partners to relevant open source projects both in terms of input documents submitted for consideration and software components integrated to the reference software implementations It will also describe the user communities and international adoption and impact of the results of the project

Full implementation of the BLOSSOM middleware

This final deliverable includes 1 a library of composable replicated data types and their support for Java as well as for a functional language running atop the JVM 2 several mechanisms to allow object composition and sharding while preserving correctness 3 a modular consensus layer able to adjust to the consistency of each object paying the price of synchronization only where neededThe final evaluation is made using a largescale serverless applications from one of the partners

Initial specs of the Serverless Compute and Execution Engine

This deliverable includes initial specifications and design of the Serverless Compute and Execution Engine. During the first six months of the project an assessment of the leading open source serverless technologies, such as Apache OpenWhisk, PyWren, OpenFaaS, nuclio, and other, will be performed. The results of this assessment will be reported in the specification and design document and an initial prototype will be built on the best of breed technology with the highest potential for impact.

CloudButton Initial API Definition

This deliverable includes a detailed definition of the exact abstractions and API we will be exposing. It will outline the methods by which users can mark up their code for parallelization as well pseudocode sketches of how all the custom data structures will behave. The deliverable will also provide initial observations on the complexity of the porting tools we have proposed, including experiments with static analysis and an audit of the required OpenMP and MPI functionality.

Serverless Compute Engine Design and Prototypes

This deliverable describes the Serverless Compute and Execution Engine software components. It also includes initial monitoring and instrumentation efforts to acquire telemetry and log data for the workloads across various services using real experiments. The collected data will be used to validate the software and to identify candidate performance improvements and I/O problems. Finally, it also includes a performance evaluation study on the data from the use cases.

Communication report

Description of the dissemination activities with lessons learned and progress reporting. It will also describe community involvement activities. Includes the initial version of the exploitation plan.

Communication plan

Definition of the required process and strategy for dissemination activities. Description of the planned dissemination activities and expected progress reporting. It will also describe planned community involvement activities.

CloudButton Architecture Specs and Early Prototypes

Specification of the Architecture and APIs. Documentation, early tutorials, and automated tests for the early prototypes of the different software components. First description and evaluation of results obtained from validation in use cases using different experiments and workloads.

Specification and partial support for degradable objects

This mid-term deliverable includes a full implementation of the client-side of BLOSSOM and a prototype of the modular consensus-based library. The server side of BLOSSOM is able to implement any user-defined replicated data type. It runs in at least into three degraded modes, including linearizability and update consistency. A library of replicated data types is available, some of which can be sharded if required by the programmer. At that stage, we plan a detailed evaluation of the prototype using a relevant middle-scale serverless application (e.g., data analytics).

CloudButton Toolkit Reference Implementation

This final deliverable encompasses a working implementation of the CloudButton toolkit and the associated porting tools It will primarily focus on demonstrating the performance and robustness of several detailed reference implementations along with indepth comparisons to their equivalent implementations in existing frameworksThe deliverable will also include examples of porting existing HPC applications written in Java into those that run on CloudButton both using our static analysis tools as well as our suite of libraries and standardised patterns The comparison between CloudButton and the current stateof theart here will be a key output of our work

Initial prototype for stateful serverless computation

This deliverable includes a detailed specification of the programming language support for Java and an initial prototype of the server side. This initial prototype relies on the Creson framework and uses Infinispan for data distribution, replication and persistence.

CloudButton Prototype of Abstractions, Fault-tolerance and Porting Tools

This deliverable depends on the Serverless Compute Engine design (D3.1), as well as the programming abstractions built to deal with mutable state (D4.1). It will include an implementation of a significant portion of the mark-up annotations and custom data structures, evaluated via two prototypes of non-trivial big data and machine-learning problems. These prototypes will demonstrate automatically parallelized Java code based on stateful dataflow graphs, processed and executed on CloudButton. A simple fault-tolerance mechanism and associated configuration mechanism will also be provided.

Reference implementation of architectural building blocks

Public release of stable software components of the CloudButton Toolkit Complete specification and APIs Final description and evaluation of results obtained from validation in use cases using different workloads

Experiments and Initial Specifications

Description of use case scenarios, experiments and benchmarking framework. Initial specifications of the architecture.

Serverless Compute Engine Reference Implementation

This deliverable includes the final design and reference implementation for all tasksDescription and specification of the the stable release of the system including all APIs and the software components that implement these APIs the admin toolset runtime APIs Execution Engine and resource scheduler This deliverable also presents the integrated system architecture and consolidated results on the performance achieved by the joint operation of monitoring and deployment tools It also includes tutorials to facilitate the adoption of the platform by thirdparty developers Finally it also includes the performance evaluation study on the data and experiments from the use cases

Public Project Website

A website will be developed in order to provide a continuous update about the project progress and the results obtained during it. All public deliverables and publications (Open Access) will be uploaded on the website.

Data Management Plan, 3rd Version

This deliverable presents the third version of the project Data Management Plan DMP It is submitted on Month 39 as a Final review of the CloudButton Data Management Plan

Data Management Plan, 2nd Version

This deliverable presents the second version of the project Data Management Plan (DMP). It is submitted on Month 18 as a Mid–Term review of the CloudButton Data Management Plan.

Data Management Plan, 1st version

First version of the data management plan. The different experiments, workloads, benchmarks, and results will be delivered as Open Research Data for the community. This deliverable will evolve during the lifetime of the project in order to present the status of the project’s reflections on data management.

Publications

MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training

Auteurs: Pablo Gimeno Sarroca, Marc Sánchez-Artigas
Publié dans: 2022
Éditeur: arXiv
DOI: 10.48550/arxiv.2206.05786

Decentralize the feedback infrastructure!

Auteurs: Pedro Garcia Lopez
Publié dans: 2020
Éditeur: arXiv
DOI: 10.48550/arxiv.2010.03356

ServerMix: Tradeoffs and Challenges of Serverless Data Analytics

Auteurs: García-López, Pedro; Sánchez-Artigas, Marc; Shillaker, Simon; Pietzuch, Peter; Breitgand, David; Vernik, Gil; Sutra, Pierre; Tarrant, Tristan; Ferrer, Ana Juan
Publié dans: 2019
Éditeur: Cornell University

Serverless Predictions: 2021-2030

Auteurs: Pedro Garcia Lopez, Aleksander Slominski, Michael Behrendt, Bernard Metzler
Publié dans: 2021
Éditeur: arXiv

Using Biological Signals for Mass Recalibration of Mass Spectrometry Imaging Data

Auteurs: Raphaël La Rocca, Christopher Kune, Mathieu Tiquet, Lachlan Stuart, Theodore Alexandrov, Edwin De Pauw, Loïc Quinton
Publié dans: 2020
Éditeur: ChemRxiv
DOI: 10.26434/chemrxiv.12901679.v1

Transparent Serverless execution of Python multiprocessing applications

Auteurs: Aitor Arjona, Gerard Finol, Pedro Garcia-Lopez
Publié dans: 2022
Éditeur: arXiv

Serverless End Game: Disaggregation enabling Transparency

Auteurs: García-López, Pedro; Slominski, Aleksander; Shillaker, Simon; Behrendt, Michael; Metzler, Barnard
Publié dans: 2020
Éditeur: arXiv
DOI: 10.48550/arxiv.2006.01251

Please, do not decentralize the Internet with (permissionless) blockchains!

Auteurs: Pedro Garcia Lopez, Alberto Montresor, Anwitaman Datta
Publié dans: 2019
Éditeur: arXiv
DOI: 10.48550/arxiv.1904.13093

EGEON: Software-Defined Data Protection for Object Storage

Auteurs: Raul Saiz-Laudo, Marc Sanchez-Artigas
Publié dans: 2022
Éditeur: arXiv

Efficient replication via timestamp stability

Auteurs: Vitor Enes, Carlos Baquero, Alexey Gotsman, Pierre Sutra
Publié dans: EuroSys '21: Proceedings of the Sixteenth European Conference on Computer Systems, 2021, Page(s) 178–193, ISBN 978-1-4503-8334-9
Éditeur: Association for Computing Machinery
DOI: 10.1145/3447786.3456236

Triggerflow - trigger-based orchestration of serverless workflows

Auteurs: Pedro García López, Aitor Arjona, Josep Sampé, Aleksander Slominski, Lionel Villard
Publié dans: Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems, 2020, Page(s) 3-14, ISBN 9781450380287
Éditeur: ACM
DOI: 10.1145/3401025.3401731

FaaS Orchestration of Parallel Workloads

Auteurs: Daniel Barcelona-Pons, Pedro García-López, Álvaro Ruiz, Amanda Gómez-Gómez, Gerard París, Marc Sánchez-Artigas
Publié dans: WOSC '19: Proceedings of the 5th International Workshop on Serverless Computing, 2019, Page(s) 25-30, ISBN 978-1-4503-7038-7
Éditeur: Association for Computing Machinery
DOI: 10.1145/3366623.3368137

Bringing scaling transparency to Proteomics applications with serverless computing

Auteurs: Mariano Ezequiel Mirabelli, Pedro García-López, Gil Vernik
Publié dans: WoSC'20: Proceedings of the 2020 Sixth International Workshop on Serverless Computing, 2021, Page(s) 55–60, ISBN 978-1-4503-8204-5
Éditeur: Association for Computing Machinery
DOI: 10.1145/3429880.3430101

A milestone for FaaS pipelines; object storage-vs VM-driven data exchange

Auteurs: Germán T. Eizaguirre, Marc Sánchez-Artigas, Pedro García-López
Publié dans: Middleware '21: Proceedings of the 22nd International Middleware Conference: Demos and Posters, 2021, Page(s) 10-11, ISBN 978-1-4503-9154-2
Éditeur: Association for Computing Machinery
DOI: 10.1145/3491086.3492472

State-Machine Replication for Planet-Scale Systems

Auteurs: Vitor Enes, Carlos Baquero, Tuanir França Rezende, Alexey Gotsman, Matthieu Perrin, Pierre Sutra
Publié dans: EuroSys'20 : Fifteenth European Conference on Computer System, Numéro Article No.: 24, 2020, Page(s) 1-15, ISBN 978-1-4503-6882-7
Éditeur: Association for Computing Machinery
DOI: 10.1145/3342195.3387543

Serverless Elastic Exploration of Unbalanced Algorithms

Auteurs: Gerard París; Pedro García-López; Marc Sánchez-Artigas
Publié dans: 2020 IEEE 13th International Conference on Cloud Computing (CLOUD), 2020, ISBN 978-1-7281-8780-8
Éditeur: Institute of Electrical and Electronics Engineers
DOI: 10.1109/cloud49709.2020.00033

Primula: a Practical Shuffle/Sort Operator for Serverless Computing

Auteurs: Marc Sánchez-Artigas, Germán T. Eizaguirre, Gil Vernik, Lachlan Stuart, Pedro García-López
Publié dans: Middleware '20: Proceedings of the 21st International Middleware Conference Industrial Track, 2020, ISBN 978-1-4503-8201-4
Éditeur: Association for Computing Machinery
DOI: 10.1145/3429357.3430522

J-NVM: Off-heap Persistent Objects in Java

Auteurs: Anatole Lefort, Yohan Pipereau, Kwabena Amponsem, Pierre Sutra, Gaël Thomas
Publié dans: SOSP '21: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, 2021, Page(s) 408–423, ISBN 978-1-4503-8709-5
Éditeur: Association for Computing Machinery
DOI: 10.1145/3477132.3483579

Faasm: Lightweight Isolation for Efficient Stateful Serverless Computing

Auteurs: Simon Shillaker, Peter Pietzuch
Publié dans: USENIX Annual Technical Conference 2020, 2020, Page(s) 419-433, ISBN 978-1-939133-14-4
Éditeur: USENIX Association

On the FaaS Track - Building Stateful Distributed Applications with Serverless Architectures

Auteurs: Daniel Barcelona-Pons, Marc Sánchez-Artigas, Gerard París, Pierre Sutra, Pedro García-López
Publié dans: Proceedings of the 20th International Middleware Conference, 2019, Page(s) 41-54, ISBN 9781450370097
Éditeur: ACM
DOI: 10.1145/3361525.3361535

Leaderless State-Machine Replication: Specification, Properties, Limits (Extended Version)

Auteurs: Tuanir França Rezende, Pierre Sutra
Publié dans: DISC'20: 34th International Symposium on Distributed Computing, 2020, Page(s) 24:1--24:17, ISBN 978-3-95977-168-9
Éditeur: Schloss Dagstuhl -- Leibniz-Zentrum fur Informatik
DOI: 10.4230/lipics.disc.2020.24

The serverless shell

Auteurs: Aurèle Mahéo, Pierre Sutra, Tristan Tarrant
Publié dans: Middleware '21: Proceedings of the 22nd International Middleware Conference: Industrial Track, 2021, Page(s) 9-15, ISBN 978-1-4503-9152-8
Éditeur: Association for Computing Machinery
DOI: 10.1145/3491084.3491426

Spatial Metabolomics and Imaging Mass Spectrometry in the Age of Artificial Intelligence

Auteurs: Theodore Alexandrov
Publié dans: Annual Review of Biomedical Data Science, 2020, ISSN 2574-3414
Éditeur: Annual Reviews
DOI: 10.1146/annurev-biodatasci-011420-031537

On the correctness of Egalitarian Paxos

Auteurs: Pierre Sutra
Publié dans: Information Processing Letters, Numéro 156, 2020, Page(s) 105901, ISSN 0020-0190
Éditeur: Elsevier BV
DOI: 10.1016/j.ipl.2019.105901

Triggerflow: Trigger-based orchestration of serverless workflows

Auteurs: Aitor Arjona. Pedro García-López, Josep Sampé, Aleksander Slominski, Lionel Villard
Publié dans: Future Generation Computer Systems, Numéro Volume 124, 2021, Page(s) 215-229, ISSN 0167-739X
Éditeur: Elsevier BV
DOI: 10.1016/j.future.2021.06.004

Benchmarking parallelism in FaaS platforms

Auteurs: Daniel Barcelona-Pons, Pedro García-López
Publié dans: Future Generation Computer Systems, Numéro Volume 124, 2021, Page(s) 268-284, ISSN 0167-739X
Éditeur: Elsevier BV
DOI: 10.1016/j.future.2021.06.005

OffsampleAI: artificial intelligence approach to recognize off-sample mass spectrometry images

Auteurs: Katja Ovchinnikova, Vitaly Kovalev, Lachlan Stuart & Theodore Alexandrov
Publié dans: BMC Bioinformatics, Numéro 21, 2020, Page(s) 129, ISSN 1471-2105
Éditeur: BioMed Central
DOI: 10.1186/s12859-020-3425-x

Outsourcing Data Processing Jobs with Lithops

Auteurs: Josep Sampe, Marc Sanchez-Artigas, Gil Vernik, Ido Yehekzel, Pedro Garcia-Lopez
Publié dans: IEEE Transactions on Cloud Computing, 2021, ISSN 2168-7161
Éditeur: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/tcc.2021.3129000

Stateful Serverless Computing with Crucial

Auteurs: Daniel Barcelona-Pons, Pierre Sutra, Marc Sánchez-Artigas, Gerard París, Pedro García-López
Publié dans: ACM Transactions on Software Engineering and Methodology, Numéro Volume 31, Numéro 3, Article 39, 2022, Page(s) 1-38, ISSN 1049-331X
Éditeur: Association for Computing Machinary, Inc.
DOI: 10.1145/3490386

A compressed file partitioner for scalable Genomics analysis with Serverless technology

Auteurs: Francisco Damián Maleno González
Publié dans: 2021
Éditeur: University Rovira i Virgili

Study of the Feasibility of Serverless Access Transparency for Python Multiprocessing Applications

Auteurs: Gerard Finol Peñalver, Aitor Arjona Pérez
Publié dans: 2021
Éditeur: University Rovira i Virgili

Serverless OCaml Genomic Pipeline Parallelisation Engine

Auteurs: Gil Arasa Verge
Publié dans: 2022
Éditeur: University Rovira i Virgili

Machine Learning on a Serverless Architecture

Auteurs: Pablo Gimeno Sarroca
Publié dans: 2021
Éditeur: University Rovira i Virgili

Painless Data Analytics in the Cloud. Grouping data in serverless architectures

Auteurs: German Telmo Eizaguirre Suarez
Publié dans: 2021
Éditeur: University Rovira i Virgili

Porting Genomics pipelines to the Cloud - Serverless Computing as an avenue for scalable variant calling

Auteurs: Xavier Roca i Canals
Publié dans: 2022
Éditeur: University Rovira i Virgili

Trade-Offs and Challenges of Serverless Data Analytics

Auteurs: Pedro García-López, Marc Sánchez-Artigas, Simon Shillaker, Peter Pietzuch, David Breitgand, Gil Vernik, Pierre Sutra, Tristan Tarrant, Ana Juan-Ferrer & Gerard París
Publié dans: Technologies and Applications for Big Data Value, 2021, Page(s) 41-61, ISBN 978-3-030-78307-5
Éditeur: Springer, Cham
DOI: 10.1007/978-3-030-78307-5_3

Recherche de données OpenAIRE...

Une erreur s’est produite lors de la recherche de données OpenAIRE

Aucun résultat disponible