Scalable online machine learning for predictive analytics and real-time interactive visualization

Informazioni relative al progetto

PROTEUS

ID dell’accordo di sovvenzione: 687691

DOI

10.3030/687691

Progetto chiuso

Data della firma CE 4 Novembre 2015

Data di avvio 1 Dicembre 2015

Data di completamento 30 Novembre 2018

Finanziato da

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Costo totale

€ 3 156 525,00

Contributo UE

€ 3 156 525,00

Coordinato da

BOURNEMOUTH UNIVERSITY
United Kingdom

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Risultati finali

Demonstrators, pilots, prototypes (17)

Final demonstrator

Final demonstrator together with the final report including the evaluation of the whole technology developed during the project in the Hot Strip Mill process in the ArcelorMittal steelmaking factory.

Hybrid computation tested system

Tested system implementation of hybrid computation for Apache Flink

Second prototype (V2)

Second version of the above, for the second prototype

Optimizer Prototype

Prototype of a domain specific optimizer for the declarative language and Apache Flink

Scalable online machine learning algorithms for streaming

This deliverable will introduce Version 1 of SOLMA that encompasses new scalable online machine learning algorithms.

Optimizer finished implementation

Finished implementation of a domain specific optimizer for the declarative language and Apache Flink.

Updateable-state management prototype implementation

Implemented system for updateable state for Apache Flink

Third prototype (V3)

Third version of the above, for the third prototype

Software implementation and integration with Apache Flink

This deliverable includes the implementation of the 3 layers of the proposed technical solution. Data Collector, and Incremental Analytics Engine layers will be implemented within the core of Apache Flink technology. The Visualization layer will be implemented as client-side library

Basic scalable streaming algorithms

This deliverable is in the form of software (joint with publications) will present Version 0 of the library covering a set of basic scalable streaming algorithms produced in Task 4.2.

Scalable drift and anomaly detection

This deliverable will result in Version 2 of SOLMA covering new scalable drift and anomaly detection algorithms.

Hybrid computation prototype implementation

Prototypical implementation of hybrid computation for Apache Flink covering basic workflows

First prototype (V1)

The first version of the evolving prototype in the validation scenario. An associated evolving document will provide, for each prototype execution, the objectives definition, KPIs involved and their evaluation after the prototype execution phase.

Declarative language tested implementation

Tested implementation of a declarative language for (online) machine learning

Declarative language finished implementation

Finished implementation of a declarative language for (online) machine learning

Scalable Online algorithms in Flink

This deliverable will release the final implementation in Flink of the streaming algorithms produced earlier through D4.2-D4.4.

Declarative language prototype implementation

Implementation of a basic declarative language prototype

Documents, reports (15)

Report on scientific dissemination activities – V1

Details for scientific dissemination activities and materials along with the time line and success indicators. It includes a record of activities related to scientific dissemination that have been undertaken during the first half of the project, and those planned for the second period.

Scenario details and objectives description

This document details the Hot Strip Mill process in terms of sensor data characteristics and data workflow. It also describes the scenario objectives from the end-user perspective.

Report on community engagement and technology transfer activities – V2

The final version of the deliverable compiles a record of all the activities related to community engagement and technology transfer developed in the course of the project

Scenario development and KPI definition for the PROTEUS solution

A report that presents the review of benchmarks, the typical scenarios used to define the parameters of the PROTEUS solution and requirements, benchmarks and KPIs

PROTEUS evaluation and impact assessment

A report which details the gains associated with the PROTEUS solution, using quantitative information, and which identifies areas for further improvement and investment

Guidelines for interacting and visualization information in Big Data environments

This document presents the results of the research in new ways of presenting and working with large amount of data and stream data

Visualization requirements for massive online machine learning strategies

This deliverable defines functional and non-functional requirements for the visualization system regarding online machine learning strategies

Report on project communication and engagement activities – V2

The final report of communication and engagement activities, compiling a list of all activities developed for communication with other relevant initiatives in the course of the project.

Report on scientific dissemination activities – V2 [

Final report of scientific dissemination activities. The final version compiles a record of all activities related to scientific dissemination developed in the course of the project.

Catalogue of scientific and technical requirements

This document describes the catalogue of scientific and technical challenges/requirements derived from the industrial scenario needs.

Report on community engagement and technology transfer activities – V1

Details for the community engagement and technology transfer strategy for the project. The intermediate report includes a record of activities related to community creation and engagement, and technology transfer developed in the course of the first half of the project, and those planned for the second half

Declarative language syntax definition

Syntax definition for a declarative language based on machine learning requirements

Architecture design for supporting incremental visual methods

This deliverable defines the technical design of the 3-layer based architecture for implementing the visualization system

Report on project communication and engagement activities – V1

Details for communication and engagement activities and materials along with the time line and success indicators. It includes a record of communication activities that have been undertaken during the first half of the project, and those planned for the second period.

Investigative overview of targeted techniques and algorithms

The state of the art of scalable streaming algorithms for distributed environments, non-scalable streaming algorithms, and selected prominent non-streaming and non-scalable algorithms that can be approximated by an online version.

Websites, patent fillings, videos etc. (2)

PROTEUS factsheet leaflet

The PROTEUS factsheet will be an early dissemination leaflet for dissmeination and communication purposes, including the most relevant information of the project in a nutshell, and will be available from the very begining as an initial public brochure.

PROTEUS project website

PROTEUS project public website, to be active and regularly updated during the whole project.

Pubblicazioni

Conference proceedings (12)

Efficient Migration of Very Large Distributed State for Scalable Streaming Processing

Autori: Bonaventura Del Monte
Pubblicato in: Proceedings of the VLDB 2017 PhD Workshop, Numero 28 August 2017, 2017
Editore: N/A

Non-dominated solutions visualization in multiobjective optimization: application to assembly line balancing

Autori: Krzysztof Trawinski, Manuel Chica, David P. Pancho, Sergio Damas, and Oscar Cordón
Pubblicato in: Proceeding of the MIC and MAEB 2017 Conferences, Numero June 2017, 2017, Pagina/e 963-972, ISBN 978-84-697-4275-1
Editore: Universitat Pompeu Fabra

Scotty: Efficient Window Aggregation for Out-of-Order Stream Processing

Autori: Jonas Traub, Philipp Marian Grulich, Alejandro Rodriguez Cuellar, Sebastian Bress, Asterios Katsifodimos, Tilmann Rabl, Volker Markl
Pubblicato in: 2018 IEEE 34th International Conference on Data Engineering (ICDE), 2018, Pagina/e 1300-1303, ISBN 978-1-5386-5520-7
Editore: IEEE
DOI: 10.1109/ICDE.2018.00135

Scalable online learning for flink - SOLMA library

Autori: W. Jamil, N-C. Duong, W. Wang, C. Mansouri, S. Mohamad, A. Bouchachia
Pubblicato in: Proceedings of the 12th European Conference on Software Architecture Companion Proceedings - ECSA '18, 2018, Pagina/e 1-4, ISBN 9781-450364836
Editore: ACM Press
DOI: 10.1145/3241403.3241438

Benchmarking Distributed Stream Data Processing Systems

Autori: Jeyhun Karimov, Tilmann Rabl, Asterios Katsifodimos, Roman Samarev, Henri Heiskanen, Volker Markl
Pubblicato in: 2018 IEEE 34th International Conference on Data Engineering (ICDE), 2018, Pagina/e 1507-1518, ISBN 978-1-5386-5520-7
Editore: IEEE
DOI: 10.1109/ICDE.2018.00169

Aggregation Algorithm Vs. Average for Time Series Prediction

Autori: Bouchachia, Abdelhamid; Kalnishkan, Y; Jamil, W.
Pubblicato in: ECML/PKDD 2016 Workshop on Large-scale Learning from Data Streams in Evolving Environments (STREAMEVOLV-2016), Numero 1, 2016, Pagina/e 69-82
Editore: N/A

Bridging the gap: towards optimization across linear and relational algebra

Autori: Andreas Kunft, Alexander Alexandrov, Asterios Katsifodimos, Volker Markl
Pubblicato in: BeyondMR '16 Proceedings of the 3rd ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond, Numero BeyondMR '16 26-06-2016, 2016, ISBN 978-1-4503-4311-4
Editore: ACM
DOI: 10.1145/2926534.2926540

Emma in Action: Declarative Dataflows for Scalable Data Analysis

Autori: Alexander Alexandrov , Andreas Salzmann , Georgi Krastev , Asterios Katsifodimos , Volker Markl
Pubblicato in: ACM SIGMOD '16 Proceedings of the 2016 SIGMOD International Conference on Management of Data, Numero Sigmod16, 26-06-2016, 2016, Pagina/e 2073-2076, ISBN 978-1-4503-3531-7
Editore: ACM
DOI: 10.1145/2882903.2899396

Implicit Parallelism through Deep Language Embedding

Autori: Alexander Alexandrov, Asterios Katsifodimos, Georgi Krastev, Volker Markl
Pubblicato in: ACM SIGMOD Record, Numero Volume 45, Number 1, March 2016, 2016, Pagina/e 51-58, ISSN 0163-5808
Editore: ACM
DOI: 10.1145/2949741.2949754

An Incremental Approach for Real-Time Big Data Visual Analytics

Autori: Ignacio Garcia, Ruben Casado, Abdelhamid Bouchachia
Pubblicato in: 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), 2016, Pagina/e 177-182, ISBN 978-1-5090-3946-3
Editore: IEEE
DOI: 10.1109/W-FiCloud.2016.46

A non-parametric hierarchical clustering model

Autori: Saad Mohamad, Abdelhamid Bouchachia, Moamar Sayed-Mouchaweh
Pubblicato in: 2015 IEEE International Conference on Evolving and Adaptive Intelligent Systems (EAIS), Numero STREAMEVOLV-2016, 23 September 2016, 2015, Pagina/e 1-7, ISBN 978-1-4673-6698-4
Editore: IEEE
DOI: 10.1109/EAIS.2015.7368803

Active Learning for Data Streams under Concept Drift and concept evolution

Autori: Saad Mohamad, Moamar Sayed-Mouchaweh and Abdelhamid Bouchachia
Pubblicato in: ECML/PKDD 2016 Workshop on Large-scale Learning from Data Streams in Evolving Environments, Numero STREAMEVOLV-2016, 23 September 2016, 2016, Pagina/e 51-68
Editore: -

Peer reviewed articles (7)

LIBIRWLS: A parallel IRWLS library for full and budgeted SVMs

Autori: Roberto Díaz-Morales, Ángel Navia-Vázquez
Pubblicato in: Knowledge-Based Systems, Numero 136, 2017, Pagina/e 183-186, ISSN 0950-7051
Editore: Elsevier BV
DOI: 10.1016/j.knosys.2017.09.007

Batch-based active learning: Application to social media data for crisis management

Autori: Daniela Pohl, Abdelhamid Bouchachia, Hermann Hellwagner
Pubblicato in: Expert Systems with Applications, Numero 93, 2018, Pagina/e 232-244, ISSN 0957-4174
Editore: Pergamon Press Ltd.
DOI: 10.1016/j.eswa.2017.10.026

Active learning for classifying data streams with unknown number of classes

Autori: Saad Mohamad, Moamar Sayed-Mouchaweh, Abdelhamid Bouchachia
Pubblicato in: Neural Networks, Numero 98, 2018, Pagina/e 1-15, ISSN 0893-6080
Editore: Pergamon Press Ltd.
DOI: 10.1016/j.neunet.2017.10.004

MSAFIS: an evolving fuzzy inference system

Autori: José de Jesús Rubio, Abdelhamid Bouchachia
Pubblicato in: Soft Computing, Numero 21/9, 2017, Pagina/e 2357-2366, ISSN 1432-7643
Editore: Springer Verlag
DOI: 10.1007/s00500-015-1946-4

Blockjoin

Autori: Andreas Kunft, Asterios Katsifodimos, Sebastian Schelter, Tilmann Rabl, Volker Markl
Pubblicato in: Proceedings of the VLDB Endowment, Numero 10/13, 2017, Pagina/e 2061-2072, ISSN 2150-8097
Editore: ACM
DOI: 10.14778/3151106.3151110

Improving the efficiency of IRWLS SVMs using parallel Cholesky factorization

Autori: Díaz Morales, R. , & Navia Vázquez, Á
Pubblicato in: Pattern Recognition Letters, Numero Volume 84, 1 December 2016, 2016, Pagina/e 91-98, ISSN 0167-8655
Editore: Elsevier BV
DOI: 10.1016/j.patrec.2016.08.015

A Bi-Criteria Active Learning Algorithm for Dynamic Data Streams

Autori: Mohamad, S., Bouchachia, A. and Sayed-Mouchaweh, M.
Pubblicato in: IEEE Transactions on Neural Networks and Learning Systems, Numero N/A (early access), 2016, Pagina/e 1-13, ISSN 2162-2388
Editore: IEEE
DOI: 10.1109/TNNLS.2016.2614393

Book chapters (2)

Model Selection in Online Learning for Times Series Forecasting

Autori: Waqas Jamil, Abdelhamid Bouchachia
Pubblicato in: Advances in Computational Intelligence Systems - Contributions Presented at the 18th UK Workshop on Computational Intelligence, September 5-7, 2018, Nottingham, UK, Numero 840, 2019, Pagina/e 83-95, ISBN 978-3-319-97981-6
Editore: Springer International Publishing
DOI: 10.1007/978-3-319-97982-3_7

Fuzzy Classifiers

Autori: Abdelhamid Bouchachia
Pubblicato in: Handbook on Computational Intelligence, Numero May 2016, 2016, Pagina/e 185-207, ISBN 978-981-4675-00-0
Editore: WORLD SCIENTIFIC
DOI: 10.1142/9789814675017_0005

Monographic books (2)

Advances in Computational Intelligence Systems - Contributions Presented at the 18th UK Workshop on Computational Intelligence, September 5-7, 2018, Nottingham, UK

Autori: Ahmad Lotfi, Hamid Bouchachia, Alexander Gegov, Caroline Langensiepen, Martin McGinnity
Pubblicato in: Advances in Intelligent Systems and Computing, 2019, ISBN 978-3-319-97982-3
Editore: Springer
DOI: 10.1007/978-3-319-97982-3

ECML/PKDD 2017 Workshop on IoT Large Scale Learning from Data Streams

Autori: M.S. Mouchaweh, A. Bifet, A. Bouchachia, J. Gama, R. Ribeiro
Pubblicato in: 2017
Editore: CEUR-WS.org

Other (1)

Apache Flink: Stream and Batch Processing in a Single Engine

Autori: Paris Carbone, Stephan Ewen, Seif Haridi, Asterios Katsifodimos, Volker Markl, Kostas Tzoumas
Pubblicato in: Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, Numero December 2015 Vol. 38 No. 4, Numero on Next-Generation Stream Processing Systems, 2015, Pagina/e 28-38
Editore: IEEE