The EUBra-BIGSEA project has produced three novel components (a QoS Data Analytics Platform, a Data Analytics Development Framework, and a toolbox of models for building applications on traffic data) demonstrated on three applications for urban transportation data management and disseminated in 60 Publications and 50 contributions in conferences. The software is available in the project GitHub (
https://github.com/eubr-bigsea(opens in new window)) and DockerHub (
https://hub.docker.com/u/eubrabigsea/(opens in new window)) as well as in the EUBra-BIGSEA website (
http://www.eubra-bigsea.eu(opens in new window)) along with papers and presentations. Video demos are available on the youtube channel of the project (
https://goo.gl/FTCq3g(opens in new window)).
These components and applications target cloud providers, data analysis application developers, Data Scientists and Municipalities.
The 20 EUBra-BIGSEA building blocks are organised in 6 layers (infrastructure services, Big Data Services, Programming Frameworks, Security Services, High-Level services and Applications). Out of this 20 building blocks, 15 are new developments and 5 legacy components notably improved during the project.
System Administrators and cloud infrastructure providers can benefit from the QoS Data Analytics Platform, which comprises a set of applications and services that can be conveniently deployed, which provide transparent horizontal and vertical elasticity, performance modelling and optimisation. The platform enables the automatic deployment of services to run Big Data application ensuring deadline enactment through the automatic reconfiguration of the infrastructure and it is tailored for OpenStack, OpenNebula and Mesos frameworks, using IM, CLUES and MONASCA.
Data Scientists and Data Analytics Application developers can benefit from the Data Analytics Development Framework, which integrates a graphics framework (LEMONADE) capable of building up data analytic parallel workflows on Spark and COMPSs and supporting OLAP functions through Ophidia with privacy and QoS constraints. The programming framework includes a broad suite of pre-built tools and descriptive and predictive models for traffic data analytics, ready to be integrated on the applications developed Data Scientists. The programming models include the support to privacy policies and generic Data Quality and Entity Matching services and a toolbox of models for building applications on traffic data, with models for extracting routes, predicting crowdedness, estimating traffic jams, sentiment analysis, computing trajectories among others.
The dissemination of results have produced 60 publications in scientific and technical journals (some of them are high-impact journals, such as the Future Generation of Computer Systems, the Journal of Grid Computing, the Journal of Systems and Software or the Journal of Parallel and Distributed Computing), as well as 50 participation in events (including the organisation of a satellite workshop in CCGRID 2017). The outcome of the publications is expected to increase if joint publications submitted are finally accepted.
The Europe-Brazil collaboration has been crucial to achieving:
- The QoS Cloud services, integrating vertical elasticity framework from UFCG with horizontal elasticity and convenient deployment from UPV and the performance modelling and optimisation of the configuration by applications from POLIMI and UFMG.
- The Data Analytic development framework, which includes LEMONADE developed from UFMG executing workflows in parallel through COMPSs developed by BSC and connecting to processing functions from Ophidia, developed by CMCC. The framework provides privacy preservation through PRIVaaS developed jointly by UNICAMP and UC, Entity Matching, developed by UFCG and Data Quality developed by POLIMI.
- The toolbox includes models for Extracting routes, predicting crowdedness, estimating traffic jams, sentiment analysis, computing trajectories, developed by UFMG, UFCG and UFTPR, integrated into applications developed by CMCC, UFCG and UPV.
