Periodic Reporting for period 1 - ELASTIC (A Software Architecture for Extreme-ScaLe Big-Data AnalyticS in Fog CompuTIng ECosystems)
Período documentado: 2018-12-01 hasta 2020-05-31
Current cities are facing the challenge of extracting valuable knowledge from the vast amount of data generated by public and private transportation, city infrastructures, IoT devices, etc. This challenge is complicated with the additional needs inherited from the operational requirements of cities, i.e. information must be processed in real-time, consuming as less energy as possible and guaranteeing a secure environment with changing communication conditions.
With total funding of € 5.9 million, the ELASTIC project is developing a novel SW architecture framework for the development and execution of advanced mobility applications for smart cities. One of the key innovations of the ELASTIC framework will be its capability of distributing extreme-scale big-data analytics workflows across the compute continuum (from edge to cloud in a fog computing environment) while guaranteeing the city operational requirements. To do so, ELASTIC defines two strategic goals:
1. Provide SW developers with the right level of abstraction to facilitate the development of advanced mobility applications including complex extreme-scale big-data analytics workflows.
2. Provide the data throughput speed-up and data analytics accuracy needed, while fulfilling the operational requirements of cities.
ELASTIC will be evaluated in the Florence tramway network, featuring a sensor, communication and computing infrastructure to collect, exchange and process massive amounts of data in real-time. Moreover, it includes three tramway vehicles to give innovative capabilities to drivers, and vehicle-to-infrastructure communication solutions applied to private cars to inform about potential hazards.
Overall, the ELASTIC framework will allow to generate a common knowledge base between the tramway infrastructure, the tram vehicles and private cars and upon which three advanced mobility applications will operate:
- Next Generation Autonomous Positioning for an autonomous localization tram vehicles, and Advanced Driving Assistant System, for obstacle detection capabilities.
- Predictive maintenance, for monitoring variables related to the tramway infrastructure with the objective of identifying changes in equipment behaviour, before the equipment starts to fail.
- Public and private transport interaction to provide outputs for users and/or operators to identify critical situations (e.g. vehicle crossing junction despite red) and (2) optimize local traffic regulation strategies (e.g. overcrowding at the tram stop that might delay departure).
With total funding of € 5.9 million, the ELASTIC project is developing a novel SW architecture framework for the development and execution of advanced mobility applications for smart cities. One of the key innovations of the ELASTIC framework will be its capability of distributing extreme-scale big-data analytics workflows across the compute continuum (from edge to cloud in a fog computing environment) while guaranteeing the city operational requirements. To do so, ELASTIC defines two strategic goals:
1. Provide SW developers with the right level of abstraction to facilitate the development of advanced mobility applications including complex extreme-scale big-data analytics workflows.
2. Provide the data throughput speed-up and data analytics accuracy needed, while fulfilling the operational requirements of cities.
ELASTIC will be evaluated in the Florence tramway network, featuring a sensor, communication and computing infrastructure to collect, exchange and process massive amounts of data in real-time. Moreover, it includes three tramway vehicles to give innovative capabilities to drivers, and vehicle-to-infrastructure communication solutions applied to private cars to inform about potential hazards.
Overall, the ELASTIC framework will allow to generate a common knowledge base between the tramway infrastructure, the tram vehicles and private cars and upon which three advanced mobility applications will operate:
- Next Generation Autonomous Positioning for an autonomous localization tram vehicles, and Advanced Driving Assistant System, for obstacle detection capabilities.
- Predictive maintenance, for monitoring variables related to the tramway infrastructure with the objective of identifying changes in equipment behaviour, before the equipment starts to fail.
- Public and private transport interaction to provide outputs for users and/or operators to identify critical situations (e.g. vehicle crossing junction despite red) and (2) optimize local traffic regulation strategies (e.g. overcrowding at the tram stop that might delay departure).
During the first 18 month, ELASTIC has performed three main activities: (1) The development of the ELASTIC SW architecture framework; (2) the development of three advanced mobility applications based extreme-scale big-data analytics (using the ELASTIC framework); and (3) the design of the tramway network infrastructure for the extraction of the data needed to by the three applications.
With respect to the ELASTIC framework, the project has selected the set of SW components that will form the framework, prioritising those owned by the members of the ELASTIC consortium or offered as open-source components with a large community behind them, with the objective of reducing the time-to-market and maximize exploitation opportunities. The ELASTIC framework consists of four layers:
1. A Distributed Data Analytics Platform layer, for data accessibility across the compute continuum, covering aspects such as data-in-motion and data-at-rest analytics.
2. An Orchestration layer, responsible of deploying and distributing extreme-scale big-data analytics workflows across the compute continuum, with the objective of guaranteeing their operational requirements.
3. An Operation Requirement Analysis layer, that monitors the execution of analytics workflows and informs to the Orchestrator layer about the operational requirements, addressing time, energy, communication quality and security.
4. A Fog Computing Platform layer that implements the compute continuum, including monitoring, communication and data routing capabilities needed by rest of the ELASTIC SW architecture layers.
With respect to the three advanced mobility applications, ELASTIC has identified the set of big-data analytics methods that will form the extreme-scale analytics workflows, including deep neural networks for the detection of objects, analytics methods for GPS object location, trajectory predictors, collision detectors, etc. Moreover, the data input and the data-transfer requirements between the different analytics have been analysed.
With respect to the tramway infrastructure, new wifi antennas have been installed to connect the multiple sensors surrounding the tram stops, and edge computing nodes (based on GPUs) and a cloud facility have been setup at tram stops and the tramway depot respectively, all connected through an optical fiber network. Regarding the tram vehicles, they have been equipped with a camera, a radar and a LiDAR for object detection, and GPS, an IMU and a radar for autonomous position. Moreover, wifi and 4G antenna receivers have been also installed to transmit information captured and processed by the vehicles. Finally several edge computing platforms based on GPUs have been installed at vehicles as well.
With respect to the ELASTIC framework, the project has selected the set of SW components that will form the framework, prioritising those owned by the members of the ELASTIC consortium or offered as open-source components with a large community behind them, with the objective of reducing the time-to-market and maximize exploitation opportunities. The ELASTIC framework consists of four layers:
1. A Distributed Data Analytics Platform layer, for data accessibility across the compute continuum, covering aspects such as data-in-motion and data-at-rest analytics.
2. An Orchestration layer, responsible of deploying and distributing extreme-scale big-data analytics workflows across the compute continuum, with the objective of guaranteeing their operational requirements.
3. An Operation Requirement Analysis layer, that monitors the execution of analytics workflows and informs to the Orchestrator layer about the operational requirements, addressing time, energy, communication quality and security.
4. A Fog Computing Platform layer that implements the compute continuum, including monitoring, communication and data routing capabilities needed by rest of the ELASTIC SW architecture layers.
With respect to the three advanced mobility applications, ELASTIC has identified the set of big-data analytics methods that will form the extreme-scale analytics workflows, including deep neural networks for the detection of objects, analytics methods for GPS object location, trajectory predictors, collision detectors, etc. Moreover, the data input and the data-transfer requirements between the different analytics have been analysed.
With respect to the tramway infrastructure, new wifi antennas have been installed to connect the multiple sensors surrounding the tram stops, and edge computing nodes (based on GPUs) and a cloud facility have been setup at tram stops and the tramway depot respectively, all connected through an optical fiber network. Regarding the tram vehicles, they have been equipped with a camera, a radar and a LiDAR for object detection, and GPS, an IMU and a radar for autonomous position. Moreover, wifi and 4G antenna receivers have been also installed to transmit information captured and processed by the vehicles. Finally several edge computing platforms based on GPUs have been installed at vehicles as well.
The existing big-data systems are designed to provide either quick and reactive responses in real time (data-in-motion) with very limited performance capabilities, or thorough and more computationally intensive feedback based on the analysis of a vast amount of information (data-at-rest). Although these options have been tackled separately, combining the two into a unified big-data analytics workflow provides complementary capabilities, and can be especially beneficiary for extreme-scale scenarios in which data-sources are geographically distributed. Combining these two approaches requires to enrich the compute continuum concept: On the edge computing side, the availability of new parallel heterogeneous embedded processor architectures, may enable the processing of complex data-in-motion analytics with a reduced pressure on communication and cost within vehicles or in cabinets installed across the city. On the cloud computing side, data-in-motion analysis can be completed (and enriched) with data-at-rest analysis with more complex workloads.
Moreover, the fulfillment of operational requirements such as real-time, energy-efficiency, security and variability in communications have been tackled separately: addressing them in a holistic manner is of paramount importance to guarantee a correct functionality of the systems operating in the cities. ELASTIC is promoting a new set of technologies capable of efficiently distributing complex extreme-scale big-data analytics workflows across the compute continuum, from edge to cloud. Moreover, this distribution is guided based on the information provided by analytics tools that constantly monitor the execution of workflows to guarantee the fulfillment of requirements.
These distribution capabilities are aimed to be applied to the ELASTIC use-cases, which will generate several beneficial outcomes by a more efficient use of computing and communication resources, including a reduction on the number of yearly incidents, a reduction of preventive and standard maintenance costs, a reduction in the average waiting time for cars at tramway road crossings and an awareness increment of potential risks.
Moreover, the fulfillment of operational requirements such as real-time, energy-efficiency, security and variability in communications have been tackled separately: addressing them in a holistic manner is of paramount importance to guarantee a correct functionality of the systems operating in the cities. ELASTIC is promoting a new set of technologies capable of efficiently distributing complex extreme-scale big-data analytics workflows across the compute continuum, from edge to cloud. Moreover, this distribution is guided based on the information provided by analytics tools that constantly monitor the execution of workflows to guarantee the fulfillment of requirements.
These distribution capabilities are aimed to be applied to the ELASTIC use-cases, which will generate several beneficial outcomes by a more efficient use of computing and communication resources, including a reduction on the number of yearly incidents, a reduction of preventive and standard maintenance costs, a reduction in the average waiting time for cars at tramway road crossings and an awareness increment of potential risks.