Periodic Reporting for period 1 - MobiSpaces (New data spaces for green mobility)
Periodo di rendicontazione: 2022-09-01 al 2024-02-29
An immense number of measurements are collected by an ever-growing number of ubiquitous sensors and devices every second capturing a plethora of high-value data in real-time. This data is typically transferred to data centres for scalable processing and analysis, exploiting the recent advances in cloud computing, Big Data technologies and Artificial Intelligence (AI). It is timely that to exploit the huge data volumes produced in secure and trustworthy digital infrastructures, effective data governance solutions are required that allow data sharing, reuse and interoperability using standardized protocols across different organizations.
In contrast, the observed situation in Europe clearly indicates that companies and -established procedures that facilitate standardized data sharing and interoperability, leading to a waste of resources due to unnecessary and repetitive data-related operations. This is intensified in the domain of mobility and transportation, which constitutes one of the main pillars of the global economy, because mobility-related data:
(a) is (by nature) produced and collected in a completely decentralized manner,
(b) contain ssensitive information about the individual person or object tracked,
(c) can be analysed to discover hidden mobility patterns and extract invaluable insights and knowledge, and
(d) constitutes a complex data type, whose management and manipulation requires advanced processing algorithms.
Therefore, the challenge refers to developing an innovative, effective, robust, and green ecosystem for the entire lifecycle of mobility data (from data acquisition, processing and aggregation to data analysis and Machine Learning (ML)), aiming at secure and efficient AI-based data operations, further supporting interoperability, sharing, and re-use of high-quality and high-value data, and contributions towards the expansion of common European data spaces (e.g. mobility). Obviously, compliance, privacy, security, and trustworthiness are extremely important aspects of this novel ecosystem, meaning that the data management and analysis methods that will be introduced will make the solution highly performant and trustworthy, towards the development of a secure and dynamic data-agile economy, through in-situ processing on raw data and the selective transfer only of the most important data to a central location. Currently, data management in Internet of Things (IoT) and edge computing platforms rely primarily on data acquisition and collection, while intelligent processing and analysis is performed in the cloud, after having accumulated data from different devices. Although this data management paradigm has been successful so far, it is completely mobility-agnostic, whereas it is clear that mobility is a critical factor that affects data generation and raises privacy concerns. Yet, the current situation cannot exploit mobility patterns to optimize and decentralize data operations. In this context, the first challenge concerns the interplay between mobility analytics and data management decisions, which will be exploited to optimize resource consumption and enhance privacy. Furthermore, it is vital for the prosperity and growth of European companies and organizations to be able to share and exchange data in secure and easy way. Despite the existence of few data markets, this goal is still very far to be achieved. As such, the second challenge is to establish seamless data interoperability and sharing, following a common reference model, and provide the tools that facilitate declarative data manipulation regardless of the underlying storage system. To harness the merits of such a decentralized data ecosystem, distributed learning algorithms are necessary that operate on local data and derive local models, which can be communicated to a centralized location only when necessary. For certain applications that involve mobility, locally derived models may produce accurate descriptions (e.g. the movement of vessels near specific coasts and islands), but in other cases the derived models can be improved by exploiting global models built from remote data (i.e. the movement of vessels at open sea). Aiming at incremental model construction and privacy preservation, the third challenge is to produce actionable insights from distributed data by exploiting Federated Learning (FL) algorithms, aiming at building local models that describe the surrounding data more accurately, while incrementally pushing model updates to a centralized location that maintains the complete model, but not the complete data, in a way that respects data privacy.
In contrast, the observed situation in Europe clearly indicates that companies and -established procedures that facilitate standardized data sharing and interoperability, leading to a waste of resources due to unnecessary and repetitive data-related operations. This is intensified in the domain of mobility and transportation, which constitutes one of the main pillars of the global economy, because mobility-related data:
(a) is (by nature) produced and collected in a completely decentralized manner,
(b) contain ssensitive information about the individual person or object tracked,
(c) can be analysed to discover hidden mobility patterns and extract invaluable insights and knowledge, and
(d) constitutes a complex data type, whose management and manipulation requires advanced processing algorithms.
Therefore, the challenge refers to developing an innovative, effective, robust, and green ecosystem for the entire lifecycle of mobility data (from data acquisition, processing and aggregation to data analysis and Machine Learning (ML)), aiming at secure and efficient AI-based data operations, further supporting interoperability, sharing, and re-use of high-quality and high-value data, and contributions towards the expansion of common European data spaces (e.g. mobility). Obviously, compliance, privacy, security, and trustworthiness are extremely important aspects of this novel ecosystem, meaning that the data management and analysis methods that will be introduced will make the solution highly performant and trustworthy, towards the development of a secure and dynamic data-agile economy, through in-situ processing on raw data and the selective transfer only of the most important data to a central location. Currently, data management in Internet of Things (IoT) and edge computing platforms rely primarily on data acquisition and collection, while intelligent processing and analysis is performed in the cloud, after having accumulated data from different devices. Although this data management paradigm has been successful so far, it is completely mobility-agnostic, whereas it is clear that mobility is a critical factor that affects data generation and raises privacy concerns. Yet, the current situation cannot exploit mobility patterns to optimize and decentralize data operations. In this context, the first challenge concerns the interplay between mobility analytics and data management decisions, which will be exploited to optimize resource consumption and enhance privacy. Furthermore, it is vital for the prosperity and growth of European companies and organizations to be able to share and exchange data in secure and easy way. Despite the existence of few data markets, this goal is still very far to be achieved. As such, the second challenge is to establish seamless data interoperability and sharing, following a common reference model, and provide the tools that facilitate declarative data manipulation regardless of the underlying storage system. To harness the merits of such a decentralized data ecosystem, distributed learning algorithms are necessary that operate on local data and derive local models, which can be communicated to a centralized location only when necessary. For certain applications that involve mobility, locally derived models may produce accurate descriptions (e.g. the movement of vessels near specific coasts and islands), but in other cases the derived models can be improved by exploiting global models built from remote data (i.e. the movement of vessels at open sea). Aiming at incremental model construction and privacy preservation, the third challenge is to produce actionable insights from distributed data by exploiting Federated Learning (FL) algorithms, aiming at building local models that describe the surrounding data more accurately, while incrementally pushing model updates to a centralized location that maintains the complete model, but not the complete data, in a way that respects data privacy.
1: TRANSPARENT DATA GOVERNANCE PLATFORM FOR TRUSTWORTHY AND REUSABLE DATA OPERATIONS ON THE EDGE
2:An energy-efficient decentralized data management toolset for mobility data supporting green transition
requirements
3:Declarative querying of heterogeneous data sources, irrespective of their schema, format, and type,supporting" SQL on everything"
4:AI-based approaches that enhance data management solutions towards both increasing the speed of data throughput and data access, and decreasing the energy consumption and cost
5:Complete federated learning (FL) architecture for accurate detecting and forecasting over massively distributed datasets in a decentralised, explainable, and user-friendly visual manner
6:Challenge and showcase MobiSpaces innovations through various commercial use cases in different application sectors
7:Define clear exploitation strategy and goals, communicate, and disseminate the project outcomes towards early adoption of results
2:An energy-efficient decentralized data management toolset for mobility data supporting green transition
requirements
3:Declarative querying of heterogeneous data sources, irrespective of their schema, format, and type,supporting" SQL on everything"
4:AI-based approaches that enhance data management solutions towards both increasing the speed of data throughput and data access, and decreasing the energy consumption and cost
5:Complete federated learning (FL) architecture for accurate detecting and forecasting over massively distributed datasets in a decentralised, explainable, and user-friendly visual manner
6:Challenge and showcase MobiSpaces innovations through various commercial use cases in different application sectors
7:Define clear exploitation strategy and goals, communicate, and disseminate the project outcomes towards early adoption of results
1.MobiSpaces will address the case of secure data access and data sharing across organizations and companies, thereby enabling seamless exchange of data and information.
2.MobiSpaces will extend MobilityDB towards being an edge mobility database, enabling the large PostgreSQL community to build applications for the edge, using their usual ecosystem of tools.
3.MobiSpaces will extend declarative access to heterogeneous storage systems, focusing on the declarative management of spatiotemporal data, being partially supported by NoSQL stores and in a non-optimized way.
4.MobiSpaces will provide an AutoML approach based on meta-learning tailored for mobility data, learning from previous datasets and ML algorithms to build a model that automatically selects the best performing ML algorithm
5.MobiSpaces will provide a stable bedrock upon which multiple data processing and analytics methods can be built and tested, allowing the research community to both focus on useful work instead of repetitive infrastructure management, while employing pre-existing tools and services that work seamlessly with one another.
6.MobiSpaces will extend current technologies to address the edge/fog challenges and integrate the results in the Edge Analytics Suite.
7.MobiSpaces will develop methods with minimal communication overhead for the orchestration and coherent aggregation of distributed movement models with varying spatial and temporal overlap.
8.MobiSpaces will enhance trustworthiness, fairness and explainability, by enabling humans to reason about the outcomes of AI-based models.
9.MobiSpaces will focus on energy-efficient data operations, aiming at techniques for query execution and query optimization that are not only efficient in terms of processing time, but also consider the energy consumption by designing cost models that allow multi-objective optimization.
2.MobiSpaces will extend MobilityDB towards being an edge mobility database, enabling the large PostgreSQL community to build applications for the edge, using their usual ecosystem of tools.
3.MobiSpaces will extend declarative access to heterogeneous storage systems, focusing on the declarative management of spatiotemporal data, being partially supported by NoSQL stores and in a non-optimized way.
4.MobiSpaces will provide an AutoML approach based on meta-learning tailored for mobility data, learning from previous datasets and ML algorithms to build a model that automatically selects the best performing ML algorithm
5.MobiSpaces will provide a stable bedrock upon which multiple data processing and analytics methods can be built and tested, allowing the research community to both focus on useful work instead of repetitive infrastructure management, while employing pre-existing tools and services that work seamlessly with one another.
6.MobiSpaces will extend current technologies to address the edge/fog challenges and integrate the results in the Edge Analytics Suite.
7.MobiSpaces will develop methods with minimal communication overhead for the orchestration and coherent aggregation of distributed movement models with varying spatial and temporal overlap.
8.MobiSpaces will enhance trustworthiness, fairness and explainability, by enabling humans to reason about the outcomes of AI-based models.
9.MobiSpaces will focus on energy-efficient data operations, aiming at techniques for query execution and query optimization that are not only efficient in terms of processing time, but also consider the energy consumption by designing cost models that allow multi-objective optimization.