Periodic Reporting for period 1 - Waterverse (Water Data Management Ecosystem for Water Data Spaces)
Période du rapport: 2022-10-01 au 2024-03-31
The water sector is undergoing a fourth revolution, embracing digital transformation with IoT, AI, and big data to create a Smart Water Society. However, digitalization presents significant gaps and challenges that need to be addressed. Specifically, significant challenges in interoperability and data exchange due to its fragmentation should be faced. This fragmentation leads to tailored-made solutions, hindering the application of industry-wide standards and seamless data sharing. Disparate data exchange methods and inconsistent semantic definitions further impede interoperability, complicating the adoption of digital solutions. Additionally, comprehensive physical and cyber security measures are lacking. Social and environmental limitations include stakeholder unawareness of digital solutions' importance, inadequate traditional control measures, and regulations lacking energy-saving targets through ICT solutions. Waterverse aims to overcome Data Management limitations by creating an efficient ecosystem, the Water Data Management Ecosystem (WDME), ensuring accessibility, affordability, security, fairness, and user-friendliness. This enhances data usability, interoperability, resilience of water utilities, and market opportunities. The project’s specific objectives are the following:
SO1: Actively engage end-users and stakeholders to assess the main gaps and challenges the water sector must overcome to effectively be part of and contribute to quality European data spaces. Waterverse aims to improve data management and sharing in Europe's water sector by organising MSFs, encouraging stakeholder behavior change, and identifying gaps and opportunities in EU data governance frameworks that impede data sharing, sovereignty, and accessibility in the water sector.
SO2: Identify, extend, and integrate a set of data management tools to implement the Water Sector Data Spaces Ecosystem based on the FIWARE software Building Blocks. The goal is to create a customizable, user-friendly ecosystem supporting data management, ensuring compatibility with European data spaces, and addressing data interoperability, quality, fairness, provenance and anonymity.
SO3: Setup and demonstrate the WATERVERSE WDME in real environments with relevant and diverse case studies involving water sector stakeholders. The WDME will be implemented in two iterations across 6 EU pilot areas. Use cases for data sharing and management will be evaluated, and guidelines provided to help SMEs and start-ups leverage project results.
SO4: Ensure security and energy efficiency of the WATERVERSE WDME. WDME will integrate advanced cybersecurity, adopting CTI for enhanced intrusion detection, and incorporate energy-efficient machine learning for centralized and distributed data management.
SO5: Set clear and measurable indicators for assessing FAIRness of data in water-related data spaces. The project aims to define concepts for FAIR Digital Objects and Ecosystems, encompassing data, services, and entities. Additionally, it will provide guidelines, metrics, and tools to assess FAIR maturity in the water sector.
SO6: Ensure the viability and sustainability of the WATERVERSE WDME, as well as its replicability, scalability and business applicability. WDME will be bolstered through targeted communication, dissemination, outreach, and clustering activities. A detailed exploitation and business plan will promote its adaptation, scalability, and replicability within the water sector. Furthermore, policy and governance recommendations will support its widespread adoption and business applicability.
SO1: Actively engage end-users and stakeholders to assess the main gaps and challenges the water sector must overcome to effectively be part of and contribute to quality European data spaces. Waterverse aims to improve data management and sharing in Europe's water sector by organising MSFs, encouraging stakeholder behavior change, and identifying gaps and opportunities in EU data governance frameworks that impede data sharing, sovereignty, and accessibility in the water sector.
SO2: Identify, extend, and integrate a set of data management tools to implement the Water Sector Data Spaces Ecosystem based on the FIWARE software Building Blocks. The goal is to create a customizable, user-friendly ecosystem supporting data management, ensuring compatibility with European data spaces, and addressing data interoperability, quality, fairness, provenance and anonymity.
SO3: Setup and demonstrate the WATERVERSE WDME in real environments with relevant and diverse case studies involving water sector stakeholders. The WDME will be implemented in two iterations across 6 EU pilot areas. Use cases for data sharing and management will be evaluated, and guidelines provided to help SMEs and start-ups leverage project results.
SO4: Ensure security and energy efficiency of the WATERVERSE WDME. WDME will integrate advanced cybersecurity, adopting CTI for enhanced intrusion detection, and incorporate energy-efficient machine learning for centralized and distributed data management.
SO5: Set clear and measurable indicators for assessing FAIRness of data in water-related data spaces. The project aims to define concepts for FAIR Digital Objects and Ecosystems, encompassing data, services, and entities. Additionally, it will provide guidelines, metrics, and tools to assess FAIR maturity in the water sector.
SO6: Ensure the viability and sustainability of the WATERVERSE WDME, as well as its replicability, scalability and business applicability. WDME will be bolstered through targeted communication, dissemination, outreach, and clustering activities. A detailed exploitation and business plan will promote its adaptation, scalability, and replicability within the water sector. Furthermore, policy and governance recommendations will support its widespread adoption and business applicability.
The main achievements and the work performed in the reporting period can be categorised into the following pivotal technical and scientific aspects:
Need & Requirements: A stakeholder engagement framework was developed to guide multi-stakeholder forums (MSF). 6 initial MSFs identified 14 water sector challenges, including data sharing, interoperability, quality, cybersecurity, and energy efficiency. User requirements for all pilots were specified, refined and reported.
Design: A Data Quality Framework was developed to systematically manage, assess and ensure data quality throughout their lifecycle within an organization. The technical specifications for designing the WDME were based on user requirements, resulting in functional logical blocks distributed across these layers: Data Collection, Interoperability, Data Harmonization, Data Management, Data Processing, Identity Access, (Cyber) Security Access, and Application. Additionally, external components in the Data Sources and Third-Party layers enable interaction for data collection and distribution. Moreover, the Waterverse FAIR Ecosystem design, defining FAIR Digital Objects, was conducted alongside the Waterverse Fair Implementation Profile (FIP).
Development and Deployment: The development and release of the 1st version of the WDME was carried out by incorporating tools and services that are laid in the following layers: The Data Collection Layer gathers data from diverse sources using Data Connectors for non-IoT devices via APIs, IoT Agents for sensor communication, and an IoT Device Manager. A Synthetic IoT Data Generator creates artificial data for specific dataset attributes. The Interoperability and Data Harmonization Layer harmonizes data per Smart Data Models (SDM), water ontologies, and FAIR Assessments. The FIWARE Data Converters tool unifies data from 16 sources using 8 SDMs, transforming it into NGSI-LD format. Water Ontology and SDMs ensure semantic data representation. The Data Management Layer stores and distributes data using tools like Data Management System, Context Broker, Historical Context Data, Open Data Federator, and Blockchain-based Data Provenance tool for integrity and traceability. The Data Processing Layer processes data, adding value with tools for validation, reconciliation, anonymization, data balancing, and clusterability assessment. The Data Preparation Pipeline Editor integrates these tools. The Application Layer catalogs, publicizes, and displays applications, including the WDME Data Portal, Open Data Catalog, Data Exploration, and Dashboard Builder (Dashram) for data visualization and interactive dashboards.
WDME integrates the Identity Access and Management (IdM) tool for authorized access, the Cyber-Security solutions with CTI module and IDS agents, the CL-prediction model for water quality, the Flood prediction model, the Satellite-based data analysis tool for algae-map concentrations, the Helpdesk service aids users and gathers feedback and aspects of the Fairness Ecosystem.
Demonstration & Evaluation: The Pilot Evaluation Plan outlines 18 use cases across 6 sites. Online training materials and feedback questionnaires have been distributed. Three pilot evaluations are completed; three are in preparation.
Need & Requirements: A stakeholder engagement framework was developed to guide multi-stakeholder forums (MSF). 6 initial MSFs identified 14 water sector challenges, including data sharing, interoperability, quality, cybersecurity, and energy efficiency. User requirements for all pilots were specified, refined and reported.
Design: A Data Quality Framework was developed to systematically manage, assess and ensure data quality throughout their lifecycle within an organization. The technical specifications for designing the WDME were based on user requirements, resulting in functional logical blocks distributed across these layers: Data Collection, Interoperability, Data Harmonization, Data Management, Data Processing, Identity Access, (Cyber) Security Access, and Application. Additionally, external components in the Data Sources and Third-Party layers enable interaction for data collection and distribution. Moreover, the Waterverse FAIR Ecosystem design, defining FAIR Digital Objects, was conducted alongside the Waterverse Fair Implementation Profile (FIP).
Development and Deployment: The development and release of the 1st version of the WDME was carried out by incorporating tools and services that are laid in the following layers: The Data Collection Layer gathers data from diverse sources using Data Connectors for non-IoT devices via APIs, IoT Agents for sensor communication, and an IoT Device Manager. A Synthetic IoT Data Generator creates artificial data for specific dataset attributes. The Interoperability and Data Harmonization Layer harmonizes data per Smart Data Models (SDM), water ontologies, and FAIR Assessments. The FIWARE Data Converters tool unifies data from 16 sources using 8 SDMs, transforming it into NGSI-LD format. Water Ontology and SDMs ensure semantic data representation. The Data Management Layer stores and distributes data using tools like Data Management System, Context Broker, Historical Context Data, Open Data Federator, and Blockchain-based Data Provenance tool for integrity and traceability. The Data Processing Layer processes data, adding value with tools for validation, reconciliation, anonymization, data balancing, and clusterability assessment. The Data Preparation Pipeline Editor integrates these tools. The Application Layer catalogs, publicizes, and displays applications, including the WDME Data Portal, Open Data Catalog, Data Exploration, and Dashboard Builder (Dashram) for data visualization and interactive dashboards.
WDME integrates the Identity Access and Management (IdM) tool for authorized access, the Cyber-Security solutions with CTI module and IDS agents, the CL-prediction model for water quality, the Flood prediction model, the Satellite-based data analysis tool for algae-map concentrations, the Helpdesk service aids users and gathers feedback and aspects of the Fairness Ecosystem.
Demonstration & Evaluation: The Pilot Evaluation Plan outlines 18 use cases across 6 sites. Online training materials and feedback questionnaires have been distributed. Three pilot evaluations are completed; three are in preparation.
So far, there are no significant results or technological achievements beyond the state of the art with potential impact. The project, currently in its 18th month, is still in the development and evaluation phase. Hence no mature services and tools have been developed with the appropriate TRL level enables to commercialise them and reach the market.