Periodic Reporting for period 1 - TaRDIS (Trustworthy and Resilient Decentralised Intelligence for Edge Systems)
Reporting period: 2023-01-01 to 2024-06-30
Developing and managing distributed systems is an intricate task that demands expertise across various domains, such as programming languages, hardware, networking, and data management. This complexity is further amplified when dealing with swarm systems, which are highly dynamic and necessitate decentralised solutions to handle their scale and inherent heterogeneity. Configuring a swarm of satellites, coordinating devices in a smart city while ensuring privacy-aware machine learning, or managing autonomous machines in a factory are challenges that exceed the capabilities of the typical software engineer. Developing dependable, secure, and efficient systems in these scenarios demands developers to consider multiple aspects of the system.
TaRDIS addresses these challenges by offering innovative programming models and tools designed specifically for heterogeneous swarms—intelligent, dynamic, decentralised, and heterogeneous systems. Developers need the ability to reason about the overall behaviour of a system where decisions are made on devices or in edge servers while disconnected from other parts of the network. To tackle this challenge, TaRDIS constructs a toolbox of programming tools, libraries, middleware, and theories.
The project is driven by industry leaders' four distinct use cases— from satellite swarms and decentralised energy marketplaces to privacy-preserving ML applications and smart factory controls.
TaRDIS development environment aims to assist developers in building correct and performant systems through advanced software verification techniques. These techniques automatically analyse and validate interactions between components of a distributed system. Our programming environment is supported by a toolset that offers fundamental services and abstractions for constructing and running swarm applications.
Our vision is made possible by the unique expertise of the TaRDIS consortium, which brings together top experts in programming languages, automatic verification, behavioural types, model checking, privacy and security, distributed systems, decentralised protocols, distributed data management solutions, distributed and federated machine learning, and reinforcement learning mechanisms. This project bridges two previously unconnected research communities - behavioural types and machine learning - while involving industry leaders in decentralised systems.
The TaRDIS system represents a significant advancement in the development of distributed and decentralised systems, particularly in managing complex, heterogeneous swarms. By integrating innovative programming models, advanced verification tools, and cutting-edge technologies, TaRDIS addresses the current challenges in the field and lays the groundwork for future developments.
TaRDIS addresses these challenges by offering innovative programming models and tools designed specifically for heterogeneous swarms—intelligent, dynamic, decentralised, and heterogeneous systems. Developers need the ability to reason about the overall behaviour of a system where decisions are made on devices or in edge servers while disconnected from other parts of the network. To tackle this challenge, TaRDIS constructs a toolbox of programming tools, libraries, middleware, and theories.
The project is driven by industry leaders' four distinct use cases— from satellite swarms and decentralised energy marketplaces to privacy-preserving ML applications and smart factory controls.
TaRDIS development environment aims to assist developers in building correct and performant systems through advanced software verification techniques. These techniques automatically analyse and validate interactions between components of a distributed system. Our programming environment is supported by a toolset that offers fundamental services and abstractions for constructing and running swarm applications.
Our vision is made possible by the unique expertise of the TaRDIS consortium, which brings together top experts in programming languages, automatic verification, behavioural types, model checking, privacy and security, distributed systems, decentralised protocols, distributed data management solutions, distributed and federated machine learning, and reinforcement learning mechanisms. This project bridges two previously unconnected research communities - behavioural types and machine learning - while involving industry leaders in decentralised systems.
The TaRDIS system represents a significant advancement in the development of distributed and decentralised systems, particularly in managing complex, heterogeneous swarms. By integrating innovative programming models, advanced verification tools, and cutting-edge technologies, TaRDIS addresses the current challenges in the field and lays the groundwork for future developments.
During the first 18 months of the TaRDIS project, significant progress was made across technical work packages, laying a solid foundation for achieving the project's goals.
Requirements and Architecture focused on defining the foundational requirements and modular architecture for the TaRDIS platform, incorporating detailed analysis of programming trends and end-user needs to support the integration of diverse components in decentralized swarm systems.
Programming Abstractions for the Cloud-Edge Continuum advanced event-driven programming models and APIs, and released the first iteration of the TaRDIS Integrated Development Environment (IDE) after evaluating various platforms.
Programming Logic and Analysis Framework enhanced swarm systems' security, reliability, and data integrity by developing advanced communication behaviour analysis techniques, including extensions to multiparty session types (MPST) and frameworks for secure software development that verify communication channels and information flow.
Decentralized Learning and Inference Framework enabled decentralized AI/ML operations within swarm systems, introducing federated learning frameworks such as PTB-FLA, MPT-FLA, and Fedra, along with lightweight machine learning methods like early exit inference, knowledge distillation, and pruning to optimize ML models in resource-constrained environments.
Data Management and Distribution Primitives delivered solutions for decentralized data management, including Babel-Swarm for self-configuration, PotionDB for geo-replicated storage with strong eventual consistency, Nimbus for decentralized storage, and Arboreal for scalable data management across cloud-edge continuums, ensuring data integrity and efficient distribution.
Use Case Implementation validated the TaRDIS platform by deploying baseline scenarios across all use cases. This provided insights into the challenges of deploying intelligent heterogeneous swarm systems and guided the refinement of the TaRDIS toolbox for real-world applications.
Requirements and Architecture focused on defining the foundational requirements and modular architecture for the TaRDIS platform, incorporating detailed analysis of programming trends and end-user needs to support the integration of diverse components in decentralized swarm systems.
Programming Abstractions for the Cloud-Edge Continuum advanced event-driven programming models and APIs, and released the first iteration of the TaRDIS Integrated Development Environment (IDE) after evaluating various platforms.
Programming Logic and Analysis Framework enhanced swarm systems' security, reliability, and data integrity by developing advanced communication behaviour analysis techniques, including extensions to multiparty session types (MPST) and frameworks for secure software development that verify communication channels and information flow.
Decentralized Learning and Inference Framework enabled decentralized AI/ML operations within swarm systems, introducing federated learning frameworks such as PTB-FLA, MPT-FLA, and Fedra, along with lightweight machine learning methods like early exit inference, knowledge distillation, and pruning to optimize ML models in resource-constrained environments.
Data Management and Distribution Primitives delivered solutions for decentralized data management, including Babel-Swarm for self-configuration, PotionDB for geo-replicated storage with strong eventual consistency, Nimbus for decentralized storage, and Arboreal for scalable data management across cloud-edge continuums, ensuring data integrity and efficient distribution.
Use Case Implementation validated the TaRDIS platform by deploying baseline scenarios across all use cases. This provided insights into the challenges of deploying intelligent heterogeneous swarm systems and guided the refinement of the TaRDIS toolbox for real-world applications.
1. Improved Actyx Middleware: This is a decentralized middleware for manufacturing that ensures resilience by supporting fully decentralized operations with strong eventual consistency, allowing critical processes to continue even without central infrastructure.
2. Babel Framework and Ecosystem: Simplifies distributed protocol development by abstracting complexities like message passing and concurrency. It includes Babel-Swarm for self-configuration and security, and Babel-Android, extending these functionalities to Android.
3. Privacy-Preserving and Efficient Decentralized Training: Enhances FLaaS with privacy techniques like differential privacy, handling heterogeneous data, and enabling decentralized AI, making it suitable for sectors prioritizing data privacy and decentralization.
4. Decentralized Data Management Framework: Offers distributed storage solutions like PotionDB for scalable performance and Arboreal for dynamic replication extending cloud storage to the edge, ensuring data consistency and security in decentralized systems.
5. Decentralized Federated Learning Platform: This platform supports privacy-preserving ML across distributed nodes without centralizing data, making it ideal for smart homes and IoT. It includes advanced techniques like Split Learning and Reinforcement Learning, optimizing AI deployment in resource-constrained environments.
6.Swarm Verification Framework: Provides tools for verifying the behaviour of decentralized swarms, focusing on safety, liveness, and fault tolerance, crucial for mission-critical applications in aerospace, defence, and autonomous systems.
7. TaRDIS Integrated Development Environment: A comprehensive IDE for developing, deploying, and managing decentralized swarm systems, integrating tools for decentralized intelligence, formal verification, and secure data management.
8. Nimbus: A fully decentralized storage solution using CRDTs and anti-entropy protocols for eventual consistency, built on the Babel framework, providing a resilient and scalable option for high-availability applications without dedicated infrastructure.
2. Babel Framework and Ecosystem: Simplifies distributed protocol development by abstracting complexities like message passing and concurrency. It includes Babel-Swarm for self-configuration and security, and Babel-Android, extending these functionalities to Android.
3. Privacy-Preserving and Efficient Decentralized Training: Enhances FLaaS with privacy techniques like differential privacy, handling heterogeneous data, and enabling decentralized AI, making it suitable for sectors prioritizing data privacy and decentralization.
4. Decentralized Data Management Framework: Offers distributed storage solutions like PotionDB for scalable performance and Arboreal for dynamic replication extending cloud storage to the edge, ensuring data consistency and security in decentralized systems.
5. Decentralized Federated Learning Platform: This platform supports privacy-preserving ML across distributed nodes without centralizing data, making it ideal for smart homes and IoT. It includes advanced techniques like Split Learning and Reinforcement Learning, optimizing AI deployment in resource-constrained environments.
6.Swarm Verification Framework: Provides tools for verifying the behaviour of decentralized swarms, focusing on safety, liveness, and fault tolerance, crucial for mission-critical applications in aerospace, defence, and autonomous systems.
7. TaRDIS Integrated Development Environment: A comprehensive IDE for developing, deploying, and managing decentralized swarm systems, integrating tools for decentralized intelligence, formal verification, and secure data management.
8. Nimbus: A fully decentralized storage solution using CRDTs and anti-entropy protocols for eventual consistency, built on the Babel framework, providing a resilient and scalable option for high-availability applications without dedicated infrastructure.