Periodic Reporting for period 1 - Fediversity (Fediversity)
Período documentado: 2023-12-01 hasta 2025-05-31
The innovation and ambition of NGI Fediversity is to make a libre hosting platform that can in a decentralised manner scale to internet size, based on a modern packaging solution. A trustworthy and technically fully transparent way for organisations to run software. As such, the level of ambition of the project is significant: within this project we are essentially trying to establish a completely new software supply chain from developer to consumer, tailored towards empowering even SME’s to reliably provision high availability services based on state-of-the-art free and open source software.
In the last decade, in terms of IT procurement Europe has been leaning heavily on a strategy revolving on "cloud" or more specifically on Structural Computer Rental And Proprietary Services. This "SCRAPS" approach that denies the importants of inhouse expertise of IT has obviously backfired: while intended as a cost reduction and "simplification", this outsourcing strategy has resulted in lock-in, with overall cost that has risen dramatically. Even worse, our digital sovereignty has been severely damaged, and the lack of attention to capacity building in Europe is now reducing our chances of catching up. In the past two years, the developments in particular in social media have shown how harmful and problematic the current situation is. Something like NGI Fediversity is therefore even more necessary and urgent then it was when we got started, and we only wish that we had progressed faster than planned rather than slower.
There are other efforts in this space such as the €1.2 billion “Next-Generation Cloud Infrastructure and Services” IPCEI supported by the EC. However, NGI Fediversity is aiming at much more concrete end user services - the kind of user-centric services that are developed within the Next Generation Internet initiative, from standards-based social media applications to videoconferencing and productivity tools.
The project has four main objectives:
1: Hosting open social platforms
2: Offer a federated approach to social media and communication
3: Fully open-source
4: Achieve service portability
Service portability means that users will be able to (easily and fully) switch between different hosting providers and platforms without losing their data or online presence. We believe Fediversity will establish the feasibility of service portability as a critical consumer right in the modern internet landscape.
In the last decade, in terms of IT procurement Europe has been leaning heavily on a strategy revolving on "cloud" or more specifically on Structural Computer Rental And Proprietary Services. This "SCRAPS" approach that denies the importants of inhouse expertise of IT has obviously backfired: while intended as a cost reduction and "simplification", this outsourcing strategy has resulted in lock-in, with overall cost that has risen dramatically. Even worse, our digital sovereignty has been severely damaged, and the lack of attention to capacity building in Europe is now reducing our chances of catching up. In the past two years, the developments in particular in social media have shown how harmful and problematic the current situation is. Something like NGI Fediversity is therefore even more necessary and urgent then it was when we got started, and we only wish that we had progressed faster than planned rather than slower.
There are other efforts in this space such as the €1.2 billion “Next-Generation Cloud Infrastructure and Services” IPCEI supported by the EC. However, NGI Fediversity is aiming at much more concrete end user services - the kind of user-centric services that are developed within the Next Generation Internet initiative, from standards-based social media applications to videoconferencing and productivity tools.
The project has four main objectives:
1: Hosting open social platforms
2: Offer a federated approach to social media and communication
3: Fully open-source
4: Achieve service portability
Service portability means that users will be able to (easily and fully) switch between different hosting providers and platforms without losing their data or online presence. We believe Fediversity will establish the feasibility of service portability as a critical consumer right in the modern internet landscape.
OID created a physical hosting environment for Fediversity, consisting of (used) hardware spread over two data centers. OID further invented a basic ‘nixops’ deployment system, a complete rewrite of a declarative way to deploy IT infrastructure.
The second step was to start to ‘nixify’ this setup. In the first half of 2024, Tweag contributed tested customised NixOS configurations of selected social media applications that would scale to the targeted deployment environment, and helped project partners assessing issues and risks with upstream and support tools. In the second half of 2024, Tweag integrated these NixOS services configurations into a sample deployment that demonstrated viability of the overall approach. In the first half of 2025, Tweag contributed an end-to-end prototype for configuring and deploying the social media applications. Tweag also built a complete integration test suite that ensures the existing demo workflows work as advertised, and allows iterating on the prototype without accidental regressions.
OID additionally collaborated with Tweag to on a demo environment showcasing user-friendly deployment of reproducible modules for Fediverse service Mastodon, Peertube and Pixelfed. The front-end we prepared for this may serve as a simple reference implementation of Fediversity for further integration by hosting providers, which at last one hoster (ProcoliX) has committed to picking up as part of their open-source panel for hosting providers.
OID has designed a step-by-step process covering service portability, extending prior art in the Nix ecosystem. This approach covers loading of data from existing deployments, updating DNS settings, and running application-specific migration scripts, which we plan to standardise and upstream. Details on this design may be found at our issue tracker, as well as in related developer meeting notes.
Tweag built prototypes and led technical discussions to explore options for enabling portable services based on declarative NixOS configurations, experimenting with the selected social media applications. A general solution to application state management from a packaging perspective is still a matter of active research, and there are only rudimentary, isolated approaches to service portability in the Nix ecosystem.
Tweag also contributed a NixOS design pattern and accompanying prototype that enables service portability across deployment infrastructures at the configuration level], and currently leads technical discussion to provide an application-agnostic interface for service portability at the data level for migrating application state. The envisioned portability architecture builds on and extends the vast collection of existing software components from the Nix ecosystem, and will allow uniformly using the purely functional Nix programming language to describe UIs and APIs for integrating with other services.
The source code related to the project is publicly visible from:
https://git.fediversity.eu(se abrirá en una nueva ventana)
So far six projects have been allocated subgrants through the open calls from the Fediversity fund. The results of these can be publicly tracked online:
https://nlnet.nl/thema/NGIFediversityFund.html(se abrirá en una nueva ventana)
The second step was to start to ‘nixify’ this setup. In the first half of 2024, Tweag contributed tested customised NixOS configurations of selected social media applications that would scale to the targeted deployment environment, and helped project partners assessing issues and risks with upstream and support tools. In the second half of 2024, Tweag integrated these NixOS services configurations into a sample deployment that demonstrated viability of the overall approach. In the first half of 2025, Tweag contributed an end-to-end prototype for configuring and deploying the social media applications. Tweag also built a complete integration test suite that ensures the existing demo workflows work as advertised, and allows iterating on the prototype without accidental regressions.
OID additionally collaborated with Tweag to on a demo environment showcasing user-friendly deployment of reproducible modules for Fediverse service Mastodon, Peertube and Pixelfed. The front-end we prepared for this may serve as a simple reference implementation of Fediversity for further integration by hosting providers, which at last one hoster (ProcoliX) has committed to picking up as part of their open-source panel for hosting providers.
OID has designed a step-by-step process covering service portability, extending prior art in the Nix ecosystem. This approach covers loading of data from existing deployments, updating DNS settings, and running application-specific migration scripts, which we plan to standardise and upstream. Details on this design may be found at our issue tracker, as well as in related developer meeting notes.
Tweag built prototypes and led technical discussions to explore options for enabling portable services based on declarative NixOS configurations, experimenting with the selected social media applications. A general solution to application state management from a packaging perspective is still a matter of active research, and there are only rudimentary, isolated approaches to service portability in the Nix ecosystem.
Tweag also contributed a NixOS design pattern and accompanying prototype that enables service portability across deployment infrastructures at the configuration level], and currently leads technical discussion to provide an application-agnostic interface for service portability at the data level for migrating application state. The envisioned portability architecture builds on and extends the vast collection of existing software components from the Nix ecosystem, and will allow uniformly using the purely functional Nix programming language to describe UIs and APIs for integrating with other services.
The source code related to the project is publicly visible from:
https://git.fediversity.eu(se abrirá en una nueva ventana)
So far six projects have been allocated subgrants through the open calls from the Fediversity fund. The results of these can be publicly tracked online:
https://nlnet.nl/thema/NGIFediversityFund.html(se abrirá en una nueva ventana)
NGI Fediversity is an effort to amplify the technological development of digital public goods and digital commons, by significantly lowering the cost and complexity of deployment and maintenance. It cannot and should not be seen apart from Nix/Nixpkgs. Outside of the mobile app stores from Apple and Google, Nix/Nixpkgs is globally the largest repository of software in the world, and also the mainstream repository with the most up-to-date packages. Nixpkgs also leads in reproducibility and transparency, delivering the state of the art in declarative packaging - reproducible, immutable and capable of atomic updates and rollbacks. Fediversity is extending this from software to services.