Accelerated Ordering Service for Distributed Ledgers

Project Information

ACCORD

Grant agreement ID: 842956

DOI

10.3030/842956

Project terminated on 30 September 2020

EC signature date 23 April 2019

Start date 1 May 2019

End date 30 April 2021

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 172 932,48

EU contribution

€ 172 932,48

Coordinated by

FUNDACION IMDEA SOFTWARE
Spain

Periodic Reporting for period 1 - ACCORD (Accelerated Ordering Service for Distributed Ledgers)

Reporting period: 2019-05-01 to 2021-04-30

Summary of the context and overall objectives of the project

The ACCORD project focused on the question on how can Permissioned Blockchains (PB), an emerging technology, be adapted to the needs of Enterprise data-management workloads running in the cloud and datacenters. This project contributed to a potential disruption of how companies do business: by distributing the trust over a Blockchain there is no more need for trusted third parties and this will reduce cost and impediments in doing business. The results in ACCORD make it possible for PBs to efficiently utilize the network and computational resources, as well as to achieve low latency high throughput behavior; this improved performance has a significant positive impact on the cost of transacting across companies.

The starting insight of this project relies in the fact that even though it is commonly assumed that PB systems are inherently less performant than traditional Database Management Systems, by re-imagining their underlying processing, they can be made competitive and evaluated as an alternative to traditional DBMS systems. We have identified a significant challenge of achieving this goal: the high cost of the distributed communication steps of PB (the so called Ordering). Finding ways of reducing the cost of these operations was the focus of this project. The results and deliverables of the project can be used to accelerate existing, widely used PBs, such as Hyperledger Fabric, and, more importantly, they can be used provide a sound foundation for building the next generation of PBs well suited to the workloads and the execution environment typical for large Enterprises.

The methodology of the project was to combine benchmarking of widely used PBs with analytical modeling and bottleneck analysis, and, finally, implementation on modern server hardware that combines regular CPUs with programmable hardware, namely Field Programmable Gate Arrays (FPGAs). Thanks to this multi-pronged approach, the project has generated valuable insights at various levels (i.e. providing not only general recommendations for designing future Ordering services for blockchains but also hardware implementation lessons, etc.). We have released our results as Technical Reports and the source code can be found in Open Repositories hosted by the IMDEA Software Institute.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The technical work of the project has followed the initial planning closely and can be divided into two phases: 1) working on accelerating a Crash Fault Tolerant (CFT) ordering service and 2) accelerating a Byzantine Fault Tolerant (BFT) ordering service.

The first phase had as its starting point a proof of concept implementation of a re-imagined Permissioned Blockchain system that the Marie Curie Fellow built with collaborators from IBM Research before the beginning of the fellowship. The fellow has ported the main ideas behind the proof of concept to Hyperledger Fabric in a way that the resulting system, StreamChain, is fully functional and compatible with the unmodified Fabric version (the repository of this system can be found at https://gitlab.software.imdea.org/zistvan-public/streamchain). In order to benchmark this system and to identify its remaining bottlenecks, a benchmarking suite has been developed based on a real-life economics use-case for Blockchains (the repository for the benchmark is at https://gitlab.software.imdea.org/zistvan-public/streamchain-benchmarks). Based on the results of the benchmarking, an FPGA-based Ordering service has been developed that can be transparently plugged into StreamChain, and cuts end-to-end latencies by a factor of 2x (and by two orders of magnitude when compared to off-the-shelf Fabric). The implementation of the CFT Ordering service is a first of its kind and combines existing know-how of the Fellow (i.e. consensus algorithms with FPGAs) with new functionality that had to be developed (i.e. hashing incoming requests and chaining them together to form a blockchain).

The second phase of the technical work focused on BFT Ordering. Since currently most production-grade implementations of BFT algorithms are used in widely geo-distributed systems, as a first step, we undertook an experimental evaluation of a seminal BFT consensus algorithm implemented with modern datacenters and clouds in mind, without assuming very high latencies typical for geo-distributed settings. The experimental results motivate the need for incorporating specialized hardware in future BFT Ordering services and allow us to choose between the many possible approaches to acceleration (we summarize in a technical report why software will not be able to scale to emerging faster networks in the cloud: https://arxiv.org/abs/2007.12637). We have started working on the FPGA-based implementation of a BFT consensus algorithm when the project was ended. Nonetheless, we managed to gather initial insights on the resource requirements of this application that we will present at a workshop later in 2020.

The fellow has also carried out dissemination activities related to the use of Blockchains in data-management scenarios to the wider scientific public. These activities can be split into two categories: organizing events targeted at fellow systems-researchers (co-organizing the SPMA workshop at EuroSys'20 and the SERIAL workshop at Middleware'19 and '20) and events targeted at Computer Science students. The latter consisted of: First, the co-organization of a Winter School/Hackathon at IMDEA Software that focused on the use of Blockchain technology for fighting climate change, which incorporated an in-depth tutorial on the underlying Blockchain technology and touched on the core subjects of this project. Second, the organization of a tutorial at a leading FPGA conference, FPL'20, which introduced PhD students in the FPGA community to the opportunities that Blockchains present for this particular technology.

Finally, during the project the fellow has also successfully recruited and supervised three interns, one at the Bachelor's, on at the Master's level and one at the PhD level. Their internship topics were chosen in a way that complemented the efforts of the fellow in this project and allowed for in-depth exploration of various what-if questions that arose during implementation.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The project has improved upon the state of the art in three important ways. First, it provides a fully-functional implementation of a low-latency and high-throughput Permissioned Blockchain system (StreamChain) based on Hyperledger Fabric that, even though re-imagines the underlying processing, does not forfeit the security or privacy guarantees that Fabric offers. This system achieves orders of magnitudes lower latency than Fabric while matching its throughput, making it a realistic candidate to replace traditional database management systems in Enterprise use-cases.

Second, as part of the project, we built, to our knowledge, the first CFT Ordering service for Permissioned Blockchains that runs on FPGAs. This solution has been integrated in StreamChain and we showed that latencies can be further reduced to millisecond level.

Third, through the detailed experimental evaluation of the various cost factors and bottlenecks in BFT consensus on fast networks we were able to provide clear guidelines on when and how to include specialized hardware. While some of our results match the "qualitative" rules of thumb of the community, we believe there is value in providing a sound quantitative analysis.

High level overview of Permissioned Blockchains and the Focus of the ACCORD project