Community Research and Development Information Service - CORDIS

Optimised distributed storage systems for more effective data storage

As information increases steadily, it is vital to ensure that digitally stored data withstands device failures, and that it is readily available, reliable and consistent. An EU initiative sought innovative solutions for effective data storage.
Optimised distributed storage systems for more effective data storage
The EU-funded ATOMICDFS (Seeking efficient atomic implementations of distributed data storage) project studied the existence of efficient, strongly consistent distributed file systems in asynchronous environments where fail-prone processes communicate via message exchanges.

Project partners developed efficient algorithms for atomic file objects. First they identified the weaknesses of current atomic consistency algorithms to support such objects and mitigated those shortcomings by defining a new property for concurrent objects called coverability. This precisely specifies the evolution tree of a versioned object (like file objects), leading to the creation of a single path from the initial version of the object to any written version of that object.

Researchers then focused on improving the computation and communication costs in atomic file object algorithms. In total, five new algorithms were introduced that reduce computation demands while preserving consistency. To reduce communication demands, the ATOMICDFS team considered three different fragmentation techniques. The first involved simple division into blocks, the second division into encoded blocks using erasure coding approaches, while the third used operation logs (journals).

Lastly, researchers implemented some of the proposed algorithms in simulation and real-life environments. Six algorithms were developed for the former and two for the latter. Experimental results showed that the proposed approaches outperformed previous solutions and demonstrated the practicality of consistent storage systems. The team also developed a preliminary working prototype of a journaling distributed file system. The prototype was based on a distributed journaling system architecture, which was also developed during the project.

ATOMICDFS contributed towards the production of practical file systems that provide precise, provable, consistency guarantees. Distributed storage systems that are highly reliable, consistent, collaborative, functional and global could soon become a reality. Thanks to the project, Europe is now at the forefront of collaborative applications and one step closer to a future global computing platform.

Related information

Keywords

Distributed storage systems, data storage, ATOMICDFS, atomic implementations, distributed file systems
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top