Future-Proof Data Systems in the Post-Moore Era

Descripción del proyecto

Eficiencia de datos para la informática del futuro

A medida que el volumen de datos se dispara, las infraestructuras informáticas actuales se enfrentan a una inmensa presión. El aumento de la demanda coincide con un cambio hacia la computación en nube, que exige que los centros de datos mejoren su eficiencia. Esos retos se ven agravados por las leyes decrecientes de la informática, en particular el fin de la escala de Dennard, que impulsa la especialización del «hardware». Estos factores influyen en el desarrollo de «software». En este sentido, el proyecto FDS, financiado por el Consejo Europeo de Investigación, creará una nueva infraestructura de «software». Propone abstracciones de «software» de sistema innovadoras para optimizar la ejecución de trabajos intensivos en datos en el «hardware» moderno. Al sustituir las estructuras monolíticas tradicionales por un marco flexible de optimización y compilación, el equipo de FDS pretende acortar la brecha entre la demanda de datos y la capacidad de cálculo, impulsando la eficiencia y la especialización en los centros de datos que consumen muchos recursos.

Objetivo

Three major trends put immense pressure on existing infrastructure. First, there is an increased demand for compute given the exponential growth of data volume we are collecting. Second, as most data- and compute-intensive systems move to the cloud, it is vital to make data-centers more efficient. Third, pressured by the waning laws of computing infrastructure (end of Dennard scaling and its impact on Moores' law) the hardware landscape has embarked on a major push towards specialization. Unsurprisingly, all of these have major implications on software development.

FDS addresses these challenges in a holistic fashion by establishing the foundations of a new software infrastructure. More specifically, I propose new system software abstractions that enable more efficient execution of data-intensive jobs on modern hardware; and new cost-models to support reasoning about the performance/resource efficiency trade-offs when scheduling the said workloads.
To enable development of future-proof data systems, I propose replacing the traditional monoliths of a logical-graph optimizer and execution engine, with a new flexible framework for optimization and compilation, and a runtime system that can exploit the features of modern hardware, but remains amenable to its evolving nature. Doing so requires innovation on many levels, and combines work from many layers of the system stack: from HW/SW co-design, to compilers, operating and runtime systems, databases and distributed systems.

It is a non-trivial undertaking, but it is a necessary one. And it will have big and lasting impact on academia, industry, and society. In fact, I strongly believe that the only way forward towards reducing the increasing gap between data- and compute-demand in resource-hungry data centers is a software-solution that boosts efficiency, embraces specialization, removes redundancy and optimizes for data movement. FDS will be the pioneering system leading the way.