Periodic Reporting for period 3 - NEAT (A New, Evolutive API and Transport-Layer Architecture for the Internet)
Reporting period: 2017-03-01 to 2018-04-30
The Internet’s transport-layer architecture is, in some aspects, still dependent on developments dating from the 1980s. Modern online applications need a more flexible transport architecture to rest upon, given the substantial changes that the usage of the Internet has undergone in the past decades.
Smartphones and tablets can run a wide variety of Internet applications, connected via networks such as WiFi and LTE; laptops and desktop PCs add wired connections to the above mix of wireless technologies. Each application has its own requirements for how it wishes its data to be carried over the Internet. The communication needs of a distributed backup application differ from those of web browsing or remote conferencing. This diversity of needs and technologies often requires application developers to consider the diverse networks their application may run on, and how to tune transport and network parameters for those networks – not an easy task. The problem is compounded by the fact that the current Internet transport architecture – as embodied by common Application Programming Interfaces (APIs) – exposes communication protocols instead of services to applications, and ties applications to a priori choices.
NEAT offers a new approach that enables developers to specify an application’s requirements in terms of rate, delay, reliability, etc. This allows NEAT to choose, or help choose, the best communication service. By separating services offered to applications from the underlying protocols and OS features, NEAT can automatically and transparently choose the best transport options available. As network technologies and protocols keep on evolving, NEAT-enabled applications will immediately be able to take advantage of new functions to reduce web page download times, make teleconferences more responsive or reduce the cost of software downloads.
The NEAT Transport System is designed to:
• Offer an enhanced API between applications and the transport layer, so that it exposes transport services to applications.
• Provide transparent support for discovering and using the most suitable protocols and functions that are available end-to-end, and for interacting with the network for a better application experience.
• Enable user-space protocol stacks.
This allows the NEAT System to: (a) decouple applications from a priori choices of underlying protocols and technologies; (b) support incremental evolution and deployment of new transports; (c) relieve application developers from the burden of implementing common mechanisms.
NEAT therefore reduces significantly the implementation effort needed to develop an efficient Internet application, making it easier for European SMEs to compete with the “Internet Big Five”.
NEAT has paved the way for deploying novel transport protocols and mechanisms, and for providing better support of applications’ needs in networking stacks and APIs. Also, NEAT has been one of the key drivers behind the ongoing standardisation of a modern Internet transport API. By designing a novel transport-layer architecture, then developing and demonstrating a proof-of-concept prototype, NEAT has demonstrated how the Internet’s transport layer can be allowed to evolve.
Smartphones and tablets can run a wide variety of Internet applications, connected via networks such as WiFi and LTE; laptops and desktop PCs add wired connections to the above mix of wireless technologies. Each application has its own requirements for how it wishes its data to be carried over the Internet. The communication needs of a distributed backup application differ from those of web browsing or remote conferencing. This diversity of needs and technologies often requires application developers to consider the diverse networks their application may run on, and how to tune transport and network parameters for those networks – not an easy task. The problem is compounded by the fact that the current Internet transport architecture – as embodied by common Application Programming Interfaces (APIs) – exposes communication protocols instead of services to applications, and ties applications to a priori choices.
NEAT offers a new approach that enables developers to specify an application’s requirements in terms of rate, delay, reliability, etc. This allows NEAT to choose, or help choose, the best communication service. By separating services offered to applications from the underlying protocols and OS features, NEAT can automatically and transparently choose the best transport options available. As network technologies and protocols keep on evolving, NEAT-enabled applications will immediately be able to take advantage of new functions to reduce web page download times, make teleconferences more responsive or reduce the cost of software downloads.
The NEAT Transport System is designed to:
• Offer an enhanced API between applications and the transport layer, so that it exposes transport services to applications.
• Provide transparent support for discovering and using the most suitable protocols and functions that are available end-to-end, and for interacting with the network for a better application experience.
• Enable user-space protocol stacks.
This allows the NEAT System to: (a) decouple applications from a priori choices of underlying protocols and technologies; (b) support incremental evolution and deployment of new transports; (c) relieve application developers from the burden of implementing common mechanisms.
NEAT therefore reduces significantly the implementation effort needed to develop an efficient Internet application, making it easier for European SMEs to compete with the “Internet Big Five”.
NEAT has paved the way for deploying novel transport protocols and mechanisms, and for providing better support of applications’ needs in networking stacks and APIs. Also, NEAT has been one of the key drivers behind the ongoing standardisation of a modern Internet transport API. By designing a novel transport-layer architecture, then developing and demonstrating a proof-of-concept prototype, NEAT has demonstrated how the Internet’s transport layer can be allowed to evolve.
In the first year of the NEAT project, the main focus was on designing an extensible Internet transport-layer architecture. The design considered both: (a) a set of generic requirements a transport system must satisfy, to be flexible and able to evolve; (b) specific applications related to the industry partners’ use cases. These applications cover a wide range of scenarios where the NEAT approach may offer benefits, from web browsing on mobile terminals to distributed storage solutions.
In the second year, a strong effort was put on implementing the core NEAT Transport System. Core components include generic functionalities that are needed regardless of the use cases. A first prototype of these components was made as an open-source, user-space software library that implements the NEAT architecture and API. In parallel, extended components were developed, with the goal of providing either performance optimisations, better support of the project’s industrial use cases, or a simpler migration path for legacy applications.
During the last year, extensive tests were performed in relation with the different industry cases, to validate the NEAT approach and highlight the benefits it brings to those use cases.
To facilitate the broad adoption of NEAT outcomes, throughout the lifetime of the project the consortium has promoted its results at the Internet Engineering Task Force (IETF), in relevant working groups such as the Transport Services (TAPS) and the Transport Area (TSVWG) Working Groups. NEAT has been one of the leading players behind the TAPS effort. The NEAT prototype library is so far the only open-source implementation of a TAPS-compatible system. In close cooperation with industry players and academic groups, NEAT has co-authored five IETF Request for Comments (RFC) documents, with several more to be published in the next months.
NEAT has also disseminated its main results in well-known scientific venues, in FOSDEM (the biggest open-source event in the world with over 5,000 attendees) and in large trade shows such as Mobile World Congress.
NEAT contributions are already making their way into project partners’ products, and through TAPS they are being implemented in at least one major operating system.
In the second year, a strong effort was put on implementing the core NEAT Transport System. Core components include generic functionalities that are needed regardless of the use cases. A first prototype of these components was made as an open-source, user-space software library that implements the NEAT architecture and API. In parallel, extended components were developed, with the goal of providing either performance optimisations, better support of the project’s industrial use cases, or a simpler migration path for legacy applications.
During the last year, extensive tests were performed in relation with the different industry cases, to validate the NEAT approach and highlight the benefits it brings to those use cases.
To facilitate the broad adoption of NEAT outcomes, throughout the lifetime of the project the consortium has promoted its results at the Internet Engineering Task Force (IETF), in relevant working groups such as the Transport Services (TAPS) and the Transport Area (TSVWG) Working Groups. NEAT has been one of the leading players behind the TAPS effort. The NEAT prototype library is so far the only open-source implementation of a TAPS-compatible system. In close cooperation with industry players and academic groups, NEAT has co-authored five IETF Request for Comments (RFC) documents, with several more to be published in the next months.
NEAT has also disseminated its main results in well-known scientific venues, in FOSDEM (the biggest open-source event in the world with over 5,000 attendees) and in large trade shows such as Mobile World Congress.
NEAT contributions are already making their way into project partners’ products, and through TAPS they are being implemented in at least one major operating system.
Because a large majority of applications is statically bound to a single transport protocol (TCP), the whole Internet infrastructure has evolved to better support TCP. Thus, applications face difficulties when they try to use a new protocol; this problem is now commonly referred to as “network ossification”.
There exist a range of point or partway solutions that try to solve problems related to the ossification of the network infrastructure or of typical communication APIs, but many of these solutions have had little or no impact on the Internet. One important reason for this is that “de-ossifying” the Internet transport layer to re-enable its evolution is a multi-dimensional problem. This requires the enhancement of multiple components of the end-to-end communication, and effective integration of different techniques. To our knowledge, no other concrete, cohesive system besides NEAT has been designed to date that provides an overall solution.
A truly evolvable and flexible Internet transport architecture requires a comprehensive transport layer framework that can facilitate integration and cooperation of transport-layer solutions in an application-independent and flexible way. NEAT provides such a transport framework.
Building an in-house transport system can yield the best performance for applications that have special network requirements, and this is the path taken by a few industry giants. However, this path cannot be afforded by most software developers and is fraught with many technical hurdles. By designing, demonstrating and standardising a transport system with a large set of functions for efficient communication, NEAT contributes to a more level playing field for innovative small companies, from which novel Internet applications often come. NEAT will make it much easier for European software developers to write and deploy applications that efficiently communicate across the Internet, and will enable innovations across the Internet protocol stack.
There exist a range of point or partway solutions that try to solve problems related to the ossification of the network infrastructure or of typical communication APIs, but many of these solutions have had little or no impact on the Internet. One important reason for this is that “de-ossifying” the Internet transport layer to re-enable its evolution is a multi-dimensional problem. This requires the enhancement of multiple components of the end-to-end communication, and effective integration of different techniques. To our knowledge, no other concrete, cohesive system besides NEAT has been designed to date that provides an overall solution.
A truly evolvable and flexible Internet transport architecture requires a comprehensive transport layer framework that can facilitate integration and cooperation of transport-layer solutions in an application-independent and flexible way. NEAT provides such a transport framework.
Building an in-house transport system can yield the best performance for applications that have special network requirements, and this is the path taken by a few industry giants. However, this path cannot be afforded by most software developers and is fraught with many technical hurdles. By designing, demonstrating and standardising a transport system with a large set of functions for efficient communication, NEAT contributes to a more level playing field for innovative small companies, from which novel Internet applications often come. NEAT will make it much easier for European software developers to write and deploy applications that efficiently communicate across the Internet, and will enable innovations across the Internet protocol stack.