Community Research and Development Information Service - CORDIS


TRIANGLE Report Summary

Project ID: 688712
Funded under: H2020-EU.2.1.1.

Periodic Reporting for period 1 - TRIANGLE (5G Applications and Devices Benchmarking)

Reporting period: 2016-01-01 to 2016-12-31

Summary of the context and overall objectives of the project

The future 5G system is getting day by day more defined, thanks also to the recent decision of the 3rd Generation Partnership Program (3GPP) of narrowing down the scope of the standardization process in order to meet the tight deadline of 2018. This very sharp cut has then been stressing how the initial introduction of new services will be happening in the so-called 4.5/4.9G system (LTE-A Pro Evolution, LTE-M, LTE NB-IoT).
On the other side of the coin, the mobile App and services market is experiencing a constant growth in both size and revenue, even though a common ground for verifying the App performances in a standardized fashion is still lacking. Only traditional software testing, which considers the network a reliable data pipe, is performed on the code itself. While it has been an acceptable void so far, the requirements foreseen for the 5G Apps and services will make it impossible to develop software without considering its performance in varying network conditions.
This gap brings along the lack of a homogeneous testing and certification framework that encompasses a wide range of use cases, network scenarios, and Key Performance Indicators (KPIs) that are able to cover not only the different traditional telco needs, but also the End-to-End (E2E) side, given the incredibly large variety of available and foreseen Apps and services. A new and more holistic set of tools, covering also the mobile device behavior at Operating System level and the Evolved Packet Core (EPC) network would be need to support this work.
The TRIANGLE project is funded under the FIRE/FIRE+ objective, it started in January 2016, and has a duration of three years. The primary objective of the project is to promote the testing and benchmarking of mobile applications and devices in Europe as the industry moves towards 5G and to provide a pathway towards certification in order to support qualified apps and mobile developments using FIRE testbeds as testing framework.
The definition of new use cases fitting the specific mobile applications and services market is within the scope of TRIANGLE. This also opens the investigation of new and specific KPIs and metrics focused on the testing and benchmarking of detailed characteristics of mobile app, services, and devices, including the IoT ones. The final goal of these studies is the definition of a standardized testing and certification scheme that can be the forefront for creating a homogenized methodology for 5G testing and benchmarking.
A new technology platform is also needed in order to support the new testing methodology. By building on top of the FIRE community experience, the project is currently extending and enhancing the PerformNetworks testbed available at the University of Màlaga to produce the TRIANGLE testbed.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

An initial step of the project was to identify the proper 5G uses cases and scenarios to use for setting up the stage for the testing and certification framework. The initial idea was to start from what was defined by NGMN or by other H2020 projects, as nicely summarized by the 5G-PPP Association in its Issue Paper on use cases and performance evaluation. In that document the applications and the network conditions were tightly coupled, limiting the possibilities and the scope of investigation. For this reason, the TRIANGLE project has tried to de-couple the two aspects into two levels: Uses Cases and Network Scenarios.
A Use Case identifies the type of application and service that is using the 5G network. This clear cut between the two sides allows a sharper definition of the individual application performance needs and it improves the identification of the service’s characteristics features. The identified Use Cases represent the expected services that will most likely be consumed either on mobile devices such as smartphones, tablets, or as an IoT service.
The Use Cases have been abstracted in order to ensure a homogeneous certification scheme, and in TRIANGLE´s view, they represent the vast majority of the data that are expected to travel through the 4G and beyond networks, including what could be specific 5G applications not sustainable in today’s 4G networks. The other essential component for the definition of the testing methodology is the identification of when and where the services could be consumed. These factors give a clear indication on the radio environment and the network conditions that impact the overall performance.
Most of the identified scenarios represent typical conditions of mobile services use. Ranging from the traditional internet use in internet cafés or offices, to new consumption models in driving conditions or in very dense areas such as stadiums or shopping malls. The real novelty is the introduction of conditions that are mostly encountered by IoT devices or services. Behind each of the scenarios hides the full specification of the related test or the configuration of every single component of the TRIANGLE testbed that is used to build a realistic picture of what a mobile App or device could encounter while used in real operational situations.
The identified Use Cases and Scenarios heavily impacted the testbed development, given the new tight requirements that the system (and in particular the orchestration tools) had to live up to. For this reason, beside enabling a full E2E testbed through the development of a standardized S1 interface connecting the Radio Access Network testing equipment to the 4G EPC, the project exploited a completely new set of tools for mobile Apps automation and for orchestrating all the different component together.
While working on the development of the testbed in order to support the certification scheme, TRIANGLE promoted in several conferences and trade shows the concept of End-to-End testing. The published white paper “5G BaaS – Benchmark as a Service” helped strengthening the message towards the business community. This effort resulted in a very successful Open Call with a high number of experiment proposals submitted, mostly from SMEs. This fact proves the interest of the SME world to the new possibilities offered by this type of testing and testbed, opening the road for a potential market opportunity for the new 5G ecosystem.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Despite the intense work on the identification of relevant Use Cases, Scenarios, and KPIs, touching the world of certification for mobile Apps is almost like opening the Pandora’s box. Each App is somehow unique in genre and identifying the relevant features to test, together with the associated set of KPIs become a complex task. Nevertheless, the established components of the TRIANGLE certification scheme have been designed in order to accommodate such uniqueness. The inclusion of IoT and general purpose mobile devices make the TRIANGLE certification scheme the most complete ever designed, encompassing everything from the user end to the service provision end.
The creation of a clearly identifiable “label” can help not only the business actors in the 5G ecosystem in creating a more aligned and stable E2E technology, but could impact the end users of the system providing them with a tool for simply and effectively helping in the selection of the device, app, or service to purchase, exactly as the EU energy label helps purchasing everyday objects, from cars to fridges. Finally, the creation of the certification scheme would open market possibilities for the entire testing equipment and services value chain.

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top