Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision

Project Information

RETINA

Grant agreement ID: 699370

DOI

10.3030/699370

Project closed

EC signature date 24 February 2016

Start date 1 March 2016

End date 28 February 2018

Funded under

• SOCIETAL CHALLENGES - Smart, Green And Integrated Transport

Total cost

€ 1 072 910,00

EU contribution

€ 949 160,00

Coordinated by

ALMA MATER STUDIORUM - UNIVERSITA DI BOLOGNA
Italy

Periodic Reporting for period 4 - RETINA (Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision)

Reporting period: 2017-09-01 to 2018-02-28

Summary of the context and overall objectives of the project

The increasing interest in Synthetic Vision (SV) and Augmented Reality (AR) technologies has led various analysts to positively esteem the adoption of new tools enabling pilots and controllers to seamlessly operate under Visual Meteorological Conditions and Instrument Meteorological Conditions. The RETINA project investigates the potential and applicability of SV tools and Virtual/Augmented Reality (V/AR) display techniques for the Air Traffic Control (ATC) service provision by the airport control tower. Within the project, several concepts and basic principles that have been observed in different areas (e.g. Remote Tower, Synthetic Vision Systems, AR, Information Technologies, etc.) were brought to the level of maturity required for the Applied Research that will be conducted in SESAR V1-V3 (Applied Research, Industrial Research & Validation). To this end, a 3D airport model was developed, along with V/AR based human-computer interfaces. The digital model provides controllers with precise positioning for both aerial and terrestrial objects, drawing information from multiple, simulated, data sources, such as the System Wide Information Management (SWIM) network, Remote Towers sensing technologies and other well-established surveillance systems – e.g. Airport Surveillance Radar (ASR) and the Surface Movement Radar (SMR). The interface design is based on the Ecological Interface Design approach. Finally, the project investigates the impact of the newly conceived tools on the control tower air traffic management procedures. On the whole, those tasks that are negatively affected by poor visibility conditions, such as bad weather, fog, smoke, dust or any other kind of environmental occlusion, will become weather-independent. The RETINA project primarily relates to SESAR ER-06-2015 - High Performing Airport Operations - Improved Visualisation and Awareness, but also has a secondary relationship to SESAR ER-03-2015 - Information Management in ATM.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The present report covers the 2-year project duration, describing the work performed by the RETINA Consortium in the following Work Packages: WP1, WP2, WP3, WP4, WP5, WP6 and WP7.
WP1 sets up the baseline for the other WP, identifying the state of the art in terms of displays technologies, data sources and standards. The work performed was dedicated to the analysis of the relevant state of art and the production of initial concept requirements together with a task analysis for low and normal visibility conditions.
WP2 aims at generating the actual concepts for the RETINA project describing: which V/AR display technologies should be used; how they should integrate in the control tower environment and procedures; what kind of datasets should be used to visually represented the relevant information and where they come from. To meet these objectives, all the sensing technologies and data provision standards were analysed, then the V/AR technologies that can be applied in a control tower were selected. Finally it was defined when, why and how the controllers will make use of augmented visual observation in order to manage the aerodrome traffic.
WP3 aims to develop proofs of concept for the proposed solutions. The 4D airport model was developed and it was integrated with two interconnected software systems. Also, this WP sets up the basis for the subsequent Validation Activity.
WP4 evaluates the operational solutions proposed by WP3 through a specific set of human-in-the-loop experiments campaign performed in a laboratory setting.On the whole, 11 exercises were performed to validate the RETINA concept in a laboratory environment where the out of the tower view was provided to the user by means of a semi-immersive virtual environment. The validation conducted proved that the RETINA concept has a clear effect in stimulating the ATCO to work in a head-up position more than in a head-down position, with positive impact on workload and situational awareness. Those positive effects are more evident in low visibility conditions.
The main objective for WP5 is to create a public and scientific interest on RETINA outcomes and ensuring their commercial usability. In this context, partners have defined a shared a dissemination and exploitation plan, in order to broadcast step-by-step results and to make the technical and scientific panorama aware of project developments.
A specific demonstration event was held at the end of January 2018 in Forlì, Italy, to present the operational concept and the final assessment results to potential stakeholders.
The overall objective of WP6, as indicated in the DOA, is the smooth implementation of the RETINA project. To this aim, an effort was put in the first semester to define a shared Project Management Plan and facilitate its practical implementation. This resulted in a punctual and harmonized accomplishment of the project objectives set for the first eighteen months of the project. The foreseen risks did not materialize and the whole consortium has met periodically by webex and six times by face. The Final Review was successfully passed in February 2018.
A specific Work Package (WP7) dedicated to Ethics Requirement was added to the RETINA Project during the negotiation phase and 6 ethics deliverables were produced by the Consortium during the first semester. During the second semester, the deliverables were updated according to the request of the JU and finally approved.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The proposed tools provide the operator with a unique conformal representation of all the needed information that's currently provided by means of several visual inputs. When low visibility conditions apply, the use of RETINA tools leads to the reduction of current restrictions due to LVP, with consequent increased throughput. Moreover, the proposed solution provides quantified benefits in terms of mental/temporal workload, performance, effort, frustration, information accessibility, and head-down time. As far as safety is concerned, the RETINA tools proved to preserve safety, leading to its improvement as they enhance situational awareness (SA).
All in all, the development of the proposed solution will provide high-quality 4D information (position, height and speed over time) to concerned actors in any operational condition, thus improving the overall controllers’ SA.
Significant benefits are expected for the whole aviation system, including, but not limited to:
financial savings for carriers and ANSP, improved safety for passengers, environmental pollution reduction, and increased resilience and efficacy for the control tower IT systems.
The proposed solution will allow Secondary Surveillance Radar (SSR) equipped aircraft to operate at synthetic vision equipped airports thus improving the overall air traffic system capacity and, indirectly, alleviating congestion on nearby airports. Complex airports will benefit from the implementation of such technology by preserving airport capacity level in all weather conditions, even when Low Visibility Procedures apply. Smaller airports will benefit from the implementation of the RETINA concepts by enabling a visual tracking/surveillance system, by means of limited investments with respect to those required for a an A-SMGCS system.
This will result in substantial financial savings for carriers and larger incomes for ANSP. Also, with fewer delay, a reduction of the environmental impact of flights in terms of fuel burnt, emissions, CO2, etc. will be achieved.

merge.jpg