PJ05-W2 Digital Technologies for Tower

Project Information

PJ05-W2 DTT

Grant agreement ID: 874470

DOI

10.3030/874470

Project closed

EC signature date 26 November 2019

Start date 1 December 2019

End date 31 March 2023

Funded under

• SOCIETAL CHALLENGES - Smart, Green And Integrated Transport

Total cost

€ 13 994 186,40

EU contribution

€ 9 217 244,00

Coordinated by

DEUTSCHES ZENTRUM FUR LUFT - UND RAUMFAHRT EV
Germany

Periodic Reporting for period 4 - PJ05-W2 DTT (PJ05-W2 Digital Technologies for Tower)

Reporting period: 2023-01-01 to 2023-03-31

Summary of the context and overall objectives of the project

The Project PJ.05-W2 "Digital technologies for Tower" with its three solutions proposed the development of a remotely provided aerodrome air traffic service by a "multiple" and/or "center" setting and further innovative HMI modes and associated technologies to facilitate the air traffic controller's job and to make air traffic service more cost effective and sustainable for the future.
Solution PJ.05-W2-35 aimed for the development of remotely provided Air Traffic Service (ATS), through Multiple Remote Tower (MRT) and Remote Tower Centre (RTC). The main driver for the RTC concept development was increased cost efficiency by an increase of Air Traffic Control Officer (ATCO) productivity, achieved by a flexible allocation of aerodromes between the MRT Moduls (MRTM). The provision of remote ATS service to the remote aerodromes can be flexibly assigned (over time) to other MRTM within a RTC. Supervisor Planning tools support an efficient deployment of staff in an RTC. Solution PJ.05-W2-35 brought the MRT and RTC concept to a matured level ready for industrialization. All Enablers for solution PJ.05-W2-35 were positively validated. V3 was successfully reached.
Solution PJ.05-W2-97.1 ”Virtual/Augmented Reality applications for tower” aimed for to validate Virtual and Augmented Reality (V/AR) applications, including tracking labels, in different Airport operating environments, blending real world images with computer-generated data in real-time, so to allow a reduction of ATCOs head-down time, better balanced workload and an increase of situational awareness. The developed solutions include human-system interaction through the use of in-air gestures, and the attention control for V/AR applications to monitor and drive the attention of the controller. The solution proved the technical feasibility of the AR concept in an Air Traffic Control (ATC) Tower. Technology Readiness Level (TRL) 4 was successfully reached.
Solution PJ.05-W2-97.2 ”ASR at the TWR CWP supported by AI and Machine Learning” aimed for to validate Automatic Speech Recognition (ASR) technologies to allow the recognition and translation of spoken language (e.g. ATCO utterances) as commands input into the ATC system. The ASR engine is supported by Artificial Intelligence / Machine Learning algorithms, which drive and boost the ASR process through a set of command hypotheses derived from the contextual knowledge, with a benefit on workload, HMI usability and efficiency in ATCO – pilot interactions. This solution is expected to positively contribute mainly to the Cost Efficiency, Safety KPAs with increased heads up time and better balanced workload, as well as increased situation awareness and controllers' productivity. TRL 4 was successfully reached.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The operational concept of a flexible allocation of aerodromes to different MRTM´s in solution PJ.05-W2-35 was validated in eight exercises using Real Time Simulations and a Shadow Mode trial on different validation platforms. Different characteristics on weather, traffic and airport layouts were covered. End results were a consolidated report on validation results as well as on Human Performance and Safety.
In solution PJ.05-W2-97.1 the technical feasibility of the concept has been investigated through a set of technical validation exercises in different Airport operating environments with different layout complexity, traffic volumes and meteorological conditions. The use of the glasses was shown to be beneficial to safety at night or in LVC, provided that the surveillance data feed is reliable, without data dropout and tag jumps.
In the frame of Solution PJ.05-W2-97.2 ASR applications for ATC Tower have been validated through a set of three technical exercises by INDRA in Asker, by DLR at Braunschweig simulating a multiple remote airport controller working position and by LEONARDO at Rome, all proving the technical feasibility of the ASR technology to capture Aerodrome ATC instructions and clearances transmitted by radio to flight crews and to use them to automate ATC system inputs.
For all three solutions communication, dissemination and exploitation of results have been fostered to ensure awareness amongst the Air Transport community, through: a dedicated website (www.remote-tower.eu) a number of presentations organised in the frame of SESAR Academy webinars or ATM global community events (e.g. at World ATM Congress, ICNS, NextGen Tech Talk Speaker Series), videos, workshops (with NSA, PSO), Open Days, papers, Press Releases, Exhibits, Articles/Interviews and a Final Event.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Solution PJ.05-W2-35 positively validated the enable concept ‘Highly Flexible Allocation of Aerodromes to Remote Tower Modules’: The provision of remote ATS service to the remote aerodromes can be flexibly assigned (over time) to different MRTMs within an RTC. Results showed that each ATCO could get a better balance of traffic of the connected aerodromes to that MRTM compared to a more fixed allocation, which only uses split and merge for traffic balance. Supervisor Planning tools support an efficient deployment of staff in an RTC. Impact for local airport owners will lead to savings as the concept is exploited. Airliners will benefit from a greater service level achieved through synergies enabled by MRT and RTCs in place. Benefits for ANSPs and ATS staff is that working in a larger environment with an easier way to have back up resources available which is a problem in many small rural areas, which contributes to safety.
The activities performed in solution PJ.05-W2-97.1 proved its technical feasibility. The breakthrough of this technology relies in the new modality of interaction in the conventional tower, which reliefs the controllers need to continuously switch the gaze from head up to head down position, therefore supporting the controllers experience and making it more immediate and immersive. The integration of ASR could be investigated in the future to identify certain situations in the system. Additionally, stripless working methods could be investigated adding planning aspects to the outside view, making it superfluous to build a mental picture with flight status strips. Usability of HMI should allow the clearance input by means of hand gestures into the ATC System in a timely manner.
Solution PJ.05-W2-97.2 proved the technical feasibility of the ASR technology to capture Aerodrome ATC instructions and clearances transmitted by radio to flight crews and to use them to automate ATC system inputs. ATCOs saw the potential in applying speech recognition in a TWR environment and foster to go further in maturity level. In the next future, to evolve the solution, it could be possible to consider a larger amount of representative training data (especially speech data from ATC operations’ rooms), to intensify the use and enhance European-wide agreed ontology for annotation of ATC utterances and to foster standardization of ABSR input and output content as well as format in order to improve system interoperability and comparability. The investigation of ASR applied to pilot utterances could enable reasonable callsign highlighting at ATCO side and readback error detection. Moreover, the ASR engine can enable a long series of further applications that use the speech recognition and understanding output such as pre-filling of radar labels and flight strips, advanced readback error detection, incident analysis, on-the-job training support.

https://youtu.be/q8o1ixZ0vyE

Closing Event of PJ.05-W2 Digital Technologies for Tower