MOTO project explored the concept of the Remote Tower platforms enhanced by considering human performance in control towers from the perspective of “embodied cognition”, with the goal of augmenting the human-system interaction using multimodal feedback.
The first stream of work performed in MOTO, ‘Realistic Remote Tower’, was explored through the virtual reality platform using the HTC Vive Head Mounted Display. The outcomes of this stream of work showed that ATCOs performance could be enhanced by integrating visual channel with one sensory modality at a time (either audio or haptic). In line with this finding, workload assessment showed that by integrating only Audio channel to the Video one induced a decreasing in ATCO’ experienced workload.
However, attention should pay to not “overload” ATCOs providing visual and both Audio and Vibrotactile stimuli. Results seem to suggest that the full combination may induce a degradation of performance, likely due to distracting stimuli contribution that is higher than their informative content. Finally, results suggest that in the ‘Visual + Auditory’ condition ATCOs sense of presence was improved, consistently with performance and workload improvement. These results are detailed in D4.2 (First Validation Report).
The second stream of work, ‘Augmented Remote Tower’, exploited the possibilities offered by the multimodal stimuli, translating them into a technologies to enhance human performance, for instance by augmenting perception.
A set of augmented multimodal Remote Tower concepts (TRL2) were designed and a subset were actually tested during the second MOTO validation providing an initial insight of the potential benefits in terms of performance. The augmented multimodal concepts validated in the second MOTO validation showed performance advantage using a spatialized auditory warning: ATCO reported faster reaction time in detecting abnormal situation such as an unauthorized clearance in the apron or an on-going runway incursion (in this latter a vibration feedback was reinforced by an auditory spatialized warning). However, both subjective and neurophysiological results suggested an increase of experienced workload if augmented solutions were activated. Such perception could also be attributed to the fact that ATCOs needed a longer familiarization process with a more enriched sensorial environment for tower operations, since they are usually used to work only with information provided by the visual sensory channel. Further studies are then recommended to verify that performance benefits could be extended to the whole set of proposed solutions. A set of exploratory concepts for multimodal multiple Remote Tower featured by augmented interactions have been also defined at lower TRL1.
Another outcome of the project was the neurophysiological classifiers developed to monitor cognitive workload and sense of presence. These classifiers were used in both validation experiments to measure the ATCOs mental states during the execution of air traffic control tasks. The Sense of Presence Index was explored more into detailed during the first validation. This index is intended as an objective measure of the sense of presence in an immersive remote environment (in this case also virtual). The index was estimated from the possible modulations of the GSR responses induced by an emotionally salient virtual context in RTO environment, and results from GSR and EEG were consistent along the whole activity. Some limitations have been identified during the validatoins and they need to be carefully considered in the interpretation of the results.
Hovewer, MOTO outcomes in terms of requirements and solutions for multimodal Remote Towers represents a first step to integrate “realistic” multimodal feedback in current industrial solutions and to drive further R&D activities.
