Initially, the states of the art and the definition of the use cases for these WPs were generated. Later, requirements were defined and confirmed by the Topic Manager and the solutions for each Work Package were designed. For WP1 this entailed the study of various possible Augmented Reality technologies, centred around the use of Microsoft HoloLens headset. Also, investigation and development of necessary improvements to the Indoor Positioning System (IPS) presented were performed, focused on improving its performance in the 3D space and when tracking moving objects. For WP2, the design of the hardware elements needed to satisfy the communication requirements was performed with a combination of off-the-shelf and custom designed elements. Communications with the Topic Manager’s server infrastructure where also defined, as well as integration with the IPS system.
Later, the solutions were implemented for both Work Packages.
For WP1, the focus was the development of an innovative Augmented Reality HMI. The HMI is based mainly on Microsoft HoloLens, the Unity engine and the MRTK (Mixed Reality Tool Kit) plugin. A first prototype version of the HMI was developed and functionalities such as user interaction, voice recognition and positioning and guidance assistance were validated successfully. Moreover, a Middleware Network Protocol (MNP), designed to work over TCP/IP, for communication was defined and implemented as a standalone application first. Once the first prototypes were validated, the full implementation and integration of this technologies were performed. For the HMI, a full upgrade of its design was performed, and new capabilities were included. For MNP, both the integration of the protocol with the CATS test system and with SKYLIFE’s own HMI were developed. Due to lack of compatibility between the standard socket libraries used on HoloLens and CATS (standard C#), an additional development was necessary, using one of the multiplayer communication protocols from Unity (UNET) to serve as a bridge between the CATS application and HoloLens. Finally, in the validation phase, work was done to provide an adequate testing environment at Skylife’s facilities, satisfying the requirements that would be presented by the real FAL as much as possible. For this final validation tests, the test scenario was improved with the inclusion of an Augmented Reality model of a real aircraft cockpit and project was upgraded to also work on the more recent HoloLens 2 hardware, to prevent obsolescence and guarantee the impacts of the project.
For WP 2, the full development of a custom hardware capable of IoT communication (based on ThingWorx), communication with a number of existing tests means and the ability to position itself on an industrial environment was performed. Regarding infrastructure development, the SILEME Indoor Positioning System was adapted to work on the COTS hardware selected and its position estimation was improved with the development of new filters. Moreover, IoT communication via ThingWorx was implemented and tested with the data coming from the IPS itself. Hardware and Software development of the ARIESS HUB system developed. This includes the assembly of a prototype unit based on the Raspberry Pi Zero platform, the integration of all previous developments. The implementation of the IoT/Thingworx communication was completed and fully specified. To improve usability, the ARIESS HUB was equipped with an OLED screen to display information to the user such as error codes or data gathered. Finally, in the validation phase, tests were conducted at Skylife’s facilities, against a testing ThingWorx server deployed for the occasion. Tests were performed focusing in communication with test sensors, battery life, IPS precision, communication with ThingWorx, etc. As an additional work on this phase, a 3D printed enclosure for the ARIESS HUB has been developed, to serve as the basis for a future production run.
