Although localisation and 3D mapping algorithms based on SLAM techniques are now industrialised in many AR devices, it is a very active research area as some hard challenges remain for addressing large scale and dynamic environments. Augmented reality working anywhere, anytime and on any device is far from being solved, yet it is essential to the adoption of this type of technology, whether for B2B or B2C uses. While most recent research work implements machine learning approaches that can hardly be generalised for any type of environment (learning is required for each usage environment), very few research work focus on the distribution into the cloud of vision pipelines to address both large scale and dynamic environments challenges with a crowd-mapping approach that maintains an up-to-date global map. The ARtwin project provided a sovereign AR cloud platform implementing a crowd mapping approach at the scale for a factory or a construction site, compliant with any AR device. A set of innovative services based on the most recent state-of-the-art work will be deployed onto this platform to provide end-users with robust and seamless AR applications anywhere in the factory/construction site. Moreover, the 3D maps used for AR device relocalization will be reused to update the 3D mock-ups of the digital twin based on last observations captured by vision sensors embedded in AR devices. This CAD/BIM update service is able to segment the 3D map built from AR device observations based on a supervised machine learning approach requiring the building of annotated learning data bases specific to the execution environment. To ease the process, the ARtwin project has implemented a solution based on semantic dense reconstruction to automatically generate a huge amount of annotated 2D observations controlled by virtual agents. Finally, an innovative remote rendering service, that leverages a depth map to enhance the quality of experience on none ultra-low latency network, allows the display of very complex CAD/BIM models on low-resources AR devices by moving this costly computation to remote GPU servers. The use of 5G has provided the required low-latency connectivity ensuring a high quality of experience. The results of the ARtwin project pave the way for a new generation of AR technologies that work anywhere, anytime and on any device. The impact on industry and construction 4.0 could be decisive, as the use of AR could be democratised to a wide range of tasks, leading to significant gains in productivity and quality, and thus improving the competitiveness of European companies. Moreover, although the project is mainly targeted at industry and construction, its results could have a significant impact on many other professional fields (e.g. Aeronautics, shipyards, energy, military, and health) as well as on many mass market applications (e.g. entertainment, cultural heritage, retailing, and social networks) while preserving data confidentiality on a sovereign European platform.
