Making The Invisible Visible for Off-Highway Machinery by Conveying Extended Reality Technologies

Información del proyecto

THEIA-XR

Identificador del acuerdo de subvención: 101092861

DOI

10.3030/101092861

Proyecto cerrado

Fecha de la firma de la CE 15 Noviembre 2022

Fecha de inicio 1 Enero 2023

Fecha de finalización 31 Diciembre 2025

Financiado con arreglo a

• Digital, Industry and Space

Coste total

€ 6 192 686,50

Aportación de la UE

€ 6 192 686,00

Coordinado por

TTCONTROL GMBH
Austria

CORDIS proporciona enlaces a los documentos públicos y las publicaciones de los proyectos de los programas marco HORIZONTE.

Los enlaces a los documentos y las publicaciones de los proyectos del Séptimo Programa Marco, así como los enlaces a algunos tipos de resultados específicos, como conjuntos de datos y «software», se obtienen dinámicamente de OpenAIRE .

Resultado final

Documents, reports (23)

Impact activities (first version)

Deliverable D7.2 will be the first report on the communication, dissemination, exploitation activities as well as on business value and market propositions.

Technology Analysis and Requirements Specification (final version)

D2.5 will include the state-of-the-art technology analysis and the final requirements specification for the final version of the THEIA-XR technologies and the use cases.

Interaction Scenarios (final version)

Deliverable D3.7 provides the final version of documented interaction scenarios with integrated concepts for ethical, self-efficacious, and meaningful work.

Evaluation Report (final version)

Deliverable D3.10 is the final evaluation report detailing goal attainment, optimization potential and recommendations.

Design guidelines and validation reports for privacy (first version)

Deliverable D7.3 will be a document with design guidelines for privacy and security, ready to be implemented in XR products in work context.

Integrated THEIA-XR technologies and methodologies and use case implementation (first version)

Deliverable D6.1 will provide the first version of the integrated technology and methodology components (from WP3, WP4 and WP5) as well as first implementation of the use case applications and services.

Use Case Specification (first version)

D2.1 will contain a detailed specification of the use cases, highlighting the individual activities and scenarios to be tackled inside the use cases.

Design guidelines and validation reports for privacy (final version)

Deliverable D7.6 will be a validation report of ethics-related questions in deliverables from other work packages.

Context-of-use Analysis (first version)

First report on main user groups descriptions including tasks, resources, environment and threats.

Evaluation Report (first version)

Deliverable D3.5 is the first evaluation report detailing goal attainment, optimization potential and recommendations.

XR Design guidelines

Completed and published set of scientifically validated design guidelines

Integrated THEIA-XR technologies and methodologies and use case implementation (final version)

Deliverable D6.4 will provide the final version of the integrated technology and methodology components (from WP3, WP4 and WP5) as well as the final implementation of the use case applications and services.

Deployment, testing and demonstration of the THEIA-XR use cases (final version)

Deliverable D6.5 will report on the final version of deployed and tested demonstrators.

Context-of-use Analysis (final version)

Final report on main user groups descriptions including tasks, resources, environment and threats.

Deployment, testing and demonstration of the THEIA-XR use cases (first version)

Deliverable D6.3 will report on the first version of deployed and tested demonstrators.

Validation and evaluation (first version)

Deliverable D6.3 will report on the first validation and evaluation results within the projects, through the defined set of KPIs.

Privacy assessment results

A report on the validation of privacy perceptions for each of the tested use cases will be created.

Privacy Requirements

The privacy requirements will outline vital privacy considerations, derived from the process outline in the task description.

Interaction Scenarios (first version)

Deliverable D3.2 provides the first version of documented interaction scenarios with integrated concepts for ethical, self-efficacious, and meaningful work.

Impact activities (final version)

Deliverable D7.5 will be the final report on the communication, dissemination, exploitation activities as well as on business value and market propositions.

Validation and evaluation (final version)

Deliverable D6.6 will report on the final validation and evaluation results and lessons learnt within the projects, through the defined set of KPIs.

Technology Analysis and Requirements Specification (first version)

D2.2 will include the state-of-the-art technology analysis and the initial requirements specification for the 1st version of the THEIA-XR technologies and the use cases.

Use Case Specification (final version)

The final version, D2.4, will update the use case specifications.

Other (5)

Interaction modality assignment

Deliverable D4.2 provides the interaction modality framework with criteria-based recommendations for information presentation.

HMI content list (final version)

Deliverable D4.6 provides the final overview of prioritized and clustered information embedded in the use case descriptions.

HMI content list (first version)

Deliverable D4.1 provides a first overview of prioritized and clustered information embedded in the use case descriptions.

Interaction sequence models (final version)

Final version of the modeled interaction workflows for identified operator activities.

Interaction sequence models (first version)

First version of the modeled interaction workflows for identified operator activities.

Demonstrators, pilots, prototypes (6)

Prototypes for privacy-related questions (first version)

Deliverable D3.3 provides first versions of prototypes and validation reports of concepts investigating privacy-related questions in XR contexts.

Concept of virtual user interface (first version)

First concept of an human-machine interaction with simulated machine model

Prototypes for privacy-related questions (final version)

Deliverable D3.3 provides final versions of prototypes and validation reports of concepts investigating privacy-related questions in XR contexts.

Interaction concepts (final version)

Final playable immersive demonstration illustrating the different interaction concepts

Interaction concepts (first version)

First playable immersive demonstration illustrating the different interaction concepts

Concept of virtual user interface (final version)

Final concept of an human-machine interaction with simulated machine model

Publicaciones

Thesis and dissertations (17)

Steering Wheel Control for Virtual Toy Car Driving

Autores: Johannes Aigner
Publicado en: 2024
Editor: Technische Universität Graz

XR application for remote operations

Autores: Andrea Alesani
Editor: Politecnica di Milano

Visionäre Arbeitsumgebungen für Snowgroomer: Methodische Ansätze und prototypische Implementierung

Autores: Felix Wickert
Publicado en: 2025
Editor: Hochschule Reutlingen

Spotlight Control for Real-Time Targeting

Autores: Mika Heinemann
Publicado en: 2024
Editor: Technische Universität Graz

Laser-Projection based Augmented Reality for Snow Grooming

Autores: Maximilian Tschulik
Publicado en: 2023
Editor: Technische Universität Graz

Mobile Visual Search revisited

Autores: Timon Kanik
Editor: Graz University of Technology

Combining DLP and Laser Projection for Augmented Reality Visualizations

Autores: Maximilian Tschulik
Editor: Graz University of Technology

Distance and Odometry Measurements on Toy Car with VR Visualization

Autores: Jakob Khom
Publicado en: 2024
Editor: Technische Universität Graz

In-Situ Virtual Reality Tour using a realtime 360° video stream

Autores: Simon Fussi
Publicado en: 2024
Editor: Technische Universität Graz

Bridge XR: Towards a Cross-Realities Student/Teacher System - A mixed reality training environment

Autores: Franz Feichtinger
Editor: Graz University of Technology

Container Bridge Model using ESP cameras

Autores: Franz Feichtinger
Publicado en: 2024
Editor: Technische Universität Graz

Design of a low-cost FMCW Ground-Penetrating Radar based on a single-board computer

Autores: Bernhard Englmair
Editor: Graz University of Technology

3D Depth Experience in VR from Monocular Video

Autores: Christoph Nickas
Editor: Graz University of Technology

How to Build a VR Headset

Autores: Alexander Albin Streißnig
Publicado en: 2025
Editor: Technische Universität Graz
DOI: 10.3217/FPGR3-NEM31

Optimizing H.265 Real-Time Streaming

Autores: Lukas Fuchs
Publicado en: 2025
Editor: Technische Universität Graz
DOI: 10.3217/K72F3-HS602

Rear-View Camera System for Trailer Parking

Autores: Daniel Kreimer
Editor: Graz University of Technology

MIPI-Camera based Panoramic Camera Design and Implementation

Autores: Lukas Fuchs
Publicado en: 2024
Editor: Technische Universität Graz

Conference proceedings (18)

Entwicklung eines Methodenkartensets zur Steigerung der Multimodalität von Bedienkonzepten

Autores: null null, Julius Röhlig, Sebastian Lorenz, null null
Publicado en: Entwerfen Entwickeln Erleben 2024 : Menschen, Technik und Methoden in Produktentwicklung und Design, 2024
Editor: Technische Universität Dresden
DOI: 10.25368/2024.EEE.009

International Journal of Human Computer Studies

Autores: Philippe Valoggia, Anastasia Sergeeva, Arianna Rossi, Wilhelmina Maria Botes
Publicado en: Proceedings of the 10th International Conference on Information Systems Security and Privacy, 2024, 2024, ISSN 1071-5819
Editor: Academic Press
DOI: 10.5220/0012393100003648

Almost 28 Years later: Mobile Visual Search Using Large Datasets on Handheld Hardware

Autores: Timon Kanik, Philipp Fleck, Thomas Kernbauer, Clemens Arth
Publicado en: Proceedings of the 21st International Conference on Computer Vision Theory and Applications, 2026
Editor: SCITEPRESS - Science and Technology Publications
DOI: 10.5220/0014334200004084

Change-Resilient Localization Estimation

Autores: Fernando Reyes-Aviles, Philipp Fleck, Dieter Schmalstieg, Clemens Arth
Publicado en: 2026 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2026
Editor: IEEE
DOI: 10.1109/VR67842.2026.00041

Extended Reality Simulator to Evaluate Remote Control Interactions for Reach Stackers

Autores: Andrea Alesani,Kaj Helin,Vladimir Goriachev, Timo Kuula
Editor: VTT Technical Research Centre of Finland
DOI: 10.5281/ZENODO.19860128

Spatial Augmented Reality for Heavy Machinery Using Laser Projections

Autores: Maximilian Tschulik, Thomas Kernbauer, Philipp Fleck, Clemens Arth
Publicado en: 2024
Editor: Elsevier BV
DOI: 10.2139/ssrn.4942184

Learning from the Dark Side About How (not) to Engineer Privacy: Analysis of Dark Patterns Taxonomies from an ISO 29100 Perspective

Autores: Philippe Valoggia, Anastasia Sergeeva, Arianna Rossi, Marietjie Botes
Publicado en: Proceedings of the 10th International Conference on Information Systems Security and Privacy, 2024
Editor: SCITEPRESS - Science and Technology Publications
DOI: 10.5220/0012393100003648

Assisted Trailer Parking using a Reverse Camera System and Inverse Kinematics

Autores: Kreimer, Daniel; Fleck, Philipp; Kernbauer, Thomas; Arth, Clemens
Publicado en: 2025 IEEE Intelligent Vehicles Symposium (IV), 2025
Editor: IEEE
DOI: 10.1109/IV64158.2025.11097420

3D Depth Experience in VR from Monocular Video

Autores: Nickas', Christoph; Fleck, Philipp; Kernbauer, Thomas; Arth, Clemens
Publicado en: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2025
Editor: IEEE
DOI: 10.1109/VRW66409.2025.00086

Power to the operators: Transdisciplinary co-design of XR applications in off-highway vehicle work domains

Autores: Manuel Kulzer, Tanja Brodbeck, Michael Burmester
Publicado en: EuroXR 2024: Proceedings of the 21st EuroXR International Conference, 2024
Editor: VTT Technical Research Centre of Finland
DOI: 10.5281/ZENODO.19859995

PanoTherm: Panoramic Thermal Imaging for Object Detection and Tracking

Autores: Kernbauer, Thomas; Fleck, Philipp; Arth, Clemens
Publicado en: Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, ISBN 978-989-758-679-8
Editor: SciTePress
DOI: 10.5220/0012330400003660

Augmenting Controls for Tracked Vehicles with Force and Tactile Feedback: a Work in Progress

Autores: Thomas Howard, Benjamin Geslot, Jerome Perret
Publicado en: 2024
Editor: VTT Technical Research Centre of Finland
DOI: 10.5281/ZENODO.19858824

Making the invisible visible for off-highway machinery by conveying extended reality technologies

Autores: Martijn Rooker, Michael Burmester, Gerald Fritz-Mayer, Clemens Arth, Sebastian Lorenz, Volker Waurich, Manuel Kulzer, Anastasia Sergeeva, Kaj Helin, Markku Pusenius, Benjamin Geslot, Daniel Röck, Pekka Yli-Paunu
Publicado en: Proceedings of the 20th EuroXR International Conference, ISSN 2442-122X
Editor: VTT Technical Research Centre of Finland Ltd.
DOI: 10.32040/2242-122X-2023.T422

Spotlight Control for Real-Time Targeting

Autores: Heinemann, Mika; Thomas, Kernbauer; Fleck, Philipp; Arth, Clemens
Publicado en: 32. International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, ISSN 2464-4617
Editor: University of West Bohemia, Czech Republic
DOI: 10.5281/ZENODO.12658308

Early-stage user experience design of the remote operation concept of the harbour's reachstacker by expoiting eXtended Reality

Autores: Helin, K., Alesani, A., Kuula, T., & Goriachev, V.,
Publicado en: International Conference on Applied Human Factor and Ergonomics (AHFE) 2024, 2024
Editor: AHFE Open Access
DOI: 10.54941/ahfe1005411

Identifying Ethical Challenges in XR Implementations in the Industrial Domain: A Case of Off-Highway Machinery

Autores: Anastasia Sergeeva, Claudia Negri-Ribalta, Gabriele Lenzini
Publicado en: CHI 2025, 2025
Editor: Cornell University

Shape-Shifting Splats: Realtime Context Translation for Gaussian Splatting in VR

Autores: Thomas Kernbauer, Simon Fussi, Philipp Fleck, Clemens Arth
Publicado en: 2026 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2026
Editor: IEEE
DOI: 10.1109/VR67842.2026.00046

Thermochromic Temperature Measurement: Towards an Alternative to Thermal Cameras

Autores: Nussbacher, Thomas; Fleck, Philipp; Arth, Clemens
Publicado en: 2025 IEEE Sensors Applications Symposium (SAS), 2025
Editor: IEEE
DOI: 10.1109/SAS65169.2025.11105166

Peer reviewed articles (9)

Bag of World Anchors for Instant Large-Scale Localization

Autores: Reyes Aviles, Fernando; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Publicado en: IEEE Transactions on Visualization and Computer Graphics, ISSN 1941-0506
Editor: IEEE
DOI: 10.1109/TVCG.2023.3320264

An Inexpensive Chest Belt for measuring Respiratory Effort

Autores: Maximilian Geiger, Philipp Fleck, Clemens Arth
Publicado en: IEEE Sensors Journal, 2026, ISSN 1530-437X
Editor: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/JSEN.2026.3668016

Exploring the Impact of Modality and Speech Rate Manipulation in Voice Permission Requests—Limits of Applicability and Potential for Influencing Decision-Making

Autores: Anna Leschanowsky, Anastasia Sergeeva, Judith Bauer, Sheetal Vijapurapu, Mateusz Dubiel
Publicado en: International Journal of Human-Computer Studies, Edición 205, 2026, ISSN 1071-5819
Editor: Elsevier BV
DOI: 10.1016/J.IJHCS.2025.103590

Computers and Graphics

Autores: Tschulik, Maximilian; Kernbauer, Thomas; Fleck, Philipp; Arth, Clemens
Publicado en: Computers & Graphics, 2024, ISSN 0097-8493
Editor: Pergamon Press Ltd.
DOI: 10.1109/VRW62533.2024.00047

Computers and Graphics

Autores: Kernbauer, Thomas; Tschulik, Maximilian; Fleck, Philipp; Arth, Clemens
Publicado en: Computers & Graphics, 2024, ISSN 0097-8493
Editor: ScienceDirect
DOI: 10.1109/VRW62533.2024.00047

Virtual memory for 3D Gaussian Splatting

Autores: Jonathan Haberl, Philipp Fleck, Clemens Arth
Publicado en: Computers & Graphics, Edición 137, 2026, ISSN 0097-8493
Editor: Elsevier BV
DOI: 10.1016/J.CAG.2026.104598

Instant Segmentation and Fitting of Excavations in Subsurface Utility Engineering

Autores: Marco Stranner, Philipp Fleck, Dieter Schmalstieg, Clemens ARth
Publicado en: Transactions on Vizualization and Computer Graphics (TVCG), 2024, ISSN 1077-2626
Editor: IEEE
DOI: 10.1109/TVCG.2024.3372064

Spotlight Control for Real-Time Targeting

Autores: Heinemann, Mika; Thomas, Kernbauer; Fleck, Philipp; Arth, Clemens
Publicado en: Computer Science Research Notes, 2024, ISSN 2464-4625
Editor: University of West Bohemia
DOI: 10.5281/ZENODO.12658307

International Journal of Human Computer Studies

Autores: Anna Leschanowsky; Anastasia Sergeeva; Judith Bauer; Sheetal Vijapurapu; Mateusz Dubiel
Publicado en: International Journal of Human-Computer Studies, 2025, ISSN 1071-5819
Editor: Academic Press
DOI: 10.24406/PUBLICA-5669

Book chapters (2)

Extended Reality simulator to evaluate remote control interactions for reach stackers

Autores: Alesani, Andrea; Helin, Kaj; Goriachev, Vladimir; Kuula, Timo
Publicado en: 2024
Editor: VTT Technical Research Centre of Finland
DOI: 10.5281/ZENODO.18338766

Making the Invisible Visible - AR supported situation awareness in the snow groomer operations

Autores: Helin, Kaj; Alesani, Andrea; Kuula, Timo
Publicado en: 2025
Editor: VTT Technical Research Centre of Finland
DOI: 10.32040/2242-122X.2025.T440

Other (1)

Entwicklung und Evaluation eines Extended-Reality-Bedieninterfaces für Baumaschinen zur Interaktion mit digitalen Baugrubendaten im Kontext BIM

Autores: Breitbarth, Tim
Publicado en: 2024
Editor: Technische Universität Dresden
DOI: 10.5281/ZENODO.18377690