Periodic Reporting for period 1 - THEIA-XR (Making The Invisible Visible for Off-Highway Machinery by Conveying Extended Reality Technologies)
Berichtszeitraum: 2023-01-01 bis 2024-06-30
THEIA-XR (Making the invisible visible for off-highway machinery by conveying extended reality technologies) aims at improving human-machine interaction in mobile machinery by enhancing user technology fit and extending reality technologies and functionalities. The results will make the invisible visible respectively the non-perceivable perceivable to the human operator, extending the perceivable range of the operator without negatively influencing the performance of the human while controlling the machinery. The project will ensure positive impact of eXtended Reality technologies on the safety, security trustworthiness and societal aspects of the interaction between the machine, the human operator and public/non-users, and furthermore create an XR workplace that enables positive user experiences of self-efficacy and meaningfulness of work.
THEIA-XR will leverage a human-centred transdisciplinary and scenario-based co-design methodology with users, stakeholders and multidisciplinary researchers as a fully integrated project team. This methodology serves to collect human requirements and to design, develop and deploy the extended reality technologies for information presentation and interaction in the off-highway domain. The targeted extended reality technologies will enhance conventional human-machine interfaces through multi-modal information and interaction technologies, deploying innovative data visualization methods, force feedback technology and acoustic information.
The THEIA-XR approach for improving the human operator tasks by deploying extended reality technologies will be initially validated and tested in three uses cases in the off-highway domain, targeting snow grooming, logistics and construction scenarios, integrating real end-users, public/non-users and real-life data from dedicated industrial environments.
Objective #1: Increase of productivity, safety, responsibility and perceived presence in demanding workplaces through experience design
Objective #2: Increase of interaction quality through predictable and comprehensible machine behaviour and communication within an XR HMI
Objective #3: Development of advanced multi-modal interaction technologies (e.g. augmented and/or virtual reality, tactile/haptics, spatial acoustics, etc.) to convey seamless and persistent physical-digital experiences
Objective #4: Ensuring humane workplaces by regarding ethical, self-efficacious, and meaningful work conditions within an XR HMI
Objective #5: Design, implement, integrated and validate a set of real-life use cases targeting new domains for eXtended Reality
THEIA-XR will leverage a human-centred transdisciplinary and scenario-based co-design methodology with users, stakeholders and multidisciplinary researchers as a fully integrated project team. This methodology serves to collect human requirements and to design, develop and deploy the extended reality technologies for information presentation and interaction in the off-highway domain. The targeted extended reality technologies will enhance conventional human-machine interfaces through multi-modal information and interaction technologies, deploying innovative data visualization methods, force feedback technology and acoustic information.
The THEIA-XR approach for improving the human operator tasks by deploying extended reality technologies will be initially validated and tested in three uses cases in the off-highway domain, targeting snow grooming, logistics and construction scenarios, integrating real end-users, public/non-users and real-life data from dedicated industrial environments.
Objective #1: Increase of productivity, safety, responsibility and perceived presence in demanding workplaces through experience design
Objective #2: Increase of interaction quality through predictable and comprehensible machine behaviour and communication within an XR HMI
Objective #3: Development of advanced multi-modal interaction technologies (e.g. augmented and/or virtual reality, tactile/haptics, spatial acoustics, etc.) to convey seamless and persistent physical-digital experiences
Objective #4: Ensuring humane workplaces by regarding ethical, self-efficacious, and meaningful work conditions within an XR HMI
Objective #5: Design, implement, integrated and validate a set of real-life use cases targeting new domains for eXtended Reality
The first phase of the THEIA-XR project has focused mainly on three parts, namely (a) establishing detailed use case specifications and requirements (b) interviews focusing on the transdisciplinary co-design process with end users of the targeted domains, which are snow-grooming, logistics and construction, and (c) first developments of eXtended Reality solutions. At the beginning of the project, it was mainly important to get a common understanding within the consortium and a clear understanding of what potential end-users are expecting from the project. The use cases have gathered many end users, that were involved in multiple workshops gathering information them, including their daily work routine and what kind of technologies that can imagine that can be supportive for them in the future. By M6, the first version of the use case specification was available, that provided the scientific and technical partners a much more detailed insight on how these machines work and what the difficulties are with handling these machines. Additionally, first requirements (functional, non-functional and technical) have been defined and are used as the basis for the technical developments performed mainly in WP3, WP4 and WP5. Based on this, the first developments of different XR methodologies was initiated, thereby focusing on collecting various data from the environment and the machines, especially to gather data that is not directly available. Additionally, investigations are ongoing on how to present this "unknown" data to the human operator. The focus here has been either on using industrial displays and investigating on displaying information in improved fashion, displaying information in front of the machines using lasers or projectors or applying haptic feedback to the human operators. Different approaches are being investigated by the different partners, working either directly with the machines or also demonstrating technologies first in a simulated environments, which are also part of the development process taking place within the project. Finally, during the technology developments and the interaction with the human operators, concepts of ethics and legal usage of data is taken into consideration.
Finally, the three use cases are working on the following aspects to be handled within the project:
Use Case 1 (Snow Grooming):
- Providing information to the human operator performing its task in various challenging environments (e.g. darkness, white-out)
- Information provision exploiting new technologies, going away from classical AR technologies using glasses.
Use Case 2 (Logistics, Reach Stacker)
- Remote operation of Reach Stackers providing information to operators that are not inside the machines
- Providing information about handling containers in different environmental conditions
Use Case 3 (Construction, Excavator)
- Informing human operator about environmental situations around the machine that are not visible to the human operator
- Displaying information in a non-intrusive way that will not influence the work of the operator
Finally, the three use cases are working on the following aspects to be handled within the project:
Use Case 1 (Snow Grooming):
- Providing information to the human operator performing its task in various challenging environments (e.g. darkness, white-out)
- Information provision exploiting new technologies, going away from classical AR technologies using glasses.
Use Case 2 (Logistics, Reach Stacker)
- Remote operation of Reach Stackers providing information to operators that are not inside the machines
- Providing information about handling containers in different environmental conditions
Use Case 3 (Construction, Excavator)
- Informing human operator about environmental situations around the machine that are not visible to the human operator
- Displaying information in a non-intrusive way that will not influence the work of the operator
For the first period this is too early and will be elaborated upon in final reporting period.