Computational Design of Multimodal Tactile Feedback within Immersive Environments

Projektinformationen

ADVHANDTURE

ID Finanzhilfevereinbarung: 101088708

DOI

10.3030/101088708

EK-Unterschriftsdatum 5 April 2023

Startdatum 1 Oktober 2023

Enddatum 30 September 2028

Finanziert unter

• European Research Council (ERC)

Gesamtkosten

€ 1 999 750,00

EU-Beitrag

€ 1 999 750,00

Koordiniert durch

INSTITUT NATIONAL DES SCIENCES APPLIQUEES DE RENNES
France

Periodic Reporting for period 1 - ADVHANDTURE (Computational Design of Multimodal Tactile Feedback within Immersive Environments)

Berichtszeitraum: 2023-10-01 bis 2026-03-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

These last years have seen many progresses in haptic technologies for providing various tactile stimuli through innovative haptic interfaces that users could wear across all their body. However, despite these advances, virtual reality systems are today still facing a fundamental lack of rich tactile feedback able to convey compelling sensations. ADVHANDTURE aims at introducing an original computational approach for handling multimodal tactile feedback within immersive environments through the design of computational models dedicated to immersive tactile haptics. The hand -by its fundamental role in the sense of touch- is at the heart of the adventure across the tactile exploration of immersive environments. Three main scientific objectives are targeted in the project: 1) the design of rendering models for combining in a spatially-coherent manner tactile modalities within immersive environments, 2) the development of efficient algorithms for the temporal integration of various tactile stimuli within immersive environments, 3) the design of 3D interaction paradigms for combining multimodal tactile stimuli with multi-sensory integration.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

Given the scientific objectives of the project, different research directions have been initiated, following the path of the 3D interaction loop relating the user to the virtual environment he is working with.

Design of innovative haptic interfaces. As our main goal is to understand how to combine tactile stimuli on the hand, a significant part of our research activities in this first period was dedicated to the
design of haptic interfaces in order to generate various types of haptic stimuli. We explored several types of stimuli, from vibrations to ultrasounds, combined with various shapes of interfaces.

Design of perception studies. The main objective of our project is to improve our understanding of the human perception about the combination of haptic stimuli applied on the hand. We first started to
study how we could combine the feedback of a real object with a haptic stimulus. For that purpose, we conducted different experiments where we evaluated how real textures could be enhanced with
vibrotactile stimuli and if we could observe differences in terms of perception between Augmented Reality and Virtual Reality environments. Regarding the combination of haptic stimuli and their effects on the hand, we also explored how we could use vibrotactile stimuli on the arm and their effects on the hand perception.

Design of 3D rendering techniques. Given our innovative interfaces and the perception studies previously conducted, we proposed several 3D rendering techniques for 3D interaction within
virtual environments. For ultrasound mid-air haptics, we designed a specific rendering for bimanual interactions, providing a wider space for 3D manipulation in Virtual Reality. We also combined our

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

Fundamentally, ADVHANDTURE will significantly contribute to pave the way for computational haptic models able to render a wide variety of tactile sensations within immersive environments, a still under-explored challenge of computer graphics, virtual reality and haptics fields. The ground-breaking nature of the proposal relies on its multidisciplinary aspect, gathering knowledge of different scientific domains to leverage synergistic outcomes that would be beneficial to the different research communities. It will also enrich our understanding of rendering haptic sensations through the use of combined tactile modalities, as well as the role of haptic feedback during 3D interaction with virtual worlds.