Computational Design of Multimodal Tactile Feedback within Immersive Environments

Información del proyecto

ADVHANDTURE

Identificador del acuerdo de subvención: 101088708

DOI

10.3030/101088708

Fecha de la firma de la CE 5 Abril 2023

Fecha de inicio 1 Octubre 2023

Fecha de finalización 30 Septiembre 2028

Financiado con arreglo a

• European Research Council (ERC)

Coste total

€ 1 999 750,00

Aportación de la UE

€ 1 999 750,00

Coordinado por

INSTITUT NATIONAL DES SCIENCES APPLIQUEES DE RENNES
France

Periodic Reporting for period 1 - ADVHANDTURE (Computational Design of Multimodal Tactile Feedback within Immersive Environments)

Período documentado: 2023-10-01 hasta 2026-03-31

Resumen del contexto y de los objetivos generales del proyecto

These last years have seen many progresses in haptic technologies for providing various tactile stimuli through innovative haptic interfaces that users could wear across all their body. However, despite these advances, virtual reality systems are today still facing a fundamental lack of rich tactile feedback able to convey compelling sensations. ADVHANDTURE aims at introducing an original computational approach for handling multimodal tactile feedback within immersive environments through the design of computational models dedicated to immersive tactile haptics. The hand -by its fundamental role in the sense of touch- is at the heart of the adventure across the tactile exploration of immersive environments. Three main scientific objectives are targeted in the project: 1) the design of rendering models for combining in a spatially-coherent manner tactile modalities within immersive environments, 2) the development of efficient algorithms for the temporal integration of various tactile stimuli within immersive environments, 3) the design of 3D interaction paradigms for combining multimodal tactile stimuli with multi-sensory integration.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

Given the scientific objectives of the project, different research directions have been initiated, following the path of the 3D interaction loop relating the user to the virtual environment he is working with.

Design of innovative haptic interfaces. As our main goal is to understand how to combine tactile stimuli on the hand, a significant part of our research activities in this first period was dedicated to the
design of haptic interfaces in order to generate various types of haptic stimuli. We explored several types of stimuli, from vibrations to ultrasounds, combined with various shapes of interfaces.

Design of perception studies. The main objective of our project is to improve our understanding of the human perception about the combination of haptic stimuli applied on the hand. We first started to
study how we could combine the feedback of a real object with a haptic stimulus. For that purpose, we conducted different experiments where we evaluated how real textures could be enhanced with
vibrotactile stimuli and if we could observe differences in terms of perception between Augmented Reality and Virtual Reality environments. Regarding the combination of haptic stimuli and their effects on the hand, we also explored how we could use vibrotactile stimuli on the arm and their effects on the hand perception.

Design of 3D rendering techniques. Given our innovative interfaces and the perception studies previously conducted, we proposed several 3D rendering techniques for 3D interaction within
virtual environments. For ultrasound mid-air haptics, we designed a specific rendering for bimanual interactions, providing a wider space for 3D manipulation in Virtual Reality. We also combined our

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

Fundamentally, ADVHANDTURE will significantly contribute to pave the way for computational haptic models able to render a wide variety of tactile sensations within immersive environments, a still under-explored challenge of computer graphics, virtual reality and haptics fields. The ground-breaking nature of the proposal relies on its multidisciplinary aspect, gathering knowledge of different scientific domains to leverage synergistic outcomes that would be beneficial to the different research communities. It will also enrich our understanding of rendering haptic sensations through the use of combined tactile modalities, as well as the role of haptic feedback during 3D interaction with virtual worlds.