A Computational Design Approach to Haptic Synthesis

Informacje na temat projektu

TouchDesign

Identyfikator umowy o grant: 772738

DOI

10.3030/772738

Projekt został zamknięty

Data podpisania przez KE 27 Kwietnia 2018

Data rozpoczęcia 1 Września 2018

Data zakończenia 31 Sierpnia 2023

Finansowanie w ramach

• EXCELLENT SCIENCE - European Research Council (ERC)

Koszt całkowity

€ 1 968 812,00

Wkład UE

€ 1 968 812,00

Koordynowany przez

UNIVERSIDAD REY JUAN CARLOS
Spain

Periodic Reporting for period 4 - TouchDesign (A Computational Design Approach to Haptic Synthesis)

Okres sprawozdawczy: 2023-03-01 do 2023-08-31

Podsumowanie kontekstu i ogólnych celów projektu

We use touch permanently to explore, manipulate and interact with the world around us, but also to feel and transmit affection. Haptic synthesis, i.e. the ability to design and control what we feel, either on a computer application or with a consumer product, bears an immense scientific, industrial, and social impact. However, touch is still poorly understood and underexploited in today’s digital era. The state of the art in computational haptic synthesis lags well behind the technological and scientific progress in additive manufacturing, computational design for fabrication, virtual reality displays, or cutaneous haptic interfaces.

TouchDesign defines a formal and comprehensive computational design methodology for haptic synthesis, applied to both tactile digital communication and to computational design and fabrication of objects with desired tactile properties. Haptic synthesis is formulated as an optimization problem, with the objective function defined based on haptic perceptual metrics, and with the design space defined by the high-dimensional parameter space of a fabrication process or a haptic interface.

TouchDesign has introduced multiple breakthroughs in four major scientific pillars. (i) In contact biomechanics: develop measurement-based and data-driven models that enable interactive evaluation of detailed deformations resulting from contact mechanics. (ii) In perceptual modeling: establish a connection between high-resolution biomechanics and psychophysics, through machine-learning analysis of exhaustive simulated and experimental data. (iii) In numerical optimization: develop methods that control in an optimal way the high-dimensional design space of haptic fabrication and haptic display problems. (iv) In computational design: introduce methods and interfaces to visualize, explore, and define perceptual objective functions and haptic design spaces.

Prace wykonane od początku projektu do końca okresu sprawozdawczego oraz najważniejsze dotychczasowe rezultaty

We split the achievements of TouchDesign along two lines of research: (1) computational models of touch and (2) touch synthesis methods.

** Computational models of touch:

The work has addressed several levels of granularity, building bottom-up from fundamental contact and deformation mechanics to high-level interaction.
- Fundamental modeling of multi-scale deformation [Casafranca and Otaduy 2022, Chan-Lock et al. 2022].
- Computational efficiency of contact mechanics. TouchDesign has pioneered the development of reduced-order deformation models for contact, integrating machine-learning methodologies [Romero et al. 2021, 2022, 2023].
- Covering full-body biomechanical contact models. TouchDesign has created the first realistic physics-based models that integrate the skeletal structure of the body with soft skin deformation and reaction to contact [Romero et al. 2020, Santesteban et al. 2020, Tapia et al. 2021, Ramón et al. 2023].
- Connection between contact-touch and high-level cognition [Tapia et al., under review].
- Understanding of the impact of biomechanical models on interaction algorithms [Sorli et al. 2021, Lobo and Otaduy 2020].

** Touch synthesis methods:

The project has explored a broad set of applications where touch-oriented design is highly relevant, in the context of both virtual reality and real-world touch.
- Optimization-based synthesis for tactile haptics [Verschoor et al. 2020].
- Optimization-based synthesis for ultrasound haptics [Barreiro et al. 2019, 2020, 2021].
- Optimization-based synthesis for glove haptics [Tong et al. 2023].
- Touch synthesis for touchscreens [Torres et al. 2021].
- Utilization of computational touch models for hand tracking [Mueller et al. 2019, Wang et al. 2020].
- Modeling of high-resolution cloth deformations [Sánchez-Banderas et al. 2020, Pizana et al. 2020, Casafranca et al. 2020, Sperl et al. 2022].
- Synthesis of cloth deformation for virtual try-on. TouchDesign has pioneered the design of learning-based cloth-body simulation models for virtual try-onSantesteban et al. 2019, 2021, 2022a, 2022b].
- Design of soft real-world devices, for soft robotics [Tapia et al. 2020] and for scoliosis braces [Koutras et al., under review].

Innowacyjność oraz oczekiwany potencjalny wpływ (w tym dotychczasowe znaczenie społeczno-gospodarcze i szersze implikacje społeczne projektu)

The project and the results are now completed, as discussed earlier.

Touch models applied to hand tracking [Mueller et al. ACM TOG 2019]

Cloth models for ergonomics design of clothing [Casafranca et al. CGF 2020]

Hand tracking methods using computational models of touch [Wang et al. ACM TOG 2020]

Garment fit and drape for ergonomics design [Santesteban et al. CGF 2019]

Learning-based full-body biomechanics [Romero et al. CGF 2020]

Ultrasound-based haptic rendering [Barreiro et al. IEEE ToH 2020]

Learning-based modelling of contact-induced deformations [Romero et al. ACM TOG 2021]

Low-level modelling of contact with fabric materials [Sánchez-Banderas et al. ACM TOG 2020]

Learning-based solutions to garment interaction with the body [Santesteban et al. CVPR 2021]

Reduced-order models of skeletal and contact-induced deformations [Tapia et al. CGF 2021]

Optimization-based haptic rendering with wearable thimbles [Verschoor et al. ACM TOG 2020]

Haptic rendering with underactuated devices [Lobo and Otaduy IEEE ToH 2020]

Touch models applied to robot sensing [Tapia et al. Soft Robotics 2020]

Measurement-based full-body biomechanics [Romero et al. CGF 2020]