A Computational Design Approach to Haptic Synthesis

Informations projet

TouchDesign

N° de convention de subvention: 772738

DOI

10.3030/772738

Projet clôturé

Date de signature de la CE 27 Avril 2018

Date de début 1 Septembre 2018

Date de fin 31 Août 2023

Financé au titre de

• EXCELLENT SCIENCE - European Research Council (ERC)

Coût total

€ 1 968 812,00

Contribution de l’UE

€ 1 968 812,00

Coordonné par

UNIVERSIDAD REY JUAN CARLOS
Spain

Periodic Reporting for period 4 - TouchDesign (A Computational Design Approach to Haptic Synthesis)

Période du rapport: 2023-03-01 au 2023-08-31

Résumé du contexte et des objectifs généraux du projet

We use touch permanently to explore, manipulate and interact with the world around us, but also to feel and transmit affection. Haptic synthesis, i.e. the ability to design and control what we feel, either on a computer application or with a consumer product, bears an immense scientific, industrial, and social impact. However, touch is still poorly understood and underexploited in today’s digital era. The state of the art in computational haptic synthesis lags well behind the technological and scientific progress in additive manufacturing, computational design for fabrication, virtual reality displays, or cutaneous haptic interfaces.

TouchDesign defines a formal and comprehensive computational design methodology for haptic synthesis, applied to both tactile digital communication and to computational design and fabrication of objects with desired tactile properties. Haptic synthesis is formulated as an optimization problem, with the objective function defined based on haptic perceptual metrics, and with the design space defined by the high-dimensional parameter space of a fabrication process or a haptic interface.

TouchDesign has introduced multiple breakthroughs in four major scientific pillars. (i) In contact biomechanics: develop measurement-based and data-driven models that enable interactive evaluation of detailed deformations resulting from contact mechanics. (ii) In perceptual modeling: establish a connection between high-resolution biomechanics and psychophysics, through machine-learning analysis of exhaustive simulated and experimental data. (iii) In numerical optimization: develop methods that control in an optimal way the high-dimensional design space of haptic fabrication and haptic display problems. (iv) In computational design: introduce methods and interfaces to visualize, explore, and define perceptual objective functions and haptic design spaces.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

We split the achievements of TouchDesign along two lines of research: (1) computational models of touch and (2) touch synthesis methods.

** Computational models of touch:

The work has addressed several levels of granularity, building bottom-up from fundamental contact and deformation mechanics to high-level interaction.
- Fundamental modeling of multi-scale deformation [Casafranca and Otaduy 2022, Chan-Lock et al. 2022].
- Computational efficiency of contact mechanics. TouchDesign has pioneered the development of reduced-order deformation models for contact, integrating machine-learning methodologies [Romero et al. 2021, 2022, 2023].
- Covering full-body biomechanical contact models. TouchDesign has created the first realistic physics-based models that integrate the skeletal structure of the body with soft skin deformation and reaction to contact [Romero et al. 2020, Santesteban et al. 2020, Tapia et al. 2021, Ramón et al. 2023].
- Connection between contact-touch and high-level cognition [Tapia et al., under review].
- Understanding of the impact of biomechanical models on interaction algorithms [Sorli et al. 2021, Lobo and Otaduy 2020].

** Touch synthesis methods:

The project has explored a broad set of applications where touch-oriented design is highly relevant, in the context of both virtual reality and real-world touch.
- Optimization-based synthesis for tactile haptics [Verschoor et al. 2020].
- Optimization-based synthesis for ultrasound haptics [Barreiro et al. 2019, 2020, 2021].
- Optimization-based synthesis for glove haptics [Tong et al. 2023].
- Touch synthesis for touchscreens [Torres et al. 2021].
- Utilization of computational touch models for hand tracking [Mueller et al. 2019, Wang et al. 2020].
- Modeling of high-resolution cloth deformations [Sánchez-Banderas et al. 2020, Pizana et al. 2020, Casafranca et al. 2020, Sperl et al. 2022].
- Synthesis of cloth deformation for virtual try-on. TouchDesign has pioneered the design of learning-based cloth-body simulation models for virtual try-onSantesteban et al. 2019, 2021, 2022a, 2022b].
- Design of soft real-world devices, for soft robotics [Tapia et al. 2020] and for scoliosis braces [Koutras et al., under review].

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The project and the results are now completed, as discussed earlier.

Touch models applied to hand tracking [Mueller et al. ACM TOG 2019]

Cloth models for ergonomics design of clothing [Casafranca et al. CGF 2020]

Hand tracking methods using computational models of touch [Wang et al. ACM TOG 2020]

Garment fit and drape for ergonomics design [Santesteban et al. CGF 2019]

Learning-based full-body biomechanics [Romero et al. CGF 2020]

Ultrasound-based haptic rendering [Barreiro et al. IEEE ToH 2020]

Learning-based modelling of contact-induced deformations [Romero et al. ACM TOG 2021]

Low-level modelling of contact with fabric materials [Sánchez-Banderas et al. ACM TOG 2020]

Learning-based solutions to garment interaction with the body [Santesteban et al. CVPR 2021]

Reduced-order models of skeletal and contact-induced deformations [Tapia et al. CGF 2021]

Optimization-based haptic rendering with wearable thimbles [Verschoor et al. ACM TOG 2020]

Haptic rendering with underactuated devices [Lobo and Otaduy IEEE ToH 2020]

Touch models applied to robot sensing [Tapia et al. Soft Robotics 2020]

Measurement-based full-body biomechanics [Romero et al. CGF 2020]