Periodic Reporting for period 4 - TouchDesign (A Computational Design Approach to Haptic Synthesis)
Période du rapport: 2023-03-01 au 2023-08-31
We use touch permanently to explore, manipulate and interact with the world around us, but also to feel and transmit affection. Haptic synthesis, i.e. the ability to design and control what we feel, either on a computer application or with a consumer product, bears an immense scientific, industrial, and social impact. However, touch is still poorly understood and underexploited in today’s digital era. The state of the art in computational haptic synthesis lags well behind the technological and scientific progress in additive manufacturing, computational design for fabrication, virtual reality displays, or cutaneous haptic interfaces.
TouchDesign defines a formal and comprehensive computational design methodology for haptic synthesis, applied to both tactile digital communication and to computational design and fabrication of objects with desired tactile properties. Haptic synthesis is formulated as an optimization problem, with the objective function defined based on haptic perceptual metrics, and with the design space defined by the high-dimensional parameter space of a fabrication process or a haptic interface.
TouchDesign has introduced multiple breakthroughs in four major scientific pillars. (i) In contact biomechanics: develop measurement-based and data-driven models that enable interactive evaluation of detailed deformations resulting from contact mechanics. (ii) In perceptual modeling: establish a connection between high-resolution biomechanics and psychophysics, through machine-learning analysis of exhaustive simulated and experimental data. (iii) In numerical optimization: develop methods that control in an optimal way the high-dimensional design space of haptic fabrication and haptic display problems. (iv) In computational design: introduce methods and interfaces to visualize, explore, and define perceptual objective functions and haptic design spaces.
TouchDesign defines a formal and comprehensive computational design methodology for haptic synthesis, applied to both tactile digital communication and to computational design and fabrication of objects with desired tactile properties. Haptic synthesis is formulated as an optimization problem, with the objective function defined based on haptic perceptual metrics, and with the design space defined by the high-dimensional parameter space of a fabrication process or a haptic interface.
TouchDesign has introduced multiple breakthroughs in four major scientific pillars. (i) In contact biomechanics: develop measurement-based and data-driven models that enable interactive evaluation of detailed deformations resulting from contact mechanics. (ii) In perceptual modeling: establish a connection between high-resolution biomechanics and psychophysics, through machine-learning analysis of exhaustive simulated and experimental data. (iii) In numerical optimization: develop methods that control in an optimal way the high-dimensional design space of haptic fabrication and haptic display problems. (iv) In computational design: introduce methods and interfaces to visualize, explore, and define perceptual objective functions and haptic design spaces.
We split the achievements of TouchDesign along two lines of research: (1) computational models of touch and (2) touch synthesis methods.
** Computational models of touch:
The work has addressed several levels of granularity, building bottom-up from fundamental contact and deformation mechanics to high-level interaction.
- Fundamental modeling of multi-scale deformation [Casafranca and Otaduy 2022, Chan-Lock et al. 2022].
- Computational efficiency of contact mechanics. TouchDesign has pioneered the development of reduced-order deformation models for contact, integrating machine-learning methodologies [Romero et al. 2021, 2022, 2023].
- Covering full-body biomechanical contact models. TouchDesign has created the first realistic physics-based models that integrate the skeletal structure of the body with soft skin deformation and reaction to contact [Romero et al. 2020, Santesteban et al. 2020, Tapia et al. 2021, Ramón et al. 2023].
- Connection between contact-touch and high-level cognition [Tapia et al., under review].
- Understanding of the impact of biomechanical models on interaction algorithms [Sorli et al. 2021, Lobo and Otaduy 2020].
** Touch synthesis methods:
The project has explored a broad set of applications where touch-oriented design is highly relevant, in the context of both virtual reality and real-world touch.
- Optimization-based synthesis for tactile haptics [Verschoor et al. 2020].
- Optimization-based synthesis for ultrasound haptics [Barreiro et al. 2019, 2020, 2021].
- Optimization-based synthesis for glove haptics [Tong et al. 2023].
- Touch synthesis for touchscreens [Torres et al. 2021].
- Utilization of computational touch models for hand tracking [Mueller et al. 2019, Wang et al. 2020].
- Modeling of high-resolution cloth deformations [Sánchez-Banderas et al. 2020, Pizana et al. 2020, Casafranca et al. 2020, Sperl et al. 2022].
- Synthesis of cloth deformation for virtual try-on. TouchDesign has pioneered the design of learning-based cloth-body simulation models for virtual try-onSantesteban et al. 2019, 2021, 2022a, 2022b].
- Design of soft real-world devices, for soft robotics [Tapia et al. 2020] and for scoliosis braces [Koutras et al., under review].
** Computational models of touch:
The work has addressed several levels of granularity, building bottom-up from fundamental contact and deformation mechanics to high-level interaction.
- Fundamental modeling of multi-scale deformation [Casafranca and Otaduy 2022, Chan-Lock et al. 2022].
- Computational efficiency of contact mechanics. TouchDesign has pioneered the development of reduced-order deformation models for contact, integrating machine-learning methodologies [Romero et al. 2021, 2022, 2023].
- Covering full-body biomechanical contact models. TouchDesign has created the first realistic physics-based models that integrate the skeletal structure of the body with soft skin deformation and reaction to contact [Romero et al. 2020, Santesteban et al. 2020, Tapia et al. 2021, Ramón et al. 2023].
- Connection between contact-touch and high-level cognition [Tapia et al., under review].
- Understanding of the impact of biomechanical models on interaction algorithms [Sorli et al. 2021, Lobo and Otaduy 2020].
** Touch synthesis methods:
The project has explored a broad set of applications where touch-oriented design is highly relevant, in the context of both virtual reality and real-world touch.
- Optimization-based synthesis for tactile haptics [Verschoor et al. 2020].
- Optimization-based synthesis for ultrasound haptics [Barreiro et al. 2019, 2020, 2021].
- Optimization-based synthesis for glove haptics [Tong et al. 2023].
- Touch synthesis for touchscreens [Torres et al. 2021].
- Utilization of computational touch models for hand tracking [Mueller et al. 2019, Wang et al. 2020].
- Modeling of high-resolution cloth deformations [Sánchez-Banderas et al. 2020, Pizana et al. 2020, Casafranca et al. 2020, Sperl et al. 2022].
- Synthesis of cloth deformation for virtual try-on. TouchDesign has pioneered the design of learning-based cloth-body simulation models for virtual try-onSantesteban et al. 2019, 2021, 2022a, 2022b].
- Design of soft real-world devices, for soft robotics [Tapia et al. 2020] and for scoliosis braces [Koutras et al., under review].
The project and the results are now completed, as discussed earlier.