Periodic Reporting for period 3 - TOUCHLESS (Touchless Haptic Experiences with Neurocognitive AI)
Reporting period: 2023-07-01 to 2024-12-31
The COVID-19 pandemic highlighted the profound importance of physical touch for human well-being and the challenges posed by its absence in digital interactions. Despite advances in haptic technology, replicating the wealth of touch sensations possible—essential for emotional connection and care—remains a significant hurdle. TOUCHLESS aimed to address this by developing next-generation touchless haptic technologies capable of evoking tactile sensations without physical contact, and creating intelligent algorithms that know when and how to apply touchless touch. To that end, by blending a mix of innovative hardware and neurocognitive models of experience, TOUCHLESS has created systems that stimulate touch receptors remotely, inducing not only functional but also soothing, affective, and social experiences. Through these explorations this project has developed technologies that open pathways to new, immersive digital interactions, that can enrich emotional connections and enhancing the quality of virtual engagements across various domains. Importantly however, the project has also identified timely questions around responsible and ethical haptic AI technology development, thus initiating a dialogue around such topics.
WP1: TOUCHLESS has significantly advanced ultrasonic phased arrays for haptic stimulation, publishing comprehensive guides for system development. Achievements include designing and testing a large array for mid-air sensations, pioneering simulation techniques like PhantomFields for acoustic holography, and conducting foundational studies such as PiloNape on electrostatic stimulation. The Sparkitect demonstration showcased ultrasonic-guided high-voltage sparks with potential applications like mid-air Braille. Thermal, infrasound, and multimodal haptics saw XR and VR-integrated experiences culminating in the Portal experience, integrating diverse technologies.
WP2: Quantitative studies on mid-air haptics characterised the spatial and temporal precision of tactile perception, confirming the ability to selectively stimulate different tactile receptors via ultrasonic vibrations. These findings contribute a robust evidence base for designing functional touch experiences. In addition, foundational work on neurocognitive models linked sensory inputs to high-level experiences like agency and bonding, with significant results published in leading journals.
WP3: A multimodal sensing platform integrated touchless sensors to monitor human states in real time through machine learning analysis of biosignals and body movements. Highlights also included a hybrid AI-powered text-to-haptics demonstrator translating textual emotions into synchronized mid-air haptic, audio, and visual AI generated effects. Validated through user studies, this system showed potential in storytelling, mindfulness, and communication.
WP4: WP4 created a rich touch library and novel interaction techniques, enabling immersive applications like mid-air gestures and multisensory storytelling. Multiple demonstrators—such as Feelustrator, Knuckles Notifications, and AcoustoBots—showcased the technology’s versatility across sectors, including automotive, gaming, and healthcare. These efforts culminated in significant publications and presentations at top-tier venues, such as ACM CHI, UIST, and IEEE ToH.
WP5: This work package developed tailored Responsible Research and Innovation (RRI) training, blending generic frameworks with specific challenges in haptics and body-centric technology. Ethical insights included a taxonomy of tactile devices, considerations for sensory autonomy, and frameworks for addressing power asymmetries in social haptics. Industry partners incorporated these findings into their practices, and public resources like the "Ethics of Digital Touch" flyer and web materials extended the project's reach.
WP6: Dissemination and outreach activities flourished, with live updates on the project website and significant engagement through the Haptics Helix innovation platform, which grew to include over 130 organisations and experts. TOUCHLESS showcased its innovations at major academic and industrial events, including CES, WebSummit, and EuroHaptics, while organising impactful workshops on AI, ethics, and XR. The video demonstrators and editorial special issues further amplified the project's reach, fostering cross-sector collaboration and public engagement.
Overall, TOUCHLESS has made ground-breaking contributions across its WPs, addressing core challenges in touchless haptics and delivering tools, technologies, and insights that will shape the future of human-computer interaction.
WP2: Quantitative studies on mid-air haptics characterised the spatial and temporal precision of tactile perception, confirming the ability to selectively stimulate different tactile receptors via ultrasonic vibrations. These findings contribute a robust evidence base for designing functional touch experiences. In addition, foundational work on neurocognitive models linked sensory inputs to high-level experiences like agency and bonding, with significant results published in leading journals.
WP3: A multimodal sensing platform integrated touchless sensors to monitor human states in real time through machine learning analysis of biosignals and body movements. Highlights also included a hybrid AI-powered text-to-haptics demonstrator translating textual emotions into synchronized mid-air haptic, audio, and visual AI generated effects. Validated through user studies, this system showed potential in storytelling, mindfulness, and communication.
WP4: WP4 created a rich touch library and novel interaction techniques, enabling immersive applications like mid-air gestures and multisensory storytelling. Multiple demonstrators—such as Feelustrator, Knuckles Notifications, and AcoustoBots—showcased the technology’s versatility across sectors, including automotive, gaming, and healthcare. These efforts culminated in significant publications and presentations at top-tier venues, such as ACM CHI, UIST, and IEEE ToH.
WP5: This work package developed tailored Responsible Research and Innovation (RRI) training, blending generic frameworks with specific challenges in haptics and body-centric technology. Ethical insights included a taxonomy of tactile devices, considerations for sensory autonomy, and frameworks for addressing power asymmetries in social haptics. Industry partners incorporated these findings into their practices, and public resources like the "Ethics of Digital Touch" flyer and web materials extended the project's reach.
WP6: Dissemination and outreach activities flourished, with live updates on the project website and significant engagement through the Haptics Helix innovation platform, which grew to include over 130 organisations and experts. TOUCHLESS showcased its innovations at major academic and industrial events, including CES, WebSummit, and EuroHaptics, while organising impactful workshops on AI, ethics, and XR. The video demonstrators and editorial special issues further amplified the project's reach, fostering cross-sector collaboration and public engagement.
Overall, TOUCHLESS has made ground-breaking contributions across its WPs, addressing core challenges in touchless haptics and delivering tools, technologies, and insights that will shape the future of human-computer interaction.
TOUCHLESS has advanced the integration of hardware, psychophysics, neuroscience, AI, human-computer interaction, and ethical innovation, pushing the boundaries of touchless haptic systems. By uniting tactile technology with emotional experience, the project demonstrated how mid-air haptics and AI-driven adaptive systems can go beyond functional notifications in a new branch of affective computing.
Key technological advancements include the development of sophisticated mid-air haptics for delivering nuanced tactile sensations across body regions like the forearm, palm, and face. These were complemented by AI-driven systems that utilize multimodal sensing and neurocognitive models to adapt feedback dynamically based on user context, preferences, and emotional states. This combination allows applications in gaming, education, healthcare, and automotive systems to deliver more intuitive, user-friendly interactions.
The project also addressed high-level challenges like enhancing emotional engagement, user trust, and agency through tactile feedback. For instance, in automotive contexts, mid-air haptics improved driver awareness and interaction with infotainment systems. Similarly, in extended reality (XR) environments, TOUCHLESS demonstrated how multisensory integration could create more immersive and personalized experiences by combining tactile, visual, and auditory feedback.
An important contribution of TOUCHLESS is its ethics framework for touchless technologies, which offers actionable guidelines for addressing privacy, informed consent, and user autonomy in tactile interaction. This framework ensures that innovations in touchless haptics are designed responsibly, aligning with societal values and regulatory requirements, such as the EU AI Act.
The potential impacts of TOUCHLESS extend into multiple sectors. The project has catalyzed opportunities for European innovation by equipping researchers and companies with tools like the Style Touch Library and AI-based tactile modulation engines. These tools also serve as platforms for further research and development.
TOUCHLESS has thus not only delivered cutting-edge technologies but also laid the groundwork for responsible exploitation and widespread adoption. By tackling technical, ethical, and application challenges, the project has established touchless haptics as a transformative solution for enhancing digital interactions across industries.
Key technological advancements include the development of sophisticated mid-air haptics for delivering nuanced tactile sensations across body regions like the forearm, palm, and face. These were complemented by AI-driven systems that utilize multimodal sensing and neurocognitive models to adapt feedback dynamically based on user context, preferences, and emotional states. This combination allows applications in gaming, education, healthcare, and automotive systems to deliver more intuitive, user-friendly interactions.
The project also addressed high-level challenges like enhancing emotional engagement, user trust, and agency through tactile feedback. For instance, in automotive contexts, mid-air haptics improved driver awareness and interaction with infotainment systems. Similarly, in extended reality (XR) environments, TOUCHLESS demonstrated how multisensory integration could create more immersive and personalized experiences by combining tactile, visual, and auditory feedback.
An important contribution of TOUCHLESS is its ethics framework for touchless technologies, which offers actionable guidelines for addressing privacy, informed consent, and user autonomy in tactile interaction. This framework ensures that innovations in touchless haptics are designed responsibly, aligning with societal values and regulatory requirements, such as the EU AI Act.
The potential impacts of TOUCHLESS extend into multiple sectors. The project has catalyzed opportunities for European innovation by equipping researchers and companies with tools like the Style Touch Library and AI-based tactile modulation engines. These tools also serve as platforms for further research and development.
TOUCHLESS has thus not only delivered cutting-edge technologies but also laid the groundwork for responsible exploitation and widespread adoption. By tackling technical, ethical, and application challenges, the project has established touchless haptics as a transformative solution for enhancing digital interactions across industries.