Skip to main content

Levitation with localised tactile and audio feedback for mid-air interactions

Deliverables

Demonstrator v1 illustrating interaction techniques for atomic selection action

Demonstrator v1 illustrating interaction techniques for atomic selection action

Demonstrator v3 presenting two applications combining all modalities

Demonstrator v3 presenting two applications combining all modalities

A working prototype of the transducer array with haptics + audio + visual

The software API concept will be taken from Python toolbox D12 requirements regarding functionality will be identified in WP3 and WP4

"Organization of demonstrators and workshops as part of outreach activities #1"

Organization of demonstrators and workshops as part of outreach activities

"Organization of demonstrators and workshops as part of outreach activities #3"

Organization of demonstrators and workshops as part of outreach activities 3

Demonstrator v2 illustrating interaction techniques for positioning, rotation, scaling and deformation

Demonstrator v2 illustrating interaction techniques for positioning, rotation, scaling and deformation

"Organization of demonstrators and workshops as part of outreach activities #4"

"Organization of demonstrators and workshops as part of outreach activities #4"

Prototype with levitation + haptics + visual

requirements for this identified in WP3 and WP4

"Organization of demonstrators and workshops as part of outreach activities #2"

Organization of demonstrators and workshops as part of outreach activities 2

Prototype with all modalities

API concept taken from D1.2; requirements identified in WP3 and WP4

Data managment plan

plan on how we will manage data generated from the project

Report on multisensory integration, including ventriloquism and interaction techniques

Report on multisensory integration, including ventriloquism eddefct

Database of broadband radiation characteristics across multiple transducer types

Database of broadband radiation characteristics across multiple transducer types

Evaluation report on final demonstrators

Evaluation report on demonstrators

Catalogue of effective haptic experiences, together with guidelines on how to create them

Catalogue of effective haptic experiences together with guidelines on how to create them

Signal processing model for de-modulating non-linearity for a range of operating points

Signal processing model for de-modulating non-linearity for a range of operating points

Python toolbox of sound field/beamformer design methods

This will support the full functionality and API of the realtime implementations in WP2

Python tool box - final version

Python tool box - final version

Publications

HaptiRead - Reading Braille as Mid-Air Haptic Information

Author(s): Viktorija Paneva, Sofia Seinfeld, Michael Kraiczi, Jörg Müller
Published in: Proceedings of the 2020 ACM Designing Interactive Systems Conference, 2020, Page(s) 13-20, ISBN 9781450369749
Publisher: ACM
DOI: 10.1145/3357236.3395515

Measurement-Based Modeling of Higher-Order Non-Linearities of the Parametric Loudspeaker

Author(s): Carl Andersson, Jens Ahrens
Published in: 'Fortschritte der Akustik - DAGA 2019, 2019, 2019, Page(s) 330-333, ISBN 978-3-939296-14-0
Publisher: DEGA

Levitation Simulator: Prototyping Ultrasonic Levitation Interfaces in Virtual Reality

Author(s): Viktorija Paneva, Myroslav Bachynskyi, Jörg Müller
Published in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, Page(s) 1-12, ISBN 9781450367080
Publisher: ACM
DOI: 10.1145/3313831.3376409

UltraPower: Powering Tangible & Wearable Devices with Focused Ultrasound

Author(s): Rafael Morales González, Asier Marzo, Euan Freeman, William Frier, Orestis Georgiou
Published in: Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, 2021, Page(s) 1-13, ISBN 9781450382137
Publisher: ACM
DOI: 10.1145/3430524.3440620

Floating Widgets - Interaction with Acoustically-Levitated Widgets

Author(s): Euan Freeman, Ross Anderson, Carl Andersson, Julie Williamson, Stephen Brewster
Published in: Proceedings of the Interactive Surfaces and Spaces on ZZZ - ISS '17, 2017, Page(s) 417-420, ISBN 9781-450346917
Publisher: ACM Press
DOI: 10.1145/3132272.3132294

Levitate - interaction with floating particle displays

Author(s): Julie R. Williamson, Euan Freeman, Stephen Brewster
Published in: Proceedings of the 6th ACM International Symposium on Pervasive Displays - PerDis '17, 2017, Page(s) 1-2, ISBN 9781-450350457
Publisher: ACM Press
DOI: 10.1145/3078810.3084347

TastyFloats - A Contactless Food Delivery System

Author(s): Chi Thanh Vi, Asier Marzo, Damien Ablart, Gianluca Memoli, Sriram Subramanian, Bruce Drinkwater, Marianna Obrist
Published in: Proceedings of the Interactive Surfaces and Spaces on ZZZ - ISS '17, 2017, Page(s) 161-170, ISBN 9781-450346917
Publisher: ACM Press
DOI: 10.1145/3132272.3134123

Textured surfaces for ultrasound haptic displays

Author(s): Euan Freeman, Ross Anderson, Julie Williamson, Graham Wilson, Stephen A. Brewster
Published in: Proceedings of the 19th ACM International Conference on Multimodal Interaction - ICMI 2017, 2017, Page(s) 491-492, ISBN 9781-450355438
Publisher: ACM Press
DOI: 10.1145/3136755.3143020

Point-and-Shake - Selecting from Levitating Object Displays

Author(s): Euan Freeman, Julie Williamson, Sriram Subramanian, Stephen Brewster
Published in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 2018, Page(s) 1-10, ISBN 9781-450356206
Publisher: ACM Press
DOI: 10.1145/3173574.3173592

Touchless Tactile Displays for Digital Signage - Mid-air Haptics meets Large Screens

Author(s): Loic Corenthy, Sriram Subramanian, Marcello Giordano, Richard Hayden, Daniel Griffiths, Craig Jeffrey, Hannah Limerick, Orestis Georgiou, Tom Carter, Jörg Müller
Published in: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 2018, Page(s) 1-4, ISBN 9781-450356213
Publisher: ACM Press
DOI: 10.1145/3170427.3186533

A Method for Simultaneous Creation of an Acoustic Trap and a Quiet Zone

Author(s): Carl Andersson, Jens Ahrens
Published in: 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), 2018, Page(s) 622-626, ISBN 978-1-5386-4752-3
Publisher: IEEE
DOI: 10.1109/sam.2018.8448949

Levitating Particle Displays with Interactive Voxels

Author(s): Euan Freeman, Julie R. Williamson, Praxitelis Kourtelos, Stephen Brewster
Published in: Proceedings of the 7th ACM International Symposium on Pervasive Displays - PerDis '18, 2018, Page(s) 1-6, ISBN 9781-450357654
Publisher: ACM Press
DOI: 10.1145/3205873.3205878

Pulsed schlieren imaging of ultrasonic haptics and levitation using phased arrays

Author(s): Michele Iodice, William Frier, William Frier, James Wilcox, Ben Long, Orestis Georgiou
Published in: Proceedings of the ICSV 2018, 2018, 2018
Publisher: Elsevier / SCOPUS

LeviCursor - Dexterous Interaction with a Levitating Object

Author(s): Myroslav Bachynskyi, Viktorija Paneva, Jörg Müller
Published in: Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces - ISS '18, 2018, Page(s) 253-262, ISBN 9781-450356947
Publisher: ACM Press
DOI: 10.1145/3279778.3279802

Touchless Haptic Feedback for VR Rhythm Games

Author(s): Orestis Georgiou, Craig Jeffrey, Ziyuan Chen, Bao Xiao Tong, Shing Hei Chan, Boyin Yang, Adam Harwood, Tom Carter
Published in: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018, Page(s) 553-554, ISBN 978-1-5386-3365-6
Publisher: IEEE
DOI: 10.1109/vr.2018.8446619

Fibonacci Spiral Arranged Ultrasound Phased Array for Mid-Air Haptics

Author(s): Adam Price, Benjamin Long
Published in: 2018 IEEE International Ultrasonics Symposium (IUS), 2018, Page(s) 1-4, ISBN 978-1-5386-3425-7
Publisher: IEEE
DOI: 10.1109/ultsym.2018.8580072

SoundBender - Dynamic Acoustic Control Behind Obstacles

Author(s): Mohd Adili Norasikin, Diego Martinez Plasencia, Spyros Polychronopoulos, Gianluca Memoli, Yutaka Tokuda, Sriram Subramanian
Published in: The 31st Annual ACM Symposium on User Interface Software and Technology - UIST '18, 2018, 2018, Page(s) 247-259, ISBN 9781-450359481
Publisher: ACM Press
DOI: 10.1145/3242587.3242590

Acoustic Levitation from Superposition of Spherical Harmonics Expansions of Elementary Sources: Analysis of Dependency on Wavenumber and Order

Author(s): Carl Andersson, Jens Ahrens
Published in: 2019 IEEE International Ultrasonics Symposium (IUS), 2019, 2019, Page(s) 920-923, ISBN 978-1-7281-4596-9
Publisher: IEEE
DOI: 10.1109/ultsym.2019.8926167

Enhancing physical objects with actuated levitating particles

Author(s): Euan Freeman, Asier Marzo, Praxitelis B. Kourtelos, Julie R. Williamson, Stephen Brewster
Published in: Proceedings of the 8th ACM International Symposium on Pervasive Displays, 2019, 2019, Page(s) 1-7, ISBN 9781-450367516
Publisher: ACM
DOI: 10.1145/3321335.3324939

HaptiGlow: Helping Users Position their Hands for Better Mid-Air Gestures and Ultrasound Haptic Feedback

Author(s): Euan Freeman, Dong-Bach Vo, Stephen Brewster
Published in: 2019 IEEE World Haptics Conference (WHC), 2019, 2019, Page(s) 289-294, ISBN 978-1-5386-9461-9
Publisher: IEEE
DOI: 10.1109/whc.2019.8816092

Sampling Strategy for Ultrasonic Mid-Air Haptics

Author(s): William Frier, Dario Pittera, Damien Ablart, Marianna Obrist, Sriram Subramanian
Published in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, 2019, 2019, Page(s) 1-11, ISBN 9781-450359702
Publisher: ACM Press
DOI: 10.1145/3290605.3300351

Three-in-one - Levitation, Parametric Audio, and Mid-Air Haptic Feedback

Author(s): Gözel Shakeri, Euan Freeman, William Frier, Michele Iodice, Benjamin Long, Orestis Georgiou, Carl Andersson
Published in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA '19, 2019, 2019, Page(s) 1-4, ISBN 9781-450359719
Publisher: ACM Press
DOI: 10.1145/3290607.3313264

User engagement for mid-air haptic interactions with digital signage

Author(s): Hannah Limerick, Richard Hayden, David Beattie, Orestis Georgiou, Jörg Müller
Published in: Proceedings of the 8th ACM International Symposium on Pervasive Displays, 2019, 2019, Page(s) 1-7, ISBN 9781-450367516
Publisher: ACM
DOI: 10.1145/3321335.3324944

Using Ultrasonic Mid-air Haptic Patterns in Multi-Modal User Experiences

Author(s): Damien Ablart, William Frier, Hannah Limerick, Orestis Georgiou, Marianna Obrist
Published in: 2019 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), 2019, 2019, Page(s) 1-6, ISBN 978-1-7281-2355-4
Publisher: IEEE
DOI: 10.1109/have.2019.8920969

LeviProps - Animating Levitated Optimized Fabric Structures using Holographic Acoustic Tweezers

Author(s): Rafael Morales, Asier Marzo, Sriram Subramanian, Diego Martínez
Published in: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, 2019, Page(s) 651-661, ISBN 9781-450368162
Publisher: ACM
DOI: 10.1145/3332165.3347882

Avoiding Collisions when Interacting with Levitating Particle Displays

Author(s): Maxime Reynal, Euan Freeman, Stephen Brewster
Published in: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, Page(s) 1-7, ISBN 9781450368193
Publisher: ACM
DOI: 10.1145/3334480.3382965

Minimum trap separation for acoustical levitation using phased ultrasonic transducer arrays

Author(s): Carl Andersson, Jens Ahrens
Published in: Proceedings of ICA, 2019, 2019, Page(s) 1117-1123, ISBN 978-3-939296-15-7
Publisher: DEGA
DOI: 10.18154/rwth-conv-239242

Levi-loop: A Mid-Air Gesture Controlled Levitating Particle Game

Author(s): Rafael Morales González, Euan Freeman, Orestis Georgiou
Published in: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, Page(s) 1-4, ISBN 9781450368193
Publisher: ACM
DOI: 10.1145/3334480.3383152

Reducing Spiraling in Transducer Array Based Acoustic Levitation

Author(s): Carl Andersson, Jens Ahrens
Published in: 2020 IEEE International Ultrasonics Symposium (IUS), 2020, Page(s) 1-4, ISBN 978-1-7281-5448-0
Publisher: IEEE
DOI: 10.1109/ius46767.2020.9251489

Impact of Information Placement and User Representations in VR on Performance and Embodiment

Author(s): Sofia Seinfeld, Tiare Feuchtner, Johannes Pinzek, Jorg Muller
Published in: IEEE Transactions on Visualization and Computer Graphics, 2020, Page(s) 1-1, ISSN 1077-2626
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tvcg.2020.3021342

Acoustic levitation with optimized reflective metamaterials

Author(s): Spyros Polychronopoulos, Gianluca Memoli
Published in: Scientific Reports, 10/1, 2020, ISSN 2045-2322
Publisher: Nature Publishing Group
DOI: 10.1038/s41598-020-60978-4

Impact of visuomotor feedback on the embodiment of virtual hands detached from the body

Author(s): Sofia Seinfeld, Jörg Müller
Published in: Scientific Reports, 10/1, 2020, ISSN 2045-2322
Publisher: Nature Publishing Group
DOI: 10.1038/s41598-020-79255-5

User Representations in Human-Computer Interaction

Author(s): Sofia Seinfeld, Tiare Feuchtner, Antonella Maselli, Jörg Müller
Published in: Human–Computer Interaction, 2020, 2020, Page(s) 1-39, ISSN 0737-0024
Publisher: Lawrence Erlbaum Associates Inc.
DOI: 10.1080/07370024.2020.1724790

A volumetric display for visual, tactile and audio presentation using acoustic trapping

Author(s): Ryuji Hirayama, Diego Martinez Plasencia, Nobuyuki Masuda, Sriram Subramanian
Published in: Nature, 575/7782, 2019, Page(s) 320-323, ISSN 0028-0836
Publisher: Nature Publishing Group
DOI: 10.1038/s41586-019-1739-5

GS-PAT

Author(s): Diego Martinez Plasencia, Ryuji Hirayama, Roberto Montano-Murillo, Sriram Subramanian
Published in: ACM Transactions on Graphics, 39/4, 2020, ISSN 0730-0301
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3386569.3392492

Using Spatiotemporal Modulation to Draw Tactile Patterns in Mid-Air

Author(s): William Frier, Damien Ablart, Jamie Chilles, Benjamin Long, Marcello Giordano, Marianna Obrist, Sriram Subramanian
Published in: Haptics: Science, Technology, and Applications, 10893, 2018, Page(s) 270-281, ISBN 978-3-319-93444-0
Publisher: Springer International Publishing
DOI: 10.1007/978-3-319-93445-7_24

Datasets

Database Of Ultrasonic Transducer Radiation Characteristics

Author(s): Andersson, Carl; Ahrens, Jens
Published in: Zenodo

User Study Data from "Point-and-Shake: Selecting from Levitating Object Displays"

Author(s): Freeman, Euan; Williamson, Julie; Subramanian, Sriram; Brewster, Stephen
Published in: Zenodo