Periodic Reporting for period 5 - SenseX (Sensory Experiences for Interactive Technologies)
Período documentado: 2020-07-01 hasta 2021-05-31
Despite the fact that interactive technologies have permeated our environment (e.g. mobile, ubiquitous computing) and have become an essential part of our everyday life (e.g. work, leisure, education, health, etc.), the senses we call upon to interact with technology are still very limited relying mostly on visual and auditory senses. Given the immediacy of touch and the ubiquity of taste and smell and their importance to health, safety, leisure and work, and a persons emotional wellbeing, future multi-sensory experiences with interactive technologies can have a major impact on society and consumer markets creating entirely new product, technology, and service opportunities. More importantly, multi-sensory experience research can make a step-change in our understanding of the human senses as interaction modalities and revolutionize existing interaction paradigms within Human-Computer Interaction (HCI).
The grand challenge and vision of this project is to gain a rich and integrated understanding on touch, taste, and smell experiences for interactive technologies. We aim to achieve this ambitious grand vision by 1) creating a ‘sensory interaction framework’ on the bases of a systematic empirical investigation of touch, taste, and smell experiences, 2) integrating the generated understanding on the three senses into meaningful and efficient experiential cross-sensory gamuts and interaction principles, and 3) demonstrating the added value of the created experiential understanding on touch, taste, and smell – aka the experiential gamuts – through their integration into the development of multi-sensory systems verifying the short-, mid- and long-term societal and scientific impact (short-term: multi-sensory media experiences; mid-term: interaction concepts for partially sensory impaired people; long-term: multi-sensory interaction approach for life beyond Earth).
This research will pioneer novel interaction concepts for interactive technologies in relation to essential components of multisensory experiences. This project will transform existing interaction paradigm in Human-Computer Interaction (HCI) and likewise impact other disciplines such as sensory and cognitive sciences by delivering ground-breaking new insights on the experiential dimensions underlying neurological processes and human perception.
This ERC SenseX project contributed to this basic understanding of multisensory experiences and the design and future development of multisensory technologies. Those emerging technologies not just stimulate our eyes (think of screens) and ears (and audio systems), but also consider how and what we touch, smell, and taste, among others, in our lives. Multisensory digital technology enables the creation of multisensory experiences that enrich and augment the way we interact with the worlds around us.
This project has resulted in
- over 75 publications in peer-reviewed journals and conferences, that won several best paper and honourable mention awards (~ six awards)
- lead to the organisation of various international events including workshops, tutorials, panels
- a range of keynote talks by the PI in renowned international conferences, and invited talks to global leading research organisations and industries, including MIT, Stanford, Facebook/Oculus
- a successful University spin-out company (OWidgets Ltd) that has been supported by an initial ERC PoC 2017, and obtained a second PoC 2021
Above all, this project has truly advance our ability to understand, design, and develop multisensory interfaces in a broad range of application domains, from entertainment, immersive media, to automotive and healthcare applications. With July 2020, the PI has moved her research laboratory to a world-leading University, becoming Professor of Multisensory Interfaces at UCL (University College London), Department of Computer Science and was further selected as Deputy Director (Digital Health) for the UCL Institute of Healthcare Engineering. She is also a Visiting Professor at the Material Science Research Centre at the Royal College of Art in London and was a Visiting Professor at the HCI Engineering Group at MIT CSAIL in summer 2019.
Most recently, she published a book on ‘Multisensory Experiences: where the senses meet technology’, published by Oxford University Press as a popular science book; thus making the research of this project accessible to the wider public.