Skip to main content

Amplifying Human Perception Through Interactive Digital Technologies

Periodic Reporting for period 3 - AMPLIFY (Amplifying Human Perception Through Interactive Digital Technologies)

Reporting period: 2019-06-01 to 2020-11-30

In the project AMPLIFY we research how digital technologies can enhance, augment, and amplify human perception. Over the last decade technical sensor systems have been created that are superior to human perception; e.g. cameras can be built that provide higher spatial and temporal resolutions than the human eye and that cover a wider spectral range. However, a human sense is much more than a sensor. The vision in the project is to lay a foundation for the creation of digital technologies that provide novel sensory experiences and new perceptual capabilities for humans. Hereby the central design goal is to create new and amplified artificial senses that are natural and intuitive to use. Augmenting, amplifying, and extending human perception measurably will open up new perspectives for human interaction with the environment and these advances have the potential to revolutionize human perception without creating information overload. The way we perceive our environment, what we see, hear, and feel, is fundamental to our experience and to the way we think. Amplified senses have the potential to make us perceptive of things we could not experience before and by this make us aware of new phenomena, ranging from environmental conditions, to relationships between people, or the bias of a piece of information.
The project has four main objectives: (1) Researching the Foundation for Implementing New Human Senses, (2) Researching the Foundation for Amplifying Human Senses, (3) Developing User Interfaces for Explicit and Implicit Control of Technical Amplified Perception, and (4) Developing Measures to Quantify the Effectiveness and Quality of Perceptual Amplification. Using a human centered and experimental approach that relies heavily on creating and studying functional prototypes we conduct research in human computer interaction and ubiquitous computing to achieve our goals. We employ an iterative design process with the main elements of inspiration, exploration, and validation.
At its core, the project is a research topic in computer science, but has strong interdisciplinary aspects. The key driver for innovation is in the idea of amplifying the human mind and improving human perception, which is at the intersection of understanding humans and creating new technologies. Inspiration is taken from human psychology and neuro science. For the technical exploration and in order to create fundamental and usable systems we require expertise in computer science and electronics. Our interdisciplinary research team includes scientists and students with backgrounds in neuro science, psychology, electronics, and computer science.
In the first part of the project, we had a stronger focus on inspiration and exploration, but also conducted initial experimental validations. Especially exploratory prototyping in a multidisciplinary setup has proven to be very successful and it has helped to create new insights that are contributing to the consolidated set of results. We focused our efforts around specific prototypical systems and conducted our process iteratively. To enhance and amplify human sensory experiences we were inspired by technical sensor systems, such as cameras that can capture a spectrum that is wider than visible light and high-speed cameras that can show movements which are invisible to the human eye as well as directional microphones which can pick up sounds at long distances. For new senses, we explored new opportunities for orientation and created prototypes to “see” the cognitive and mental state.
During the creation of the prototypes, we particularly looked at the feasibility of creating artificial human senses that provide new perceptual channels to the human mind, without increasing the experienced cognitive load of the person. Our focus was on creating intuitive and natural control mechanisms for amplified senses using eye gaze, muscle activity, and brain signals. Especially here, new approaches to neuro control are important.
So far, we have worked towards systematically researching and exploring new means of amplified perception through various prototypes. These prototypes have specific functions, but their main purpose is to (1) study the process of creating perceptual amplification for humans, and (2) to map out the technical design space systematically. In the coming period, we will build on this and assess and explore how to increase the human intake of information, how to create robust and reliable control mechanisms, and work towards the creation of artificial reflexes.
The specific prototypes have been widely published in the human computer interaction community as well as in the ubiquitous computing community. Through events and workshops, we have started to foster an international community that conducts research into the perceptual and cognitive augmentation and amplification of humans.
So far we have created a set of prototypes that show the feasibility of creating amplify senses. These prototypes include amplification of the visual and auditory sense as well as novel senses (e.g. for weather).
In the context of the project we have explored and prototyped several concepts and systems for implicit control based on physiological sensing, here in particular using gaze tracking, EEG and EMG.
Please see the set of publications that describe the results so far.
On the practical side, we expect that by the end of the project we can demonstrate several examples for sensory amplification and examples of new artificial reflexes. On the conceptual and theoretical side, we hope to create a deeper understanding of the feasibility and limitations for amplified human senses, a systematic design space for implicit control of sensors, and methods for developing and evaluating sensor amplifications.
User study with amplifyed directional hearing
Sketch of a envisioned body-word device
Sketch of a envisioned head-worn device
Prototype of a device to provide a sense of orientation while swimming
Prototype of an enhanced visual sense
Supporting orientation in open water