Skip to main content

Presence: Research Encompassing Sensory Enhancement, Neuroscience and Cognition, with Interactive Applications

Objective

This project aims to develop a theory of presence in virtual environments (VEs) based on empirical observations from neuroscience and other experimentation. The data will go far beyond the usual subjective questionnaires that have been at the root of presence research to date, and will include brain imaging studies, physiological measures, eye tracking, behaviour and subjective reporting. The second aim is to extend 'presence' beyond similarity of response between virtual and real stimuli, in order to exploit the capability of virtual reality for sensory enhancement and the plasticity of the brain in learning to carry out actions in new ways. This will enable carrying out of activity by 'thought'. Another application will be communications between remotely sited people sharing a VE. The goal is to portray not just the geometry of individuals, and their motor actions, but a range of other information such as empathic responses and physiological condition. This will explore how communication is affected by visualisation of as much as the total state of a person as is possible (and desirable). The impact of such sensory enhancement on presence will be explored. This project aims to develop a theory of presence in virtual environments (VEs) based on empirical observations from neuroscience and other experimentation. The data will go far beyond the usual subjective questionnaires that have been at the root of presence research to date, and will include brain imaging studies, physiological measures, eye tracking, behaviour and subjective reporting. The second aim is to extend 'presence' beyond similarity of response between virtual and real stimuli, in order to exploit the capability of virtual reality for sensory enhancement and the plasticity of the brain in learning to carry out actions in new ways. This will enable carrying out of activity by 'thought'. Another application will be communications between remotely sited people sharing a VE. The goal is to portray not just the geometry of individuals, and their motor actions, but a range of other information such as empathic responses and physiological condition. This will explore how communication is affected by visualisation of as much as the total state of a person as is possible (and desirable). The impact of such sensory enhancement on presence will be explored.

OBJECTIVES
a) A neural and physiological characterisation of presence: To execute psychophysiological and brain imaging studies that provide a characterisation of the physiological and neuronal signatures associated with switches between different presence states. The key goal is to implement fMRI experiments, using event-related designs, where the presence state (or switches in state of presence) is indexed by (i) phenomenological report from subjects (ii) a change in bodily state indexed by indepenent psychophysiogical markers;
b) Neurophysiological and behavioural studies: To execute electrophysiological and behavioural experiments, with the goal of understanding the pattern of electrical brain activity when there is a choice of behaviour under conditions of changing environmental cues. How does behaviour change, and what pattern of neuron firings are associated with these changes?
c) Neurophysiological based interaction methods: To develop a Human-Computer-interface based on a real-time neuro-physiological device to be constructed in the project that participants can consciously control and utilize to navigate through a virtual environment. The same device will be used to portray the neuro-physiological state of a people communicating in a shared VE;
d) Measurement techniques for presence: To construct robust measures of presence that can assess presence in relation to many different display and interaction media. The measurements techniques will follow from the theory to be developed during the project;
e) A theory of presence: To develop a theory of presence that characterises presence and attempts to explain how and why the presence state changes in response to changes in display and interaction parameters, and across different media, and including the role of perceptual and cognitive factors, including attention. This theory will be elaborated in response to real data accumulated in pursuance of the other and tested against the execution of experiments.

DESCRIPTION OF WORK
Workpackage 1 (WP1) Brain Imaging Studies will design functional neuroimaging experiments initially to test hypotheses relating presence and bodily state. Subsequently, the work will focus on implementing a series of functional neuroimaging experiments to describe the neuronal correlates of sense of presence.

WP2 Behavioural and Neurophysiological Studies: will carry out animal experiments that will allow us to understand the neurological basis of presence. This will be devoted to a detailed study of the cellular and network properties that may underlie presence.

WP3 Presence Theory and Experimentation: will involve experiments that attempt to understand the relationship between presence, immersion, minimal sensory cues. It will manipulate immersion variables, study changes in presence state, and test and develop the current state of presence theory as it evolves during the project. It will carry out experiments on doing by thinking, and sensory enhancement in communications.

WP4: Physiological recording system and HCI: will deliver a physiological recording and real-time monitoring device. This will be used both to gather data during the course of experimentation about the physiological state of experimental subjects, and also be used for biofeedback experiments where subjects learn to carry out actions by thought. This WP will provide the software, off-line and real-time data analysis associated with this device.

WP5 Haptics: will study how visual and haptics are integrated based on the idea that visual sensations often induce haptic sensations and vice-versa. An attempt will be made to exploit this interrelationship to maximise haptic sensations when there is incomplete sensory input. This will also study how co-presence and task performance is enhanced by haptics when remotely sited individuals cooperate in a shared VE over a network.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

UNIVERSITY COLLEGE LONDON
Address
Gower Street
WC1E 6BT London
United Kingdom

Participants (4)

GUGER TECHNOLOGIES OEG
Austria
Address
Plabutscherstrasse 63
8051 Graz
TECHNION - ISRAEL INSTITUTE OF TECHNOLOGY
Israel
Address
Technion City
32000 Haifa
TECHNISCHE UNIVERSITAET GRAZ
Austria
Address
Rechbauerstrasse 12
8010 Graz
UNIVERSIDAD MIGUEL HERNANDEZ DE ELCHE
Spain
Address
Avenida Del Ferrocarril S/n
03202 Elche