Foundations of Perception Engineering

Descripción del proyecto

Aplicación de principios de ingeniería para crear ilusiones perceptivas

La realidad virtual (RV) crea ilusiones perceptivas que pueden mejorar el teletrabajo, la educación en línea, la atención sanitaria e incluso las interacciones sociales, todo ello a distancia. En este contexto, el equipo del proyecto ILLUSIVE, financiado por el Consejo Europeo de Investigación, creará bases matemáticas y científicas para una disciplina de nueva aparición que los investigadores denominan «ingeniería de la percepción». El dato fundamental es que el objetivo son las ilusiones perceptivas en sí mismas, que deben desarrollarse utilizando los principios de ingeniería avalados con el paso del tiempo. El proyecto, que reúne a un equipo de expertos en RV, neurociencia, psicología perceptiva, robótica, teoría del control y matemáticas puras, empleará la noción de los «espacios de información», que se remonta al trabajo de John von Neumann sobre la teoría de juegos.

Objetivo

Virtual reality (VR) technology has enormous potential to transform society, especially as the world faces unprecedented challenges of remote work and social distancing brought on by COVID 19. Through telepresence it uniquely enhanced health care, collaborative design, education, and social interaction, all from a distance. Further benefits arise by incorporating synthetic models, which advance computer interfaces, data visualization, and immersive storytelling.

In spite of its potential, the VR industry has struggled through hype cycles as devices built from co-opted component technologies fail to meet the expectations of each generation. As an industry leader in Huawei and early Oculus VR, I concluded that the struggle is largely due to the lack of a rigorous foundation that draws from scientific disciplines of perceptual psychology, neuroscience, and human physiology. I propose to reduce the gap between engineering and these human-centered sciences by developing a new field of perception engineering. In this pioneering view, the object being engineered is the illusion itself, and the physical devices that achieve it are auxiliary.

This project focuses on developing mathematical foundations for VR that unify engineering concepts from robotics and control theory with their biological-system counterparts, especially predictive coding and the free energy principle. A cornerstone is von Neumann-Morgenstern information spaces, introduced for games with hidden information, which I have leveraged for fundamental problems in robotics throughout my research career. Our team will include experts from VR, neuroscience, perceptual psychology, robotics, control theory, and pure mathematics. The work will lay a foundation for principled engineering approaches to design, simulation, prediction, and analysis of sustained, targeted perceptual illusions. This would offer valuable guidance and deeper insights into VR, robotics, and possibly the sciences that study perception.