Periodic Reporting for period 3 - HOWPER (An open or closed process: Determining the global scheme of perception) Reporting period: 2021-06-01 to 2022-11-30 Summary of the context and overall objectives of the project The HOWPER project addresses the question of perception – what does it mean for a brain (any brain) to perceive. That is, what are the processes involved, and what determines what is perceived and when in a given active brain. Answering these questions is important primarily in two domains. In one, understanding how perception is typically achieved will help devising methods to improve perception of impaired people, such as blind, deaf or other sensory or cognitive impaired individuals. In the other, such understanding will be instrumental in constructing efficient artificial perceivers for use in autonomic or semi-autonomic vehicles or robots. The major objective of the project is to determine whether perception emerges in the human brain in a closed- or open-loop manner (two alternative hypotheses of perception). To this end, we have defined and formulated the contrasting schemes, derived discriminative testable predictions, and are testing them to find which scheme is consistent with the empirical data and which is not. Secondary objectives are to improve current visual-to-tactile sensory substitution approaches for the visually impaired and to develop novel biomimetic algorithms for autonomous robotic agents. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far 1. Testing open- and closed-loop schemes of perception with human vision. Using eye tracking and gaze-contingent displays we have found that human vision follows the predictions of closed-loop perception and not those of open-loop perception. Part of WP2.2. Analyzing the peripheral coding scheme of human vision. Using eye tracking, near-threshold vision and simulations of retinal dynamics we have found that the while visual perception cannot be explained based on the image details or eye movements alone, it can be explained by the simulated retinal activation that is generated by the convolution of both. Part of WP2.3. Examining the efficiency and strategies of use of an active-sensing based sensory substitution device (ASenSub). We have found that blind and blindfolded sighted participants learn and perceive through ASenSub significantly faster than previously reported with passive-sensing based devices. Their strategies resembled natural tactile and visual strategies, respectively. Part of WP5.4. Analyzing perceptual behavior of freely moving rodents. We have found that perceptual behavior is sensitive to the available degrees of freedom in moving the sensory organs. Thus, constraining head motion results in redirecting attention from explorative to defensive mode, and compensating with larger whisker movements. Part of WP4.5. Studying sensory processing in a rat-machine hybrid. We have found that sophisticated sensory processing, including complex interactions between whisker motion and touch, occurs already at the brainstem. Part of WP4.6. Mimicking a fast reflex loop in a rat-machine hybrid. We have found that a closed-loop touch reflex, termed touch-induced pumping (TIP), can be generated by a simple brainstem feedback loop and the mechano-elastic properties of the sensory organ. Part of WP4.7. Analyzing strategies of human touch. We have found that human individuals employ stereotypical tactile scanning strategies with idiosyncratic characteristics that are matched with their sensory thresholds (spatial resolution and temporal adaptation). Part of WP2. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The results obtained so far extend the state of the art in the field of perception. Together, they provide a significant support for the closed-loop scheme of human perception, and provide quantitative descriptions of the underlying processes. We expect that until the end of the project we will be able to provide a definitive answer for the major question addressed in this project: does human tactile and visual perception function within an open-loop or closed-loop scheme? The answer will be accompanied with a formal theoretical description, a synthetic demonstration, and dynamical characterization of the biological processes. In addition, an efficient sensory substitution device for the blind will be described in detail. Related documents ASenSub cover4.jpg
