Periodic Reporting for period 4 - BRISC (Bounded Rationality in Sensorimotor Coordination)
Reporting period: 2021-04-01 to 2022-09-30
Machines can repeat the same movement a million times with relentless precision, while humans struggle even after years of practise to hit the bull’s-eye twice in a row. Compared to humans, machines seem to have superior resources ranging from more reliable hardware, better signal-to-noise ratios in their signal processing, less sensorimotor delays, and higher computation speed. Yet, despite their successful application in well-structured and predictable environments, machines are still blatantly outperformed by humans in unstructured and uncertain environments that require flexible behavior through real-time sensorimotor interactions like playing football or navigating a disaster zone. The goal of this project is to elucidate the effect of limited computational resources in biological sensorimotor processing and to investigate how control strategies that take such limitations into account might cope better in real-world scenarios with model uncertainty and computational constraints. In order to achieve this goal, the project has advanced a bounded rational framework for sensorimotor processing based on information-processing costs as the fundamental currency along three complementary objectives investigating predictions in single-agent, multi-agent and hierarchical motor control. In single-agent motor control, we could show how such a framework provides a bridge between Bayes-optimal and heuristic behavioural strategies in human sensorimotor decision-making and how human behaviour can be quantified in terms of information efficiency. Moreover, we could show how the stipulated information costs can be related to brain activity, which allows distinguishing different information processing paradigms. In multi-agent motor control, we could explain characteristic deviations from Nash equilibrium behaviour as well as adaptation paths to equilibrium, and develop novel paradigms for studying group coordination. In the study of hierarchical control systems, we found that hierarchically organized multi-expert systems with information constraints naturally partition the task space and specialize, which can be exploited for meta-learning and continuous learning. Across the three objectives, we were able to gather considerable experimental evidence of information-based bounded rationality in human sensorimotor processing and to show the merits of such constraints in hierarchical decision-making systems.
The bounded rational framework for sensorimotor processing advanced in this project can be conceptualized as a trade-off between task utility and information-processing costs, occurring at one or multiple information-processing nodes, that takes the shape of a free energy (Gottwald & Braun, PLoS Comp. Biol., 2020). We have tested this framework in the context of (A) single agent sensorimotor processing, (B) multi-agent sensorimotor processing and (C) hierarchical sensorimotor processing. The main results of these tests are:
(A) Bounded rationality in single agent sensorimotor processing
When quantifying the information efficiency in a number of sensorimotor tasks, we found human behaviour often close to bounded optimal, and variability pattern of actions characteristically adapted to prior information and permissible movement planning times [Schach, Gottwald, Braun, Frontiers Neurosci., 2018; Lindig-Leon, Gottwald, Braun, Frontiers Neurosci., 2019]. By adapting the degree of boundedness in our model, we could model transitions between Bayes-optimal behavior and heuristic strategies depending on available decision time (Lindig-Leon, Kaur, Braun, Frontiers Dec. Neurosci., 2022). Moreover, by relating information costs to brain activity in fMRI, we found that our framework could distinguish between different modi of information processing (Schach, Lindner, Braun, PLoS Comp. Biol., 2022), in this case providing evidence for parallel prospective motor planning when the motor system is faced with multiple options. Finally, by studying human visuomotor adaptation in changing environments, we could show characteristic hysteresis effects in quantitative agreement with our framework (Hack, Gottwald, Braun, Entropy, 2022; Hack, Lindig-Leon, Gottwald, Braun, arxiv, 2022).
(B) Bounded rationality in multi-agent sensorimotor processing
In sensorimotor experiments with haptically coupled human players we found characteristic deviations from theoretically optimal Nash equilibrium behaviour that can be explained by bounded optimality (Lindig-Leon, Schmid, Braun, Proc. Roy. Soc. B, 2021). Moreover, we could show that the learning process to reach this distorted equilibrium behaviour is itself also best understood by taking into account information constraints (Lindig-Leon, Schmid, Braun, Sci. Rep., 2021). When studying group coordination in sensorimotor tasks with more than two actors, we found that resource-efficient sampling-based Bayesian methods provided the best explanation for group learning (Schmid, Braun, Sci. Rep., 2020).
(C) Bounded rationality in hierarchical sensorimotor processing
In several studies, we could show how abstractions arise in simulated agents when introducing latent variables with information constraints (Peng, Genewein, Leibfried, Braun, IROS 2017; Hihn, Braun, CDC 2019). In particular, we found that hierarchical organizations are beneficial for decision systems that consist of multiple experts with information constraints and that such systems naturally partition the decision space among themselves and thereby specialize (Gottwald, Braun, Neural Comp., 2018; Hihn, Braun, Neural Inf. Proc. Lett., 2020). Crucially, this specialization can be harnessed for generalization and meta-learning, including continual learning in open-ended environments (Hihn, Braun, Mach. Learning, 2022).
(A) Bounded rationality in single agent sensorimotor processing
When quantifying the information efficiency in a number of sensorimotor tasks, we found human behaviour often close to bounded optimal, and variability pattern of actions characteristically adapted to prior information and permissible movement planning times [Schach, Gottwald, Braun, Frontiers Neurosci., 2018; Lindig-Leon, Gottwald, Braun, Frontiers Neurosci., 2019]. By adapting the degree of boundedness in our model, we could model transitions between Bayes-optimal behavior and heuristic strategies depending on available decision time (Lindig-Leon, Kaur, Braun, Frontiers Dec. Neurosci., 2022). Moreover, by relating information costs to brain activity in fMRI, we found that our framework could distinguish between different modi of information processing (Schach, Lindner, Braun, PLoS Comp. Biol., 2022), in this case providing evidence for parallel prospective motor planning when the motor system is faced with multiple options. Finally, by studying human visuomotor adaptation in changing environments, we could show characteristic hysteresis effects in quantitative agreement with our framework (Hack, Gottwald, Braun, Entropy, 2022; Hack, Lindig-Leon, Gottwald, Braun, arxiv, 2022).
(B) Bounded rationality in multi-agent sensorimotor processing
In sensorimotor experiments with haptically coupled human players we found characteristic deviations from theoretically optimal Nash equilibrium behaviour that can be explained by bounded optimality (Lindig-Leon, Schmid, Braun, Proc. Roy. Soc. B, 2021). Moreover, we could show that the learning process to reach this distorted equilibrium behaviour is itself also best understood by taking into account information constraints (Lindig-Leon, Schmid, Braun, Sci. Rep., 2021). When studying group coordination in sensorimotor tasks with more than two actors, we found that resource-efficient sampling-based Bayesian methods provided the best explanation for group learning (Schmid, Braun, Sci. Rep., 2020).
(C) Bounded rationality in hierarchical sensorimotor processing
In several studies, we could show how abstractions arise in simulated agents when introducing latent variables with information constraints (Peng, Genewein, Leibfried, Braun, IROS 2017; Hihn, Braun, CDC 2019). In particular, we found that hierarchical organizations are beneficial for decision systems that consist of multiple experts with information constraints and that such systems naturally partition the decision space among themselves and thereby specialize (Gottwald, Braun, Neural Comp., 2018; Hihn, Braun, Neural Inf. Proc. Lett., 2020). Crucially, this specialization can be harnessed for generalization and meta-learning, including continual learning in open-ended environments (Hihn, Braun, Mach. Learning, 2022).
Research on sensorimotor processing has largely relied on Bayes-optimal models both for studying human behaviour in laboratory tasks as well as for designing artificial control systems. In this project we go beyond Bayes-optimal models and advance a bounded rational framework with information constraints for the analysis of sensorimotor systems. Such a framework based on information constraints is closely related in its structure to thermodynamic models that trade off energy and entropy, and we could show experimentally that this analogy carries over to non-equilibrium systems, thus allowing the study of learning systems. By considering the level of available information-processing resources, this framework goes beyond the state-of-the-art in that it can form a bridge between Bayes-optimal and heuristic models of sensorimotor decision-making. This also includes a new methodology for analysing brain activity and behaviour together, as the bounded rational analysis allows relating behaviour and brain activity to a normative baseline that takes information-processing constraints into account. As the impact of limited resources is of particular significance in multi-agent systems, we advance the state-of-the-art in this area, by studying a new paradigm for sensorimotor coordination in groups, as well as theoretical investigations that address the catastrophic forgetting problem by studying mixture-of-expert models with information-constraints that naturally partition the task space for meta-learning and continual learning.