Periodic Reporting for period 1 - BASE (Biologically-inspired Autonomous Systems for Space Exploration)
Periodo di rendicontazione: 2024-09-01 al 2026-08-31
Sintesi del contesto e degli obiettivi generali del progetto
Artificial intelligence has become an integral part of our lifes, but it comes with an increasingly unsustainable cost of resources. In contrast, the mammalian brain, which was the main inspiration behind modern artificial intelligence, is highly energy efficient, consuming only around 20W - far less than even a single, modern GPU.
BASE (Biologically-inspired Autonomous Systems for Space Exploration) investigated two properties of biological systems that could help make future artificial intelligence systems more efficient. First, the cells that make up our brain ("neurons") communicate with each other using brief electrical pulses, so-called "spikes". Most of the time neurons remain relatively inactive and only emit a spike when sufficient input has been received from other neurons. This is fundamentally different from many conventional artificial neural networks, where large numbers of operations are performed continuously. Artificial spiking neural networks aim to exploit this sparse, event-driven form of computation and could therefore provide a route towards more energy-efficient artificial intelligence. Second, biological systems operate in a highly distributed manner. Information processing and memory are not separated into a central processor and a separate memory unit as in conventional computers. Instead, individual components process information that is available to them locally, while useful behaviour emerges from their interaction. Similar principles can be used to design artificial systems in which many simple components collectively reorganise to achieve a given objective.
BASE explored these ideas from two complementary directions. The first objective was to understand the computational capabilities and limitations of spiking neural networks. New mathematical tools were developed to study how well such networks can learn and approximate functions, and how the number and timing of spikes affect their computational capabilities. This contributes to establishing a theoretical foundation for understanding the potential and limitations of computing with event-based spikes. The second objective was to investigate how modular systems can autonomously reorganise themselves to attain a desired shape or functionality. This included methods for controlling ensembles of moving robotic cubes using only locally available information, as well as methods for continuously reconfiguring mechanical lattice structures. Such systems could eventually adapt to damage, repair themselves or change their properties depending on their current task.
Space applications provided a particularly demanding test case for both research directions. Spacecraft have strictly limited energy and computing resources, while communication delays can make continuous control from Earth impossible during deep-space missions. Future spacecraft therefore need increasingly capable onboard artificial intelligence that can make decisions autonomously and efficiently. At the same time, reconfigurable structures could enable future systems that assemble large telescopes or other infrastructure in space, adapt their configuration to different tasks, or recover functionality after damage.
BASE (Biologically-inspired Autonomous Systems for Space Exploration) investigated two properties of biological systems that could help make future artificial intelligence systems more efficient. First, the cells that make up our brain ("neurons") communicate with each other using brief electrical pulses, so-called "spikes". Most of the time neurons remain relatively inactive and only emit a spike when sufficient input has been received from other neurons. This is fundamentally different from many conventional artificial neural networks, where large numbers of operations are performed continuously. Artificial spiking neural networks aim to exploit this sparse, event-driven form of computation and could therefore provide a route towards more energy-efficient artificial intelligence. Second, biological systems operate in a highly distributed manner. Information processing and memory are not separated into a central processor and a separate memory unit as in conventional computers. Instead, individual components process information that is available to them locally, while useful behaviour emerges from their interaction. Similar principles can be used to design artificial systems in which many simple components collectively reorganise to achieve a given objective.
BASE explored these ideas from two complementary directions. The first objective was to understand the computational capabilities and limitations of spiking neural networks. New mathematical tools were developed to study how well such networks can learn and approximate functions, and how the number and timing of spikes affect their computational capabilities. This contributes to establishing a theoretical foundation for understanding the potential and limitations of computing with event-based spikes. The second objective was to investigate how modular systems can autonomously reorganise themselves to attain a desired shape or functionality. This included methods for controlling ensembles of moving robotic cubes using only locally available information, as well as methods for continuously reconfiguring mechanical lattice structures. Such systems could eventually adapt to damage, repair themselves or change their properties depending on their current task.
Space applications provided a particularly demanding test case for both research directions. Spacecraft have strictly limited energy and computing resources, while communication delays can make continuous control from Earth impossible during deep-space missions. Future spacecraft therefore need increasingly capable onboard artificial intelligence that can make decisions autonomously and efficiently. At the same time, reconfigurable structures could enable future systems that assemble large telescopes or other infrastructure in space, adapt their configuration to different tasks, or recover functionality after damage.
Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti
BASE was composed of two main research parts: developing theory for spiking neural networks, and developing methods for continuously reorganising mechanical structures to attain a target shape or function.
For the first part, we developed novel approaches for bounding the generalisation and approximation errors of spiking neural networks. The approximation error tells us how well a spiking neural network can, in principle, fit a given dataset, while the generalisation error tells us how well the spiking neural network will perform on data that was not available during training. In addition, we investigated how often neurons should spike through a mathematical transference principle, which showed that neural networks composed of neurons that spike at most once and networks composed of neurons that spike more than once are equivalent in their approximation capabilities.
For the second part, we looked into two different systems: an ensemble of connected cubes that can move along each other by pivoting around their edges, and a mechanical lattice composed of strings and beams that can reconfigure its geometry without deforming its beams (a “Totimorphic” lattice). For the Totimorphic lattice, we derived a computational method that allows continuous reconfiguration of the lattice until it reaches a configuration with a target property, e.g. a mechanical or optical property. We used this approach to demonstrate self-repair capabilities: in the case of a mirror telescope made from such a lattice, meteorite damage to one of its mirrors would interfere with the focusing properties of the telescope. We demonstrated that focusing can be restored by reconfiguring the lattice with our method. For the cube ensembles, we developed novel neural network architectures that respect the symmetries inherent to the cubes, such as rotation and mirroring. Since all cubes are identical, they are all controlled by the same small neural network. However, cubes only have access to information from other cubes in their local surroundings. We trained such networks using reinforcement learning to reconfigure from any initial configuration to a given target configuration, while all cubes are identical and can only sense their local environment.
Finally, the original research programme was extended towards additional space-relevant computing concepts. Bio-inspired neural network architectures derived from the fruit-fly connectome were evaluated for astrodynamics applications, and memristor-based neural-network accelerators were investigated as a potential low-power and radiation-tolerant hardware platform for onboard artificial intelligence. Methods were developed to mitigate the effects of hardware imperfections such as noise and drift. For the Totimorphic lattices, a simple prototype was developed in collaboration with ESA’s Advanced Concepts Team and EPFL to validate the theoretical self-repair framework.
For the first part, we developed novel approaches for bounding the generalisation and approximation errors of spiking neural networks. The approximation error tells us how well a spiking neural network can, in principle, fit a given dataset, while the generalisation error tells us how well the spiking neural network will perform on data that was not available during training. In addition, we investigated how often neurons should spike through a mathematical transference principle, which showed that neural networks composed of neurons that spike at most once and networks composed of neurons that spike more than once are equivalent in their approximation capabilities.
For the second part, we looked into two different systems: an ensemble of connected cubes that can move along each other by pivoting around their edges, and a mechanical lattice composed of strings and beams that can reconfigure its geometry without deforming its beams (a “Totimorphic” lattice). For the Totimorphic lattice, we derived a computational method that allows continuous reconfiguration of the lattice until it reaches a configuration with a target property, e.g. a mechanical or optical property. We used this approach to demonstrate self-repair capabilities: in the case of a mirror telescope made from such a lattice, meteorite damage to one of its mirrors would interfere with the focusing properties of the telescope. We demonstrated that focusing can be restored by reconfiguring the lattice with our method. For the cube ensembles, we developed novel neural network architectures that respect the symmetries inherent to the cubes, such as rotation and mirroring. Since all cubes are identical, they are all controlled by the same small neural network. However, cubes only have access to information from other cubes in their local surroundings. We trained such networks using reinforcement learning to reconfigure from any initial configuration to a given target configuration, while all cubes are identical and can only sense their local environment.
Finally, the original research programme was extended towards additional space-relevant computing concepts. Bio-inspired neural network architectures derived from the fruit-fly connectome were evaluated for astrodynamics applications, and memristor-based neural-network accelerators were investigated as a potential low-power and radiation-tolerant hardware platform for onboard artificial intelligence. Methods were developed to mitigate the effects of hardware imperfections such as noise and drift. For the Totimorphic lattices, a simple prototype was developed in collaboration with ESA’s Advanced Concepts Team and EPFL to validate the theoretical self-repair framework.
Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)
One of the main achievements of BASE is the introduction of "causal pieces". We showed that every spiking neural network is a piecewise function, meaning that its input space decomposes into many different "causal pieces". For a simple spiking neuron model, we showed that the number of causal pieces a spiking neural network possesses provides a lower bound on its approximation error. Thus, the number of causal pieces is a measure of unlearnability: if the neural network only has a few pieces, it is not flexible enough to sufficiently fit the data. Conversely, if the neural network has many causal pieces, it is highly expressive. The concept of causal pieces is applicable to many different types of spiking neuron models, and thus has high potential for future research on the computational properties of spiking neural networks.
However, causal pieces were originally designed for spiking neural networks with neurons that spike at most once. To answer the question of whether our results also apply to spiking neurons that can spike more often, we developed a transference principle. This principle showed that approximation bounds derived for single-spike neural networks are also valid for multi-spike neural networks, and vice versa, for a large class of spiking neuron models. Put simply: when it comes to approximation capability, allowing neurons to spike more often does not fundamentally increase what the neural network can represent.
BASE also extended its research towards memristor-based neural network accelerators for onboard AI. We studied how hardware imperfections such as noise and drift affect neural network performance and developed mitigation methods, providing a step towards low-power and radiation-tolerant AI hardware for space applications.
Our work on reconfigurable cubes is inspired by neural cellular automata: a method where "cells" on a grid change their state depending on their local surroundings, which is computed by a local neural network. Different from cellular automata, where cells can be "created" and "deleted" out of thin air, our cubes are physical objects and therefore conserved, and reconfiguration has to obey physical principles, i.e. cubes cannot overlap or collide with each other, and the ensemble is not allowed to disconnect. BASE therefore extends the idea of local neural control to physically constrained reconfigurable systems, where useful global behaviour has to emerge while respecting collision, connectivity, and movement constraints.
Totimorphic lattices have only recently been proposed, and our work is the first to propose their use in space applications. In these types of systems, it is common to solve for configurations that have a certain property, without providing a way to morph the structure into this configuration. Our approach solves this by continuously morphing the structure until a target property is reached, yielding both the desired target configuration and the trajectory to it. We further demonstrated how autonomous reconfiguration can be used to enable self-repair, which was also validated in real experiments with 3D-printed lattice configurations.
The causal piece concept in particular has high potential and will require future research extending the results to a broader class of spiking neuron models, as well as studying in more depth which computational properties can be predicted from a spiking neural network's causal piece structure. Eventually, this might enable a principled comparison of spiking and non-spiking neural networks, highlighting where spiking provides benefits and what its limitations are.
However, causal pieces were originally designed for spiking neural networks with neurons that spike at most once. To answer the question of whether our results also apply to spiking neurons that can spike more often, we developed a transference principle. This principle showed that approximation bounds derived for single-spike neural networks are also valid for multi-spike neural networks, and vice versa, for a large class of spiking neuron models. Put simply: when it comes to approximation capability, allowing neurons to spike more often does not fundamentally increase what the neural network can represent.
BASE also extended its research towards memristor-based neural network accelerators for onboard AI. We studied how hardware imperfections such as noise and drift affect neural network performance and developed mitigation methods, providing a step towards low-power and radiation-tolerant AI hardware for space applications.
Our work on reconfigurable cubes is inspired by neural cellular automata: a method where "cells" on a grid change their state depending on their local surroundings, which is computed by a local neural network. Different from cellular automata, where cells can be "created" and "deleted" out of thin air, our cubes are physical objects and therefore conserved, and reconfiguration has to obey physical principles, i.e. cubes cannot overlap or collide with each other, and the ensemble is not allowed to disconnect. BASE therefore extends the idea of local neural control to physically constrained reconfigurable systems, where useful global behaviour has to emerge while respecting collision, connectivity, and movement constraints.
Totimorphic lattices have only recently been proposed, and our work is the first to propose their use in space applications. In these types of systems, it is common to solve for configurations that have a certain property, without providing a way to morph the structure into this configuration. Our approach solves this by continuously morphing the structure until a target property is reached, yielding both the desired target configuration and the trajectory to it. We further demonstrated how autonomous reconfiguration can be used to enable self-repair, which was also validated in real experiments with 3D-printed lattice configurations.
The causal piece concept in particular has high potential and will require future research extending the results to a broader class of spiking neuron models, as well as studying in more depth which computational properties can be predicted from a spiking neural network's causal piece structure. Eventually, this might enable a principled comparison of spiking and non-spiking neural networks, highlighting where spiking provides benefits and what its limitations are.