In the project, three analog neuromorphic devices -- ROLLS, Dynap-se, and SCAMP smart camera were interfaced to three robotic vehicles -- Pushbot, Omnibot, and Khepera. In particular, a software framework was developed to, on the one hand, configure the neuromorphic devices to realise different cognitive neuromorphic architectures and, on the other hand, to stream sensory inputs to the silicon neurons on chip and read-out spiking activity of the neurons to drive the robots' motors. Cognitive architectures for sequence learning and spatial scene representation were realised in the project both in spiking network simulations and in hardware. For testing the architectures, the navigation scenario was selected, in which robots had to navigate an unknown environment, avoiding obstacles and approaching targets, and building a representation of this environment.
First, a neural-dynamic architecture for sequence learning was implemented, in which sequences of states or events are stored in plastic connections between neuronal populations on the ROLLS chip. The sequence to be learned was perceived through a robotic camera (a neuromorphic artificial retina, Dynamic Vision Sensor). Interface that linked the DVS, the robot, and the neuromorphic chip was realised on the miniature computing board Parallella. We have demonstrated functionality of this system in a closed sensorimotor loop in an exemplary scenario, in which one robotic agent "teaches" a sequence of turns to the second robot. In a spiking neural network simulation, we have explored an extension of the sequence learning architecture towards complex, hierarchical sequences, taking as inspiration bird song learning.
In the second line of work, we developed architectures for spatial representations of an environment. We have implemented different components of a neuronal architecture for simultaneous localisation and mapping (SLAM) in neuromorphic device, while simulating the overall architecture in a spiking neural network simulator BRAIN2. Moreover, in simulation, we have developed a new structure for elementary behaviours, which allows the robot to detect and analyse mismatch (in a ``hypothesis-testing'' network) between the perceived objects and the ones previously stored, autonomously. This new structure is a key component for autonomous, online learning in neuromorphic hardware.
We have also developed methods to tune parameters of neuromorphic chips autonomously, using intrinsic plasticity for Dynamic Neural Fields and evolutionary optimisation techniques. During this work, a number of principles were elaborated, which allow to realised cognitive architectures in neuromorphic hardware that uses analog neurons and synapses. These principles are: (1) using population code to represent behavioural variables; (2) using different number of randomly assigned weights between populations to emulate weights of different strength in hardware that has a limited number of synaptic weights; (3) using dynamic neural fields-connectivity to stabilise spatial representations against sensory and neuronal noise.
Thus, in the project the following critical core-stones for further development of neuromorphic cognitive robotics systems were achieved: 1. A software framework for configuring neuromorphic hardware to realise neuronal architectures for sequence learning and space representation. 2. An interface between neuromorphic hardware and robotic sensors (e.g. the neuromorphic camera DVS, Inertial Measurement Unit (IMU), Gyroscope, wheel encoders, microphones) and motors (wheels of a vehicle for controlling speed and turning).
Results of the projects were disseminated at several conferences (ISCAS 2017, RSS 2017, ICDSC 2016), Workshops/Symposiums (e.g. Neuroscience Center Zurich Symposium; Swiss Society for Neuroscience symposium; Robotics in the 21st Century Workshop), public events (e.g. Brain Fair 2017; Alpbach Technology Forum 2016; microTalk at CSEM, Switzerland), as well as at the Capo Caccia Workshop for Cognitive Neuromorphic Engineering.
