Deliverables A model of sensory motor integration applied to a mobile robot The control of behaviour, like navigating to a goal, is understood as a top down process. This control of behaviour is usually understood in terms of three distinct components: sensory processing, decision-making and movement control. Recently, this view has been questioned on the basis of physiological and behavioural data, blurring the distinction between these three stages. This raises the question to what extent the motor system itself can contribute to the interpretation of behavioural situations. To investigate this question we use a neural model of sensory motor integration applied to a behaving mobile robot performing a navigation task. We show that the population response of the motor system provides a substrate for the categorization of behavioural situations. This categorization allows for the assessment of the complexity of a behavioural situation and regulates whether higher-level decision making is required to resolve behavioural conflicts. Our model lends credence to an emerging reconceptualization of behavioural control where the motor system can be considered as part of a high-level perceptual system. Data of behavioural experiments and corresponding spike trains We recorded neuronal activity from barrel cortex while rats discriminated between a rough and a smooth texture without visual cues. Whiskers moved across both textures with a "stick and slip" behaviour and the spikes of cortical neurons were associated with distinctive kinetic events - high bending angles or the release from bending. In the 200-ms interval before the animal's decision, spike counts were higher on average on rough trials than on smooth trials, and we propose firing rate as a cortical code for texture coarseness under these conditions. Spike count of a neuronal cluster carried, on average, 0.05 bits of information about the texture, about one-tenth of the information necessary to account for behaviour. The evidence indicates that firing rate is the neuronal substrate for the rats' discrimination of texture. Coding of high frequency vibrations in the awake animal Whisker vibration at a wide range of frequencies is encoded in the awake barrel cortex (tested between 10 and 700Hz). Preliminary data indicate that 1 to 1 responses are present continuously for at least 1 s (stimulus duration) for the entire range of frequencies. The same vibratory frequencies also elicited responses under anaesthesia (isoflurane), however 1 to 1 responses were present only up to 320Hz. Phase-locking of high frequency responses appears to be more precise in the awake cortex. The initial 50-100ms of a response may be altered under anaesthesia. The On-response is enhanced, followed by an inhibitory pause of about 40 ms and a rebound activity. Thus, entrainment to the vibratory stimulus begins not before 80 -100ms after response onset. In the awake animal, high frequency components of whisker movements are precisely encoded in the barrel cortex within the first 10ms (as well as for seconds) of cortical activity and thus are features available for texture discrimination. Artificial whisker system Construction of an active whisker system, which closely corresponds to the external parts of the whisker system in rodents. The artificial whisker sensor developed by Partner 2 is based on a capacitor microphone sensor. It has the advantage of being a simple, robust, and low-cost transducer with a good signal-to-noise ratio. Furthermore, it allows different type of artificial or natural fibres to be used as whisker shaft. The miniature size is also ideally suited for combining a number of sensors into active multiple-whisker arrays. The design of the single whisker system of Partner 5 coarsely replicates the design of the follicle sheath of rodents and other mammals. This general design is copied in all our technical whisker systems. Our whiskers comprise two pairs of sensors and can therefore measure deflection in two orthogonal directions. This is a valuable property for the recognition of slanted surfaces. On the other hand, developed a simple, robust, and low-cost technical solution, which could be replicated multiple times in a whisker array with reasonable effort. The magnetic whisker design is best suitable for shape recognition since it is measuring the static deflection signal (tonic response in biological terms), whereas our whisker using piezo sensors responds only to changes (phasic response) and therefore has its preferred application in texture recognition. Magnetic whiskers can of course also provide texture-related signals, and we also applied piezo sensors to shape recognition tasks (by measurement of deflection speed). Experiments on motor categorization and forward models We investigated numerous strategies for active sensing in distance estimation and shape recognition, and tested these in robot experiments. We found that the integration of motor signals (whisker protraction) and sensory signals (whisker deflection) provides useful information for above-mentioned tasks. Sensorimotor loops for the repositioning of the mobile robot with respect to the object were developed. We also used an integrated sensorimotor architecture based on reciprocal cross-modal coupling, and highlighted cognitive abilities that spontaneously arose in tactile discrimination task and predictive reafference cancellation (forward models). The AMOUSE robot of partner 2 was used to collect naturalistic stimuli. The robot is equipped with 12 whiskers (6 on each side in two rows of 3 whiskers each) and a camera pointing to a hyperbolic overhead mirror. The whisker tips are located within the visual field. While the robot explores the environment, we record the 360-degree view around the robot and the simultaneous whisker deflections. Using this bimodal sensory input dataset we train networks to optimise for sparseness or stability of the output layer. We thereby generalize the learning algorithms (developed by partner 4) that where successfully used to explain e.g. viewpoint invariance in the visual system and the emergence of whisker representations for texture discrimination in the somato-sensory system to learn multi-modal representations. Model of somato-sensory processing We implemented a model of the somato-sensory system processing complex natural stimuli that can serve the decision system of the "AMOUSE" as input. We advance the understanding of what features of whisker movement signals are relevant for texture discrimination. Furthermore, we advanced our whisker set up. Action potential data from the somato-sensory pathway Vibration of whiskers activates mechanoreceptors in sinus hair follicles. Neurons of the thalamic ventrobasal complex (VB) receive glutamatergic excitatory inputs from the principal trigeminal nucleus and primary somatosensory cortex. The thalamic reticular nucleus (TRN), activated by collaterals of thalamocortical projections, provides GABAergic inhibitory feedback. The experiments addressed to the somatosensory pathway show an exquisite sensitivity to vibratory stimuli with 1:1 responses up to high frequencies and phase-locking precision in the sub-millisecond range. Mobile robot with vision and whisker array The central technological achievement of the project was the construction of an active artificial whisker system and its test on a mobile robot. By incorporating biological insights (material properties, morphology and active sensing), this constituted a significant technological advancement of whisker-like artificial touch sensors, and contributed to a better understanding of the role of the sensor morphology for information processing. Virtual reality environment for an artificial rodent We developed a simulation for Khepera robots. This simulation was mainly built for the processing sensory stimuli with the help of neural networks. The main part of this simulation was to adapt the real behaviour of a Khepera robot in a simulated environment. This program can be used to optimise for different kind of objectives for different input, as for example visual and whisker sensory data. For example with the help of this program we were able to record the visual stimuli of robot, while it drove through an artificial environment. This visual stimulus was used to train a hierarchical neural network for smoothness. This resulted in a complex representation of the environment, like place cell. Comparison of temporal dynamics in visual and somato-sensory system By parallel experiments in the visual and the somato-sensory system of rodents, we have achieved a comparative analysis of temporal dynamics in the two modalities. We suggest a common coding framework, which takes into account both stimulus-locked and intrinsically generated parts of the signals. We conclude that the similarities between the two modalities in encoding of related sensory information and in the build-up of representational states may be greater than previously expected. Several publications have appeared or are in preparation. Active sensing through whisker and body movements Our artificial whiskers comprise two perpendicular channels. This allows us to recognize the vertical shape of objects by observing the time course of the two channels (X, Y) during protraction / retraction while the whisker slides along the surface. Alternatively, several whiskers arranged perpendicularly to the sweeping plane will detect differences in contact time and deflection amplitude, thus providing information on the vertical shape of objects (e.g. cone vs. Inverted cone). We summarize that an active whisking strategy can reveal a high amount of information on the distance and shape of an object. Searching for OpenAIRE data... There was an error trying to search data from OpenAIRE No results available
