Predictive Neural Information for Proactive Actions: From Monkey Brain to Smart House Control

Deliverables

Websites, patent fillings, videos etc. (1)

Website

This deliverable will a short report on the Website of the project.

Documents, reports (17)

Dissemination strategy and plan.

This deliverable will report on how the Plan4Act project will set out to disseminate its results.

Report on neural network model encoding sequences.

This deliverable will report on the design of a neural network encoder which is able to model neural sequences belonging to the monkey recordings from WP1. This is the basic starting point of the model for emulating data from Tests 1 and 2 (see Part-B).

Short report and specification data sheet about the implementation of the interfaces for the Smart House.

This deliverable will provide a report in the form of a data sheet specifying the actual implementations of the required interfaces that allow connecting Smart House devices to the experimental data from WP1 via the controllers developed in WP2 and WP3.

First report on the adaptable network controller for complex action sequence generation and smart house control.

This deliverable will report how the adaptive control units from D3.1 can be combined into an adaptable network controller that allows encoding complex action sequence information and can be used for the controlling of a Smart House.

Report on neural encoding of complex action sequences in SmartCage for Test 2 and update on Test 1.

This deliverable will report how complex action sequences are encoded by the neurons in response to monkey predictive behavior in the SmartCage. This concerns Test 2 as described in Paert-B. In addition the report will provide an update on Test 1.

Final report on the hardware controller and integration into the demonstrator for proactive neural-based smart house control.

This deliverable is the continuation from D3.2 and represents the final report on how to create an adaptable network controller from the adaptive control units from D3.1. It shall report how to finally encode complex action sequence information with this network and how to use it for smart house control.

First Report on status of experimental setup (SmartCage), training, behavioral testing, and neural recording.

This deliverable will report how the Smart Cage will be set up, which functions it will support and how it shall operate in conjunction with neural recordings. It will address the training and behavioral testing aspects as well as the actual recording setup for monkey neural recordings.

Report on the status of Smart House devices and interfaces for connecting to the controllers of WP2 and WP3.

This deliverable will report on the different devices and interfaces that exist in the here used Smart House in order to allow us to connect them with the controllers from WP2 and WP3.

Pro Memori: reporting as per Grant Agreement

Covers all reporting obligations (periodic and final) as required by the Grant Agreement

Report on behavioral testing and status of recording with SmartCage action sequence planning for Test 1.

"This deliverable will report about the specific experiments for the action sequence planning in ""Test 1"" as described in Part-B. This concerns behavioral testing as well as the corresponding neural recordings in the SmartCage."

Data sheet of definitions of experimental conditions for the Smart House as depending on the setup and data of WP1.

This deliverable will provide a report in the form of a data sheet specifying the different required definitions of the experimental conditions which would allow controlling a Smart House with neural data from Tests 1 and 2 (see Part-B). Hence it provides the link to the experimental data from WP1.

Report on neural network model enabling the prediction of complex planned sequences of actions.

This deliverable will report on the next extension of the neural network controller to model the prediction of complex planned sequences of actions. This concerns Test 2 as described in Part-B.

Final report on all dissemination activities.

This deliverable will be the final report on all dissemination activities of Plan4Act

Report on neural network model which predicts distractor-robustly the planned sequence of actions

This deliverable will report on the final version of the neural network controller to model the data in a way which is robust against distractors in the planned sequence of actions. This concerns Tests 1 and 2 as described in Part-B.

Report on neural network model using the sequential structure to predict the present sequence.

This deliverable will report on the neural network model extended to be able to predict the data from Test 1 (see Part-B), which will use a sequential structure with the goal to be able to predict the present behavioral sequence.

Report on generic reduced control units for complex action sequence formation.

This deliverable reports on the electronic implementation and it will describe how the cell assemblies in the neural network from WP2 can be generically reduced into electronic control units that allow encoding of complex action sequence information.

Report of all final specifications of the Smart House setup and interfaces.

This deliverable will provide a report in the form of a data sheet and will be the continuation of D4.3 where we shall specify how all required interfaces are constituted and actually set up for Smart House control.

Demonstrators, pilots, prototypes (2)

Final demo of system using the hardware controller and data from Tests 1 and 2, at least for one Test an online demo is planned.

This deliverable will be the final demonstrator of this project and shall show how to use the hardware controller to control Smart House devices using data from both Tests (1 and 2). It is planned to show at least for the simpler Test 1 an online demo.

Demonstration of Smart House control using the software controller from WP2 and the Test 1 condition from WP1.

This deliverable will be a demonstrator that shows the functionality of the software based controller from WP2 using the simpler Test 1 experimental data.

Open Research Data Pilot (1)

Data Management Plan

This deliverable will describe the Data Managment Plan as required by the Pilot on Open Research Data. This plan will discuss: 1) What types of data will the project generate/collect? 2) What standards will be used? 3) How this data will be exploited and/or shared/made accessible for verification and re-use? And how this data will be curated and preserved?

Publications

Peer reviewed articles (25)

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

Author(s): Nan-Sheng Huang, Yi-Chung Chen, Jørgen Christian Larsen, Poramate Manoonpong
Published in: Energies, 13/9, 2020, Page(s) 2180, ISSN 1996-1073
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/en13092180

The cone method: Inferring decision times from single-trial 3D movement trajectories in choice behavior

Author(s): Philipp Ulbrich, Alexander Gail
Published in: Behavior Research Methods, 2021, ISSN 1554-3528
Publisher: Springer
DOI: 10.3758/s13428-021-01579-5

Self-Organized Structuring of Recurrent Neuronal Networks for Reliable Information Transmission

Author(s): Daniel Miner, Florentin Wörgötter, Christian Tetzlaff and Michael Fauth
Published in: Biology, 2021, ISSN 2079-7737
Publisher: MDPI
DOI: 10.3390/biology10070577

The Interplay of Synaptic Plasticity and Scaling Enables Self-Organized Formation and Allocation of Multiple Memory Representations

Author(s): Johannes Maria Auth, Timo Nachstedt, Christian Tetzlaff
Published in: Frontiers in Neural Circuits, 14, 2020, ISSN 1662-5110
Publisher: Frontiers Research Foundation
DOI: 10.3389/fncir.2020.541728

Memory consolidation and improvement by synaptic tagging and capture in recurrent neural networks

Author(s): Jannik Luboeinski, Christian Tetzlaff
Published in: Communications Biology, 4/1, 2021, ISSN 2399-3642
Publisher: Nature Publishing Group
DOI: 10.1038/s42003-021-01778-y

Evolving artificial neural networks with feedback

Author(s): Sebastian Herzog, Christian Tetzlaff, Florentin Wörgötter
Published in: Neural Networks, 123, 2020, Page(s) 153-162, ISSN 0893-6080
Publisher: Pergamon Press Ltd.
DOI: 10.1016/j.neunet.2019.12.004

A Secure and Scalable Smart Home Gateway to Bridge Technology Fragmentation

Author(s): Ezequiel Simeoni, Eugenio Gaeta, Rebeca I. García-Betances, Dave Raggett, Alejandro M. Medrano-Gil, Diego F. Carvajal-Flores, Giuseppe Fico, María Fernanda Cabrera-Umpiérrez, María Teresa Arredondo Waldmeyer
Published in: Sensors, 21/11, 2021, Page(s) 3587, ISSN 1424-8220
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/s21113587

Embodied Synaptic Plasticity With Online Reinforcement Learning

Author(s): Jacques Kaiser, Michael Hoff, Andreas Konle, J. Camilo Vasquez Tieck, David Kappel, Daniel Reichard, Anand Subramoney, Robert Legenstein, Arne Roennau, Wolfgang Maass, Rüdiger Dillmann
Published in: Frontiers in Neurorobotics, 13, 2019, ISSN 1662-5218
Publisher: Frontiers Research Foundation
DOI: 10.3389/fnbot.2019.00081

Generic Neural Locomotion Control Framework for Legged Robots

Author(s): Mathias Thor, Tomas Kulvicius, Poramate Manoonpong
Published in: IEEE Transactions on Neural Networks and Learning Systems, 2020, Page(s) 1-13, ISSN 2162-237X
Publisher: IEEE Computational Intelligence Society
DOI: 10.1109/tnnls.2020.3016523

Robust Trajectory Generation for Robotic Control on the Neuromorphic Research Chip Loihi

Author(s): Carlo Michaelis, Andrew B. Lehr, Christian Tetzlaff
Published in: Frontiers in Neurorobotics, 14, 2020, ISSN 1662-5218
Publisher: Frontiers Research Foundation
DOI: 10.3389/fnbot.2020.589532

Hey, look over there: Distraction effects on rapid sequence recall

Author(s): Daniel Miner, Christian Tetzlaff
Published in: PLOS ONE, 15/4, 2020, Page(s) e0223743, ISSN 1932-6203
Publisher: Public Library of Science
DOI: 10.1371/journal.pone.0223743

A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

Author(s): Dennis Goldschmidt, Poramate Manoonpong, Sakyasingha Dasgupta
Published in: Frontiers in Neurorobotics, 11, 2017, ISSN 1662-5218
Publisher: Frontiers Research Foundation
DOI: 10.3389/fnbot.2017.00020

Principles underlying the input-dependent formation and organization of memories

Author(s): Juliane Herpich, Christian Tetzlaff
Published in: Network Neuroscience, 3/2, 2019, Page(s) 606-634, ISSN 2472-1751
Publisher: The MIT Press
DOI: 10.1162/netn_a_00086

Peri-hand space expands beyond reach in the context of walk-and-reach movements

Author(s): Michael Berger, Peter Neumann, Alexander Gail
Published in: Scientific Reports, 9/1, 2019, ISSN 2045-2322
Publisher: Nature Publishing Group
DOI: 10.1038/s41598-019-39520-8

Recognition and prediction of manipulation actions using Enriched Semantic Event Chains

Author(s): Fatemeh Ziaeetabar, Tomas Kulvicius, Minija Tamosiunaite, Florentin Wörgötter
Published in: Robotics and Autonomous Systems, 110, 2018, Page(s) 173-188, ISSN 0921-8890
Publisher: Elsevier BV
DOI: 10.1016/j.robot.2018.10.005

A Fast Online Frequency Adaptation Mechanism for CPG-Based Robot Motion Control

Author(s): Mathias Thor, Poramate Manoonpong
Published in: IEEE Robotics and Automation Letters, 4/4, 2019, Page(s) 3324-3331, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2019.2926660

A Theoretical Framework to Derive Simple, Firing-Rate-Dependent Mathematical Models of Synaptic Plasticity

Author(s): Janne Lappalainen, Juliane Herpich, Christian Tetzlaff
Published in: Frontiers in Computational Neuroscience, 13, 2019, ISSN 1662-5188
Publisher: Frontiers Research Foundation
DOI: 10.3389/fncom.2019.00026

Comparing Open-Source Toolboxes for Processing and Analysis of Spike and Local Field Potentials Data

Author(s): Valentina A. Unakafova, Alexander Gail
Published in: Frontiers in Neuroinformatics, 13, 2019, ISSN 1662-5196
Publisher: Frontiers Research Foundation
DOI: 10.3389/fninf.2019.00057

Symbol Emergence in Cognitive Developmental Systems: a Survey

Author(s): Tadahiro Taniguchi, Emre Ugur, Matej Hoffmann, Lorenzo Jamone, Takayuki Nagai, Benjamin Rosman, Toshihiko Matsuka, Naoto Iwahashi, Erhan Oztop, Justus Piater, Florentin Worgotter
Published in: IEEE Transactions on Cognitive and Developmental Systems, 2018, Page(s) 1-1, ISSN 2379-8920
Publisher: IEEE
DOI: 10.1109/tcds.2018.2867772

Error-Based Learning Mechanism for Fast Online Adaptation in Robot Motor Control

Author(s): Mathias Thor, Poramate Manoonpong
Published in: IEEE Transactions on Neural Networks and Learning Systems, 2019, Page(s) 1-10, ISSN 2162-237X
Publisher: IEEE Computational Intelligence Society
DOI: 10.1109/tnnls.2019.2927737

Neural computational model GrowthEstimate: A model for studying living resources through digestive efficiency

Author(s): Krisna Rungruangsak-Torrissen, Poramate Manoonpong
Published in: PLOS ONE, 14/8, 2019, Page(s) e0216030, ISSN 1932-6203
Publisher: Public Library of Science
DOI: 10.1371/journal.pone.0216030

The self-organized learning of noisy environmental stimuli requires distinct phases of plasticity

Author(s): Steffen Krüppel; Christian Tetzlaff
Published in: Network Neuroscience, 1, 2020, ISSN 2472-1751
Publisher: MIT Press
DOI: 10.1101/612341

Editorial: Neural Computation in Embodied Closed-Loop Systems for the Generation of Complex Behavior: From Biology to Technology

Author(s): Poramate Manoonpong, Christian Tetzlaff
Published in: Frontiers in Neurorobotics, 12, 2018, ISSN 1662-5218
Publisher: Frontiers Research Foundation
DOI: 10.3389/fnbot.2018.00053

General Distributed Neural Control and Sensory Adaptation for Self-Organized Locomotion and Fast Adaptation to Damage of Walking Robots

Author(s): Aitor Miguel-Blanco, Poramate Manoonpong
Published in: Frontiers in Neural Circuits, 14, 2020, ISSN 1662-5110
Publisher: Frontiers Research Foundation
DOI: 10.3389/fncir.2020.00046

Wireless recording from unrestrained monkeys reveals motor goal encoding beyond immediate reach in frontoparietal cortex

Author(s): Michael Berger, Naubahar Shahryar Agha, Alexander Gail
Published in: eLife, 9, 2020, ISSN 2050-084X
Publisher: eLife Sciences Publications
DOI: 10.7554/elife.51322

Conference proceedings (5)

Autobot for Effective Design Space Exploration and Agile Generation of RBFNN Hardware Accelerator in Embedded Real-time Computing

Author(s): Nan-Sheng Huang, Jorgen Christian Larsen, Poramate Manoonpong
Published in: 2020 IEEE International Conference on Real-time Computing and Robotics (RCAR), 2020, Page(s) 339-344, ISBN 978-1-7281-7293-4
Publisher: IEEE
DOI: 10.1109/rcar49640.2020.9303043

End-to-End Rapid FPGA Prototyping for Embedded Proactive BMI Control

Author(s): Nan-Sheng Huang, Jan-Matthias Braun, Ricardo Rodrigues do Carmo, Jorgen Christian Larsen, Poramate Manoonpong
Published in: 2020 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan), 2020, Page(s) 1-2, ISBN 978-1-7281-7399-3
Publisher: IEEE
DOI: 10.1109/icce-taiwan49838.2020.9258186

Teaching Hardware Implementation of Neural Networks using High-Level Synthesis in Less Than Four Hours for Engineering Education of Intelligent Embedded Computing

Author(s): Nan-Sheng Huang, Jan-Matthias Braun, Jorgen Christian Larsen, Poramate Manoonpong
Published in: 2019 20th International Carpathian Control Conference (ICCC), 2019, Page(s) 1-7, ISBN 978-1-7281-0702-8
Publisher: IEEE
DOI: 10.1109/carpathiancc.2019.8765994

scalable Echo State Networks hardware generatorfor embedded systems using high-level synthesis

Author(s): Nan-Sheng Huang, Jan-Matthias Braun, Jørgen Christian Larsen, Poramate Manoonpong
Published in: 8th Mediterranean Conference on Embedded Computing, 2019
Publisher: IEEE

A scalable Echo State Networks hardware generator for embedded systems using high-level synthesis

Author(s): Nan-Sheng Huang, Jan-Matthias Braun, Jorgen Christian Larsen, Poramate Manoonpong
Published in: 2019 8th Mediterranean Conference on Embedded Computing (MECO), 2019, Page(s) 1-6, ISBN 978-1-7281-1740-9
Publisher: IEEE
DOI: 10.1109/meco.2019.8760065

Book chapters (1)

Development of a Real-Time Motor-Imagery-Based EEG Brain-Machine Interface

Author(s): Gal Gorjup, Rok Vrabič, Stoyan Petrov Stoyanov, Morten Østergaard Andersen, Poramate Manoonpong
Published in: Neural Information Processing - 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13–16, 2018, Proceedings, Part VII, 11307, 2018, Page(s) 610-622, ISBN 978-3-030-04238-7
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-04239-4_55

Other (4)

The Reach Cage environment for wireless neural recordings during structured goal-directed behavior of unrestrained monkeys

Author(s): Michael Berger, Alexander Gail
Published in: bioRxiv, 2018
Publisher: Cold Spring Harbor Laboratory
DOI: 10.1101/305334

Symbol Emergence in Cognitive Developmental Systems: a Survey

Author(s): Taniguchi, Tadahiro; Ugur, Emre; Hoffmann, Matej; Jamone, Lorenzo; Nagai, Takayuki; Rosman, Benjamin; Matsuka, Toshihiko; Iwahashi, Naoto; Oztop, Erhan; Piater, Justus; Wörgötter, Florentin
Published in: 1, 2018
Publisher: arxiv

Action Prediction in Humans and Robots

Author(s): Florentin Wörgötter, Fatemeh Ziaeetabar, Stefan Pfeiffer, Osman Kaya, Tomas Kulvicius, Minija Tamosiunaite
Published in: 2019
Publisher: arxiv

Generation of Paths in a Maze using a Deep Network without Learning

Author(s): Kulvicius, Tomas; Herzog, Sebastian; Tamosiunaite, Minija; Wörgötter, Florentin
Published in: 7, 2021
Publisher: arxiv