Periodic Reporting for period 2 - B-CRATOS (Wireless Brain-Connect inteRfAce TO machineS: B-CRATOS)
Período documentado: 2022-03-01 hasta 2023-08-31
Challenges Addressed:
Overcoming challenges in brain-machine interfaces, including tethered systems and implants with high power consumption and low bandwidth.
Addressing limitations such as a limited number of channels, low bandwidth, and low spatiotemporal resolution in current brain-machine interfaces.
Tackling vulnerabilities in wireless solutions and improving the dexterity of prosthetics.
Societal Benefits:
Advancing Brain-Machine Interface: Enabling direct connections between humans and machines for enhanced learning and brain plasticity, supporting applications like exoskeletons and sensors with full IoT functionality.
Enhancing full-immersion gaming and augmented virtual reality experiences.
Facilitating neuroprosthetics for restoration of various functions, including vision, audition, speech movement, sensation, and neuromodulation/brain stimulation for various disorders.
Achieving homeostasis through closed-loop control over internal organs, addressing conditions like heart pacemakers, bladder control, and sleep apnea.
Objectives:
Integrating Battery-Free BCI: Integrating a battery-less, fully implantable brain interface to sense high-resolution neural signals, deliver electrical stimulation, and enable two-way wireless communication.
Seamless Integration of Fat-IBC Technology: Integrating the high-speed Fat intra-body communication platform efficiently into the overall system, minimizing external interference.
Integration of High-Performance AI Computing: Incorporating high-performance AI computing based on machine learning and deep learning algorithms for enhanced pattern recognition and classification.
Integration of Biomechatronic Prosthetic Hand: Integrating robotic hands with optimized power consumption algorithms, enhancing autonomy, and sensorizing an IH2 Azzurra hand for experiments.
Integration of Novel Sensory System: Incorporating the B-Cratos sensory system, utilizing triboelectric nanogenerators and graphene-based hydrogels, into the platform without external power requirements.
Overcoming challenges in brain-machine interfaces, including tethered systems and implants with high power consumption and low bandwidth.
Addressing limitations such as a limited number of channels, low bandwidth, and low spatiotemporal resolution in current brain-machine interfaces.
Tackling vulnerabilities in wireless solutions and improving the dexterity of prosthetics.
Societal Benefits:
Advancing Brain-Machine Interface: Enabling direct connections between humans and machines for enhanced learning and brain plasticity, supporting applications like exoskeletons and sensors with full IoT functionality.
Enhancing full-immersion gaming and augmented virtual reality experiences.
Facilitating neuroprosthetics for restoration of various functions, including vision, audition, speech movement, sensation, and neuromodulation/brain stimulation for various disorders.
Achieving homeostasis through closed-loop control over internal organs, addressing conditions like heart pacemakers, bladder control, and sleep apnea.
Objectives:
Integrating Battery-Free BCI: Integrating a battery-less, fully implantable brain interface to sense high-resolution neural signals, deliver electrical stimulation, and enable two-way wireless communication.
Seamless Integration of Fat-IBC Technology: Integrating the high-speed Fat intra-body communication platform efficiently into the overall system, minimizing external interference.
Integration of High-Performance AI Computing: Incorporating high-performance AI computing based on machine learning and deep learning algorithms for enhanced pattern recognition and classification.
Integration of Biomechatronic Prosthetic Hand: Integrating robotic hands with optimized power consumption algorithms, enhancing autonomy, and sensorizing an IH2 Azzurra hand for experiments.
Integration of Novel Sensory System: Incorporating the B-Cratos sensory system, utilizing triboelectric nanogenerators and graphene-based hydrogels, into the platform without external power requirements.
Over the past 18 months in the second report period, significant progress has been made in the following key areas:
• Successful completion of functional stimulation and reading Brain-Computer Interface (BCI) prototypes, along with the implementation of power transfer mechanisms.
• Thorough testing of the overall system to validate information connectivity and ensure reliability.
• Design and development of a wireless power transfer system and a wireless communication system tailored for Fat-IBC (In-Body Communication) and E-skin devices.
• Real-world testing of the Fat-IBC system using a non-human primate (NHP) model involves transmitting representative brain signal data and performance evaluation of the modems.
• Optimize the Fat-IBC system to achieve lower power consumption and higher data transmission rates.
• Fabrication of E-skins equipped with an array of tactile sensors and readout circuits, followed by extensive testing on a robotic hand for object classification and manipulation.
• Integration of the E-skins with the wireless interface and the Fat-IBC system.
• Successful completion of two robotic hand prototypes.
Notably, all objectives outlined in the previous reporting period have been successfully achieved during this reporting period. This progress places us in a strong position for the integration phase scheduled for report period III.
• Successful completion of functional stimulation and reading Brain-Computer Interface (BCI) prototypes, along with the implementation of power transfer mechanisms.
• Thorough testing of the overall system to validate information connectivity and ensure reliability.
• Design and development of a wireless power transfer system and a wireless communication system tailored for Fat-IBC (In-Body Communication) and E-skin devices.
• Real-world testing of the Fat-IBC system using a non-human primate (NHP) model involves transmitting representative brain signal data and performance evaluation of the modems.
• Optimize the Fat-IBC system to achieve lower power consumption and higher data transmission rates.
• Fabrication of E-skins equipped with an array of tactile sensors and readout circuits, followed by extensive testing on a robotic hand for object classification and manipulation.
• Integration of the E-skins with the wireless interface and the Fat-IBC system.
• Successful completion of two robotic hand prototypes.
Notably, all objectives outlined in the previous reporting period have been successfully achieved during this reporting period. This progress places us in a strong position for the integration phase scheduled for report period III.
Progress beyond state of the art:
B-CRATOS has several significant advantages for neural recording and stimulation, and for signal transmission through subdermal fat tissue. The high-bandwidth (>32 Mbps) wireless communications system proposed in B-CRATOS is not available in any commercial product and will enable high 64-channel recording of neural action potentials. The B-CRATOS project will implement, for the first time in a neurological medical device application, high-bandwidth RF backscatter and fat channel communication (Fat-IBC) to transmit motor control signals from motor cortex to a prosthetic arm, and sensor information from the prosthetic arm back to the primary sensory cortex of a non-human primate (NHP). A novel low-power stim/record ASIC and implantable titanium module will enable this functionality. This proof-of-concept demonstration in NHP leads to subsequent human clinical trials. The Utah Array has been demonstrated in both humans and non-human primates using percutaneous pedestal connectors . Motor intention signals will be read from the motor cortex with microelectrodes, and transmitted through the skin wirelessly to a processing unit worn on the head. The digital data will be converted microwaves and propagation through subdermal fat layer to the dexterous mechatronic arm where they will be converted into control sequences for the motors using machine learning and AI algorithms. A feedback signal conveying the tactile, position and force information from proprioceptive and exteroceptive sensors in the artificial skin on the arm will be sent through the same Fat-IBC channel. The head-worn device will receive the feedback through the Fat-IBC and convert to stimulation signals for electrodes implanted in the sensory cortex. Bi-directional communication will be first demonstrated on the bench-top and then with NHP.
Expected results and impact:
Scientific and technological contributions to the foundation of new future technology. We are an inflection point with regard to machine learning and human-machine interaction. B-CRATOS establishes a new platform for fully implanted wireless direct communication and control with the internet and machines. Paradigm shift is underway from pharmaceuticals with global actions and subsequent side effects to medical devices with local, targeted action. Communications previously used in physics and telecommunications (microwave, RFID backscatter) will be brought into the human body. Two-way, always-on wireless implant for recording and stimulation/neuromodulation of neurons in any location in the central or peripheral nervous system with individual neuron resolution and continuous 24/7 operation without medical supervision, enabling learning and brain plasticity necessary for complete integration of sensors and prosthetics. Advanced machine learning algorithms will accelerate this integration and improve the user’s quality of life. Secure, encrypted intra-body data network: Will enable brain-organ communication, intra-body sensor network, wearable exoskeletons, bionic arm control, etc. Two-way high-speed brain-computer link: Real-time sensing and stimulation for augmented reality, learning (cognition) and gaming (sensation and movement). Electronic Skin: Self-powered mechano-electric transduction-based tactile sensing enabling large-area, self-powered electronic skin with high spatial resolution. Beneficial for human-like perception capability with unique material and form recognition features. Can be used for treating burn injuries, augmented reality, gaming, etc. The mechanical to electrical power generation could be used to power implanted medical devices. Impact on innovation in Neural Disease, Electroceutical, AI, Exoskeleton and Robotics sectors: The B-CRATOS wireless platform is compatible with all electrode sensor and effector types, and can be ported into existing in-body closed-loop systems to increase the number of electrode channels, speed, and resolution.
B-CRATOS has several significant advantages for neural recording and stimulation, and for signal transmission through subdermal fat tissue. The high-bandwidth (>32 Mbps) wireless communications system proposed in B-CRATOS is not available in any commercial product and will enable high 64-channel recording of neural action potentials. The B-CRATOS project will implement, for the first time in a neurological medical device application, high-bandwidth RF backscatter and fat channel communication (Fat-IBC) to transmit motor control signals from motor cortex to a prosthetic arm, and sensor information from the prosthetic arm back to the primary sensory cortex of a non-human primate (NHP). A novel low-power stim/record ASIC and implantable titanium module will enable this functionality. This proof-of-concept demonstration in NHP leads to subsequent human clinical trials. The Utah Array has been demonstrated in both humans and non-human primates using percutaneous pedestal connectors . Motor intention signals will be read from the motor cortex with microelectrodes, and transmitted through the skin wirelessly to a processing unit worn on the head. The digital data will be converted microwaves and propagation through subdermal fat layer to the dexterous mechatronic arm where they will be converted into control sequences for the motors using machine learning and AI algorithms. A feedback signal conveying the tactile, position and force information from proprioceptive and exteroceptive sensors in the artificial skin on the arm will be sent through the same Fat-IBC channel. The head-worn device will receive the feedback through the Fat-IBC and convert to stimulation signals for electrodes implanted in the sensory cortex. Bi-directional communication will be first demonstrated on the bench-top and then with NHP.
Expected results and impact:
Scientific and technological contributions to the foundation of new future technology. We are an inflection point with regard to machine learning and human-machine interaction. B-CRATOS establishes a new platform for fully implanted wireless direct communication and control with the internet and machines. Paradigm shift is underway from pharmaceuticals with global actions and subsequent side effects to medical devices with local, targeted action. Communications previously used in physics and telecommunications (microwave, RFID backscatter) will be brought into the human body. Two-way, always-on wireless implant for recording and stimulation/neuromodulation of neurons in any location in the central or peripheral nervous system with individual neuron resolution and continuous 24/7 operation without medical supervision, enabling learning and brain plasticity necessary for complete integration of sensors and prosthetics. Advanced machine learning algorithms will accelerate this integration and improve the user’s quality of life. Secure, encrypted intra-body data network: Will enable brain-organ communication, intra-body sensor network, wearable exoskeletons, bionic arm control, etc. Two-way high-speed brain-computer link: Real-time sensing and stimulation for augmented reality, learning (cognition) and gaming (sensation and movement). Electronic Skin: Self-powered mechano-electric transduction-based tactile sensing enabling large-area, self-powered electronic skin with high spatial resolution. Beneficial for human-like perception capability with unique material and form recognition features. Can be used for treating burn injuries, augmented reality, gaming, etc. The mechanical to electrical power generation could be used to power implanted medical devices. Impact on innovation in Neural Disease, Electroceutical, AI, Exoskeleton and Robotics sectors: The B-CRATOS wireless platform is compatible with all electrode sensor and effector types, and can be ported into existing in-body closed-loop systems to increase the number of electrode channels, speed, and resolution.