Final Report Summary - ICONNECT (Intracranial COnnection with Neural Networks for Enabling Communication in Total paralysis) ICONNECT aims to develop a new Brain-Computer Interface (BCI) solution for paralyzed people. The central goal is to prove that attempted speech and hand gestures (for using sign language to communicate) could be decoded in people who lost the faculty of speech or communication due to severe paralysis, by recording the electrical signals generated in the human sensorimotor cortex. The sensorimotor cortex is the single gateway for the brain to generate muscle movements, and is therefore the most attractive brain region to decode attempted actions from. Attempted actions can only be studied in paralyzed people, or in some cases amputees, and they generate the same neural activity that is generated by actual movements. It is only the nerves that convey movement instructions to the muscles, that are not functioning. We found strong evidence for this in a study with people who had lost one arm, which makes it possible to conduct decoding research in abled people and extrapolate the findings to paralyzed people. Most of the research in iCONNECT was conducted with abled people, including healthy volunteers and amputees for 7 Tesla high-resolution functional MRI research, and epilepsy patients for research with high-density electrode grids implanted together with their clinical grids for seizure localization. With these methods we interrogated the brain with high detail, looking for ways to identify when our subjects were making instructed movements, and which movements they made from a limited set.Earlier work from our lab and others have indicated feasibility, but to justify use of a brain implant requires exceptionally high decoding performance. In iCONNECT we investigated all aspects of the use of attempted speech and hand gestures for BCI, and divided the work in 4 work packages addressing optimal recording parameters, decoding algorithms, feedback parameters, and the interaction between user and BCI implant, while recognizing the importance of the inclusion of the BCI user in our developmental work. We made use of the opportunity to conduct experiments with implanted electrodes in patients who are implanted for finding the source of seizures in epilepsy but have time to participate in our research, and for the other subjects we used one of only 40 existing high-field 7 Tesla MRI scanners to obtain the resolution we needed. In addition, we implanted two people with Locked-in Syndrome with the worldwide first fully implantable BCI system that works at home without expert help, that gave us the unique opportunity to work closely together with the first BCI implant users for several years. In a total of 35 research projects, we made significant progress in all 4 work packages. Whereas in the first half of the project we primarily focused on recording and decoding, in the second half we also focused on the implanted participants and how the BCI system could meet their needs (feedback and interaction). We significantly improved decoding algorithms for high density ECoG data, and confirmed feasibility of decoding selectively from somatosensory cortex. Key to this is the incorporation of neurobiologically driven constraints, based on spatial, temporal and spectral features of electrical brain activity. The confirmation provided by our research that brain activity generated during executed movements in healthy volunteers closely matches brain activity generated during attempted movements in amputees made it possible to identify the regions on the sensorimotor cortex that provide the most decodable information. We also confirmed that functional MRI is a very reliable tool for identifying those informative regions in great detail. Together, iCONNECT has provided evidence that a new BCI system with more amplifier channels and electrodes than our current device (which has 4) will provide a significant improvement of the type of movements that can be decoded. This is crucial because to implant such a system in severely paralyzed people we need to provide very strong evidence that it will work.Apart from discovering the neural codes for attempted movements and how we can decode them, we learned a lot about the interaction between BCI system and the user. The many recording, decoding and feedback parameters we managed to optimize using models and advanced multivariate analyses, did not always match what the user preferred. Fast processing and displaying of decoded brain signals provided the best performance mathematically, but for users was much too ‘nervous’. Hence, we developed additional software allowing the user to adjust parameters to their liking, thereby slightly sacrificing performance but greatly improving user experience.For speech, we exhaustively investigated various basic speech elements (phonemes), the results of which will be incorporated in a library of elementary neural speech features. Decoding realtime speech will require significantly more research, but with iCONNECT we discovered how to conduct that research, and gained confidence that it will be possible to decode attempted speech in the future. The challenge lies in the speed with which complex muscle movements follow one another, and the fact that the same phoneme in one word can be generated with different muscle sequences in another word, and this will require an expansion of the library. With the evidence we generated in iCONNECT that a new, high-channel-count BCI implant will produce the detail and richness that we expect, and the parameter spaces we can now define, the next step is to implant such systems in paralyzed people. Chronic implantation is the only way that enough data can be collected and parameter settings can be tested, to comprehensively build the speech library.
