i) Develop electronic technologies for brain mapping to record from large number of active sites over large areas of the cortex. The g-SGFET technology has reached high maturity, achieving high performance and homogeneity, with yields close to 100%. We have validated the 2 different multiplexing strategies, time-domain and frequency-domain multiplexing (TDM and FDM), with arrays of 8x8 g-SGFET in acute in vivo experiments. The g-SGFETs technology has been used in chronically in vivo, demonstrating the potential of g-SGFET arrays for the study of wide frequency band LFP activity in freely moving animals. Regarding the scalability of the technology, arrays of 256 and 512 transistors have been already validated in saline. In addition, MoS2/Graphene flexible technology for multiplexing has been demonstrated by modelling, fabricating and testing monolithically integrated arrays.
Two 1024-channel compact ASICs have been designed, and fabricated in 0.18µm CMOS technology for the TDM and FDM multiplexing strategies of 32x32 GFET sensor arrays. Thanks to these novel strategies, the large-scale sensor arrays can be built from GFET-only devices with the corresponding simplification and reduction in costs of their manufacturability. The modular FPGA-based headstages incorporating the TDM and FDM ASICs have been also tested in vitro.
ii) Advance the fundamental understanding of the link between surface and intracortical signals and dynamics in cortical circuits. We have developed flexible cranial interface and performed first of a kind distributed recordings together with 3D behaviour, and a Green-function-based frequency domain procedure for LFP decomposition, and we have identified sparse and distributed anatomical basis for decoding of spatial representation from hippocampal LFP in theta frequency band. We have established joined depth and ECoG recordings in motor cortex combined with 3D limb analysis in head-fixed mice and identified contribution of theta dynamics to motor cortex activity; we have performed first wireless recordings of vocal and cortical ECoG activity in minipigs and found activation of motor cortex at single trial level.
iii) Gain new fundamental understanding of the distributed brain circuits of speech and their plastic flexibility before and after lesions. We characterized brain dynamics underlying overt and inner speech production in humans, both at the whole brain level using non-invasive, and more locally using invasive large-scale electrophysiology. ECoG datasets were collected in patients for word level speech production, characterizing the covert speech network. We have also developed original decoding methods, and characterization of best neural signals to be decoded. Intracortical activity was collected during covert and covert speech in Broca area, and found significant differences in the ensemble dynamics between both conditions, suggesting that this area of speech production is differently engaged between overt and covert production.
iv) Development of speech BCI proof of concept. We developed a software framework to customize real-time processing of data streams, incorporating real-time DNN-based articulatory-to-acoustic speech synthesis for closed-loop BCI applications. A first closed-loop speech BCI paradigm was tested in an epileptic patient. We developed a neuromorphic approach to perform online spike sorting that anticipates the advent of BrainCom technology for large scale neural recordings with low power. We developed deep-learning methods to decode neural activity underlying overt speech production. Intelligible speech reconstruction from cortical activity could be obtained for a number of speech items and in particular sequences of vowels. We have obtained preliminary decoding of covert speech production from cortical activity.
v) Developed an ethics framework to identify and explore issues linked to the use of brain implants. Collaborative research addressed questions generated by neural decoding techniques, including the prospect for involuntary speech and inadvertent ‘mind reading’. Further work addressed questions concerning user responsibility, shared control of devices, and the status of ‘brain data’. We addressed ethical policymaking for neurotechnologies with, e.g. contributions to think tanks, and to UK parliamentary understanding of neuroprosthetic technology – its opportunities and challenges. Wider audiences were reached via media interviews and debates, and via a substantial interactive ethics section of the BrainCom virtual reality app.
