Multivariate analysis for the Imaging of Neuronal activity using Deep architectureS

Project Information

MINDS

Grant agreement ID: 659860

DOI

10.3030/659860

Project closed

EC signature date 30 March 2015

Start date 11 January 2016

End date 10 January 2018

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 212 194,80

EU contribution

€ 212 194,80

Coordinated by

DANMARKS TEKNISKE UNIVERSITET
Denmark

Periodic Reporting for period 1 - MINDS (Multivariate analysis for the Imaging of Neuronal activity using Deep architectureS)

Reporting period: 2016-01-11 to 2018-01-10

Summary of the context and overall objectives of the project

Functional magnetic resonance imaging (fMRI) is the dominating approach to research in the mapping of neural activity in the human brain. State of the art data analysis techniques employ a statistical parametric mapping (SPM) strategy to convert raw signal into interpretable images by processing data in a pipeline of task-specific modules. This approach, despite its simplicity and reliability, presents a set of inconveniences, including low interconnectivity among modules, resulting in suboptimal solutions. The main problem is that each individual module within the pipeline works in isolation; that is, the required parameters are (often manually) chosen by an expert without any feedback about the implications of this choice to the subsequent modules.

The goal of the action is to make a major contribution to the field by replacing the step-by-step data processing pipeline by a deep neural network. We hypothesise that this new approach will achieve better solutions by propagating the effects of module-based decisions through the network, jointly optimising the whole processing pipeline. We aim at establishing a new paradigm that greatly improves optimisation and interpretability of state of the art data processing pipelines for fMRI.

By approaching the problem in the way it is explained in the previous paragraph, we are also tackling a set of other subgoals, as the use of multivariate techniques to account for interdependencies among features, as well as the ability to capture existing non-linear relationships in the data. Moreover, the proposed architecture can be used to analyse the resulting data processing pipelines, a tool that helps in better understanding the influence that each module has to the final task being investigated.

It is also important to assess the complexity of transferring these networks to other data acquisition modalities, such as electroencephalography (EEG), where temporal resolution plays an important role, and evaluate the technology that best adapts to these constraints. In this respect, we developed a system using cutting-edge computer vision techniques to automatically score and interpret sleep stages by means of EEG data, showing promising results towards its applicability in a neurofeedback environment, where subjects learn relaxation strategies guided by EEG technology.

Success in tackling the aforementioned goals has a great impact in improving the quality of data processing pipelines for fMRI and EEG; by providing a set of improved tools for analysing neurological data; which ultimately will aid in better understanding the brain functioning for both healthy and pathologic brains, and even help in the treatment of certain diseases.

In the current project, we have demonstrated the feasibility of achieving the proposed goals and released a freely accessible easy-to-use GPU-powered framework to perform automatic multivariate non-linear data analysis of neurological signals. We think that these achievements will contribute to paving the path for other research projects to adopt a similar approach and take advantage of the rapidly growing deep neural network field to be applied in those areas where advanced data processing of neurological measurements is required.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have created specific modules for the realignment, corregistration, normalisation and smoothing steps of the fMRI data processing pipeline as layers in a deep neural network, which are optimised as a whole, all parameters being selected automatically based on the task performed by the last layer of the network (i.e. the specific data analysis under investigation).

We have used these pre-processing modules for brain state prediction by designing a layer in the neural network dealing with this data analysis task. The approach used is multivariate in nature and easily transferrable to other data analysis tasks, by defining specific layers for each analytical task of interest. Data non-linear relationships have been modelled through the activation functions in each layer.

EEG modality has also been approached by treating it as a visualisation task using Convolutional Neural Networks. Recurrent Neural Networks have become a successful approach to deal with time-aware signals, being it a good candidate to be adapted in fMRI to alleviate its bad time resolution.

We have designed a tool based on the gradient flow in the neural networks (both fMRI and EEG pipelines) to analyse and understand the functioning of such networks, hence understanding the impact that each of the steps in the pipeline has with respect to the final task.

The software obtained can be freely downloaded from the project’s website, together with the published technical manuscripts and presentations. All the achievements in the project are being presented in international conferences to professionals from a variety of backgrounds.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

To the best of our knowledge, this has been the first attempt to formalise the successful SPM data processing pipeline in form of a neural network. Therefore, the approach itself is a change of paradigm with respect to what has been done in the field, and all the obtained results represent a novelty.

As such, this new approach provides a successful use case in form of the architectural design and implementation of the data processing pipeline for fMRI as a neural network, fostering interest of the community to continue improving the pipelines following our strategy. The added value of the provided tools to analyse the impact of each module within the pipeline is a key element of its success.

From an application point of view, the data processing pipeline for EEG we have created to score sleep stages is to be used by the medical community to alleviate tedious manual scoring, easing their daily practise and, therefore, improving quality in the diagnosis of sleep disorders. Moreover, it presents a novel approach to build systems that are called to improve the state of the art technology for neurofeedback treatment.

fMRI data processing pipeline