Project description
Decoding your thoughts from brain activity
Recent findings reveal that the training data of deep neural networks (DNNs) can be decoded directly from their parameters, raising significant data privacy concerns. This breakthrough challenges the assumption that training data is secure, while also shedding light on the performance of DNNs. In this context, the ERC-funded MindReading project explores a parallel question: can we decode a person’s sensory experiences or thoughts from brain activity? This could revolutionise communication for ‘locked-in’ patients and advance brain-machine interfaces. Despite differences, both DNNs and human brains share similarities. The project aims to develop tools to translate between brain activity and DNN activations, offering profound insights into both fields and promising innovations in privacy and machine learning.
Objective
Can we decode the training data of a Deep Neural Network (DNN) directly from its parameters?
Training data of DNNs are assumed safe. Recent findings by us and by others indicate that this is not the case, with severe implications on Data-Privacy. Yet, such findings shed light on why DNNs perform so well.
On a different front: Can we decode what a person saw/heard/thinks directly from their brain activity?
This may have huge benefits: communicate with “locked-in” patients, explore dreams, man-machine interfaces, enhance our understanding of the human brain. No risk of violating human privacy here, as thoughts do not “float” in the air and person’s collaboration is required.
Each of those 2 questions is intriguing on its own, with far-reaching implications. Despite the inherent differences between Human Brains & DNNs, they also have much in common. Exploring the two in-tandem can lead to significant breakthroughs in both fields. Recent advancements in both areas, with recent incorporation of Deep-Learning (DL) tools to analyze brain activity, opens the door to explore the two jointly. Our expertise in both domains will enable explicit Encoding/Decoding between Brain activity & DNN activations, allowing to directly learn/infer from one about the other. Initial explorations indicate that our proposed goals, although ambitious, are within reach. Our intermediate goals in each domain are worthwhile on their own, forming a strong safety net.
Expected outcomes include:
•Deep Data-Privacy
•Insights on DNNs, their Generalization & Vulnerabilities
•Insights on of “what is encoded where” in the brain
•New scientific tools for brain-scientists to explore the brain
•Allow “locked-in” (ALS) patients to communicate their thoughts/needs
•Use Brain scanning to improve DNNs & DNNs to improve Brain scanning
Our project requires no human subjects nor BrainScience expertise; only publicly available datasets. All methods lie in Computer-Vision & DL, with impact on both DL & BrainScience.
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.1 - European Research Council (ERC)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-ERC - HORIZON ERC Grants
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) ERC-2023-ADG
See all projects funded under this callHost institution
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
7610001 Rehovot
Israel
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.