Project description
Solving the ‘black box’ challenge
As machine learning and AI advance, the complexity of models grows, making them increasingly opaque to human understanding. This phenomenon is referred to as the ‘black box’ problem. This opacity arises from an identifiability problem, where models can express the same pattern in countless ways within their internal representations. This lack of transparency hinders our ability to comprehend the insights generated by these models, leading to a dilemma: either accept the inscrutability of the results or sacrifice model complexity. In this context, the ERC-funded NoKnow project shifts the focus from the representations themselves to the tasks they solve. Using tools from differential geometry and Bayesian inference, the NoKnow project will ensure identifiable outcomes within the representation manifold.
Objective
Machine learning and artificial intelligence techniques are progressing at a tremendous pace and impressive results appear across scientific fields. However, as machine learning models grow in capacity, they become increasingly ‘black box’, and it becomes harder for humans to reason about the patterns discovered by the machine. A root cause of this difficulty is that most machine learning models can express the same pattern in infinitely many, equally good, ways within their internal representations of the world. This is known as an identifiability problem.
Today we lack a general solution to identifiability problems, and either give up on understanding the patterns discovered by the machine or reduce model complexity to lessen the problem. The latter also reduces the fidelity and applicability of the model. NoKnow rephrase the question of identifiability to be concerned with tasks solved by the representation rather than the representation itself. Using tools from differential geometry and Bayesian inference, we develop the theoretical tools to ensure that tasks solved in the representation have an identifiable outcome even if the representations themselves are not identifiable.
To turn theory into practice, we develop state-of-the-art algorithms for assessing the uncertainty of learned representations in order to indirectly estimate the topology of the representation manifold. We further develop novel, high-fidelity predictive models that have identifiable outcomes when trained on learned representations.
NoKnow provides the fundamental tools needed to engage with learned representations to guarantee identifiable outcomes. This, in turn, increase trust in findings as they are not dependent on arbitrariness in learned representations. As society increasingly automates decisions, this trust in machine learning becomes ever more important.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- natural sciences mathematics pure mathematics topology
- natural sciences mathematics pure mathematics geometry
You need to log in or register to use this function
We are sorry... an unexpected error occurred during execution.
You need to be authenticated. Your session might have expired.
Thank you for your feedback. You will soon receive an email to confirm the submission. If you have selected to be notified about the reporting status, you will also be contacted when the reporting status will change.
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.1 - European Research Council (ERC)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-ERC - HORIZON ERC Grants
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) ERC-2023-COG
See all projects funded under this callHost institution
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
2800 KONGENS LYNGBY
Denmark
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.