Project description
Unlocking the power of multimodal AI
AI models are becoming more powerful, but using them across different types of data (like text, images, and medical signals) remains a complex task. Multimodal foundation models (MMFM) could change that by enabling flexible, efficient learning across domains. With this in mind, the EU-funded DVPS project is working to make MMFM more accessible and robust by tackling challenges such as cross-modal alignment and architecture design. It will deliver practical tools, including a model design toolkit and benchmarking suite, to support development and evaluation. Applications range from cardiology to geo-intelligence, with experimental domains added to push innovation further. The project also promotes ethical AI practices and open collaboration to strengthen Europe’s position in cutting-edge AI research.
Objective
"Foundation models represent a paradigm shift in AI, exhibiting remarkable capabilities across multiple tasks. Their true potential lies in generalizing across diverse domains and modalities, a largely untapped frontier. DVPS advances this frontier by focusing on multimodal foundation models (MMFM), aiming to harness their capabilities across various application domains.
DVPS emphasizes three core benefits of MMFM: label efficiency, compute reusability, and engineering efficiency. However, achieving these benefits in multimodal settings presents challenges such as modality-specific architecture and cross-modal alignment. To overcome these, DVPS aims to develop generalizable methods that work across diverse modalities and domains, creating a unified framework for MMFM development and integrating new modalities into existing models.
The project focuses on generating foundational knowledge, delivering tested methods, and creating algorithms to expand MMFM capabilities across domains like cardiology, geo-intelligence, and language communication. DVPS also includes two ""surprise domains"" to drive innovation by challenging initial assumptions.
Key objectives include the development of AutoDVPS, a toolkit for automated MMFM design, and the creation of DVPSBench, a benchmarking suite for evaluating MMFM across tasks and domains. DVPS aims to foster a European ecosystem for MMFM research, promoting transparency, fairness, and ethical compliance in line with European values. Through collaboration and open-source contributions, DVPS seeks to standardize and advance MMFM as a scientifically rigorous discipline."
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- natural sciences biological sciences ecology ecosystems
- medical and health sciences clinical medicine cardiology
You need to log in or register to use this function
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.2.4 - Digital, Industry and Space
MAIN PROGRAMME
See all projects funded under this programme -
HORIZON.2.4.5 - Artificial Intelligence and Robotics
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-RIA - HORIZON Research and Innovation Actions
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) HORIZON-CL4-2024-HUMAN-03
See all projects funded under this callCoordinator
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
00144 ROMA
Italy
See on map
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Participants (17)
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
00144 Rome Rm
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
OX1 2JD Oxford
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
38122 Trento
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
2400 Mol
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
69120 Heidelberg
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
30-059 Krakow
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
44121 Ferrara
See on map
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
1070 WIEN
See on map
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
08007 Barcelona
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
1081 HV Amsterdam
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
76131 Karlsruhe
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
20122 MILAN
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
00198 Rome
See on map
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
NW1 2DB London
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
08035 BARCELONA
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
SW7 2AZ London
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
10115 Berlin
See on map
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Partners (2)
Partner organisations contribute to the implementation of the action, but do not sign the Grant Agreement.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
1015 Lausanne
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.
Partner organisations contribute to the implementation of the action, but do not sign the Grant Agreement.
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
8092 Zuerich
See on map
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.