Project description
Unlocking the power of multimodal AI
AI models are becoming more powerful, but using them across different types of data (like text, images, and medical signals) remains a complex task. Multimodal foundation models (MMFM) could change that by enabling flexible, efficient learning across domains. With this in mind, the EU-funded DVPS project is working to make MMFM more accessible and robust by tackling challenges such as cross-modal alignment and architecture design. It will deliver practical tools, including a model design toolkit and benchmarking suite, to support development and evaluation. Applications range from cardiology to geo-intelligence, with experimental domains added to push innovation further. The project also promotes ethical AI practices and open collaboration to strengthen Europe’s position in cutting-edge AI research.
Objective
"Foundation models represent a paradigm shift in AI, exhibiting remarkable capabilities across multiple tasks. Their true potential lies in generalizing across diverse domains and modalities, a largely untapped frontier. DVPS advances this frontier by focusing on multimodal foundation models (MMFM), aiming to harness their capabilities across various application domains.
DVPS emphasizes three core benefits of MMFM: label efficiency, compute reusability, and engineering efficiency. However, achieving these benefits in multimodal settings presents challenges such as modality-specific architecture and cross-modal alignment. To overcome these, DVPS aims to develop generalizable methods that work across diverse modalities and domains, creating a unified framework for MMFM development and integrating new modalities into existing models.
The project focuses on generating foundational knowledge, delivering tested methods, and creating algorithms to expand MMFM capabilities across domains like cardiology, geo-intelligence, and language communication. DVPS also includes two ""surprise domains"" to drive innovation by challenging initial assumptions.
Key objectives include the development of AutoDVPS, a toolkit for automated MMFM design, and the creation of DVPSBench, a benchmarking suite for evaluating MMFM across tasks and domains. DVPS aims to foster a European ecosystem for MMFM research, promoting transparency, fairness, and ethical compliance in line with European values. Through collaboration and open-source contributions, DVPS seeks to standardize and advance MMFM as a scientifically rigorous discipline."
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: https://op.europa.eu/en/web/eu-vocabularies/euroscivoc.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: https://op.europa.eu/en/web/eu-vocabularies/euroscivoc.
- natural sciences biological sciences ecology ecosystems
- medical and health sciences clinical medicine cardiology
You need to log in or register to use this function
We are sorry... an unexpected error occurred during execution.
You need to be authenticated. Your session might have expired.
Thank you for your feedback. You will soon receive an email to confirm the submission. If you have selected to be notified about the reporting status, you will also be contacted when the reporting status will change.
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.2.4 - Digital, Industry and Space
MAIN PROGRAMME
See all projects funded under this programme -
HORIZON.2.4.5 - Artificial Intelligence and Robotics
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-RIA - HORIZON Research and Innovation Actions
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) HORIZON-CL4-2024-HUMAN-03
See all projects funded under this callCoordinator
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
00144 Roma
Italy
The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.