Project description
Smarter household robots through vision, touch and language
While robotics has transformed the factory floor, household robots have not kept pace. They still struggle to navigate the unpredictable chaos of a home, largely because they cannot fully understand human intent or handle objects. Supported by the Marie Skłodowska-Curie Actions programme, the VITAL-Rim project will develop a tactile-vision-language-action (TVLA) framework that combines visual perception, natural language understanding, and real-time touch sensing to enable robots to perform everyday household tasks with greater precision and adaptability. The goal is to teach robots to interpret spoken instructions and refine their movements using constant tactile feedback. Ultimately, VITAL-Rim is moving us closer to a world where helpful robots are a fixture of our daily lives.
Objective
The VITAL-Rim project, short for VIsion-TActile-Language integration for Robotic interactive manipulation, aims to enhance household robots' ability to assist with daily chores by endowing three key capabilities: scene perception through vision, semantic understanding via language, and real-time force control through touch. Currently, the development of household robots lags behind that of industrial robots, primarily due to two VITAL RIMs in homes: the barrier between humans and robots, and the complicated interaction between robots and their environments. To address these challenges, this project will propose a novel slow-fast tactile-vision-language-action (TVLA) model that facilitates high-rate, dexterous control of robots in homes. To achieve this, the project will implement a three-tier hierarchical framework consisting of the VLA Semantic Layer, Slow Policy Layer, and Fast Action Layer. The VLA Semantic Layer will interpret language instructions and outline task sequences. The Slow Policy Layer will generate scene-adaptive action blocks, bridging the operational frequency gap between semantic actions and real contact scenarios. The Fast Action Layer will refine these actions in real-time using a proposed lightweight vision-tactile fusion model, ensuring precise force control and rapid responsiveness. The project will design a portable handheld data collector that collects multimodal data from real household chores, thereby generating a comprehensive dataset and providing a benchmark for the TVLA model training. Ultimately, this trained model will undergo thorough evaluation in both simulations and real-world robots.
Overall, the VITAL-Rim project will empower home robots to effectively understand and respond to natural language instructions, significantly improving their interactive manipulation skills in the contact-rich scenarios. This aligns with the work programme's focus on advancing robotics for practical applications in domestic environments.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- engineering and technology electrical engineering, electronic engineering, information engineering electronic engineering robotics
You need to log in or register to use this function
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.2 - Marie Skłodowska-Curie Actions (MSCA)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-TMA-MSCA-PF-EF - HORIZON TMA MSCA Postdoctoral Fellowships - European Fellowships
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) HORIZON-MSCA-2025-PF
See all projects funded under this callCoordinator
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
L69 7ZX LIVERPOOL
United Kingdom
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.