Periodic Reporting for period 2 - PILLAR-Robots (Purposeful Intrinsically motivated Lifelong Learning Autonomous Robots)
Período documentado: 2023-10-01 hasta 2025-03-31
PILLAR-Robots aims to develop a new generation of highly autonomous robots capable of setting their own goals and strategies based on acquired experience, meeting the needs of human users in real-life applications up to TRL5. The project leverages the concept of Purpose from cognitive sciences to enhance robots' autonomy and enable them to acquire pertinent knowledge and skills for specific applications. It will create algorithms for robots to acquire purpose, bias their cognitive systems, and learn representations and skills aligned with their goals. Demonstrators will be implemented and validated in Agri-food, Edutainment, and unstructured Industrial/retail domains.
During this second reporting period, research efforts have advanced significantly across WP2–WP5, consolidating the foundations laid in RP1:
WP2: focused on grounding multimodal perception modules and aligning them with purposes. Work included fine-tuning perception modules, exploring real-world implementation (e.g. in Edutainment scenarios), and adapting visual-language models (e.g. Flamingo, BLIP-2) for real-time understanding. Efforts also explored joint training across perception, language, and action for better integration and grounding.
WP3: progressed in modelling motivational mechanisms. The integration of purpose inference and motivational dynamics was deepened using tools like CLING and enhanced GRAIL modules. There was notable work on goal detection and motivational balancing strategies, leading to more autonomous learning in robots. Cross-WP integration with perception and planning components also intensified.
WP4: focused on implementing learning algorithms that align robot behaviours with defined purposes. Notable activities included bootstrapping RL strategies via LLMs, deploying planning modules to steer behaviour according to inferred goals, and ensuring skill transfer and generalization. These developments enabled more adaptive and reusable policies.
WP5: reached major milestones in refining the cognitive architecture v1 and exploring v2. Implementations were consolidated in ROS 2 Humble and integrated with modules from other WPs. Additionally, flexible software interfaces were tested to ensure modularity. Efforts also targeted long-term memory representation and the management of abstract knowledge components.
Work in WP7–WP9 has intensified since M24, focusing on deploying demonstrators in agrifood, edutainment, and industrial/retail contexts. Activity centered on developing realistic scenarios, refining KPIs, and integrating key modules in physical robots. Real-world testing started, including purpose-driven behaviour and multimodal interaction with users.
WP2: focused on grounding multimodal perception modules and aligning them with purposes. Work included fine-tuning perception modules, exploring real-world implementation (e.g. in Edutainment scenarios), and adapting visual-language models (e.g. Flamingo, BLIP-2) for real-time understanding. Efforts also explored joint training across perception, language, and action for better integration and grounding.
WP3: progressed in modelling motivational mechanisms. The integration of purpose inference and motivational dynamics was deepened using tools like CLING and enhanced GRAIL modules. There was notable work on goal detection and motivational balancing strategies, leading to more autonomous learning in robots. Cross-WP integration with perception and planning components also intensified.
WP4: focused on implementing learning algorithms that align robot behaviours with defined purposes. Notable activities included bootstrapping RL strategies via LLMs, deploying planning modules to steer behaviour according to inferred goals, and ensuring skill transfer and generalization. These developments enabled more adaptive and reusable policies.
WP5: reached major milestones in refining the cognitive architecture v1 and exploring v2. Implementations were consolidated in ROS 2 Humble and integrated with modules from other WPs. Additionally, flexible software interfaces were tested to ensure modularity. Efforts also targeted long-term memory representation and the management of abstract knowledge components.
Work in WP7–WP9 has intensified since M24, focusing on deploying demonstrators in agrifood, edutainment, and industrial/retail contexts. Activity centered on developing realistic scenarios, refining KPIs, and integrating key modules in physical robots. Real-world testing started, including purpose-driven behaviour and multimodal interaction with users.
RESULTS
(1) Novel algorithmic models;
(2) integrated cognitive architecture based on modular subsystems;
(3) ready to use solution (software + interface);
(4) adapted hardware component;
(5) innovative test and validation systems;
(6) small-scale demonstrator in 3 use cases,
(7) more structured PIMOL environment with agreements with 10 EU PPPs and platforms, and
(8) business agreements with 10 associated partners to further develop the demonstrators under TRL6 and beyond.
MEDIUM TERM IMPACTS
(1) The concept of Purpose is operational in 3 novel algorithmic models.
(2) Completion/integration of PIMOL architecture autonomy
(3) Successful validation of the demonstrators of PIMOL autonomy in lab and in real applications within 3 different application domains characterized
(4) Successful evaluation of PIMOL autonomous robots, (a) technological impact (performance, functionality), and (b) societal repercussions (socio-economic, ethics, legal dimensions).
(5) Target stakeholders: ‘friendly stakeholders’ (50) are involved and ‘associated partners’ (10) are engaged at the end of the project
LONG TERM IMPACTS
Industrial: more aligned industrial players with business agreements leading to further demonstration (10), success stories in the form of tech transfer agreements (5), products distributed among early adopters (20), maintenance / consultancy / training (500 person/days)
Business: 20% savings / productivity gain), automatised processes (5 average per chain), staff supervising (20% decrease)
PIMOL community: more structured environment with collaboration agreements with platforms, sister projects, public-private partnerships (10),
‘friendly stakeholders’ involved (50), bilateral meeting held with stakeholders, including regulators (25)
Technological: early adopters (10), endorsement by prescribers and other stakeholders (5)
Scientific: New breakthroughs presented in peer-reviewed journals (10), papers communications to international meetings (10)
Wider dissemination: Participatory events organised & attended (20), Presence impacts in the Internet (2000)
(1) Novel algorithmic models;
(2) integrated cognitive architecture based on modular subsystems;
(3) ready to use solution (software + interface);
(4) adapted hardware component;
(5) innovative test and validation systems;
(6) small-scale demonstrator in 3 use cases,
(7) more structured PIMOL environment with agreements with 10 EU PPPs and platforms, and
(8) business agreements with 10 associated partners to further develop the demonstrators under TRL6 and beyond.
MEDIUM TERM IMPACTS
(1) The concept of Purpose is operational in 3 novel algorithmic models.
(2) Completion/integration of PIMOL architecture autonomy
(3) Successful validation of the demonstrators of PIMOL autonomy in lab and in real applications within 3 different application domains characterized
(4) Successful evaluation of PIMOL autonomous robots, (a) technological impact (performance, functionality), and (b) societal repercussions (socio-economic, ethics, legal dimensions).
(5) Target stakeholders: ‘friendly stakeholders’ (50) are involved and ‘associated partners’ (10) are engaged at the end of the project
LONG TERM IMPACTS
Industrial: more aligned industrial players with business agreements leading to further demonstration (10), success stories in the form of tech transfer agreements (5), products distributed among early adopters (20), maintenance / consultancy / training (500 person/days)
Business: 20% savings / productivity gain), automatised processes (5 average per chain), staff supervising (20% decrease)
PIMOL community: more structured environment with collaboration agreements with platforms, sister projects, public-private partnerships (10),
‘friendly stakeholders’ involved (50), bilateral meeting held with stakeholders, including regulators (25)
Technological: early adopters (10), endorsement by prescribers and other stakeholders (5)
Scientific: New breakthroughs presented in peer-reviewed journals (10), papers communications to international meetings (10)
Wider dissemination: Participatory events organised & attended (20), Presence impacts in the Internet (2000)