Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications

Información del proyecto

JARVIS

Identificador del acuerdo de subvención: 101135708

DOI

10.3030/101135708

Fecha de la firma de la CE 20 Octubre 2023

Fecha de inicio 1 Enero 2024

Fecha de finalización 31 Diciembre 2027

Financiado con arreglo a

• Digital, Industry and Space

Coste total

€ 10 338 223,75

Aportación de la UE

€ 9 319 266,26

Coordinado por

PANEPISTIMIO PATRON
Greece

Periodic Reporting for period 1 - JARVIS (Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications)

Período documentado: 2024-01-01 hasta 2024-12-31

Resumen del contexto y de los objetivos generales del proyecto

The JARVIS project, as an Innovation Action, aspires to bring together a team of “Best in Class” experts that can combine all the aforementioned technological enablers to:
“develop a reusable set of tools that enable AI driven multimodal means of interaction: a) involving interfaces for physical and remote information exchange, robot control and programming, b) providing social skills to a variety of robots to achieve seamless user-centric interaction that extends human ability for complex tasks and c) demonstrating scalability of application and ability to achieve economies at scale.”

The proposed technologies will be tested for flexibility, scalability, trustworthiness and user acceptance in four use cases, aiming to demonstrate production line and cross sector applicability: Pillar 1 - Agile Manufacturing: a) Aeronautics production (COLLINS-BE), b) Automotive production (TOFAS) ii) Pillar 2 - Inspection and Maintenance (I&M): c) Energy – Nuclear plants decommissioning (EDF), d) Energy – offshore energy production (EQUINOR).

The workplan of JARVIS has as a target to comply with the following 6 objectives:
Objective O1: Development of AI enhanced means of interaction for seamless human-robot communication, control and programming
Objective O2: Development of socially interactive robots
Objective O3: Security, privacy and safety towards trustworthy AI
Objective O4: Development of cognitive and intelligent mechatronics for advanced HRI, dynamic scene modeling and understanding
Objective O5: Application of HRI in large scale pilots
Objective O6: Boosting companies, in particular SMEs and start-ups, to adopt industrial HRI solutions

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

During the first reporting period (M1-M12) the effort was focused on:
i) Definition of Industrial pilot cases scenarios & validation metrics
ii) Requirement analysis and the specifications for the technologies to be developed
iii) Definition of architecture for interoperable interaction modules, which complies with the requirements
iv) Establishment of the exploitation and dissemination strategy to be followed
v) Initialization of the development of the first prototypes of the JARVIS enabling technologies (to be delivered on M18)

Thus, consortium managed to achieve three Milestones (MS1, MS2), by the submission of the relevant deliverables.

The definition of the architecture enabled us to define 13 software and hardware modules which are currently on the developing phase towards the first prototypes, namely:
- Robot Control Module (RCM)
- Human-Robot Interaction Module (HRIM)
- VR Teleoperation (VRT)
- Generation Of Robot Programs (GORP)
- Teaching By Demonstration Module (TDM)
- Robot Behavior Adaptation (ROBA)
- User-Centric Interfaces (UCI)
- OpenFlow Orchestrator (OFO)
- Multi-Sensory Perception (MSP)
- Human Intention Perception And Prediction (HIPAP)
- Task Planner Module (TPM)
- Intelligent DTs (IDT)
- Smart Mechatronics Control (SMC)

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

The JARVIS solutions have the potential to provide wider scientific, economic and societal effects, contributing to the expected impacts as follows:
- Reduction of human exposure to harmful environments through teleoperation: reduction of up to 50%, to be demonstrated in two use cases
- Operator acceptance level: acceptance score of >60% comparing to the baseline
- Reduction of reconfiguration and programming time: decrease of 30% of robot programming time compared to conventional techniques
- Portfolio of re-usable HRI tools that can be applied in various use-cases/sectors: demonstration of application of the JARVIS tools in 4 internal and 15 external use cases
- Productivity increase: decrease of cycle time by 40% in average
- Ergonomic conditions for HRC vs manual: ergonomics scores (RULA/ OCRA/ NIOSH) and acceptance for human involvement: reduction 15%
- Impact creation in the academic community: peer review papers submitted to well-known scientific journals and conferences at the end of the project: 20
- Impact creation through pilots’ dissemination: videos of each case study published inside wide range social media and industrial/ academic networks: 10
- Impact creation through involvement of SMEs: involvement/funding of >24 SMEs through open calls