Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications

Informazioni relative al progetto

JARVIS

ID dell’accordo di sovvenzione: 101135708

DOI

10.3030/101135708

Data della firma CE 20 Ottobre 2023

Data di avvio 1 Gennaio 2024

Data di completamento 31 Dicembre 2027

Finanziato da

• Digital, Industry and Space

Costo totale

€ 10 338 223,75

Contributo UE

€ 9 319 266,26

Coordinato da

PANEPISTIMIO PATRON
Greece

Periodic Reporting for period 1 - JARVIS (Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications)

Periodo di rendicontazione: 2024-01-01 al 2024-12-31

Sintesi del contesto e degli obiettivi generali del progetto

The JARVIS project, as an Innovation Action, aspires to bring together a team of “Best in Class” experts that can combine all the aforementioned technological enablers to:
“develop a reusable set of tools that enable AI driven multimodal means of interaction: a) involving interfaces for physical and remote information exchange, robot control and programming, b) providing social skills to a variety of robots to achieve seamless user-centric interaction that extends human ability for complex tasks and c) demonstrating scalability of application and ability to achieve economies at scale.”

The proposed technologies will be tested for flexibility, scalability, trustworthiness and user acceptance in four use cases, aiming to demonstrate production line and cross sector applicability: Pillar 1 - Agile Manufacturing: a) Aeronautics production (COLLINS-BE), b) Automotive production (TOFAS) ii) Pillar 2 - Inspection and Maintenance (I&M): c) Energy – Nuclear plants decommissioning (EDF), d) Energy – offshore energy production (EQUINOR).

The workplan of JARVIS has as a target to comply with the following 6 objectives:
Objective O1: Development of AI enhanced means of interaction for seamless human-robot communication, control and programming
Objective O2: Development of socially interactive robots
Objective O3: Security, privacy and safety towards trustworthy AI
Objective O4: Development of cognitive and intelligent mechatronics for advanced HRI, dynamic scene modeling and understanding
Objective O5: Application of HRI in large scale pilots
Objective O6: Boosting companies, in particular SMEs and start-ups, to adopt industrial HRI solutions

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

During the first reporting period (M1-M12) the effort was focused on:
i) Definition of Industrial pilot cases scenarios & validation metrics
ii) Requirement analysis and the specifications for the technologies to be developed
iii) Definition of architecture for interoperable interaction modules, which complies with the requirements
iv) Establishment of the exploitation and dissemination strategy to be followed
v) Initialization of the development of the first prototypes of the JARVIS enabling technologies (to be delivered on M18)

Thus, consortium managed to achieve three Milestones (MS1, MS2), by the submission of the relevant deliverables.

The definition of the architecture enabled us to define 13 software and hardware modules which are currently on the developing phase towards the first prototypes, namely:
- Robot Control Module (RCM)
- Human-Robot Interaction Module (HRIM)
- VR Teleoperation (VRT)
- Generation Of Robot Programs (GORP)
- Teaching By Demonstration Module (TDM)
- Robot Behavior Adaptation (ROBA)
- User-Centric Interfaces (UCI)
- OpenFlow Orchestrator (OFO)
- Multi-Sensory Perception (MSP)
- Human Intention Perception And Prediction (HIPAP)
- Task Planner Module (TPM)
- Intelligent DTs (IDT)
- Smart Mechatronics Control (SMC)

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

The JARVIS solutions have the potential to provide wider scientific, economic and societal effects, contributing to the expected impacts as follows:
- Reduction of human exposure to harmful environments through teleoperation: reduction of up to 50%, to be demonstrated in two use cases
- Operator acceptance level: acceptance score of >60% comparing to the baseline
- Reduction of reconfiguration and programming time: decrease of 30% of robot programming time compared to conventional techniques
- Portfolio of re-usable HRI tools that can be applied in various use-cases/sectors: demonstration of application of the JARVIS tools in 4 internal and 15 external use cases
- Productivity increase: decrease of cycle time by 40% in average
- Ergonomic conditions for HRC vs manual: ergonomics scores (RULA/ OCRA/ NIOSH) and acceptance for human involvement: reduction 15%
- Impact creation in the academic community: peer review papers submitted to well-known scientific journals and conferences at the end of the project: 20
- Impact creation through pilots’ dissemination: videos of each case study published inside wide range social media and industrial/ academic networks: 10
- Impact creation through involvement of SMEs: involvement/funding of >24 SMEs through open calls