Periodic Reporting for period 3 - FELICE (FlExible assembLy manufacturIng with human-robot Collaboration and digital twin modEls)
Reporting period: 2023-10-01 to 2024-12-31
FELICE addresses one of the greatest challenges in robotics, i.e. that of coordinated interaction and combination of human and robot skills. The project targets the application priority area of agile production and aspires to design the next generation assembly processes required to effectively address current and pressing needs in manufacturing, specifically increase the agility and productivity of a manual assembly production system, ensure the safety and improve the physical and mental well-being of factory workers. Our framework comprises of two layers, a local one introducing i) a single collaborative assembly robot that will roam the shop floor assisting workers, and ii) adaptive workstations able to automatically adjust to the workers’ somatometries and providing multimodal informative guidance and notifications on assembly tasks, and a global layer which will sense and operate upon the real world via an actionable digital replica of the entire physical assembly line. The objectives of the project are the following:
-Develop context awareness via ubiquitous sensing
-Ensure safety in human-robot collaborative assembly
-Amplify fluency and ergonomics in human-robot collaborative assembly
-Orchestrate the assembly process in real-time
-Realize collaborative assembly lines for agile production in real production environments
-Maximize the project impact
-Develop context awareness via ubiquitous sensing
-Ensure safety in human-robot collaborative assembly
-Amplify fluency and ergonomics in human-robot collaborative assembly
-Orchestrate the assembly process in real-time
-Realize collaborative assembly lines for agile production in real production environments
-Maximize the project impact
Some of the project highlights during this period are:
Improved collaborative robot with fluent movements and enhanced self localization with more accurate tool detection capabilities. The adaptive workstation (AWS) automatically positions work objects based on ergonomic principles to reduce the physical strain for workers. Integration and evaluation activities have been carried out during two phases at CRF premises. In the final phase, the validation of the manual and automatic operational modes was conducted. In the manual mode case, workers trigger functionalities via voice and touch interfaces, whereas in the later case the system autonomously manages the assembly line activities based on Orchestrator directives.
Work highlights per WP:
WP2: Prepared the basis for the R&D tasks and formalised the FELICE framework. The final refinements and consolidation of the specifications and requirements for the adaptive workstation, the robot platform, the system architecture, security and privacy as well as the HR-collaborative requirements were completed. A training program for trustworthy, acceptable and efficient collaboration was conceptualised. Altogether aiming for the improvement of hardware and software deployment as well as user-centred ergonomics, safety, and productivity.
WP3: The implemented advanced security mechanisms ensure the security and privacy of the data held in FELICE. The FELICE incorporates intrusion detection and vulnerability scanning, in order to identify potential exploitation points and weaknesses, classifying them and suggesting countermeasures. Visual interfaces for interaction between the user workstations were built for seamless information sharing between software modules.
WP4: Developed perception mechanisms used for a) the detection and estimation of the 6D pose of objects, with an online uncertainty metric to support grasping and prune failure cases, b) real-time robot localization via a state of the art visual SLAM pipeline coupled with an absolute appearance-based localization component, c) monitoring the movements, actions and ergonomic inducing postures of human workers during assembly activities, d) identifying spoken commands and gestures for natural human-robot interaction in industrial environments.
WP5: Developed the hardware platforms (robot and adaptive workstation) and the synergetic execution of assembly tasks considering safety, productivity and ergonomic issues, programmed the robot with a repertoire of robust and reusable actions and considered temporal aspects of human-robot collaboration.
WP6: The FELICE Digital Twin system has been developed and validated. It integrates a comprehensive dashboard for real-time monitoring of multiple key performance indicators (KPIs) that can be used for evaluating in simulated scenarios the impact of the FELICE system on the operational efficiency of the assembly line.
WP7: The integration and refinement of the Orchestrator was finalized. Several functionalities according to the three level error handling taxonomy handled in the system were tested.
WP8: New demos (manual and automatic), metrics and methods for the evaluation of the results related. Execution of tests for the technologies developed based on the FELICE evaluation methodology.
WP9: The project was presented at major events such as the European Robotics Forum (ERF) and relevant trade fairs such as the Automation and Testing fair in Turin, Italy, LogiMat in Germany or the Stellantis Factory Booster Day. Two videos have been produced to showcase FELICE.The results of the project will remain visible as FELICE contributions to 13 open source software repositories, four open datasets and scientific publications. FELICE achieved its goal of high quality academic publications by publishing 15 scientific papers in conferences and journals. Furthermore, FELICE developed a long-term exploitation and sustainability plan.
Improved collaborative robot with fluent movements and enhanced self localization with more accurate tool detection capabilities. The adaptive workstation (AWS) automatically positions work objects based on ergonomic principles to reduce the physical strain for workers. Integration and evaluation activities have been carried out during two phases at CRF premises. In the final phase, the validation of the manual and automatic operational modes was conducted. In the manual mode case, workers trigger functionalities via voice and touch interfaces, whereas in the later case the system autonomously manages the assembly line activities based on Orchestrator directives.
Work highlights per WP:
WP2: Prepared the basis for the R&D tasks and formalised the FELICE framework. The final refinements and consolidation of the specifications and requirements for the adaptive workstation, the robot platform, the system architecture, security and privacy as well as the HR-collaborative requirements were completed. A training program for trustworthy, acceptable and efficient collaboration was conceptualised. Altogether aiming for the improvement of hardware and software deployment as well as user-centred ergonomics, safety, and productivity.
WP3: The implemented advanced security mechanisms ensure the security and privacy of the data held in FELICE. The FELICE incorporates intrusion detection and vulnerability scanning, in order to identify potential exploitation points and weaknesses, classifying them and suggesting countermeasures. Visual interfaces for interaction between the user workstations were built for seamless information sharing between software modules.
WP4: Developed perception mechanisms used for a) the detection and estimation of the 6D pose of objects, with an online uncertainty metric to support grasping and prune failure cases, b) real-time robot localization via a state of the art visual SLAM pipeline coupled with an absolute appearance-based localization component, c) monitoring the movements, actions and ergonomic inducing postures of human workers during assembly activities, d) identifying spoken commands and gestures for natural human-robot interaction in industrial environments.
WP5: Developed the hardware platforms (robot and adaptive workstation) and the synergetic execution of assembly tasks considering safety, productivity and ergonomic issues, programmed the robot with a repertoire of robust and reusable actions and considered temporal aspects of human-robot collaboration.
WP6: The FELICE Digital Twin system has been developed and validated. It integrates a comprehensive dashboard for real-time monitoring of multiple key performance indicators (KPIs) that can be used for evaluating in simulated scenarios the impact of the FELICE system on the operational efficiency of the assembly line.
WP7: The integration and refinement of the Orchestrator was finalized. Several functionalities according to the three level error handling taxonomy handled in the system were tested.
WP8: New demos (manual and automatic), metrics and methods for the evaluation of the results related. Execution of tests for the technologies developed based on the FELICE evaluation methodology.
WP9: The project was presented at major events such as the European Robotics Forum (ERF) and relevant trade fairs such as the Automation and Testing fair in Turin, Italy, LogiMat in Germany or the Stellantis Factory Booster Day. Two videos have been produced to showcase FELICE.The results of the project will remain visible as FELICE contributions to 13 open source software repositories, four open datasets and scientific publications. FELICE achieved its goal of high quality academic publications by publishing 15 scientific papers in conferences and journals. Furthermore, FELICE developed a long-term exploitation and sustainability plan.
The major FELICE state of the art advances and innovations are:
WP2 advanced the state of the art by defining the requirements for the direct HR collaboration on the paradigm of Joint Activity theory guiding the derivation and implementation of respective criteria for a truly human–centered design of the joint collaborative work process.
WP3 innovations include enhancement of the security and privacy of the data held in the FELICE system by orchestrating different tools and techniques. In addition, it provides user-driven interfaces that put humans in the loop of the production line management.
WP4 advanced in multiple aspects by developing: a learning-based approach for object detection and localization with online confidence scoring; a feature-based, real-time absolute localization system with decimeter accuracy; a human behavior understanding system estimating task cycle progress and ergonomically assessing postures during assembly; and a resilient speech command recognition system.
WP5 advanced the state of the art in multiple aspects including the design, development and manufacturing: of an adaptive workstation able to dynamically adapt to worker requirements; a 8 DOF mobile collaborative robot; implementing a human robot interaction module for fluent collaboration between a human and the robot.
WP6 advanced the state of the art by proposing a Digital Twin (DT) solution that is part of a larger platform with multiple modules. The state of the art highlights that simulation DT are starting to emerge only as stand-alone applications. Furthermore, the FELICE DT includes the human in the loop and makes use of Virtual Reality.
WP7 advanced the state of the art by devising novel planning models and applications for evolutionary algorithms.
WP8 innovations include the realization of the FELICE system and new methods for production process implementation and evaluation.
WP9 established the framework for exploitation and innovation by defining business models for 16 FELICE project assets as well as exploitation roadmaps for its three innovation clusters. Communication and dissemination activities support exploitation efforts.
WP2 advanced the state of the art by defining the requirements for the direct HR collaboration on the paradigm of Joint Activity theory guiding the derivation and implementation of respective criteria for a truly human–centered design of the joint collaborative work process.
WP3 innovations include enhancement of the security and privacy of the data held in the FELICE system by orchestrating different tools and techniques. In addition, it provides user-driven interfaces that put humans in the loop of the production line management.
WP4 advanced in multiple aspects by developing: a learning-based approach for object detection and localization with online confidence scoring; a feature-based, real-time absolute localization system with decimeter accuracy; a human behavior understanding system estimating task cycle progress and ergonomically assessing postures during assembly; and a resilient speech command recognition system.
WP5 advanced the state of the art in multiple aspects including the design, development and manufacturing: of an adaptive workstation able to dynamically adapt to worker requirements; a 8 DOF mobile collaborative robot; implementing a human robot interaction module for fluent collaboration between a human and the robot.
WP6 advanced the state of the art by proposing a Digital Twin (DT) solution that is part of a larger platform with multiple modules. The state of the art highlights that simulation DT are starting to emerge only as stand-alone applications. Furthermore, the FELICE DT includes the human in the loop and makes use of Virtual Reality.
WP7 advanced the state of the art by devising novel planning models and applications for evolutionary algorithms.
WP8 innovations include the realization of the FELICE system and new methods for production process implementation and evaluation.
WP9 established the framework for exploitation and innovation by defining business models for 16 FELICE project assets as well as exploitation roadmaps for its three innovation clusters. Communication and dissemination activities support exploitation efforts.