Periodic Reporting for period 1 - FELICE (FlExible assembLy manufacturIng with human-robot Collaboration and digital twin modEls)
Berichtszeitraum: 2021-01-01 bis 2022-09-30
FELICE addresses one of the greatest challenges in robotics, i.e. that of coordinated interaction and combination of human and robot skills. The project targets the application priority area of agile production and aspires to design the next generation assembly processes required to effectively address current and pressing needs in manufacturing, specifically increase the agility and productivity of a manual assembly production system, ensure the safety and improve the physical and mental well-being of factory workers. Our framework comprises of two layers, a local one introducing i) a single collaborative assembly robot that will roam the shop floor assisting workers, and ii) adaptive workstations able to automatically adjust to the workers’ somatometries and providing multimodal informative guidance and notifications on assembly tasks, and a global layer which will sense and operate upon the real world via an actionable digital replica of the entire physical assembly line.
The objectives of the project are the following:
-Develop context awareness via ubiquitous sensing
-Ensure safety in human-robot collaborative assembly
-Amplify fluency and ergonomics in human-robot collaborative assembly
-Orchestrate the assembly process in real-time
-Realize collaborative assembly lines for agile production in real production environments
-Maximize the project impact
The objectives of the project are the following:
-Develop context awareness via ubiquitous sensing
-Ensure safety in human-robot collaborative assembly
-Amplify fluency and ergonomics in human-robot collaborative assembly
-Orchestrate the assembly process in real-time
-Realize collaborative assembly lines for agile production in real production environments
-Maximize the project impact
Some of the project highlights during this period are:
Collaborative robot deployment. A collaborative robot was designed, manufactured and deployed in the CRF pilot environment comprising the mobile platform and a manipulator with 8 DOF and a 3-finger gripper.
Adaptive workstation prototype. The concept, design, and implementation of the first prototype of an adaptive workstation (AWS) was completed.
Baseline for predictable and directable collaboration. The outlined requirements for Joint Activity as part of the conceptual framework for interaction design both for the AWS as well as for HRC provided the design guidelines for the fluency of the robot-assisted task performance and additional guidance for the implementation of a safe and reliable interaction design.
Integration of the first version of software modules. Eighty installation and testing activities have been carried out during two integration rounds at CRF. The installation and integration activities were satisfactory in terms of the available prototype functionalities.
Work highlights per WP:
WP2: The work carried out led to the baseline formal requirements of the collaborative robot in terms of functionality and user safety, the definition of the conceptual framework of ergonomic requirements for the AWS and the HRC, the establishment of the productivity criteria and the specifications of the system architecture addressing the interconnections and constraints of the entire FELICE modular environment.
WP3: The work carried out led to the review of the state of the art in technologies and tools pertaining to topics investigated in the project, the design and development of system-wide mechanisms for privacy and security, knowledge storage and user-driven visualization interfaces.
WP4: Work concerned the development of perception mechanisms for use in the dynamic shop floor. These include the detection and 6D pose estimation of objects for robot grasping; the spatial mapping of the environment for robot navigation; the monitoring of worker’s actions and ergonomic body postures; and the recognition of spoken commands and gestures for natural human-robot interaction.
WP5: The work carried out led to the implementation of a first prototype of the collaborative robot and the adaptive workstation (AWS). A human robot interaction module commanding the robot ‘when’, ‘which’ and ‘where’ a task has to be performed was implemented together with an execution module that translates these commands into a series of robot executable skills.
WP6: The work carried out led to the definition of the Digital Twin (DT) requirements that led to the design of the FELICE DT, comprising the Discrete Event Simulation, the Real-Time Virtual Simulation and the Digital Mirror. The quality of the DES module was assured by the V&V activities carried out on the requirements specification, on the conceptual model and on the executable version of the DES module.
WP7: The work carried out led to the formulation of data models, assembly line planning models, reinforcement learning approaches, and novel applications for evolutionary algorithms. The orchestrator was deployed as a web service in the two pilots and interconnects with the other modules via FIWARE.
WP8: The work carried out led to the detailed definition of the two use cases with particular attention to the different technologies involved in the project. The integration strategy has been defined and the installation and testing of the different modules and technologies has been performed. An initial evaluation strategy has been developed.
WP9: The work carried out led to progress on relevant dKPIs, including activities in journals, at conferences/workshops and via dissemination and communication channels.Potentially exploitable assets were defined. Relevant Standards under development have been identified. Cooperation with two DIHs is underway.
Collaborative robot deployment. A collaborative robot was designed, manufactured and deployed in the CRF pilot environment comprising the mobile platform and a manipulator with 8 DOF and a 3-finger gripper.
Adaptive workstation prototype. The concept, design, and implementation of the first prototype of an adaptive workstation (AWS) was completed.
Baseline for predictable and directable collaboration. The outlined requirements for Joint Activity as part of the conceptual framework for interaction design both for the AWS as well as for HRC provided the design guidelines for the fluency of the robot-assisted task performance and additional guidance for the implementation of a safe and reliable interaction design.
Integration of the first version of software modules. Eighty installation and testing activities have been carried out during two integration rounds at CRF. The installation and integration activities were satisfactory in terms of the available prototype functionalities.
Work highlights per WP:
WP2: The work carried out led to the baseline formal requirements of the collaborative robot in terms of functionality and user safety, the definition of the conceptual framework of ergonomic requirements for the AWS and the HRC, the establishment of the productivity criteria and the specifications of the system architecture addressing the interconnections and constraints of the entire FELICE modular environment.
WP3: The work carried out led to the review of the state of the art in technologies and tools pertaining to topics investigated in the project, the design and development of system-wide mechanisms for privacy and security, knowledge storage and user-driven visualization interfaces.
WP4: Work concerned the development of perception mechanisms for use in the dynamic shop floor. These include the detection and 6D pose estimation of objects for robot grasping; the spatial mapping of the environment for robot navigation; the monitoring of worker’s actions and ergonomic body postures; and the recognition of spoken commands and gestures for natural human-robot interaction.
WP5: The work carried out led to the implementation of a first prototype of the collaborative robot and the adaptive workstation (AWS). A human robot interaction module commanding the robot ‘when’, ‘which’ and ‘where’ a task has to be performed was implemented together with an execution module that translates these commands into a series of robot executable skills.
WP6: The work carried out led to the definition of the Digital Twin (DT) requirements that led to the design of the FELICE DT, comprising the Discrete Event Simulation, the Real-Time Virtual Simulation and the Digital Mirror. The quality of the DES module was assured by the V&V activities carried out on the requirements specification, on the conceptual model and on the executable version of the DES module.
WP7: The work carried out led to the formulation of data models, assembly line planning models, reinforcement learning approaches, and novel applications for evolutionary algorithms. The orchestrator was deployed as a web service in the two pilots and interconnects with the other modules via FIWARE.
WP8: The work carried out led to the detailed definition of the two use cases with particular attention to the different technologies involved in the project. The integration strategy has been defined and the installation and testing of the different modules and technologies has been performed. An initial evaluation strategy has been developed.
WP9: The work carried out led to progress on relevant dKPIs, including activities in journals, at conferences/workshops and via dissemination and communication channels.Potentially exploitable assets were defined. Relevant Standards under development have been identified. Cooperation with two DIHs is underway.
The major FELICE state of the art advances and innovations are:
WP2 advanced the state of the art by defining the requirements for the direct HR Collaboration on the paradigm of Joint Activity theory guiding the derivation and implementation of respective criteria for a truly human–centered design of the joint collaborative work process.
WP3 innovations include enhancement of the security and privacy of the data held in the FELICE by orchestrating different tools and techniques. In addition, it provides user-driven interfaces that put human in the loop of the production line management.
WP4 advanced in multiple aspects developing: a learning-based approach for object detection and localization; a feature-based visual simultaneous localization and mapping system enhanced with appearance-based localization; a human behavior understanding system estimating task cycle progress and ergonomically assessing postures during assembly; and a speech command recognition algorithm.
WP5 advanced the state of the art in multiple aspects including the design, development and manufacturing: of an adaptive workstation able to dynamically adapt to worker requirements; a 8 DOF mobile collaborative robot; implementing a human robot interaction module for fluent collaboration between human and the robot.
WP6 advanced the state of the art by proposing a Digital Twin (DT) solution that is part of a larger platform with multiple modules. The state of the art highlights that simulation DT are starting to emerge only as stand-alone applications. Furthermore, the FELICE DT includes the human in the loop and makes use of Virtual Reality.
WP7 advanced the state of the art by devising novel planning models and applications for evolutionary algorithms.
WP8 innovations include the realization of the FELICE system and new methods for production process implementation and evaluation.
WP9 established the framework for exploitation and innovation by defining an innovation roadmap for potentially exploitable assets in the FELICE project. Furthermore, additional assets are defined in an iterative process. Communication and dissemination activities support exploitation efforts.
WP2 advanced the state of the art by defining the requirements for the direct HR Collaboration on the paradigm of Joint Activity theory guiding the derivation and implementation of respective criteria for a truly human–centered design of the joint collaborative work process.
WP3 innovations include enhancement of the security and privacy of the data held in the FELICE by orchestrating different tools and techniques. In addition, it provides user-driven interfaces that put human in the loop of the production line management.
WP4 advanced in multiple aspects developing: a learning-based approach for object detection and localization; a feature-based visual simultaneous localization and mapping system enhanced with appearance-based localization; a human behavior understanding system estimating task cycle progress and ergonomically assessing postures during assembly; and a speech command recognition algorithm.
WP5 advanced the state of the art in multiple aspects including the design, development and manufacturing: of an adaptive workstation able to dynamically adapt to worker requirements; a 8 DOF mobile collaborative robot; implementing a human robot interaction module for fluent collaboration between human and the robot.
WP6 advanced the state of the art by proposing a Digital Twin (DT) solution that is part of a larger platform with multiple modules. The state of the art highlights that simulation DT are starting to emerge only as stand-alone applications. Furthermore, the FELICE DT includes the human in the loop and makes use of Virtual Reality.
WP7 advanced the state of the art by devising novel planning models and applications for evolutionary algorithms.
WP8 innovations include the realization of the FELICE system and new methods for production process implementation and evaluation.
WP9 established the framework for exploitation and innovation by defining an innovation roadmap for potentially exploitable assets in the FELICE project. Furthermore, additional assets are defined in an iterative process. Communication and dissemination activities support exploitation efforts.