Periodic Reporting for period 2 - HR-Recycler (Hybrid Human-Robot RECYcling plant for electriCal and eLEctRonic equipment)
Okres sprawozdawczy: 2021-03-01 do 2021-12-31
The aim of the HR-Recycler project is to develop a "Hybrid Human-Robot RECYcling plant for electriCal and eLEctRonic equipment" operating in an indoor environment. The great innovation potential of HR-Recycler is to replace multiple currently manual, expensive, hazardous and time-consuming tasks of WEEE materials pre-processing with correspondingly automatic robotic-based procedures. HR-Recycler will offer:
- A real-world, robust and open human-robot collaborative working environment.
- Advanced collaborative factory floor modelling and real-time orchestration.
- Advanced AI-based cognitive perception both at cell and factory level.
- Smart mechatronic systems with advanced and sophisticated capabilities for robotic actions planning and control.
- Advanced mechanisms for realizing safe and efficient collaboration with humans.
- Incorporation of Social Sciences and Humanities (SSH) elements.
- Delivery, deployment, demonstration and thorough evaluation of a functional system prototype in real-world operational environments.
- Intense dissemination, cooperation with other projects.
- Business exploitation of the project tools and services.
To demonstrate the capabilities of the proposed system, four main Use Cases have been selected:
- UC1 – Emergency lamps
- UC2 – Microwave ovens
- UC3 – PC towers
- UC4 – Displays/monitors.
- A real-world, robust and open human-robot collaborative working environment.
- Advanced collaborative factory floor modelling and real-time orchestration.
- Advanced AI-based cognitive perception both at cell and factory level.
- Smart mechatronic systems with advanced and sophisticated capabilities for robotic actions planning and control.
- Advanced mechanisms for realizing safe and efficient collaboration with humans.
- Incorporation of Social Sciences and Humanities (SSH) elements.
- Delivery, deployment, demonstration and thorough evaluation of a functional system prototype in real-world operational environments.
- Intense dissemination, cooperation with other projects.
- Business exploitation of the project tools and services.
To demonstrate the capabilities of the proposed system, four main Use Cases have been selected:
- UC1 – Emergency lamps
- UC2 – Microwave ovens
- UC3 – PC towers
- UC4 – Displays/monitors.
In the context of WP2 - Regulatory, legal, ethical and societal challenges of robotics in industrial automation, an impact assessment method has been developed, which intends to describe the steps to take in order to generate useful information to consortium partners and assist them in taking informed decisions about the development and deployment of the HR-Recycler technology.
In WP3 - Use-cases, user requirements and system architecture, work was carried out in order to provide a list of use cases a user requirement and the architecture to be followed for the rest of the project. More specifically, definition of user requirements involved:
- Identifying real‐world use cases and scenarios related to WEEE recycling at project recycling partner’s facilities.
- Providing a set of specifications based on user needs, use cases and scenarios.
- Employing the use cases as an instrument in driving and giving directions to the technical developments of the project.
A functional specification of the HR-Recycler system that will lead to a clear system specification was elaborated taking into account technical issues regarding interoperability, scalability and flexibility required for a fully functional system. The overall architecture of the HR-Recycler system was provided.
Within the WP4 - Factory-level modelling, cognitive perception and orchestration, all the models involved within the factory floor environment including the Collaborative Factory Model (CFM), the Worker Model (WM), and the virtual factory studio, along with the required routines for their update, maintenance and communication, have been developed. The Collaborative Factory model (CFM) has been developed based on the analytic definition of the adaptive Cell (aCell) approach. The Worker model has been developed by organizing the worker-related data into task-relevant and interaction-relevant data. The development of the Virtual Factory Studio (VFS) has also significantly progressed. Also, he development of the workstation monitoring system has seen significant progress. The collaborative factory floor orchestration engine has been implemented.
The work within WP5 related to developing deep-learning-assisted object detection approaches provided for both the classification and the disassembly stages. Recognition of human actions, while disassembling as well as while interacting with the robot with specific gestures, has been implemented. To improve the object detection performance, especially for disassembly, novel “sensorimotor” approaches have been developed for human-like object learning. This WP focuses also on the investigation of physics-based haptic simulation to enable realistic and intuitive interactions in VR. A new dataset to support the latter activities was created.
In WP6 - Robotic actions planning and control, robot action planning and control approaches are proposed for a WEEE disassembly setup.
- manipulation skills required for disassembly of WEEE devices.
- an adaptive grasping force controller was developed and tested in simulation as well as on the final hardware platform.
- the lifelong mapping strategy for highly dynamic environment involving humans.
- the first operational version of the Workstation Task Planner.
In WP7 - Human-robot collaboration schemes, the Human Factors Worker Model has been developed. Another main achievement of the task is Human Trust classifier development. The results in this period is a prepared experiment to get data from participants to train classifier. Moreover, the development of an Interaction Manager took place. With respect to the development of learning from human input, a proactive tagging system was developed that deals with uncertainty, based on a confidence measure, which will define the need for human intervention or not.Progress has been conducted both for the optical see-through AR as well on the projective AR modules.
In WP8 - Smart mechatronics H/W for efficient HRC, the proper hardware and software, identified within RP1, was developed, tested and integrated in the test case at the technologies provider facilities.
In WP9 - System integration, the SW and HW integration activities of the HR-Recycler project have progressed.
In WP10 - Pilot studies demonstration and evaluation, preliminary layouts of the pilot scenarios have been developed and the more convenient system element distribution agreed, based on the collection and analysis of technical information both from HW developers and end-users.
With the activities carried out in WP11, we want to ensure the project’s visibility we have developed and promoted means to raise awareness amongst interested parties. The consortium members of the H-Recycler projects published 12 research articles and attended in 6 conferences, workshops and seminars during this reporting period.
In WP3 - Use-cases, user requirements and system architecture, work was carried out in order to provide a list of use cases a user requirement and the architecture to be followed for the rest of the project. More specifically, definition of user requirements involved:
- Identifying real‐world use cases and scenarios related to WEEE recycling at project recycling partner’s facilities.
- Providing a set of specifications based on user needs, use cases and scenarios.
- Employing the use cases as an instrument in driving and giving directions to the technical developments of the project.
A functional specification of the HR-Recycler system that will lead to a clear system specification was elaborated taking into account technical issues regarding interoperability, scalability and flexibility required for a fully functional system. The overall architecture of the HR-Recycler system was provided.
Within the WP4 - Factory-level modelling, cognitive perception and orchestration, all the models involved within the factory floor environment including the Collaborative Factory Model (CFM), the Worker Model (WM), and the virtual factory studio, along with the required routines for their update, maintenance and communication, have been developed. The Collaborative Factory model (CFM) has been developed based on the analytic definition of the adaptive Cell (aCell) approach. The Worker model has been developed by organizing the worker-related data into task-relevant and interaction-relevant data. The development of the Virtual Factory Studio (VFS) has also significantly progressed. Also, he development of the workstation monitoring system has seen significant progress. The collaborative factory floor orchestration engine has been implemented.
The work within WP5 related to developing deep-learning-assisted object detection approaches provided for both the classification and the disassembly stages. Recognition of human actions, while disassembling as well as while interacting with the robot with specific gestures, has been implemented. To improve the object detection performance, especially for disassembly, novel “sensorimotor” approaches have been developed for human-like object learning. This WP focuses also on the investigation of physics-based haptic simulation to enable realistic and intuitive interactions in VR. A new dataset to support the latter activities was created.
In WP6 - Robotic actions planning and control, robot action planning and control approaches are proposed for a WEEE disassembly setup.
- manipulation skills required for disassembly of WEEE devices.
- an adaptive grasping force controller was developed and tested in simulation as well as on the final hardware platform.
- the lifelong mapping strategy for highly dynamic environment involving humans.
- the first operational version of the Workstation Task Planner.
In WP7 - Human-robot collaboration schemes, the Human Factors Worker Model has been developed. Another main achievement of the task is Human Trust classifier development. The results in this period is a prepared experiment to get data from participants to train classifier. Moreover, the development of an Interaction Manager took place. With respect to the development of learning from human input, a proactive tagging system was developed that deals with uncertainty, based on a confidence measure, which will define the need for human intervention or not.Progress has been conducted both for the optical see-through AR as well on the projective AR modules.
In WP8 - Smart mechatronics H/W for efficient HRC, the proper hardware and software, identified within RP1, was developed, tested and integrated in the test case at the technologies provider facilities.
In WP9 - System integration, the SW and HW integration activities of the HR-Recycler project have progressed.
In WP10 - Pilot studies demonstration and evaluation, preliminary layouts of the pilot scenarios have been developed and the more convenient system element distribution agreed, based on the collection and analysis of technical information both from HW developers and end-users.
With the activities carried out in WP11, we want to ensure the project’s visibility we have developed and promoted means to raise awareness amongst interested parties. The consortium members of the H-Recycler projects published 12 research articles and attended in 6 conferences, workshops and seminars during this reporting period.
Progress beyond the state-of-the-art and project results:
- A Collaborative Factory model (CFM) has been developed, based on the adaptive Cell approach.
- Novel Deep Learning approachs for detection of small objects and Object Affordance Segmentation have been developed. A new dataset for detection of screws was introduced.
- Developed an adaptive grasping force controller, a haptic SLAM, an Implementation of 3D-FEM based haptic probing.
- Implemented the lifelong semantic mapping and a method for pallet detection.
- Implementation of the optical see-through interaction with the mobile manipulator.
- Development of probabilistic inference-based kinematic trajectory planner and a novel dedicated solution for collaborative robotics.
- For the the motion planning problem, an algorithm denoted as Min-sum Message Passing algorithm for Motion Planning (MS2MP) was developed combines numerical optimization with message passing to find collision- free trajectories. MS2MP improves existing work in convergence time and success rate.
- TEC has improved the real-time monitoring worker profiling process, by providing the classification error probability as an additional output from the trust classifier.
- A Collaborative Factory model (CFM) has been developed, based on the adaptive Cell approach.
- Novel Deep Learning approachs for detection of small objects and Object Affordance Segmentation have been developed. A new dataset for detection of screws was introduced.
- Developed an adaptive grasping force controller, a haptic SLAM, an Implementation of 3D-FEM based haptic probing.
- Implemented the lifelong semantic mapping and a method for pallet detection.
- Implementation of the optical see-through interaction with the mobile manipulator.
- Development of probabilistic inference-based kinematic trajectory planner and a novel dedicated solution for collaborative robotics.
- For the the motion planning problem, an algorithm denoted as Min-sum Message Passing algorithm for Motion Planning (MS2MP) was developed combines numerical optimization with message passing to find collision- free trajectories. MS2MP improves existing work in convergence time and success rate.
- TEC has improved the real-time monitoring worker profiling process, by providing the classification error probability as an additional output from the trust classifier.