Periodic Reporting for period 1 - HumanTech (Human Centered Technologies for a Safer and Greener European Construction Industry)
Okres sprawozdawczy: 2022-06-01 do 2023-11-30
The European construction industry faces three major challenges: increase the
safety and wellbeing of its workforce, improve its productivity, and become greener,
making efficient use of resources.
To address these challenges, HumanTech proposes to develop human-centred
cutting-edge technologies such as wearables for workers' safety and support and
robots that can harmoniously coexist with human workers while contributing to the
ecological transition of the sector.
These HumanTech advances will include:
● Robotic devices equipped with vision and intelligence that allow them to
navigate autonomously and safely in highly unstructured environments,
and collaborate with humans.
● Smart, unobtrusive workers protection and support equipment. From
exoskeletons activated by body sensors for posture and strain to wearable
cameras and XR glasses that provide real-time workers' location and guidance
for them to perform their tasks efficiently and accurately.
● Dynamic Semantic Digital Twins (DSDTs) of construction sites that depict in detail the current state of a construction
site at the geometric and semantic level, based on an extended BIM formulation that contains all relevant structural
and semantic dimensions (BIMxD). BIMxDs will act as a common reference for
all human workers, engineers and autonomous machines.
The HumanTech consortium is formed by 22 organisations — leading research
institutes and universities and innovative hi-tech SMEs and large enterprises.
The consortium is led by the German
Research Center for Artificial Intelligence’s Augmented Vision department.
safety and wellbeing of its workforce, improve its productivity, and become greener,
making efficient use of resources.
To address these challenges, HumanTech proposes to develop human-centred
cutting-edge technologies such as wearables for workers' safety and support and
robots that can harmoniously coexist with human workers while contributing to the
ecological transition of the sector.
These HumanTech advances will include:
● Robotic devices equipped with vision and intelligence that allow them to
navigate autonomously and safely in highly unstructured environments,
and collaborate with humans.
● Smart, unobtrusive workers protection and support equipment. From
exoskeletons activated by body sensors for posture and strain to wearable
cameras and XR glasses that provide real-time workers' location and guidance
for them to perform their tasks efficiently and accurately.
● Dynamic Semantic Digital Twins (DSDTs) of construction sites that depict in detail the current state of a construction
site at the geometric and semantic level, based on an extended BIM formulation that contains all relevant structural
and semantic dimensions (BIMxD). BIMxDs will act as a common reference for
all human workers, engineers and autonomous machines.
The HumanTech consortium is formed by 22 organisations — leading research
institutes and universities and innovative hi-tech SMEs and large enterprises.
The consortium is led by the German
Research Center for Artificial Intelligence’s Augmented Vision department.
The project has recently reached its middle point and has progressed according to plan so far, with significant achievements at different levels:
- The definition of an extended BIM format, the BIMxD has progressed through the development of an information delivery manual and the specifications of BIMxD data platform.
- A fully automated pipeline that transforms 3D point cloud scans to BIM models using the IFC format is now in place. Its development was supported by the collection of diverse datasets in construction sites using different sensors in handheld or terrestrial mode, or mounted on semi-autonomous ground and aerial vehicles. A novel RGB-D spherical capturing device prototype from RICOH is in use in the project, allowing single click reconstruction of entire rooms.
- The development of wearable devices for construction is also progressing well, with a body-sensor network for worker pose being already integrated with a helmet-mounted wide angle camera. Meanwhile, integration of an exoskeleton with the system in order to achieve selective activation for worker support is ongoing, as well as the integration of an XR-glass for the visualization of BIMxD information on-site.
- Assistance robotics for construction are also progressing significantly. The robotic perception system uses an RGB-D camera to estimate the pose of material such as bricks for robotic picking. Robotic end-effectors for construction tasks such as brick grasping or mastic application have been developed. New approaches for human-robot communication are already in place and being tested. As a next step, the main robotic platform of the project will become available for integration.
- HumanTech recognizes the importance of keeping users in the loop during the development of new technologies. Therefore, a campaign for early evaluation and feedback from construction workers and engineers was carried out with workshops at 3 construction sites in Spain and Ireland. The results of these workshops were analysed and will be taken into account in technological developments in the second half of the project.
- The ultimate objective of HumanTech is to be able to demonstrate its technologies in actual construction environments by the end of the project. 5 pilots have been planned as key final project results and are all under development through the integration of technology components from all work packages.
- Finally, HumanTech has already shown significant scientific excellence. Project results have already been published in 6 peer-reviewed, top-tier publications in different fields such as AI and computer vision, construction and robotics. HumanTech technologies such as the automated scan-to-BIM pipeline or the object pose estimation have entered international challenges in their areas and even received 1st place awards.
- The definition of an extended BIM format, the BIMxD has progressed through the development of an information delivery manual and the specifications of BIMxD data platform.
- A fully automated pipeline that transforms 3D point cloud scans to BIM models using the IFC format is now in place. Its development was supported by the collection of diverse datasets in construction sites using different sensors in handheld or terrestrial mode, or mounted on semi-autonomous ground and aerial vehicles. A novel RGB-D spherical capturing device prototype from RICOH is in use in the project, allowing single click reconstruction of entire rooms.
- The development of wearable devices for construction is also progressing well, with a body-sensor network for worker pose being already integrated with a helmet-mounted wide angle camera. Meanwhile, integration of an exoskeleton with the system in order to achieve selective activation for worker support is ongoing, as well as the integration of an XR-glass for the visualization of BIMxD information on-site.
- Assistance robotics for construction are also progressing significantly. The robotic perception system uses an RGB-D camera to estimate the pose of material such as bricks for robotic picking. Robotic end-effectors for construction tasks such as brick grasping or mastic application have been developed. New approaches for human-robot communication are already in place and being tested. As a next step, the main robotic platform of the project will become available for integration.
- HumanTech recognizes the importance of keeping users in the loop during the development of new technologies. Therefore, a campaign for early evaluation and feedback from construction workers and engineers was carried out with workshops at 3 construction sites in Spain and Ireland. The results of these workshops were analysed and will be taken into account in technological developments in the second half of the project.
- The ultimate objective of HumanTech is to be able to demonstrate its technologies in actual construction environments by the end of the project. 5 pilots have been planned as key final project results and are all under development through the integration of technology components from all work packages.
- Finally, HumanTech has already shown significant scientific excellence. Project results have already been published in 6 peer-reviewed, top-tier publications in different fields such as AI and computer vision, construction and robotics. HumanTech technologies such as the automated scan-to-BIM pipeline or the object pose estimation have entered international challenges in their areas and even received 1st place awards.
The HumanTech project reaching the middle of its duration is an important milestone
for monitoring its progress and evaluating its impact and exploitation perspectives. The
middle of the project also marks the beginning of a transition from individual technical
work of the partners into integration work and preparation for the HumanTech pilots.
Significant advances are achieved in different project areas, with 3 Key Exploitable Results being selected as the most promising regarding their exploitation perspectives:
KER #1, Body Sensor Controlled Exoskeleton was selected due to the innovativeness of the technology, KER #2, Dynamic
Semantic Digital Twin Software - Automated Scan2BIM pipeline because of the
impressive results achieved in HumanTech and the effective collaboration between DFKI
and RPTU, and finally, KER #3, Spherical 3D RGB-D camera due the high exploitation
potential of this result that can be directly brought to market by its developer
organisation and project partner, RICOH. In order to detail and strengthen the potential of these KERs the HumanTech project is making use of the Horizon Results Booster (HRB)
service.
Regarding scientific impact, HumanTech already has significant in the first 18M. In summary:
• HumanTech has 6 scientific publications (5 conference proceedings papers, 1 journal article) in highly acclaimed venues (CVPR, ICCV, WACV, Robotics&Automation Letters, EC3).
• HumanTech results have received awards in 3 scientific challenges (2x BOP Challenge 2022, 3x BOP Challenge 2023, 1x CV4AEC Scan2BIM Challenge 2023).
• HumanTech has been involved in the organization of 6 scientific workshops such as the “AI and Robotics in Construction workshop” at the European Robotics Forum 2023.
In addition, HumanTech partners have been involved in several activities related to standardization.
for monitoring its progress and evaluating its impact and exploitation perspectives. The
middle of the project also marks the beginning of a transition from individual technical
work of the partners into integration work and preparation for the HumanTech pilots.
Significant advances are achieved in different project areas, with 3 Key Exploitable Results being selected as the most promising regarding their exploitation perspectives:
KER #1, Body Sensor Controlled Exoskeleton was selected due to the innovativeness of the technology, KER #2, Dynamic
Semantic Digital Twin Software - Automated Scan2BIM pipeline because of the
impressive results achieved in HumanTech and the effective collaboration between DFKI
and RPTU, and finally, KER #3, Spherical 3D RGB-D camera due the high exploitation
potential of this result that can be directly brought to market by its developer
organisation and project partner, RICOH. In order to detail and strengthen the potential of these KERs the HumanTech project is making use of the Horizon Results Booster (HRB)
service.
Regarding scientific impact, HumanTech already has significant in the first 18M. In summary:
• HumanTech has 6 scientific publications (5 conference proceedings papers, 1 journal article) in highly acclaimed venues (CVPR, ICCV, WACV, Robotics&Automation Letters, EC3).
• HumanTech results have received awards in 3 scientific challenges (2x BOP Challenge 2022, 3x BOP Challenge 2023, 1x CV4AEC Scan2BIM Challenge 2023).
• HumanTech has been involved in the organization of 6 scientific workshops such as the “AI and Robotics in Construction workshop” at the European Robotics Forum 2023.
In addition, HumanTech partners have been involved in several activities related to standardization.