Periodic Reporting for period 1 - UWB-IODA SF-PC (Ultra Wide Band Integrated Optical-and-Digital Approach for Smart Factory and Perceptive Car)
Reporting period: 2020-03-02 to 2022-03-01
The transition to the digital era is lined with many challenges, including the broadband infrastructure. The need for high data rate communications and accurate indoor localization/identification/tracking makes the ultra-wide band (UWB) technology particularly attractive. However, the huge instantaneous frequency band to be processed results in strong constraints, beyond the specifications of the state-of-the-art digital circuits.
The EU-funded project “Ultra-Wide Band Integrated Optical-and-Digital Approach for Smart Factory and Perceptive Car” introduces a multidisciplinary approach relying on impulse radio UWB (IR-UWB) and UWB-over-fiber (UWBoF) technologies. They associate UWB waveform optimization, cutting edge signal processing techniques and deep learning to design intelligent, robust and high performance indoor wireless systems, for smart factories and perceptive cars.
This project aims at increasing the safety, security and convenience in Industry 4.0 environments, by multi-humans detection, localization and tracking, without the need for them to wear tags or any additional identification equipment. It also aims to contribute to the road safety by preventing the driver drowsiness related accidents and to facilitate the interaction with the perceptive car.
While addressing these objectives, original results have been obtained and significant progress can be reported. First, an integrated optical-and-digital architecture has been proposed, combining wireless IR-UWB and UWBoF transmission capability, and relying on UWB waveform optimization, advanced digital signal processing and deep learning algorithms. Then, people counting and accurate detection/localization of humans/machines inside an indoor environment have been achieved. Finally, robust detection of vital signs, with data loss recovery capability, has been introduced for driver condition monitoring inside a perceptive car. The UWB sensing solutions proposed in the framework of this project can also be deployed in other real-world environments (hospital, office, home) and contribute to the development of smart cities and lifestyles, for the benefit of the whole society.
The EU-funded project “Ultra-Wide Band Integrated Optical-and-Digital Approach for Smart Factory and Perceptive Car” introduces a multidisciplinary approach relying on impulse radio UWB (IR-UWB) and UWB-over-fiber (UWBoF) technologies. They associate UWB waveform optimization, cutting edge signal processing techniques and deep learning to design intelligent, robust and high performance indoor wireless systems, for smart factories and perceptive cars.
This project aims at increasing the safety, security and convenience in Industry 4.0 environments, by multi-humans detection, localization and tracking, without the need for them to wear tags or any additional identification equipment. It also aims to contribute to the road safety by preventing the driver drowsiness related accidents and to facilitate the interaction with the perceptive car.
While addressing these objectives, original results have been obtained and significant progress can be reported. First, an integrated optical-and-digital architecture has been proposed, combining wireless IR-UWB and UWBoF transmission capability, and relying on UWB waveform optimization, advanced digital signal processing and deep learning algorithms. Then, people counting and accurate detection/localization of humans/machines inside an indoor environment have been achieved. Finally, robust detection of vital signs, with data loss recovery capability, has been introduced for driver condition monitoring inside a perceptive car. The UWB sensing solutions proposed in the framework of this project can also be deployed in other real-world environments (hospital, office, home) and contribute to the development of smart cities and lifestyles, for the benefit of the whole society.
Our project is organized around 3 work packages (WP). WP1 addresses UWB waveform optimization and compressed sensing to design low cost UWBoF transceivers. WP2 is related to context awareness tasks, such as physiological parameters detection and gesture recognition inside a perceptive car. WP3 focuses on people detection and localization, as well as human and machines classification for mixed human/robot Industry 4.0 environments.
In the framework of WP1, we have developed a general system architecture and proposed optimized UWB waveforms complying with the project requirements and optimizing the spectral efficiency. Off-the-shelf UWB radar sensors were used for both WP2 and WP3. For reliable detection, localization and classification of human and machines inside Industry 4.0 a setting consisting of 4 radar sensors was deployed. The experiments were performed in multiple cluttered locations such as indoor room and university lab environments.
The WP2 addresses the physiological signal monitoring and gesture recognition for a perceptive car. However, since we could not use the experimental facilities of ZF company, we started by developing robust algorithms for physiological monitoring of a human in a bed with some natural motions. To this end, we have first deployed an IR-UWB radar sensor to extract the physiological signal in resting position without any natural body motion. The next step aimed compensating the effect of the natural motions. We have developed advanced signal processing and data fusion algorithms for the motion artifacts removal. This work can be easily extended to a perceptive car to monitor the physiological patterns of the driver and passengers.
Nevertheless, despite all the significant research progress resumed above, we have also experienced a major difficulty, since the MSCA fellow was not able to join our research lab for pandemic related reasons and had to work remotely. His visa requests being rejected, he could not carry out the planned secondment at the ZF company and implement the experimental phase related to WP2. Thus, the project had to be unfortunately stopped after 16 months instead of the initially planned 24 months.
In the framework of WP1, we have developed a general system architecture and proposed optimized UWB waveforms complying with the project requirements and optimizing the spectral efficiency. Off-the-shelf UWB radar sensors were used for both WP2 and WP3. For reliable detection, localization and classification of human and machines inside Industry 4.0 a setting consisting of 4 radar sensors was deployed. The experiments were performed in multiple cluttered locations such as indoor room and university lab environments.
The WP2 addresses the physiological signal monitoring and gesture recognition for a perceptive car. However, since we could not use the experimental facilities of ZF company, we started by developing robust algorithms for physiological monitoring of a human in a bed with some natural motions. To this end, we have first deployed an IR-UWB radar sensor to extract the physiological signal in resting position without any natural body motion. The next step aimed compensating the effect of the natural motions. We have developed advanced signal processing and data fusion algorithms for the motion artifacts removal. This work can be easily extended to a perceptive car to monitor the physiological patterns of the driver and passengers.
Nevertheless, despite all the significant research progress resumed above, we have also experienced a major difficulty, since the MSCA fellow was not able to join our research lab for pandemic related reasons and had to work remotely. His visa requests being rejected, he could not carry out the planned secondment at the ZF company and implement the experimental phase related to WP2. Thus, the project had to be unfortunately stopped after 16 months instead of the initially planned 24 months.
The most important results beyond the state of the art have been obtained in terms of multi-human detection, localization and classification in an indoor environment. We achieved better results in multi-path reduction compared to the state-of-the-art UWB radar based people detection/counting. We have worked on a combination of techniques including a novel thresholding algorithm, a multi-path reduction technique based on radar sensors geometry, and deep learning based algorithms fed with images of radar backscattered signals.
The impact of the research activities and collaborations in the framework of this project is given below.
• Collision avoidance in Industry 4.0 and beyond
Our work related to multi-human detection, positioning and classification may play a significant role in collision avoidance in a hybrid human/robot Industry 4.0 environment. The robust radar-based solution is not sensitive to the environmental conditions such as lighting, humidity and temperature unlike sensors like camera and LIDAR. This is not only useful for safety of the human participants but it can also reduce the costs associated with the damage of the autonomous vehicles during collisions in industries. Although the main goal of this work is to detect and avoid collisions in Industry 4.0 environments, this solution may also be suitable for implementing social distancing during a pandemic situation such as COVID-19. As our proposed approach provides very accurate and reliable indoor detection and positioning, it can detect the situation when two people are getting closer than a certain threshold distance. It can also help the authorities to monitor the number of people in a given indoor environment.
• Context awareness tasks inside a perceptive car
Although UWB radar sensors have already been used for different context awareness tasks such as occupancy detection and people counting, the physiological monitoring is still an unexplored area in context of perceptive car. Since the main challenge is the radar signal sensitivity to movements, we propose state-of-the-art solutions associating signal processing, deep learning and algorithm design to achieve robust detection of vital signs inside a car. Other sensors, such as audio recording, are also used to enhance the results. We believe that the multi-sensor fusion and multi-disciplinary research based algorithms provide a new direction to the radar based physiological monitoring and make the proposed solution suitable for real life applications such as in-car, hospital and home monitoring of patients and elderly people. The in-car physiological sensing is helpful for driver drowsiness detection, which may prevent accidents and thus saving many lives in the future. Moreover, since the increase of elderly population has a huge impact on the demand for hospital beds, our solution has the potential to provide continuous, non-invasive health monitoring at home and office, and help reduce the pressure on the medical facilities in hospitals.
The impact of the research activities and collaborations in the framework of this project is given below.
• Collision avoidance in Industry 4.0 and beyond
Our work related to multi-human detection, positioning and classification may play a significant role in collision avoidance in a hybrid human/robot Industry 4.0 environment. The robust radar-based solution is not sensitive to the environmental conditions such as lighting, humidity and temperature unlike sensors like camera and LIDAR. This is not only useful for safety of the human participants but it can also reduce the costs associated with the damage of the autonomous vehicles during collisions in industries. Although the main goal of this work is to detect and avoid collisions in Industry 4.0 environments, this solution may also be suitable for implementing social distancing during a pandemic situation such as COVID-19. As our proposed approach provides very accurate and reliable indoor detection and positioning, it can detect the situation when two people are getting closer than a certain threshold distance. It can also help the authorities to monitor the number of people in a given indoor environment.
• Context awareness tasks inside a perceptive car
Although UWB radar sensors have already been used for different context awareness tasks such as occupancy detection and people counting, the physiological monitoring is still an unexplored area in context of perceptive car. Since the main challenge is the radar signal sensitivity to movements, we propose state-of-the-art solutions associating signal processing, deep learning and algorithm design to achieve robust detection of vital signs inside a car. Other sensors, such as audio recording, are also used to enhance the results. We believe that the multi-sensor fusion and multi-disciplinary research based algorithms provide a new direction to the radar based physiological monitoring and make the proposed solution suitable for real life applications such as in-car, hospital and home monitoring of patients and elderly people. The in-car physiological sensing is helpful for driver drowsiness detection, which may prevent accidents and thus saving many lives in the future. Moreover, since the increase of elderly population has a huge impact on the demand for hospital beds, our solution has the potential to provide continuous, non-invasive health monitoring at home and office, and help reduce the pressure on the medical facilities in hospitals.