Intelligent robots need to be able to see and feel their environment. Like that, they are able to work together with humans in factories, work in offices or help people at home. Technically, they rely on a sophisticated perception systems with many distributed sensor modules, such as cameras, force-torque and tactile sensors. This results in high hardware complexity, high assembly and maintenance costs, as well as complex sensor calibration and synchronization. These problems prevent the use of intelligent robots especially in SMEs and private households. Within this project, a novel visuo-haptic sensor system for robotic manipulators has been developed. It allows for multimodal sensing, reduced system complexity and significantly lower costs compared to current systems. The system replaces dedicated force sensors with passive components and a camera, providing coherent measurements of both force and contact shape. Our patented technology substitutes active sensor modules for force-torque and tactile data by an external camera and a passive flexible element. Forces applied to this passive element result in a characteristic deformation. Robust image processing algorithms detect the deformation from the camera image, and provide the applied forces or contact patterns for further control and planning tasks. Compared to existing solutions, this approach results in several advantages, including lower costs, simplicity of hardware, easy adaptability and rich feedback signals.
