Holographic vision enables immersive telerobotic operations
Performing remote, delicate manoeuvres with teleguided robot arms is a critical operation required in numerous sectors, from repairing oil rigs underwater to nuclear plant reactor manoeuvres and space exploration. The more accurate and easier the operation is to execute, the greater the likelihood of a successful outcome. New advances in this field could also open up potential new applications in other sectors.
Better remote situational awareness
To this end, the EU-funded HoviTron project sought to develop a holographic headset, together with a multi-camera video feed, which enables a teleoperator to feel fully immersed in a situation while operating at distance. “We wanted to provide the teleoperator with the ability to visualise the scene from any viewpoint,” explains project coordinator Gauthier Lafruit from the Université Libre de Bruxelles (ULB) in Belgium. “Our idea was to use video processing, instead of having to reposition a camera with a pan-tilt robot arm mechanism.” To achieve this, a number of fixed cameras are set up around the scene. Videos recorded in real time are then blended into a virtual viewpoint, corresponding to the teleoperator’s head position. “This means that abrupt movements of the teleoperator’s head instantly create the corresponding images in the stereoscopic head-mounted display they are wearing,” adds Lafruit. “There is no mechanical latency, which can cause cybersickness.” Additionally, each eye captures a holographic light field wavefront. This ensures that the teleoperator’s eyes can focus on the object of interest at will, irrespective of whether the object is positioned in the foreground or background. In doing so, this perfectly mimics natural human vision. “Together, the video feed and holographic vision ensure that the telerobotic operator feels fully naturally immersed in the scene, while being far away from a situation that might be hazardous or difficult to access,” says Lafruit.
Accurate, instantaneous vision
The project team applied synthetic, video game-like content along with real images to test the technology’s viability under various working conditions. Users confirmed that they felt less visual fatigue (and ultimately less physical fatigue) when wearing the holographic headset, and that their depth perception was enhanced. The hope is that these successful trials could open up a number of interesting applications. “For example, the technology could be used on future moon missions,” notes Lafruit. “Robots on the moon could place cameras around a scene, enabling astronauts in orbit to really see through the eyes of a moving robot.” The traditional technique of putting cameras in the robot’s head means that any rapid movement by the headset-wearing astronaut creates a time lag between images, causing cybersickness. HoviTron’s ability to instantaneously calculate the correct images to project in the astronaut’s eyes does remove this obstacle.
New navigation application possibilities
Lafruit sees the technology as having potential applications in any situation where ‘free navigation’ is required. A diving robot for example could repair a deep-sea pipeline, while the teleoperator remains safely on board a ship or platform. Such specialised applications however will require more investment, in order to further develop the technology. The project team is currently in touch with a company specialised in depth-sensing devices. “Source code will soon be provided under the Apache 2.0 licence on our website,” adds Lafruit. “We can promote our technology by letting people try it out themselves in a free navigation scenario using simple pre-recorded examples, without the robot arm, high-quality depth-sensing and/or camera devices and holographic virtual reality goggles, of course.”
Keywords
HoviTron, holographic, video, teleoperators, robot, space, telerobotic, astronauts
