Hands coated in virtual skin give VR users more digital dexterity
The increasing availability of virtual reality systems has made immersive digital environments more accessible than ever, from Microsoft’s HoloLens to Facebook’s Oculus system. However, interacting physically with these environments is a stumbling block for many developers. “Imagine a person with a head-mounted device, looking at a virtual world with objects in front of them, who wants to interact with these objects,” says Miguel Otaduy, project coordinator of the EU-funded CLAP project. “We are making technology to pick up, manipulate and drop those objects in a natural way.”
Click and collect
Currently, explains Otaduy, most systems rely on a snap-into-place mechanic, where a sensor detects when the hands are near a digital object, and attaches it to the hand regardless of grip or hand orientation. “We designed a much more natural hand, which manipulates the digital object the way real hands do,” he adds. At King Juan Carlos University, Otaduy and his team created a digital skeleton which recreates the user’s hand in virtual space, covered in a soft, pliable skin which deforms depending on how the hand is flexed. This allows the calculation of contact area between the hand and an object, allowing the realistic manipulation of objects in virtual space. Simulating this with sufficient accuracy is key to achieving natural reactions, says Otaduy. “Snap-into-place is good in some games and VR training, but there are other things you cannot do,” he explains. “We’re trying to take a virtual hand that looks like a real hand, which can approach an object and introduce physical interaction, such as grasping operations. This would mimic what happens in the real world more closely.”
Project evolution
The technology is supported by all the major virtual reality systems in use today, and does not rely on any additional hardware. “We can use all options available right now: integrated sensors, tracking gloves, camera-based systems, infrared systems. It works with all of those,” notes Otaduy. The proof of concept project was supported by the European Research Council, and came out of a more fundamental project to develop the underlying algorithms. “It was crucial to carry out that research, and to attract the right talent,” says Otaduy. “The project then allowed us to integrate those fundamental algorithms with multiple technologies, such as trackers and various visualisation devices. This allowed us to demonstrate the technology as a step towards marketing it.”
A helping hand
The CLAP team is now searching for commercial partners, investors and collaborators who will help take the technology to the next stage of development. “At the moment we are trying to create a spin-off company – we’re in conversation with the university about how to do this – and trying to expand the applications that can be done with CLAP. These would be social interaction applications, telecommunications and more realistic, person-to-person interaction,” explains Otaduy. “Hand interaction allows you to take these computer models and make them real, to create virtual worlds that become truly interactive. It’s a fascinating way to bring together physics, mathematics and computer science, and play with something that only exists inside a computer.”
Keywords
CLAP, virtual, reality, hand, manipulate, skeleton, skin, snap, digital, grasp, tracking
