Touch and feel over distance: the next trend in ICT?
Haptic communication is often considered one of the next breakthrough technologies in human-computer interaction, and also allows for more in-depth interaction between users over distances. However, whilst the topic of haptic interaction via a ‘Human-machine interface’ (HMI) has been around for quite some time, there is still a lot of work to be done to find suitable methods and technologies for the efficient processing and communication of haptic signals. Enter the EU-backed PROHAPTICS (Haptic Signal Processing and Communications) project. Over the past five years, Prof. Eckehard Steinbach from TU Munich and his team have been developing a series of novel methods and technologies for haptic communication. In this exclusive interview with the research*eu results magazine, he discusses his expectations for the future of this relatively new field of research, some potential applications and his plans to further develop the technology now that the project has come to an end. How do you see haptic communication progressively taking over the ICT market? One of the main application scenarios considered is teleoperation with haptic feedback, where a user interacts with a remote environment through the HMI. In this context, the user remotely controls a robotic system typically equipped with sensors and actuators. The interaction forces/torques are captured when the teleoperator is in contact with the remote objects, and they are reflected back as haptic feedback to the operator. This way, the user is able not only to see and hear what is going on in the remote space, but also to feel the interaction. Numerous studies have shown that haptic feedback improves task performance and the feeling of being present. The long-term goal of this research is to make the teleoperation fully transparent, which means the user will no longer be able to tell if a task is carried out locally or remotely through the HMI. This research is complemented by investigations about tactile displays and tactile feedback, which for instance allow a user to feel the roughness of an object surface via the human-machine interface. Major steps have only recently been taken to also enable standard mobile devices such as smartphones to generate haptic feedback. It is already technically possible today to locally modulate the friction between the finger and the glass display of a mobile device in such a way that tactile patterns or textures can be displayed. This, in my opinion, bears tremendous potential for new types of applications which allow a user to explore objects over the internet, not only visually but also haptically. What can the concrete benefits of human-to-machine interaction based on haptic communication be? In our daily life, we as humans rely heavily and constantly on the haptic modality when interacting with our environment. Without manipulation capabilities and haptic sensing, this interaction would be extremely limited. So far, however, physical interaction happens only locally in our direct vicinity. With appropriate haptic human-machine interfaces and haptic communication approaches, this interaction can also be enabled across barriers such as distance or scale. Haptic interaction and communication of course also comes with great potential for the visually impaired. A lot of useful information can be communicated through the haptic modality. Can you give a few examples of applications that would be conceivable within the next few years? The aforementioned teleoperation systems are already in use and support applications such as telesurgery, telemaintenance, etc. With new haptic interfaces emerging, in particular tactile displays, completely novel applications become possible. Just imagine a web store where the user can not only visually explore a product but also touch it before buying it. Or imagine that your spouse is in a furniture store and sends you a photo of a sofa he/she would like to buy. Wouldn’t your first question after seeing the photo be what the material feels like? Is it soft, warm, comfortable? With haptic communication, you could for instance slide your smartphone over the surface of the sofa, record the resulting vibration signal using the built-in acceleration sensors, compress and transmit the signals and eventually display them remotely. This way a remote haptic experience becomes possible. If you don’t like this particular material, you could then search in a database for other materials which are similar or which have a specific ‘feel’. Another application would be an improved video conferencing system where you can haptically interact with your children while being on a business trip. Through these systems you could comfort them and be more present than is possible with today’s solutions. How is PROHAPTICS an important step towards such applications? PROHAPTICS has developed a series of algorithms, codecs and protocols which enable haptic communication across distances for both haptic modalities (kinesthetic and tactile). The solutions developed are human-centric in the sense that they consider and exploit the limitations of the human haptic perception system. This way, information that cannot be perceived does not have to be transmitted. The resulting haptic communication schemes are highly efficient with respect to the communication resources required and can be used for both remote interaction with a real environment and a virtual environment. We have also proposed solutions which allow multiple users to physically interact with the same object in a virtual environment in a coherent and natural manner. What would you say are the main contributions of your research to the main issues currently faced in this field? We have been among the first to address the topic of haptic communication from a technical / communication engineering point of view. Some of the approaches we have proposed are first-of-their-kind and I hope it is fair to say that we have significantly advanced the state-of-the-art in this emerging field. An example of this is the perceptual deadband coding approach that we have proposed for data reduction in haptic communication. This coding approach works for haptic signals with multiple degrees of freedom. It has also been combined with passivity-based control architectures and hence can also be used in networked teleoperation scenarios where the communication delay between the two sides jeopardises system stability. Can you tell us about the mathematical model you developed? This model combines many of the known limitations of human haptic perception with some other limitations that we have initially described in a common framework, which can be used to decide if a haptic signal change is perceivable by a human or if it stays below the perception threshold. Based on this model, highly efficient data reduction schemes can be designed. Another application of the model is the definition of objective quality metrics, which replace (at least partially) the need for costly and time-consuming user studies. Hopefully, these approaches will accelerate the progress in this field. Now that the project is completed, do you have any further plans to exploit or build upon its results? Yes, together with two of the PhD candidates involved in the PROHAPTICS project, we are currently investigating how to commercialise selected results and working towards a start-up company in this area. We are in the fortunate situation where our related ERC Proof of Concept application ROVI has been selected for funding. This will allow us to get a better idea about the market potential of our results over the next 18 months and to produce prototypes in close collaboration with potential customers. PROHAPTICS Funded under FP7-IDEAS-ERC. project page on CORDIS
Countries
Germany