Periodic Reporting for period 2 - DyViTo (DyViTo: Dynamics in Vision and Touch - the look and feel of stuff)
Reporting period: 2019-10-01 to 2022-03-31
DyViTo stands for Dynamics of Vision and Touch: the look and feel of stuff. Daily tasks as diverse as drinking tea or operating machines require us to integrate information across time and the senses rapidly and flexibly. Understanding how the human brain performs dynamic, multisensory integration is a key question with important applications in creating digital and virtual environments. Our goal is to produce a step change in the industrial challenge of creating virtual objects that look, feel, move and change like ‘the real thing’. Levering the research excellence of 9 universities across Europe, we are combining psychology and neuroscience with engineering and computer graphics, to understand and use human sensory perception to drive applied innovations. Concurrently, we utilise the insights from 6 non-academic partners, including Schlenk Metallic Pigments (Germany) and Substance by Adobe (France) to deliver the following objectives:
- understand how humans, perceive dynamic changes in shape, material properties and illumination using new behavioural and brain imaging techniques
- measure and model the sensory integration of vision and touch information as we explore objects to determine their material properties
- exploit insights from perception science to optimise the interactive rendering of virtual visual-haptic objects, and to support advances in lighting technology
Our work shall benefit the wider community and be accessible by everyone. Whether for scientific or industry application, our findings will help shape academia, create new industry standards and inspire a whole new generation of scientists.
- understand how humans, perceive dynamic changes in shape, material properties and illumination using new behavioural and brain imaging techniques
- measure and model the sensory integration of vision and touch information as we explore objects to determine their material properties
- exploit insights from perception science to optimise the interactive rendering of virtual visual-haptic objects, and to support advances in lighting technology
Our work shall benefit the wider community and be accessible by everyone. Whether for scientific or industry application, our findings will help shape academia, create new industry standards and inspire a whole new generation of scientists.
All our beneficiary institutions have employed at least one Early Stage Researcher (ESR). The University of Newcastle (UK) is moving us closer to practical applications of dynamic illumination. Their research extends our understanding of perception of dynamic changes in illumination on their own and in relation to how we perceive objects and visually identify them. The ESR won a prize at the International Colour Vision Society Meeting 2019 for this work. Research into how light affects our view of objects and spaces is also being carried out at University of Delft (Netherlands), who have two ESRs working on separate projects. The first researcher is starting to show the applications of the theoretical Delft Light Framework and revolutionising how architects think about light in a space. The second researcher has been developing a solution to help non-expert users, within the sphere online retailing to manipulate light in order to present their product best. An ESR working at the University of Giessen (Germany) is investigating how people perceive softness. To decide how soft a pillow is you may squash it however you would probably stroke something furry. They found people distinguish between more elements of softness than previously assumed, and that we use different movements and motions when it comes to exploring these different dimensions. This work won Best Student Paper at the World Haptics Conference 2019 in Tokyo. A second ESR is taking a more industry focused approach. The question our partners need to answer is whether two surfaces have the same surface gloss, matching sufficiently well to pass quality control. University of Giessen also played host to an ESR from the University of Bilkent (Turkey) on secondment. The researcher was able to study response times and accuracies of human participants in simple tasks under different conditions, relating to the effect of expectations on how we look at different moving objects. University of Southampton (UK) hosted a secondment from the University of Ulm, where the ESR was trying to better understand how vision and touch work together when materials of different compositions are manipulated. University of Southampton themselves have been making significant progress in vision and haptics, looking at the interaction of visual and touch cues for roughness and slant. The ESR at the University of Zaragoza (Spain) has presented at the ACM Symposium on Applied Perception 2019 his work on prototype software that will allow users to intuitively modify how materials appear virtually, specifically when objects are in motion. The goal of the research at the University of Cambridge (UK) is to describe the functional interactions between different cortical regions of the brain that allow multisensory signals (vision and touch) to be integrated. The ESR based at the University of Bradford (UK) has been making progress in understanding what makes a material look old, a secondment at the National Science and Media Museum (UK) allowed her to conduct a pilot study during the Bradford Science Festival, on why people decide to keep or discard objects. All DyViTo members have benefited from the many collaboration and training opportunities in particular the two network wide meetings organised by the University of Bradford (10-14 December 2018) and the University of Giessen (11-14 March 2019) were we were joined by other leading researchers from around the world for focused workshops and tutorials.
The innovative work carried out by DyViTo leads to significant impact across many sectors. The research from the University of Giessen is a potential breakthrough, developing a complete end-to-end system which would fundamentally change the way gloss is measured in automotive industry. The Delft Light Framework will be relevant for various for industry sectors; lighting design companies and architects. University of Zaragoza and the University of Delft are both pushing research further in discovering additional heuristics that can help improve how objects are rendered in computer graphics influencing how game designers and special effects experts work in the entertainment industry.The impact of DyViTo’s work benefits the researchers themselves. The ESRs have been able to take full advantage of the training provided, varying from research specific skills to responsible research and innovation. DyViTo provides a platform for the ESRs to showcase their achievements to a multidisciplinary audience of potential employers and collaborators both during network meetings and conferences. DyViTo is proud to play a part in developing a new generation of scientists, enhancing the availability of highly trained researchers who have already been exposed to working in various countries and sectors across the EU. DyViTo has added value to the wider European society by strengthening collaboration and building long-lasting relationships across beneficiaries in participating countries, as well as amassing high quality research that can be accessed by anyone. DyViTo co-develops truly international links and ensures the spread of best practice as well as increasing Europe’s visibility and attractiveness as a research location. Our researchers have presented their findings across Europe, Asia and America with overwhelmingly positive feedback. DyViTo is committed to the development and support of citizen science which has profound future cultural and socio-economic benefits. Through developing outreach activities in Museums and by the use of social media it continues to raise DyViTo’s public profile and that of the EU as a funder and supporter of this innovative training network.