Shape Understanding: On the Perception of Growth, Form and Process

Información del proyecto

SHAPE

Identificador del acuerdo de subvención: 682859

DOI

10.3030/682859

Proyecto cerrado

Fecha de la firma de la CE 7 Julio 2016

Fecha de inicio 1 Agosto 2016

Fecha de finalización 31 Enero 2022

Financiado con arreglo a

• EXCELLENT SCIENCE - European Research Council (ERC)

Coste total

€ 1 950 725,00

Aportación de la UE

€ 1 950 725,00

Coordinado por

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN
Germany

Periodic Reporting for period 4 - SHAPE (Shape Understanding: On the Perception of Growth, Form and Process)

Período documentado: 2021-02-01 hasta 2022-01-31

Resumen del contexto y de los objetivos generales del proyecto

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much in common. But in fact they all share have one important feature: they all involve the way we perceive and interpret shape. To work out the properties of draped cloth involves identifying and interpreting the specific shape of its creases and folds. Predicting how an object will deform when we pick it up involves recognizing the weak points in its shape and understanding how they will change in response to forces. Working out whether an object is a shoe or a fish involves identifying key features of its shape that it has in common with other members of each class. All these tasks involve perceiving, representing and interpreting the shape of objects to understand more about them. Indeed, shape perception pervades practically all aspects of visual behaviour.

There has been a lot of previous research on how the brain works out the local surface properties of shapes such as the depths or curvatures. However, practically nothing is known about how the brain uses its estimates of shape to work out other properties of objects. In SHAPE, we seek to understand how the brain goes beyond simply working out what shapes objects have, to working out their underlying ‘logic’, that is, how and why they have those shapes. We are developing a new field of study in the perceptual and cognitive sciences to unravel how the brain identifies and represents the origins of form.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

We have used a combination of approaches from psychology, neuroscience, computer graphics and machine learning to investigate how we perceive and interpret 2D and 3D shape.

We have used computer simulations and neural networks to create databases consisting of hundreds of thousands of shapes being created and altered by interactions with other objects. For example, we have simulated liquids flowing, and soft objects being squashed, as well as ‘grown’ novel animal-like outlines, using a neural network that was trained on examples of thousands of natural animal shapes. This allows us to characterize shape --- and the way shape changes --- in the natural world.

We have then used these stimuli to perform visual perception experiments on volunteers, who judge the shape, or the objects properties (e.g. how soft the object is) based on viewing the shape or watching how the shape changes over time. By carefully selecting stimuli from our giant datasets, we can tease apart the predictions of rival theories about how the brain processes and interprets shape. We then build computational models of visual perception processes.

We have used this approach to study the perception of material properties, causal history, categorization and perceived shape similarity. This approach has allowed us to show, for example, that when we view liquids, we use a combination of particular shape and motion cues to work out how viscous the liquid. These cues work across a wide range of viewing conditions.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

SHAPE has significantly advanced our scientific understanding of how we visually perceive shape, which is the most important source of visual information about objects. We have created tools, methods and databases that enable the study of complex naturalistic shapes on a scale never tried previously. Among other achievements, this has resulted in a computational model for comparing how similar different shapes are, which can predict human judgments of perceived similarity for novel shapes. We have also uncovered basic visual mechanisms involved in the perception of material properties from shape, motion and optical cues.

image copyright Jan Jaap Van Assen, reproduced with permission