Shape Understanding: On the Perception of Growth, Form and Process

Project Information

SHAPE

Grant agreement ID: 682859

DOI

10.3030/682859

Project closed

EC signature date 7 July 2016

Start date 1 August 2016

End date 31 January 2022

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 950 725,00

EU contribution

€ 1 950 725,00

Coordinated by

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN
Germany

Periodic Reporting for period 4 - SHAPE (Shape Understanding: On the Perception of Growth, Form and Process)

Reporting period: 2021-02-01 to 2022-01-31

Summary of the context and overall objectives of the project

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much in common. But in fact they all share have one important feature: they all involve the way we perceive and interpret shape. To work out the properties of draped cloth involves identifying and interpreting the specific shape of its creases and folds. Predicting how an object will deform when we pick it up involves recognizing the weak points in its shape and understanding how they will change in response to forces. Working out whether an object is a shoe or a fish involves identifying key features of its shape that it has in common with other members of each class. All these tasks involve perceiving, representing and interpreting the shape of objects to understand more about them. Indeed, shape perception pervades practically all aspects of visual behaviour.

There has been a lot of previous research on how the brain works out the local surface properties of shapes such as the depths or curvatures. However, practically nothing is known about how the brain uses its estimates of shape to work out other properties of objects. In SHAPE, we seek to understand how the brain goes beyond simply working out what shapes objects have, to working out their underlying ‘logic’, that is, how and why they have those shapes. We are developing a new field of study in the perceptual and cognitive sciences to unravel how the brain identifies and represents the origins of form.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have used a combination of approaches from psychology, neuroscience, computer graphics and machine learning to investigate how we perceive and interpret 2D and 3D shape.

We have used computer simulations and neural networks to create databases consisting of hundreds of thousands of shapes being created and altered by interactions with other objects. For example, we have simulated liquids flowing, and soft objects being squashed, as well as ‘grown’ novel animal-like outlines, using a neural network that was trained on examples of thousands of natural animal shapes. This allows us to characterize shape --- and the way shape changes --- in the natural world.

We have then used these stimuli to perform visual perception experiments on volunteers, who judge the shape, or the objects properties (e.g. how soft the object is) based on viewing the shape or watching how the shape changes over time. By carefully selecting stimuli from our giant datasets, we can tease apart the predictions of rival theories about how the brain processes and interprets shape. We then build computational models of visual perception processes.

We have used this approach to study the perception of material properties, causal history, categorization and perceived shape similarity. This approach has allowed us to show, for example, that when we view liquids, we use a combination of particular shape and motion cues to work out how viscous the liquid. These cues work across a wide range of viewing conditions.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

SHAPE has significantly advanced our scientific understanding of how we visually perceive shape, which is the most important source of visual information about objects. We have created tools, methods and databases that enable the study of complex naturalistic shapes on a scale never tried previously. Among other achievements, this has resulted in a computational model for comparing how similar different shapes are, which can predict human judgments of perceived similarity for novel shapes. We have also uncovered basic visual mechanisms involved in the perception of material properties from shape, motion and optical cues.

image copyright Jan Jaap Van Assen, reproduced with permission