Shape Understanding: On the Perception of Growth, Form and Process

Informations projet

SHAPE

N° de convention de subvention: 682859

DOI

10.3030/682859

Projet clôturé

Date de signature de la CE 7 Juillet 2016

Date de début 1 Août 2016

Date de fin 31 Janvier 2022

Financé au titre de

• EXCELLENT SCIENCE - European Research Council (ERC)

Coût total

€ 1 950 725,00

Contribution de l’UE

€ 1 950 725,00

Coordonné par

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN
Germany

Periodic Reporting for period 4 - SHAPE (Shape Understanding: On the Perception of Growth, Form and Process)

Période du rapport: 2021-02-01 au 2022-01-31

Résumé du contexte et des objectifs généraux du projet

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much in common. But in fact they all share have one important feature: they all involve the way we perceive and interpret shape. To work out the properties of draped cloth involves identifying and interpreting the specific shape of its creases and folds. Predicting how an object will deform when we pick it up involves recognizing the weak points in its shape and understanding how they will change in response to forces. Working out whether an object is a shoe or a fish involves identifying key features of its shape that it has in common with other members of each class. All these tasks involve perceiving, representing and interpreting the shape of objects to understand more about them. Indeed, shape perception pervades practically all aspects of visual behaviour.

There has been a lot of previous research on how the brain works out the local surface properties of shapes such as the depths or curvatures. However, practically nothing is known about how the brain uses its estimates of shape to work out other properties of objects. In SHAPE, we seek to understand how the brain goes beyond simply working out what shapes objects have, to working out their underlying ‘logic’, that is, how and why they have those shapes. We are developing a new field of study in the perceptual and cognitive sciences to unravel how the brain identifies and represents the origins of form.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

We have used a combination of approaches from psychology, neuroscience, computer graphics and machine learning to investigate how we perceive and interpret 2D and 3D shape.

We have used computer simulations and neural networks to create databases consisting of hundreds of thousands of shapes being created and altered by interactions with other objects. For example, we have simulated liquids flowing, and soft objects being squashed, as well as ‘grown’ novel animal-like outlines, using a neural network that was trained on examples of thousands of natural animal shapes. This allows us to characterize shape --- and the way shape changes --- in the natural world.

We have then used these stimuli to perform visual perception experiments on volunteers, who judge the shape, or the objects properties (e.g. how soft the object is) based on viewing the shape or watching how the shape changes over time. By carefully selecting stimuli from our giant datasets, we can tease apart the predictions of rival theories about how the brain processes and interprets shape. We then build computational models of visual perception processes.

We have used this approach to study the perception of material properties, causal history, categorization and perceived shape similarity. This approach has allowed us to show, for example, that when we view liquids, we use a combination of particular shape and motion cues to work out how viscous the liquid. These cues work across a wide range of viewing conditions.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

SHAPE has significantly advanced our scientific understanding of how we visually perceive shape, which is the most important source of visual information about objects. We have created tools, methods and databases that enable the study of complex naturalistic shapes on a scale never tried previously. Among other achievements, this has resulted in a computational model for comparing how similar different shapes are, which can predict human judgments of perceived similarity for novel shapes. We have also uncovered basic visual mechanisms involved in the perception of material properties from shape, motion and optical cues.

image copyright Jan Jaap Van Assen, reproduced with permission