Shape Understanding: On the Perception of Growth, Form and Process

Projektinformationen

SHAPE

ID Finanzhilfevereinbarung: 682859

DOI

10.3030/682859

Projekt abgeschlossen

EK-Unterschriftsdatum 7 Juli 2016

Startdatum 1 August 2016

Enddatum 31 Januar 2022

Finanziert unter

• EXCELLENT SCIENCE - European Research Council (ERC)

Gesamtkosten

€ 1 950 725,00

EU-Beitrag

€ 1 950 725,00

Koordiniert durch

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN
Germany

Periodic Reporting for period 4 - SHAPE (Shape Understanding: On the Perception of Growth, Form and Process)

Berichtszeitraum: 2021-02-01 bis 2022-01-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much in common. But in fact they all share have one important feature: they all involve the way we perceive and interpret shape. To work out the properties of draped cloth involves identifying and interpreting the specific shape of its creases and folds. Predicting how an object will deform when we pick it up involves recognizing the weak points in its shape and understanding how they will change in response to forces. Working out whether an object is a shoe or a fish involves identifying key features of its shape that it has in common with other members of each class. All these tasks involve perceiving, representing and interpreting the shape of objects to understand more about them. Indeed, shape perception pervades practically all aspects of visual behaviour.

There has been a lot of previous research on how the brain works out the local surface properties of shapes such as the depths or curvatures. However, practically nothing is known about how the brain uses its estimates of shape to work out other properties of objects. In SHAPE, we seek to understand how the brain goes beyond simply working out what shapes objects have, to working out their underlying ‘logic’, that is, how and why they have those shapes. We are developing a new field of study in the perceptual and cognitive sciences to unravel how the brain identifies and represents the origins of form.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

We have used a combination of approaches from psychology, neuroscience, computer graphics and machine learning to investigate how we perceive and interpret 2D and 3D shape.

We have used computer simulations and neural networks to create databases consisting of hundreds of thousands of shapes being created and altered by interactions with other objects. For example, we have simulated liquids flowing, and soft objects being squashed, as well as ‘grown’ novel animal-like outlines, using a neural network that was trained on examples of thousands of natural animal shapes. This allows us to characterize shape --- and the way shape changes --- in the natural world.

We have then used these stimuli to perform visual perception experiments on volunteers, who judge the shape, or the objects properties (e.g. how soft the object is) based on viewing the shape or watching how the shape changes over time. By carefully selecting stimuli from our giant datasets, we can tease apart the predictions of rival theories about how the brain processes and interprets shape. We then build computational models of visual perception processes.

We have used this approach to study the perception of material properties, causal history, categorization and perceived shape similarity. This approach has allowed us to show, for example, that when we view liquids, we use a combination of particular shape and motion cues to work out how viscous the liquid. These cues work across a wide range of viewing conditions.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

SHAPE has significantly advanced our scientific understanding of how we visually perceive shape, which is the most important source of visual information about objects. We have created tools, methods and databases that enable the study of complex naturalistic shapes on a scale never tried previously. Among other achievements, this has resulted in a computational model for comparing how similar different shapes are, which can predict human judgments of perceived similarity for novel shapes. We have also uncovered basic visual mechanisms involved in the perception of material properties from shape, motion and optical cues.

image copyright Jan Jaap Van Assen, reproduced with permission