Final Report Summary - VISUALCROWDING (Visual crowding: The paradox of position)
In our visual field, objects are typically easy to see when we look directly at them and difficult to see in the periphery (the 'edges' of our vision). This difference is not simply to do with resolution - even when a target object in the periphery is large enough to be seen in isolation, the placement of other objects in its immediate vicinity can render the target 'jumbled' and impossible to recognise. This is known as “visual crowding”. It is crowding and not simple resolution that limits our object recognition over more than 95% of the visual field.
The dominant view of crowding depicts it as an excessive integration of object features that simplifies the crowded region into efficiently processed texture. A key assumption here is that the position of objects is lost – otherwise, the true features would be seen in their true locations. This is, however, inconsistent with the high positional specificity of crowding: its magnitude relies strongly on the position of objects, both in relation to one another (there is more crowding for closely adjacent objects) and across the visual field (there is more crowding as we move out into the peripheral visual field). In addition, while crowding affects object identities, it rarely fills the spaces in between. A simple feature integration process would not retain this strong specificity for position. We call this discrepancy between gross object position and the position of their features “the paradox of position” in crowding and set out to explore it in this project.
We first set out to examine the nature of the position coding involved in crowding. We have first utilised illusions of position to demonstrate that the magnitude of crowding depends on the perceived position of object elements rather than their physical positions. This suggests that crowding is a higher-order form of position uncertainty within the visual system (Dakin, Greenwood, Carlson & Bex, 2011, Journal of Vision).
But how much uncertainty is there about crowded object locations and how does this arise? We have investigated this by comparing the spatial extent of crowding with judgements of the position of the same stimuli. For a range of position judgements, including judgements of the position of dots and the accuracy of saccadic (ballistic) eye movements, the scale of position uncertainty is well below that predicted by crowding. This demonstrates that the scale of the uncertainty regarding crowded features is far above our uncertainty about the location of the objects themselves. However, we also find that individual variations in the spatial extent of crowding are correlated with variations in saccadic accuracy. This suggests a link between position uncertainty and crowding despite the differences in scale. We suggest that the seeds of uncertainty are set early in the visual system and inherited throughout, to the later stages of visual processing where crowding occurs (Greenwood, Szinte, Sayim & Cavanagh, 2012, Journal of Vision). It is likely that this position uncertainty can also explain the co-variation between crowding and stereo-acuity that we observe in both normal children and those with amblyopia (Greenwood, Tailor, Simmers, Sloper, Bex & Dakin, 2012, Investigative Ophthalmology & Visual Science).
Given that crowding is an uncertainty about feature locations, despite the lower uncertainty about object locations, our second aim was to examine the way in which crowded features are mapped onto the apparently veridical location of objects. We did so by examining the release from crowding – the feature interactions that determine when crowding does and does not occur (and thus when the correct features are seen in the correct locations). We first demonstrate that crowding is tuned for the transient onset of events – a brief ‘blink’ at the beginning of an object presentation can dramatically reduce crowding. However, this release occurs only when the blink occurs in the target object, and not in flanking objects. We suggest it is the specific localisation of the target features in their correct location that causes this release from crowding (Greenwood, Sayim & Cavanagh, 2011, Perception).
We also report that when the release from crowding occurs, all of the features of an object (e.g. their orientation, position, and colour) are released together. Specifically, we demonstrate that when one feature type (e.g. orientation) releases crowding, others relevant to that object (e.g. their relative positions) are similarly released. This again paints crowding as a higher-order process, perhaps even ‘object-based’ in line with findings from the first research stream (Greenwood, Bex & Dakin, 2012, Journal of Vision). We extended this finding with an examination of whether distinct object parts (e.g. the legs of a chair vs. the seat) can release crowding for the whole object. Indeed, we find that with simple ‘wheel’ elements, altering the colour of one small part of the object can release the whole object, even when these parts are separated from the task-relevant object part. With this technique we have defined a small region around the target within which adjacent object parts can release crowding (Greenwood & Cavanagh, 2013). This ‘release zone’ may relate to the smaller degree of positional uncertainty observed earlier on in the visual system and measured in our first research stream. We suggest that the release from crowding may affect multiple levels of the visual system, just as we see with measurements of position uncertainty.
Our third research aim was to examine the neural basis of crowding, where we again see the multi-level nature of crowding. We examined these neural mechanisms using functional magnetic brain resonance imaging (fMRI). One of the key effects of crowding is that it changes the appearance of target objects, bringing them closer to an average of the target with surrounding object features. We have utilised this property to examine these changes using fMRI and report that crowding changes even the earliest stages of the brain, with an increasing effect for cortical areas higher up in the cortical hierarchy, V2 to V3 and V4. Crowding thus has widespread effects throughout the visual system (Anderson, Dakin, Schwarzkopf, Rees & Greenwood, 2012, Current Biology), consistent with our many behavioural observations described above.
Finally, a fourth research stream examined the processes that give rise to our perception of the density of textures. This is a process in which position information is both highly important (in defining texture boundaries) and yet seemingly lost (since the localisation of individual elements is highly difficult). The relation to crowding is thus of great interest. Our findings demonstrate that density perception is distinct from the perception of numerosity, a process that is well described by a “low-level” computational model using filters that take fine detail and map them onto coarse detail (Dakin, Tibber, Greenwood, Kingdom & Morgan, 2011, Proceedings of the National Academy of Sciences). There is strong possibility that these ideas could similarly apply to crowding and we are exploring this in ongoing work (e.g. Tibber, Greenwood & Dakin, 2012, Journal of Vision).
Together, our results give new insights into the basis of crowding. We conceptualise this phenomenon as a higher-level form of position uncertainty within the visual system that is nonetheless heavily dependent upon earlier stages of processing. An increased understanding of this phenomenon is important not only in terms of understanding our own visual perception, but also in appropriately directing treatment programs in cases where crowding becomes elevated – notably in cases of strabismic amblyopia (‘lazy eye’) and dyslexia.
The dominant view of crowding depicts it as an excessive integration of object features that simplifies the crowded region into efficiently processed texture. A key assumption here is that the position of objects is lost – otherwise, the true features would be seen in their true locations. This is, however, inconsistent with the high positional specificity of crowding: its magnitude relies strongly on the position of objects, both in relation to one another (there is more crowding for closely adjacent objects) and across the visual field (there is more crowding as we move out into the peripheral visual field). In addition, while crowding affects object identities, it rarely fills the spaces in between. A simple feature integration process would not retain this strong specificity for position. We call this discrepancy between gross object position and the position of their features “the paradox of position” in crowding and set out to explore it in this project.
We first set out to examine the nature of the position coding involved in crowding. We have first utilised illusions of position to demonstrate that the magnitude of crowding depends on the perceived position of object elements rather than their physical positions. This suggests that crowding is a higher-order form of position uncertainty within the visual system (Dakin, Greenwood, Carlson & Bex, 2011, Journal of Vision).
But how much uncertainty is there about crowded object locations and how does this arise? We have investigated this by comparing the spatial extent of crowding with judgements of the position of the same stimuli. For a range of position judgements, including judgements of the position of dots and the accuracy of saccadic (ballistic) eye movements, the scale of position uncertainty is well below that predicted by crowding. This demonstrates that the scale of the uncertainty regarding crowded features is far above our uncertainty about the location of the objects themselves. However, we also find that individual variations in the spatial extent of crowding are correlated with variations in saccadic accuracy. This suggests a link between position uncertainty and crowding despite the differences in scale. We suggest that the seeds of uncertainty are set early in the visual system and inherited throughout, to the later stages of visual processing where crowding occurs (Greenwood, Szinte, Sayim & Cavanagh, 2012, Journal of Vision). It is likely that this position uncertainty can also explain the co-variation between crowding and stereo-acuity that we observe in both normal children and those with amblyopia (Greenwood, Tailor, Simmers, Sloper, Bex & Dakin, 2012, Investigative Ophthalmology & Visual Science).
Given that crowding is an uncertainty about feature locations, despite the lower uncertainty about object locations, our second aim was to examine the way in which crowded features are mapped onto the apparently veridical location of objects. We did so by examining the release from crowding – the feature interactions that determine when crowding does and does not occur (and thus when the correct features are seen in the correct locations). We first demonstrate that crowding is tuned for the transient onset of events – a brief ‘blink’ at the beginning of an object presentation can dramatically reduce crowding. However, this release occurs only when the blink occurs in the target object, and not in flanking objects. We suggest it is the specific localisation of the target features in their correct location that causes this release from crowding (Greenwood, Sayim & Cavanagh, 2011, Perception).
We also report that when the release from crowding occurs, all of the features of an object (e.g. their orientation, position, and colour) are released together. Specifically, we demonstrate that when one feature type (e.g. orientation) releases crowding, others relevant to that object (e.g. their relative positions) are similarly released. This again paints crowding as a higher-order process, perhaps even ‘object-based’ in line with findings from the first research stream (Greenwood, Bex & Dakin, 2012, Journal of Vision). We extended this finding with an examination of whether distinct object parts (e.g. the legs of a chair vs. the seat) can release crowding for the whole object. Indeed, we find that with simple ‘wheel’ elements, altering the colour of one small part of the object can release the whole object, even when these parts are separated from the task-relevant object part. With this technique we have defined a small region around the target within which adjacent object parts can release crowding (Greenwood & Cavanagh, 2013). This ‘release zone’ may relate to the smaller degree of positional uncertainty observed earlier on in the visual system and measured in our first research stream. We suggest that the release from crowding may affect multiple levels of the visual system, just as we see with measurements of position uncertainty.
Our third research aim was to examine the neural basis of crowding, where we again see the multi-level nature of crowding. We examined these neural mechanisms using functional magnetic brain resonance imaging (fMRI). One of the key effects of crowding is that it changes the appearance of target objects, bringing them closer to an average of the target with surrounding object features. We have utilised this property to examine these changes using fMRI and report that crowding changes even the earliest stages of the brain, with an increasing effect for cortical areas higher up in the cortical hierarchy, V2 to V3 and V4. Crowding thus has widespread effects throughout the visual system (Anderson, Dakin, Schwarzkopf, Rees & Greenwood, 2012, Current Biology), consistent with our many behavioural observations described above.
Finally, a fourth research stream examined the processes that give rise to our perception of the density of textures. This is a process in which position information is both highly important (in defining texture boundaries) and yet seemingly lost (since the localisation of individual elements is highly difficult). The relation to crowding is thus of great interest. Our findings demonstrate that density perception is distinct from the perception of numerosity, a process that is well described by a “low-level” computational model using filters that take fine detail and map them onto coarse detail (Dakin, Tibber, Greenwood, Kingdom & Morgan, 2011, Proceedings of the National Academy of Sciences). There is strong possibility that these ideas could similarly apply to crowding and we are exploring this in ongoing work (e.g. Tibber, Greenwood & Dakin, 2012, Journal of Vision).
Together, our results give new insights into the basis of crowding. We conceptualise this phenomenon as a higher-level form of position uncertainty within the visual system that is nonetheless heavily dependent upon earlier stages of processing. An increased understanding of this phenomenon is important not only in terms of understanding our own visual perception, but also in appropriately directing treatment programs in cases where crowding becomes elevated – notably in cases of strabismic amblyopia (‘lazy eye’) and dyslexia.