Final Report Summary - POEM (Perceptual Organization and Eye Movements)
A summary description of the project objectives:
We consciously experience the world as made up of complete objects, e.g. a house behind a tree, and not a juxtaposition of colored blobs and edges. The process by which the brain transforms visual input on the retina into coherent objects is called perceptual organization, and understanding this process is a fundamental scientific problem. Despite of the efforts, very little progress has been made in understanding the representations and processes underlying perceptual organization, more specifically in understanding of how coherent percept arises from the discrete samples that are collected by eye movements. The major goal of this project (POEM) is to investigate the missing link, i.e. the link between discrete eye-movements and holistic perceptual organization. Connection between Perceptual Organization (PO) and Eye Movements (EM) will be investigated at three levels of processing explicit (ePOEM), implicit (iPOEM) and functional (fPOEM).
Objectives:
ePOEM: Visual correlates of PO from explicit EM
iPOEM: Quantitative measures of PO from implicit modulation of EM
fPOEM: Correlations between of PO-aesthetics and functional ease of EM
Description of the work performed since the beginning of the project:
In the 1st phase, in collaboration with Johan Wagemans (University of Leuven, Belgium), we successfully developed a saccade-based implicit measure for perceptual grouping based on goal-directed EM. In the 2nd phase, in collaboration with Stephen Palmer (Berkeley, USA) we established that explicit EM can influence the perceived perceptual grouping. Finally, we combined the goals of ePOEM and fPOEM in a project conducted in collaboration with Andreas Dengel, DKFI (Kaiserslautern, Germany) and Takeshi Suzuki (Ricoh Institute of Technology, Japan). Here, we studied the EM of expert photographers and novices to reveal task relevant visual features for different photo editing tasks, such as contrast/color adjustments. A comparison of human EM- based points of interests and those selected by various the state-of-the-art computer algorithms were made to devise plans for developing better software.
(for further details see: https://www.sowi.uni-kl.de/fileadmin/wpsy/public/POEM/POEM.htm(öffnet in neuem Fenster))
A description of the main results achieved so far:
The results of iPOEM showed that saccadic latencies are implicitly modified by the spatial location of group with respect to the target. Grouping cues of proximity and good-continuation were effective in combination but proximity or good-continuation per se failed to show any significant effect in the absence of the other one. Lastly, comparison with typically used behavioral measures for grouping reinforce the importance of EM based metrics to investigate such subtle effects. The second study showed that the perceived direction of grouping (horizontal/vertical) of a grid of dots can be influenced by the direction of prior volitional EM. That is, a prior vertical EM biased participants to report that an equally spaced ambiguous array of dots was arranged in vertical columns more often than horizontal rows. In the third study, we show that for photo editing tasks, there is a significant difference in EM patterns of experts and novices suggesting that expertise can be defined in terms of selection of task-relevant visual features. We quantified the differences in EM based on spatial distribution of fixations performed during the task. A support-vector machine was trained to classify EM-based features (luminance/color at fixation, luminance/color-variance in small/large region around fixation) into experts or novices. Classification accuracy was significantly above chance both for the contrast (60 %) and color (52 %) adjustment tasks. Luminance-features were significantly more discriminative during contrast than during color-adjustment, and vice-versa for color-features.
Based on EM based classification of observer expertise we conclude that EM encode task-relevant information (increased discriminability of color-/luminance-features in color/luminance-based tasks respectively).
The expected final results and their potential impact and use (including the socio-economic impact and the wider societal implications):
DISSEMINATION OF RESULTS IN REPUTED INTERNATIONAL CONFERENCES: The scientific results were presented at reputed international conferences (e.g. European Conference on Visual Perception (ECVP 2012, 2013, 2014, 2015, EU), Vision Sciences Society (VSS 2012, 2013, 2014, 2015, Florida, USA), European Conference on Eye Movements, 2015, Austria)), for presentation as talks or posters. The scientific results will also be submitted for publication in prestigious peer-reviewed scientific journals. A dedicated webpage is being maintained at my workgroup’s website with all the current results from this project. The stimulus and the software developed for this project is available by an email link on the website, enabling their use by anyone in the community worldwide. Our results led to the submission of two patent applications with RICOH®, Japan.
TRAINING OF STUDENT RESEARCHERS: Additionally, one main product of POEM was the set of students educated within this project. Now these students are trained in the science of mid-level vision as well as general research methodology such as devising and running experiments and data analysis and the use of sophisticated instruments such as the eye-tracker.
PUBLIC OUTREACH ACTIVITIES: The three workshops were widely advertised with eye-catching posters, flyers and newspaper advertisements. They raised a curiosity about the process of “seeing” in the city among people from various walks of life. The event “Wahrnehmung – Public lecture series on Perception” was inaugurated by the President of University of Kaiserslautern and Lord Mayor of the city of Kaiserslautern. It was covered by press and was one of the highlights of events happening in the city for that week. Sixteen world renowned researchers, seven from the US and the rest from EU, in visual perception met at my host institution for stimulating workshop enabling exchange of knowledge between my home country, the USA and the EU.
LAST BUT NOT LEAST, THIS FUNDING HAS ENABLED ME TO BE VERY SUCCESSFUL IN MY MIDTERM REVIEW LEADING TO MY APPOINTMENT AS JUNIOR PROFESSOR- PHASE-2 AND HAS HAD A TREMENDOUS POSITIVE IMPACT ON MY INTEGRATION AS A WELL ROUNDED RESEARCHER IN THE EU.
We consciously experience the world as made up of complete objects, e.g. a house behind a tree, and not a juxtaposition of colored blobs and edges. The process by which the brain transforms visual input on the retina into coherent objects is called perceptual organization, and understanding this process is a fundamental scientific problem. Despite of the efforts, very little progress has been made in understanding the representations and processes underlying perceptual organization, more specifically in understanding of how coherent percept arises from the discrete samples that are collected by eye movements. The major goal of this project (POEM) is to investigate the missing link, i.e. the link between discrete eye-movements and holistic perceptual organization. Connection between Perceptual Organization (PO) and Eye Movements (EM) will be investigated at three levels of processing explicit (ePOEM), implicit (iPOEM) and functional (fPOEM).
Objectives:
ePOEM: Visual correlates of PO from explicit EM
iPOEM: Quantitative measures of PO from implicit modulation of EM
fPOEM: Correlations between of PO-aesthetics and functional ease of EM
Description of the work performed since the beginning of the project:
In the 1st phase, in collaboration with Johan Wagemans (University of Leuven, Belgium), we successfully developed a saccade-based implicit measure for perceptual grouping based on goal-directed EM. In the 2nd phase, in collaboration with Stephen Palmer (Berkeley, USA) we established that explicit EM can influence the perceived perceptual grouping. Finally, we combined the goals of ePOEM and fPOEM in a project conducted in collaboration with Andreas Dengel, DKFI (Kaiserslautern, Germany) and Takeshi Suzuki (Ricoh Institute of Technology, Japan). Here, we studied the EM of expert photographers and novices to reveal task relevant visual features for different photo editing tasks, such as contrast/color adjustments. A comparison of human EM- based points of interests and those selected by various the state-of-the-art computer algorithms were made to devise plans for developing better software.
(for further details see: https://www.sowi.uni-kl.de/fileadmin/wpsy/public/POEM/POEM.htm(öffnet in neuem Fenster))
A description of the main results achieved so far:
The results of iPOEM showed that saccadic latencies are implicitly modified by the spatial location of group with respect to the target. Grouping cues of proximity and good-continuation were effective in combination but proximity or good-continuation per se failed to show any significant effect in the absence of the other one. Lastly, comparison with typically used behavioral measures for grouping reinforce the importance of EM based metrics to investigate such subtle effects. The second study showed that the perceived direction of grouping (horizontal/vertical) of a grid of dots can be influenced by the direction of prior volitional EM. That is, a prior vertical EM biased participants to report that an equally spaced ambiguous array of dots was arranged in vertical columns more often than horizontal rows. In the third study, we show that for photo editing tasks, there is a significant difference in EM patterns of experts and novices suggesting that expertise can be defined in terms of selection of task-relevant visual features. We quantified the differences in EM based on spatial distribution of fixations performed during the task. A support-vector machine was trained to classify EM-based features (luminance/color at fixation, luminance/color-variance in small/large region around fixation) into experts or novices. Classification accuracy was significantly above chance both for the contrast (60 %) and color (52 %) adjustment tasks. Luminance-features were significantly more discriminative during contrast than during color-adjustment, and vice-versa for color-features.
Based on EM based classification of observer expertise we conclude that EM encode task-relevant information (increased discriminability of color-/luminance-features in color/luminance-based tasks respectively).
The expected final results and their potential impact and use (including the socio-economic impact and the wider societal implications):
DISSEMINATION OF RESULTS IN REPUTED INTERNATIONAL CONFERENCES: The scientific results were presented at reputed international conferences (e.g. European Conference on Visual Perception (ECVP 2012, 2013, 2014, 2015, EU), Vision Sciences Society (VSS 2012, 2013, 2014, 2015, Florida, USA), European Conference on Eye Movements, 2015, Austria)), for presentation as talks or posters. The scientific results will also be submitted for publication in prestigious peer-reviewed scientific journals. A dedicated webpage is being maintained at my workgroup’s website with all the current results from this project. The stimulus and the software developed for this project is available by an email link on the website, enabling their use by anyone in the community worldwide. Our results led to the submission of two patent applications with RICOH®, Japan.
TRAINING OF STUDENT RESEARCHERS: Additionally, one main product of POEM was the set of students educated within this project. Now these students are trained in the science of mid-level vision as well as general research methodology such as devising and running experiments and data analysis and the use of sophisticated instruments such as the eye-tracker.
PUBLIC OUTREACH ACTIVITIES: The three workshops were widely advertised with eye-catching posters, flyers and newspaper advertisements. They raised a curiosity about the process of “seeing” in the city among people from various walks of life. The event “Wahrnehmung – Public lecture series on Perception” was inaugurated by the President of University of Kaiserslautern and Lord Mayor of the city of Kaiserslautern. It was covered by press and was one of the highlights of events happening in the city for that week. Sixteen world renowned researchers, seven from the US and the rest from EU, in visual perception met at my host institution for stimulating workshop enabling exchange of knowledge between my home country, the USA and the EU.
LAST BUT NOT LEAST, THIS FUNDING HAS ENABLED ME TO BE VERY SUCCESSFUL IN MY MIDTERM REVIEW LEADING TO MY APPOINTMENT AS JUNIOR PROFESSOR- PHASE-2 AND HAS HAD A TREMENDOUS POSITIVE IMPACT ON MY INTEGRATION AS A WELL ROUNDED RESEARCHER IN THE EU.
