Final Report Summary - IMPLICIT PERCEPTION (Finding the Neural Correlates of Implicit Object Perception)
In 1957 a market researcher named James Vicary conducted a study in which he showed that flashing subliminal advertisements for popcorn and Coca-Cola during movies increased the sales of the advertised products. Vicary himself later admitted that his finding was nothing but “a gimmick" , but the question of how and whether the brain can process visual stimuli that remain invisible, is still intriguing.
In this project we aimed at studying a specific method called “continuous flash suppression” (CFS), which is widely used to render visual stimuli invisible. CFS takes advantage of the natural competition between the two eyes, called binocular rivalry. The visual presentation includes rapid flashes of salient texture to one eye and a low-contrast vague stimulus to the other eye. The perceptual experience will include only the flashes, while the visual stimulus will remain invisible. From the subject’s point of view, the effect is quite dramatic: a stimulus which is clearly visible without the flashes “disappears” completely when the flashes are turned on. CFS is increasingly used in recent years and was used by dozens of studies on unaware perception.
The original purpose of this research was to study object recognition in conditions when the object is unaware. The plan was to present subjects with pictures of objects that are either meaningful (tools, animals, body parts, food items etc) or meaningless (scrambled versions of the same pictures). These pictures will be presented when they are either rendered invisible by CFS (presented to the eye opposite the flashes) or visible (when presented to the same eye as the flashes). This experiment was planned as a functional MRI (fMRI) experiment, where brain activity can be monitored while subjects observe the pictures. In a similar version of the same experiment, familiar and unfamiliar letters (of a known/unknown alphabet) replaced the meaningful and meaningless objects.
In my host lab at NYU, I conducted pilot experimentation with these two procedures. Functional MRI was scanned in 4 subjects when they observed either meaningful.meaningless objects or familiar/unfamiliar letters in conditions where the stimuli were either visible or invisible (with CFS). Our results showed no visual activity at all when objects were invisible, and consequently no familiarity/meaning effect in that condition. In addition, in the visible condition (without CFS) we found activity in early visual cortex (V1) which supposedly suggested that this area, known to be modulated by low physical properties of the stimulus, was actually responding to object identify. Both findings did not fit our original hypothesis and required further investigation. We then raised two new hypotheses which led us to two new research directions: First, we hypothesized that the “object-related” activity we see in V1 is due to small miniature eye movements called “microsaccades”. These involuntary eye movements occur frequently when subjects attempt to keep steady fixation. Microsaccades were shown in the past to be modulated by object identity (be more frequent when object is recognizable). It is possible that their involvement causes the V1 modulation we saw in the visible case. Second, we hypothesized that, contrary to common views on CFS, this manipulation reduces responses in V1 to a level that is almost equivalent to complete stimulus absence. We hypothesized that CFS controls visual gain and not awareness as was previously suggested. We initiated four new directions of research, which luckily turned out to be successful:
1. Microsaccades in fMRI: We initiated a research program on microsaccades. Following our preliminary fMRI results we intended to examine how microsaccades are manifested in fMRI of the visual cortex. We conducted a study where subjects either viewed a stimulus or fixated in complete darkness, while their eye movements and brain activity were measured. We showed that the visual cortex is robustly activated in correlation with each microsaccade. Even in complete darkness some residual activity is present. This study suggests that many other fMRI studies are potentially impacted by microsaccades and that these small eye movements should be seriously examined when designing fMRI experiments. The study was completed and presented in two conferences: in the Visual Science Society (VSS) and in the Israeli Society For Neuroscience (ISFN). A manuscript is currently under preparation. (see Fig 1)
2. Microsaccades and attention. We examined how microsaccades performed during passive fixation (such in the previous section) correlate with fluctuation of spatial attention. Using an innovative methodology we showed that when the direction of microsaccade match the direction of a visual target, performance is better than when they are opposite each other. This study was presented in 3 conferences: Society for Neuroscience (SFN), Cognitive Neuroscience Society (CNS) and Israeli Society for Neuroscience (ISFN). The manuscript was submitted for review (see Fig 2). Another study focusing on microsaccades and eccentricity of visual stimulation is under preparation.
CFS and visual gain: Last, we conducted a study trying to understand the physiological correlates of CFS. We claimed that CFS works by making visual neurons less responsive (reducing the gain). This study was published in the Journal of Neuroscience (Yuval-Greenberg and Heeger, 2013).
Saccadic suppression: Our research on microsaccade and visual perception has led us to study the interesting phenomenon of “saccadic suppression”: during a saccade visual input is blanked out. This probably happens to prevent a blur or perception of motion when the eye moves. In collaboration with Dr. Elisha Merriam, we are currently conducting an fMRI work on saccadic suppression with microsaccades. Recent work, which started in the Hebrew University during the third “reintegration” year of this grant, is a project on saccadic suppression and EEG recordings. Both works are still in progress.
This was a basic-science study. As such, it made a significant contribution to our current understanding of visual processing. Our research opened up new perspectives on the study of eye movements and perception and of unaware processing in the human brain. In addition, the project has advanced the cooperation and transfer of knowledge between the Israeli researchers and research groups in NYU. We are now submitting follow-up grant proposal to ensure the continuation of this collaboration. As a training program, it was also very successful, as evident by two job offers received by the junior scientist, Dr. Yuval-Greenberg upon the return to Israel, and based on the products of this fellowship. Dr. Yuval-Greenberg has recently started working as a senior lecturer at Tel Aviv University.
In this project we aimed at studying a specific method called “continuous flash suppression” (CFS), which is widely used to render visual stimuli invisible. CFS takes advantage of the natural competition between the two eyes, called binocular rivalry. The visual presentation includes rapid flashes of salient texture to one eye and a low-contrast vague stimulus to the other eye. The perceptual experience will include only the flashes, while the visual stimulus will remain invisible. From the subject’s point of view, the effect is quite dramatic: a stimulus which is clearly visible without the flashes “disappears” completely when the flashes are turned on. CFS is increasingly used in recent years and was used by dozens of studies on unaware perception.
The original purpose of this research was to study object recognition in conditions when the object is unaware. The plan was to present subjects with pictures of objects that are either meaningful (tools, animals, body parts, food items etc) or meaningless (scrambled versions of the same pictures). These pictures will be presented when they are either rendered invisible by CFS (presented to the eye opposite the flashes) or visible (when presented to the same eye as the flashes). This experiment was planned as a functional MRI (fMRI) experiment, where brain activity can be monitored while subjects observe the pictures. In a similar version of the same experiment, familiar and unfamiliar letters (of a known/unknown alphabet) replaced the meaningful and meaningless objects.
In my host lab at NYU, I conducted pilot experimentation with these two procedures. Functional MRI was scanned in 4 subjects when they observed either meaningful.meaningless objects or familiar/unfamiliar letters in conditions where the stimuli were either visible or invisible (with CFS). Our results showed no visual activity at all when objects were invisible, and consequently no familiarity/meaning effect in that condition. In addition, in the visible condition (without CFS) we found activity in early visual cortex (V1) which supposedly suggested that this area, known to be modulated by low physical properties of the stimulus, was actually responding to object identify. Both findings did not fit our original hypothesis and required further investigation. We then raised two new hypotheses which led us to two new research directions: First, we hypothesized that the “object-related” activity we see in V1 is due to small miniature eye movements called “microsaccades”. These involuntary eye movements occur frequently when subjects attempt to keep steady fixation. Microsaccades were shown in the past to be modulated by object identity (be more frequent when object is recognizable). It is possible that their involvement causes the V1 modulation we saw in the visible case. Second, we hypothesized that, contrary to common views on CFS, this manipulation reduces responses in V1 to a level that is almost equivalent to complete stimulus absence. We hypothesized that CFS controls visual gain and not awareness as was previously suggested. We initiated four new directions of research, which luckily turned out to be successful:
1. Microsaccades in fMRI: We initiated a research program on microsaccades. Following our preliminary fMRI results we intended to examine how microsaccades are manifested in fMRI of the visual cortex. We conducted a study where subjects either viewed a stimulus or fixated in complete darkness, while their eye movements and brain activity were measured. We showed that the visual cortex is robustly activated in correlation with each microsaccade. Even in complete darkness some residual activity is present. This study suggests that many other fMRI studies are potentially impacted by microsaccades and that these small eye movements should be seriously examined when designing fMRI experiments. The study was completed and presented in two conferences: in the Visual Science Society (VSS) and in the Israeli Society For Neuroscience (ISFN). A manuscript is currently under preparation. (see Fig 1)
2. Microsaccades and attention. We examined how microsaccades performed during passive fixation (such in the previous section) correlate with fluctuation of spatial attention. Using an innovative methodology we showed that when the direction of microsaccade match the direction of a visual target, performance is better than when they are opposite each other. This study was presented in 3 conferences: Society for Neuroscience (SFN), Cognitive Neuroscience Society (CNS) and Israeli Society for Neuroscience (ISFN). The manuscript was submitted for review (see Fig 2). Another study focusing on microsaccades and eccentricity of visual stimulation is under preparation.
CFS and visual gain: Last, we conducted a study trying to understand the physiological correlates of CFS. We claimed that CFS works by making visual neurons less responsive (reducing the gain). This study was published in the Journal of Neuroscience (Yuval-Greenberg and Heeger, 2013).
Saccadic suppression: Our research on microsaccade and visual perception has led us to study the interesting phenomenon of “saccadic suppression”: during a saccade visual input is blanked out. This probably happens to prevent a blur or perception of motion when the eye moves. In collaboration with Dr. Elisha Merriam, we are currently conducting an fMRI work on saccadic suppression with microsaccades. Recent work, which started in the Hebrew University during the third “reintegration” year of this grant, is a project on saccadic suppression and EEG recordings. Both works are still in progress.
This was a basic-science study. As such, it made a significant contribution to our current understanding of visual processing. Our research opened up new perspectives on the study of eye movements and perception and of unaware processing in the human brain. In addition, the project has advanced the cooperation and transfer of knowledge between the Israeli researchers and research groups in NYU. We are now submitting follow-up grant proposal to ensure the continuation of this collaboration. As a training program, it was also very successful, as evident by two job offers received by the junior scientist, Dr. Yuval-Greenberg upon the return to Israel, and based on the products of this fellowship. Dr. Yuval-Greenberg has recently started working as a senior lecturer at Tel Aviv University.