Final Report Summary - ACTPUP (The active pupil: The role of pupil size in active vision)
This is a short summary of our most important findings. Most studies have also been discussed in non-technical blog posts, YouTube videos, and news articles. Relevant links can be found on the home page of Sebastiaan Mathôt: http://www.cogsci.nl/smathot
The pupil is the entry into the eye. It shrinks when we step out into the sun, and expands when we get aroused or exert mental or physical effort. These basic pupillary responses have been recognized at least since Roman times, and have been the focus of systematic investigation since the 18th century. Currently, pupil-size measurement, or pupillometry, is an established tool for diagnosing visual defects, and is widely used as a marker of arousal or effort in psychological experiments and clinical applications.
The aim of our project was to investigate how changes in pupil size reflect our interactions with the environment: what we look at, what we attend to, and even what we think about. During the two years of the project, we have made several key discoveries, which we have summarized in a review article (Mathôt & Van der Stigchel, 2015) and are briefly discussed below.
In a first series of studies, we focused on the role of visual attention in the pupillary light response; that is, we asked whether covertly attending to (from the corner of your eye, without looking at) something bright causes the pupil to constrict, relative to covertly attending to something dark. To test this, we conducted behavioral experiments in which healthy human participants saw a display that was divided into a bright and a dark half; next, their attention was captured toward either the bright or the dark side of the display, while they kept their gaze fixated on a central fixation dot. As we had predicted, the pupil constricted when attention was captured toward the bright, compared to the dark, display (Mathôt et al., 2014). This showed that the pupillary light response does not only reflect the amount of light that enter the eye, but also how this light is processed, in this case whether it is attended or not.
In a subsequent series of studies, we asked whether preparing an eye movement toward something bright or dark would be sufficient to elicit a pupillary light response; that is, we hypothesized that a pupillary constriction would be prepared while an eye movement toward a bright object was being prepared, already before the eyes set in motion. To test this, we conducted behavioral experiments in which participants made an eye movement toward a bright or dark surface. Crucially, when the eyes started to move, we flipped the display around. Therefore, when participants prepared an eye movement toward brightness, their eyes landed on darkness, and vice versa. This allowed us to dissociate the preparatory component of the pupillary light response from the response that results from directly looking at something bright or dark. And indeed, as we had predicted, the preparation of an eye movement toward brightness causes a pupillary constriction, compared to preparation for darkness (Mathôt et al., 2015). This showed that the pupillary light response anticipates our own actions, in this case our eye movements.
In a third series of studies, we asked whether keeping bright or dark stimuli in working memory would elicit a pupillary light response; that is, we hypothesized that merely keeping a bright object in memory would cause the pupil to constrict. To test this, we conducted behavioral experiments in which participants memorized either a set of bright or a set of dark stimuli. Strikingly, we found that the pupil constricted when participants attended to the bright stimuli to encode them into working memory (in line with our previous studies) but, contrary to our prediction, not when the bright stimuli were merely kept in memory and were no longer visible on the display. In other words, keeping bright or dark stimuli in working memory does not trigger a pupillary light response (Blom, Mathôt et al., 2016). At first sight, these findings argue against the close coupling between visual attention and visual working memory that is often proposed; however, we need to conduct follow-up studies to better understand and interpret these findings.
In the final stages of the project, we developed a practical application based on our research: a human-computer interface that allows people to enter text by covertly attending to (without looking at) letters that they want to select. By varying the brightness of letters, and measuring fluctuations in pupil size, the system deduces which letter the participant wants to write (i.e. is covertly attending to). We conducted a preregistered study to test the speed and accuracy of this system, and found that it is comparable to the best covert-attention based brain-computer interfaces to date (Mathôt et al., 2016); however, our system requires only an eye tracker, and is therefore much easier to use than most existing systems.
References
Mathôt, S., Dalmaijer, E., Grainger, J., & Van der Stigchel, S. (2014). The pupillary light response reflects exogenous attention and inhibition of return. Journal of Vision, 14(14), 7. doi:10.1167/14.14.7
Mathôt, S., & Van der Stigchel, S. (2015). New light on the mind’s eye: The pupillary light response as active vision. Current Directions in Psychological Science, 24(5), 374–378. doi:10.1177/0963721415593725
Mathôt, S., van der Linden, L., Grainger, J., & Vitu, F. (2015). The pupillary light response reflects eye-movement preparation. Journal of Experimental Psychology: Human Perception and Performance, 41(1), 28–35. doi:10.1037/a0038653
Blom, T., Mathôt, S., Olivers, C.N.L. & Van der Stigchel, S. (2016). The pupillary light response reflects encoding, but not maintenance, in visual working memory. Journal of Experimental Psychology: Human Perception and Performance. doi:10.1037/xhp0000252
Mathôt, S., Melmi, J.-B van der Linden, S., & Van der Stigchel, S. (2016). The mind-writing pupil: A human-computer interface based on decoding of attention through pupillometry. PLoS ONE, 11(2), e0148805. doi:10.1371/journal.pone.0148805
The pupil is the entry into the eye. It shrinks when we step out into the sun, and expands when we get aroused or exert mental or physical effort. These basic pupillary responses have been recognized at least since Roman times, and have been the focus of systematic investigation since the 18th century. Currently, pupil-size measurement, or pupillometry, is an established tool for diagnosing visual defects, and is widely used as a marker of arousal or effort in psychological experiments and clinical applications.
The aim of our project was to investigate how changes in pupil size reflect our interactions with the environment: what we look at, what we attend to, and even what we think about. During the two years of the project, we have made several key discoveries, which we have summarized in a review article (Mathôt & Van der Stigchel, 2015) and are briefly discussed below.
In a first series of studies, we focused on the role of visual attention in the pupillary light response; that is, we asked whether covertly attending to (from the corner of your eye, without looking at) something bright causes the pupil to constrict, relative to covertly attending to something dark. To test this, we conducted behavioral experiments in which healthy human participants saw a display that was divided into a bright and a dark half; next, their attention was captured toward either the bright or the dark side of the display, while they kept their gaze fixated on a central fixation dot. As we had predicted, the pupil constricted when attention was captured toward the bright, compared to the dark, display (Mathôt et al., 2014). This showed that the pupillary light response does not only reflect the amount of light that enter the eye, but also how this light is processed, in this case whether it is attended or not.
In a subsequent series of studies, we asked whether preparing an eye movement toward something bright or dark would be sufficient to elicit a pupillary light response; that is, we hypothesized that a pupillary constriction would be prepared while an eye movement toward a bright object was being prepared, already before the eyes set in motion. To test this, we conducted behavioral experiments in which participants made an eye movement toward a bright or dark surface. Crucially, when the eyes started to move, we flipped the display around. Therefore, when participants prepared an eye movement toward brightness, their eyes landed on darkness, and vice versa. This allowed us to dissociate the preparatory component of the pupillary light response from the response that results from directly looking at something bright or dark. And indeed, as we had predicted, the preparation of an eye movement toward brightness causes a pupillary constriction, compared to preparation for darkness (Mathôt et al., 2015). This showed that the pupillary light response anticipates our own actions, in this case our eye movements.
In a third series of studies, we asked whether keeping bright or dark stimuli in working memory would elicit a pupillary light response; that is, we hypothesized that merely keeping a bright object in memory would cause the pupil to constrict. To test this, we conducted behavioral experiments in which participants memorized either a set of bright or a set of dark stimuli. Strikingly, we found that the pupil constricted when participants attended to the bright stimuli to encode them into working memory (in line with our previous studies) but, contrary to our prediction, not when the bright stimuli were merely kept in memory and were no longer visible on the display. In other words, keeping bright or dark stimuli in working memory does not trigger a pupillary light response (Blom, Mathôt et al., 2016). At first sight, these findings argue against the close coupling between visual attention and visual working memory that is often proposed; however, we need to conduct follow-up studies to better understand and interpret these findings.
In the final stages of the project, we developed a practical application based on our research: a human-computer interface that allows people to enter text by covertly attending to (without looking at) letters that they want to select. By varying the brightness of letters, and measuring fluctuations in pupil size, the system deduces which letter the participant wants to write (i.e. is covertly attending to). We conducted a preregistered study to test the speed and accuracy of this system, and found that it is comparable to the best covert-attention based brain-computer interfaces to date (Mathôt et al., 2016); however, our system requires only an eye tracker, and is therefore much easier to use than most existing systems.
References
Mathôt, S., Dalmaijer, E., Grainger, J., & Van der Stigchel, S. (2014). The pupillary light response reflects exogenous attention and inhibition of return. Journal of Vision, 14(14), 7. doi:10.1167/14.14.7
Mathôt, S., & Van der Stigchel, S. (2015). New light on the mind’s eye: The pupillary light response as active vision. Current Directions in Psychological Science, 24(5), 374–378. doi:10.1177/0963721415593725
Mathôt, S., van der Linden, L., Grainger, J., & Vitu, F. (2015). The pupillary light response reflects eye-movement preparation. Journal of Experimental Psychology: Human Perception and Performance, 41(1), 28–35. doi:10.1037/a0038653
Blom, T., Mathôt, S., Olivers, C.N.L. & Van der Stigchel, S. (2016). The pupillary light response reflects encoding, but not maintenance, in visual working memory. Journal of Experimental Psychology: Human Perception and Performance. doi:10.1037/xhp0000252
Mathôt, S., Melmi, J.-B van der Linden, S., & Van der Stigchel, S. (2016). The mind-writing pupil: A human-computer interface based on decoding of attention through pupillometry. PLoS ONE, 11(2), e0148805. doi:10.1371/journal.pone.0148805