The limits of color awareness during active, real-world vision
Edited by Dale Purves, Duke University, Durham, NC, and approved April 21, 2020 (received for review December 18, 2019)
Significance
Color is a foundational aspect of visual experience that aids in segmenting objects, identifying food sources, and signaling emotions. Intuitively, it feels that we are immersed in a colorful world that extends to the farthest limits of our periphery. How accurate is our intuition? Here, we used gaze-contingent rendering in immersive VR to reveal the limits of color awareness during naturalistic viewing. Observers explored 360° real-world environments, which we altered so that only the regions where observers looked were in color, while their periphery was black-and-white. Overall, we found that observers routinely failed to notice when color vanished from the majority of their visual world. These results show that our intuitive sense of a rich, colorful world is largely incorrect.
Abstract
Color ignites visual experience, imbuing the world with meaning, emotion, and richness. As soon as an observer opens their eyes, they have the immediate impression of a rich, colorful experience that encompasses their entire visual world. Here, we show that this impression is surprisingly inaccurate. We used head-mounted virtual reality (VR) to place observers in immersive, dynamic real-world environments, which they naturally explored via saccades and head turns. Meanwhile, we monitored their gaze with in-headset eye tracking and then systematically altered the visual environments such that only the parts of the scene they were looking at were presented in color and the rest of the scene (i.e., the visual periphery) was entirely desaturated. We found that observers were often completely unaware of these drastic alterations to their visual world. In the most extreme case, almost a third of observers failed to notice when less than 5% of the visual display was presented in color. This limitation on perceptual awareness could not be explained by retinal neuroanatomy or previous studies of peripheral visual processing using more traditional psychophysical approaches. In a second study, we measured color detection thresholds using a staircase procedure while a set of observers intentionally attended to the periphery. Still, we found that observers were unaware when a large portion of their field of view was desaturated. Together, these results show that during active, naturalistic viewing conditions, our intuitive sense of a rich, colorful visual world is largely incorrect.
Sign up for PNAS alerts.
Get alerts for new articles, or get an alert when an article is cited.
Color is a fundamental ingredient of perceptual experience. As we explore the world around us, we have the impression of a rich tapestry of color, enveloping visual experience and extending all of the way into our visual periphery. But how accurate is this intuition? This question sits at the forefront of a classic controversy in neuroscience and psychology. On the one hand, numerous empirical studies using change blindness (1, 2), inattentional blindness (3, 4), and visual crowding (5, 6) have demonstrated that observers’ awareness is much more limited than they think, particularly in the visual periphery, and our impression of a detailed visual world may simply be incorrect (7–9). On the other hand, an alternative view holds that perceptual awareness is rich, but observers are simply limited in how much information they can attend to, remember, and ultimately report (10–12). Color resides at the center of this controversy, with considerable debate surrounding the question of how much color observers perceive in the world (13–18).
Resolving this debate has proven difficult, as it has historically not been possible to probe perceptual awareness during active, real-world viewing conditions. Classic studies of peripheral color perception involve asking a head-restricted observer to discern the color of an isolated visual target in their periphery (19–21). Under these conditions, color sensitivity falls off with eccentricity for small targets (21–23), but is nearly as good in the periphery as in the fovea when the size of peripheral stimuli are scaled to account for the cortical magnification factor (15, 18, 21, 24–26). These studies provide important baseline measurements of color sensitivity under ideal visual conditions. However, it is unclear how the results would translate to natural visual experience. In contrast to classic studies of peripheral color perception, where attention is covertly directed away from fixation to the peripheral target location, spatial attention during real-world, active vision is generally tightly linked to the location of the current and upcoming fixations, likely placing critical restrictions on peripheral awareness (27, 28). Thus, on the one hand, we might expect peripheral color awareness to precipitously decline in real-world environments. On the other hand, real-world environments are complex and contain statistical regularities that aid in color perception (29–32). As a result, we might expect peripheral color awareness to be rich during naturalistic vision, as consistent with our intuitive sense of a vibrant, colorful visual field. Despite the lack of experimental evidence in favor of either side of this debate, it is often claimed that the intuitive sense of a rich experience of color may be misguided (7, 16, 17). Thus, the question remains, how colorful is perceptual experience?
Here, we exploited recent developments in head-mounted virtual reality (VR) to determine how much color observers perceive during naturalistic visual experience. We immersed observers in a variety of real-world, dynamic, audiovisual environments (e.g., a symphony rehearsal, a tour guide describing a historic site, a dance group performing in the streets, etc.). These 360° environments fully encompassed observers, allowing them to freely explore their surroundings in any direction they chose via saccades and head turns (Fig. 1). Critically, we gave observers no task; we simply encouraged them to explore and learn about their surroundings in whatever way felt natural and normal to them (Materials and Methods). Giving observers no task was an integral part of our design, as we wanted observers to deploy their attention in these virtual environments in the same manner as they would deploy their attention in the real world. While observers explored their surroundings in this manner, we used custom, in-headset eye tracking to monitor observers’ gaze while they explored their surroundings. This method allowed us to know exactly where they were fixating in the scene at any given moment within their environment. Then, to measure the richness of color experience, we slowly altered scenes in a gaze-contingent fashion so that only the parts of the environment where observers were directing their gaze were presented in color. The rest of the scene (i.e., the periphery) was entirely desaturated. This allowed us to ask: how often would observers notice the absence of color in their visual periphery? And over what spatial extent would this absence of awareness hold?
Fig. 1.
Movie S1.
Overall, we found that a surprisingly large number of observers were completely oblivious when most of their visual world lacked any color whatsoever. In the most extreme case, close to a third of observers failed to notice when only a circular aperture with a radius of 10° of visual angle was in color. For perspective, in real-world viewing conditions, 10° of visual angle corresponds to less than 5% of the area of the entire visual field. It should be stressed, however, that the spatial extent of color awareness was not a fixed limit, but rather was strongly influenced by attention: participants noticed the desaturations in the periphery more easily when attention was deployed toward the periphery. These results indicate that peripheral color awareness during naturalistic viewing is much less accurate than is generally believed.
Results
Experiment 1: Color Awareness in a Free-Viewing Paradigm.
In experiment 1, we sought to measure the spatial extent of peripheral color awareness during naturalistic viewing. On each trial, observers (n = 160, 40 per condition) were immersed in dynamic, real-world audiovisual environments using a head-mounted display (Fig. 1, SI Appendix, Fig. S1, and Movie S1). Observers were instructed to explore their environments in whatever way felt natural to them, as if they were merely “looking around” the environment in real life.
Each observer participated in five trials (20 s each): four lead-up trials and one “critical trial” (Materials and Methods). On the critical trial, we modified the color of the visual environment in a gaze-contingent fashion (Fig. 1C and Movie S1). Specifically, for the first 7 s, the trial was fully saturated. Then, over a period of 5 s, the visual periphery was slowly desaturated using gaze-contingent rendering; only the portion of the environment where observers were looking (i.e., the observer’s central visual field) was rendered in color (Figs. 1C and 2A). Gaze-contingent displays such as these have been previously used in a variety of visual experiments, specifically in the study of reading (33). After this gradual desaturation process, the periphery remained rendered entirely in grayscale for the remaining 8 s of the trial while the observer’s fixation position was dynamically tracked.
Fig. 2.
For each observer, the portion of the central visual field that was displayed in color during the critical trial had a radius of either 10°, 17.5°, 25°, or 32.5° of visual angle (Fig. 2). To put these numbers in perspective, 1° of visual angle is approximately the size of 1 cm at a distance of 57 cm/1.87 ft, which is roughly arm’s length. Thus, at arm’s length, the circular apertures had an estimated radius of ∼10, 17.5, 25, or 32.5 cm. After the critical trial ended, the observer was immediately asked a series of questions to determine if they noticed the lack of color in their periphery on the previous trial (Materials and Methods). How often will observers fail to realize that large portions of their visual world were completely devoid of color for several seconds?
In experiment 1, we found that observers were often unaware when most of their visual world lacked any color whatsoever. In the most extreme case, close to a third of observers failed to notice when only a circular aperture with a radius of 10° of visual angle around gaze center was in color (Fig. 2). As a reminder, 10° of visual angle roughly corresponds to a circle that has a radius of 10 cm at approximately arm’s length and corresponds to less than 5% of the entire visual field. The percentage of observers who failed to notice this effect scaled with the portion of the visual field that was saturated (χ2 = 20.31, P < 0.001; Fig. 2B and see SI Appendix, Tables S1 and S2 for more detailed analyses on observer responses and analyses for each individual videosphere). With our largest aperture, which had a radius of 32.5° of visual angle, the overwhelming majority of observers failed to notice the changes to the color of the environment. Importantly, these results were obtained when observers were naturally engaging with a scene and were unaware of the study’s main objectives. In contrast, after the critical trial was over and each observer was shown what had just transpired in the periphery, every single observer reported that they could clearly see the desaturated periphery and were astonished that they had not noticed it before. This shows that the results of experiment 1 represent the spatial limits of color awareness during naturalistic vision, not a hard-wired limitation on visual processing (e.g., the distribution of cone cells on the retina) (34).
Experiment 2: Color when Attending to the Periphery.
Experiment 1 shows that during naturalistic viewing conditions, observers are aware of surprisingly little color in their visual periphery. These findings raise a natural question: when observers are directing their attention toward the periphery and they are expecting changes to occur, what is the threshold at which they can detect the desaturation of their periphery (35)?
To answer this question, in experiment 2, we asked observers (n = 20) to widely distribute their attention and intentionally look for desaturations in the periphery. We then used a staircase procedure to find the spatial threshold at which observers could no longer reliably discern changes to color in their periphery above chance. Specifically, on each trial of experiment 2, the outer limit of the gaze-contingent effect was varied (1-up, 1-down staircase) to determine the spatial threshold at which a participant could detect peripheral color with 50% accuracy (Materials and Methods). After two initial practice trials, in which observers were familiarized with the effect, the staircase began. Each test trial lasted 20 s and always had a brief period in which the periphery was rendered in black-and-white (Materials and Methods). At a random timepoint between 2 and 8 s into the trial, the peripheral color was gradually desaturated over a 4-s interval, fully absent for 1 s, and gradually restored again over a 2-s interval via gaze-contingent stimulus rendering. Observers were instructed to press a button as soon as they noticed a desaturation period. A desaturation detection was only registered as accurate if the observer pressed the button within a specific time window, and test trials were interspersed with catch trials on which the effect did not occur so that the effect was not predictable (Materials and Methods).
The results from the staircase procedure are shown in Fig. 3. Overall, we found that observers reliably failed to detect desaturations beyond ∼37.5° visual angle into the periphery (t [1, 19] = 5.22, P < 0.001, difference from ceiling 45°). On average, desaturations were detected 4.67 s (+/−0.36 SD) after desaturation onset, before resaturation commenced. As further discussed below, 37.5° does not correspond to known limitations imposed by retinal or neuroanatomy. Together with experiment 1, these results suggest a large effect of attention on the spatial extent of peripheral color awareness. However, it should also be stressed that a circular region comprising 37.5° of visual angle occupies less than a third of the visual world in everyday viewing. Thus, even when actively looking for alterations to the color in their periphery, observers can often remain unaware of a lack of color in a majority of the visual world.
Fig. 3.
Discussion
Here, we used gaze-contingent rendering in immersive VR to reveal the limits of color awareness during real-world vision. When observers were naturally engaged with a visual scene, they routinely failed to notice the disappearance of color from most of their visual world. Observers’ lack of awareness was heavily influenced by attention: when intentionally looking for desaturations in the periphery, observers were far more sensitive to a lack of color in their visual environment. However, even when attending to the periphery, they still failed to detect the absence of color across large portions of the visual world. Together, these results demonstrate a surprising lack of awareness of peripheral color in everyday life and raise a number of important questions.
First, how do these findings relate to the neurobiology of the retina? It is well established that the vast majority of color-sensitive cone cells are concentrated around the fovea, with considerably fewer cone cells in the far periphery (34). However, the implications of retinal organization for peripheral color perception are thought to be relatively minor for several reasons. Contrary to popular belief, the density of cone cells in the retinal periphery is quite significant, with roughly 4,000 cone cells per mm2 (36). Moreover, although color sensitivity falls off with eccentricity for small targets (21–23), color perception in the periphery is nearly as good as in the fovea when the size of peripheral stimuli is scaled to account for the cortical magnification factor (15, 18, 21, 24–26). Consistent with these findings, in experiment 1, once we directed observers’ attention toward the periphery to show them the critical manipulations on the last trial, not a single observer had difficulty differentiating between when the scenes were fully colored and when they were partially desaturated. Thus, our findings cannot simply be attributed to limitations placed on visual performance by retinal organization.
Second, why did our observers consistently fail to notice that their periphery was desaturated if it is possible to detect color in the periphery using standard psychophysical approaches? We hypothesize that this apparent discrepancy likely arises from the way that observers engaged with their surroundings in the active, real-world viewing conditions employed in our study. In traditional psychophysical paradigms, attention is artificially separated from fixation, covertly deployed to the peripheral target location. In contrast, during real-world viewing conditions, where participants make saccades ∼3–4 times per second, spatial attention is thought to tightly track the current and upcoming fixation location, likely restricting peripheral awareness (27, 28). Thus, during active vision, attention is likely deployed in a drastically different manner relative to standard psychophysical procedures (37). Further, our study sought to test the limits of color perception in real-world environments, which are considerably more complex than standard psychophysical displays. It is very likely that our results stem, at least to some degree, from the crowding inherently imposed by natural, cluttered scenes (5, 6), despite the statistical regularities in natural images that provide information about spatial structure, spatial frequencies, and, critically, color distributions (29–32). Thus, we argue that our results are not inconsistent with prior studies of peripheral color sensitivity: the results we report here do not represent the limit of color sensitivity in the peripheral visual field measured using psychophysical approaches, but rather the limit of perceptual awareness during active vision in real-world scenes.
Third, do our findings highlight the limits of perceptual awareness, memory, or accessibility? In other words, is it possible that observers are aware of the color desaturation in their periphery during the trial, but simply forget about it by the time they are probed [i.e., inattentional amnesia (38)]? Alternatively, perhaps observers are aware of the peripheral desaturation during the trial, but their awareness is not accessible to other mechanisms like attention or working memory at the time of the probe [i.e., inattentional inaccessibility (13)]. Overall, empirically differentiating between inattentional blindness, amnesia, and inaccessibility may be logically impossible in a paradigm such as ours, where awareness is assessed based on information observers have access to and can report ex post facto (13, 17). However, it would be surprising if observers could be aware of the sustained, striking reduction of color from over 95% of their visual world, and then forget it immediately afterward. One reason why this would be surprising is that in previous studies that were designed to adjudicate between inattentional blindness and amnesia accounts of failures of visual awareness in computer-based paradigms have consistently found evidence in favor of inattentional blindness (39–41). Thus, while we recognize the logical difficulty of distinguishing between these possibilities, we favor the interpretation of our findings as a failure of perceptual awareness, rather than memory, in naturalistic environments.
If color perception in the real world is indeed as sparse as our findings suggest, the final question to consider is how this can be. Why does it intuitively feel like we see so much color when our data suggest we see so little? While we cannot offer a definitive answer, several possibilities can be explored in future research. One possibility is that as observers spend time in an environment, their brains are able to eventually “fill-in” the color of many items in the periphery (42, 43). Of course, providing direct evidence for this explanation is challenging since it is extremely difficult to differentiate between scenarios where a subject knows the color of an object (i.e., “I know the tree behind me is green even though I currently cannot see the color green”) from instances where the subject is experiencing the color of that object online (i.e., “I can see the color green at this very moment”). Alternatively, some would argue that there is no need for a filling-in mechanism at all and the intuition of a rich perceptual experience is simply misguided (44, 45). Going forward, future research will need to focus on understanding the apparent discrepancy between our subjective impressions of the visual world and aligning those impressions with results like the ones reported here.
Last, the results reported here naturally raise a variety of follow-up questions that future research could explore. For example, what are the limits of the effects reported here with respect to color? Would observers fail to notice other alterations to peripheral color beyond desaturation, such as reversals in color space (46)? Beyond color, would these findings extend to alterations of other features of the visual periphery [e.g., contrast, spatial frequency (47)]? For each of these cases, how do the effect sizes vary as a function of task (e.g., if an observer is passively looking around a scene vs. actively searching for a target item such as their keys) (37)? How might these effects vary in populations where the scope of spatial attention is thought to be altered (48)? All in all, the current results reveal a fundamental limitation on peripheral color perception in naturalistic vision and lay the foundation for future studies of naturalistic perceptual experience.
Materials and Methods
Participants.
One hundred and seventy-eight adult participants were recruited for our experiments (experiment 1 [n = 178]: 108 females; mean age 20.84 +/− 4.17 SD years; experiment 2 [n = 26]: 17 females; mean age 21.96 +/− 4.37 SD years). Participants were recruited based on having 1) no neurological or psychiatric conditions, 2) no history of epilepsy, and 3) normal or contact-corrected vision and no colorblindness, later confirmed using the Ishihara color test (49). Written consent was obtained from all participants in accordance with a protocol approved by the Dartmouth College (experiment 1, n = 138; experiment 2, n = 26) and Amherst College (experiment 1, n = 40) Institutional Review Boards. All testing took place on the Dartmouth College and Amherst College campuses.
Stimuli and Stimulus Presentation.
Stimuli consisted of 360° panoramic “videospheres” of real-world scenes. Videospheres depicted a diverse set of indoor and outdoor settings (e.g., a roller-rink, a construction site), with content including people, objects, and scenery. Each videosphere contained an audio track (narration, conversation, and ambient sound), presenting the viewer with a multisensory experience. For examples, please see SI Appendix, Fig. S1 and Movie S1
Each videosphere was later applied to a VR environment built in Unity (http://www.unity3d.com/), displayed using a head-mounted display (HMD) (Oculus Rift, Development Kit 2, https://www.oculus.com). The experiment was programmed using custom scripts written in C#. The Oculus Rift display (low-persistence OLED screen, resolution: 960 × 1080 per eye; field-of-view: ∼100°; 75-Hz refresh) was used for all of the experimental conditions described below. Participants stood and wore the head-mounted VR display and headphones. This enabled participants to actively explore the 360° environment naturally via eye-movements and head turns, providing a self-directed opportunity to explore the naturalistic real-world environment from an egocentric perspective.
Eye-Tracking Technical Specifications.
Binocular in-headset eye-trackers continuously monitored participants’ gaze position during scene viewing (Pupil Labs, 120-Hz sampling frequency, 0.6 visual degrees accuracy, 0.08 visual degrees precision, 5.7-ms camera latency, 3-ms processing latency). During calibration phases, a 33-point calibration routine was performed at three eccentricities from screen center: 5.2, 16.6, and 25.3° visual angle.
General Procedures: Experimental Phases.
At the start of each study, eye-to-screen distance was standardized for each participant based on a head-fixed visual field calibration screen to ensure that the screen occupied ∼90° visual angle horizontally and 100° visual angle vertically for all participants. Subsequently, subjects participated in three experimental phases: Practice, Calibration, and Experimental Trials.
During the practice phase, participants were shown two practice trials (20 s) to adjust to the experience of VR before the experiment started, to mitigate any potential effects of distraction due to inexperience. Eye-tracking confidence was monitored during the practice trials to ensure high-quality pupil detection. If the average confidence of pupil detection dropped below 75% for either eye during either of the practice trials, the headset was readjusted and participants were required to repeat the set of practice trials. If after three attempts of practice trials the eye-trackers could not confidently detect participants’ gaze, the study session was ended.
Next, during the calibration phase, eye-tracking accuracy was validated using a 33-point calibration routine. In the event of an unsuccessful calibration, participants repeated the calibration routine up to 2 times. If successful calibration could still not be achieved, the experiment ended, and the participant did not proceed any further.
On each experimental trial, participants were presented with a videosphere via the HMD and were instructed to stand and “fully and naturally explore each scene” using saccades and head-turns. The yaw of each videosphere was randomly rotated for each scene to prevent potential central content tendencies resulting from photographer bias.
After each trial, participants returned to a virtual home screen (a neutral platform surrounded by clouds), where they were instructed to take a break. To confirm accurate eye-tracking before the next trial, participants were presented with a pretrial fixation target at screen center. Participants were instructed to align their head-center with and fixate on the target so that gaze drift could be assessed. Participants were automatically advanced into the trial if gaze drift did not exceed 3° visual angle over a 5-s interval. Otherwise, a recalibration routine was performed. If more than two calibration routines were required during the experimental phase, the experiment ended, and that participant did not proceed any further.
In experiment 1, 18 participants were excluded from subsequent analyses due to poor eye-tracking quality (n = 17) or color blindness (n = 1). Demographic information was not collected for 4 individuals. In experiment 2, 6 participants were excluded from subsequent analyses due to poor catch trial performance.
General Procedures: Real-Time Gaze-Contingent Elimination of Peripheral Color.
Gaze position was continuously monitored throughout all trials using the Pupil Labs’ Capture software version 1.9.7 using procedures detailed in ref. 37. Custom presentation scripts were written in C# for Unity to record gaze position, accomplish gaze-contingent rendering, and establish trial structure. This pipeline enabled us to desaturate participants’ visual periphery in a gaze-contingent fashion. Specifically, a grayscale filter was applied to the region of the visual scene that was presented in the participant’s visual periphery at any given moment, based on the last recorded gaze coordinate. This coordinate continuously changed as participants looked around the visual environment. In this filter, color values located more than 4° visual angle from gaze center were linearly desaturated to grayscale as a function of a distance specified in visual degrees (specified by the annulus size on the given trial, radius 10°, 17.5°, 25°, or 32.5° visual angle).
Procedure—Experiment 1.
Experiment 1 assessed the spatial limit of peripheral color awareness in naive participants during a naturalistic, free-viewing paradigm (n = 160, 40 per condition). Before entering the headset, participants were instructed to naturally explore their environment on each trial. Critically, we gave observers no task; we simply encouraged them to explore and learn about their surroundings in whatever way felt natural and normal to them, “as if they were placed in that situation in real life.” Giving observers no task was an integral part of our design, as we wanted observers to deploy their attention in these virtual environments in the same manner as they would deploy their attention in the real world. In order to ensure that observers engaged with the scene, we informed them that questions would be asked about their experience in the scenes at the end of the study.
Following practice trials and eye-tracker calibration, participants completed five experimental trials (20 s each): four lead-up trials and one critical trial in which the participants were presented with the gaze-contingent manipulation. If a calibration routine was required due to gaze drift, trials were added to ensure that four uninterrupted lead-up trials preceded the critical trial.
On each lead-up trial, a participant was shown a novel, full-colored scene for the entire trial duration (20 s). The videosphere used in each critical trial was drawn from a bank of 40 possible videospheres.
On the critical trial, participants were shown a novel, full-colored scene for 7 s, after which peripheral color was gradually removed from the scene over a 5-s interval via gaze-contingent stimulus rendering. Ultimately all color in the periphery was removed beyond an outer limit of either: 10°, 17.5°, 25°, or 32.5° visual angle. The central visual field always remained in full color (radius: 4° visual angle from gaze center), and color was linearly desaturated between that radius and the outer limit. Participants remained in the scene with a gaze-contingent, grayscale periphery for the remaining 8 s as they continued to explore the environment.
Forty individuals participated in each condition (outer limit size), and each participant in a condition saw a unique critical trial scene.
Following the critical trial, the observer was returned to a virtual “home screen,” and the experimenter immediately asked the participant a series of questions to assess awareness of the absence of peripheral color. The series of questions started out vague and progressively became more specific:
1)
“Did you notice anything strange or different about that last trial?”
2)
“If I were to tell you that we did something odd on the last trial, would you have a guess as to what we did?”
3)
“If I were to tell you we did something different in the second half of the last trial, would you have a guess as to what we did?”
4)
“Did you notice anything different about the colors in the last scene?”
Each observer’s response was scored depending on if/when they were able to say anything indicating that they were aware of the desaturated periphery. Regardless of their score, the experiment always ended by returning observers back to the same virtual environment they had just viewed on the critical trial. This time, however, we toggled back and forth between a saturated and desaturated periphery while directly explaining to participants what had happened on the last trial (“Notice that wherever you look is in color, but the periphery is black and white. Do you see this now? Did you notice this before?”). There were two main reasons for showing observers exactly what had just transpired. First, it enabled us to get explicit confirmation from observers that they had or had not noticed the changes we made to the color of the display. Second, it enabled us to confirm that observers could notice the color changes when their attention was directed to the periphery, which was important in interpreting whether our results were due to attention or some other hard-wired property of the visual system. Only those participants who claimed not to notice the critical manipulation after having both 1) answered “no” to all questions, and 2) confirmed having not previously noticed the effect after having been made aware by the experimenter were classified as being completely unaware of the peripheral color manipulation.
Procedure—Experiment 2.
Experiment 2 measured the spatial limit of peripheral color awareness during naturalistic viewing in participants whose attention was directed to the periphery using a staircase procedure. On each trial of the staircase, the outer limit of the gaze-contingent effect was varied (1-up, 1-down) to determine the spatial threshold at which a participant could detect peripheral color with 50% accuracy. Participants were explicitly instructed to detect whenever color disappeared in their periphery. All participants had participated in experiment 1, and therefore had experience with the gaze-contingent manipulation.
As in experiment 1, participants were instructed to naturally explore each environment. In addition, participants were asked to press a button whenever they detected that color was absent in their periphery. Before the experiment began, participants were given two practice trials, during which the effect was present (annulus radius: 17.5° visual angle). All participants detected the effect and stated feeling confident in being able to perform the task.
Following eye-tracker calibration and practice trials, subjects participated in up to 66 experimental trials (up to 15 s each). On each critical trial, a participant was presented with a full-colored real-world scene. Given that the critical scenes were drawn from the same stimulus bank as used in experiment 1 (SI Appendix, Fig. S1), one of these scenes would have been presented as the critical trial in experiment 1; the rest were novel. At a random timepoint between 2 and 8 s, peripheral color was gradually desaturated over a 4-s interval, was fully absent for 1 s, and gradually restored again over a 2-s interval via gaze-contingent stimulus rendering. Participants were instructed to press a button whenever they detected a desaturation of the periphery. Responses were counted as correct if they occurred during an 8-s window (effect onset to 1 s after effect offset). All experimental trials ended 1 s after a button-press or at 15 s.
To monitor false alarms, critical trials were randomly interspersed with catch trials (40% of trials), which did not contain the gaze-contingent manipulation, and ended at a random timepoint between 7 and 15 s (some catch trials were shorter than the shortest critical trial). Participants who responded on >10% of catch trials were removed from the study (n = 6). Performance on catch trials did not affect the staircase procedure, as each step adjustment was determined based on critical trial performance alone.
The staircase procedure worked in the following way. On the first trial, the outer limit of the gaze-contingent effect was set to the maximum horizontal visual angle displayed in the headset (45° visual angle radius). The outer limit was varied by a step-size of 5° visual angle for the first five reversals, after which a step-size of 2° visual angle was used for the remaining portion of the experiment. The staircase was terminated after 12 reversals, or after completion of all experimental trials, and the threshold was defined as the average of the last six staircase reversals.
Data and Materials Availability.
The datasets generated during this study are available at https://osf.io/uv8y9/ (50).
Acknowledgments
This study was supported by a National Science Foundation Collaborative Research Award (#1829470) (to M.A.C.), a grant from the Nancy Lurie Marks Family Foundation (to C.E.R.), and the donation of a Titan V GPU by the NVIDIA Corporation (to C.E.R.). We thank Kayla Hall and Ilyas Tezekbaev for assistance with data collection. We thank Chris Baker, Patrick Cavanagh, and Nancy Kanwisher for insightful comments on the manuscript. We also thank Jeff Mentch, Tim Brady, Sarah Cormiea, Daniel Dennett, Michael Pitts, Ken Nakayama, Brad Duchaine, Adam Steel, and A.J. Haskins for helpful discussion about the project.
Supporting Information
Appendix (PDF)
- Download
- 1.25 MB
Movie S1.
- Download
- 48.46 MB
References
1
J. K. O’Regan, R. A. Rensink, J. J. Clark, Change-blindness as a result of “mudsplashes”. Nature 398, 34 (1999).
2
R. A. Rensink, J. K. O’Regan, J. J. Clark, To see or not to see: The need for attention to perceive changes in scenes. Psychol. Sci. 8, 368–373 (1997).
3
U. Neisser, R. Becklen, Selective looking: Attending to visually specified events. Cognit. Psychol. 7, 480–494 (1975).
4
A. Mack, I. Rock, Inattentional Blindness: Perception without Attention, (MIT Press, Cambridge, MA, 1998).
5
D. Whitney, D. M. Levi, Visual crowding: A fundamental limit on conscious perception and object recognition. Trends Cogn. Sci. 15, 160–168 (2011).
6
J. Freeman, E. P. Simoncelli, Metamers of the ventral stream. Nat. Neurosci. 14, 1195–1201 (2011).
7
S. Dehaene, Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts, (Penguin, 2014).
8
M. A. Cohen, D. C. Dennett, N. Kanwisher, What is the bandwidth of perceptual experience? Trends Cogn. Sci. 20, 324–335 (2016).
9
S. Rajananda, M. A. K. Peters, H. Lau, B. Odegaard, Subjective inflation of color saturation in the periphery under temporal overload. bioRxiv:10.1101/227074 (01 December 2017).
10
N. Tsuchiya, M. Wilke, S. Frässle, V. A. F. Lamme, No-report paradigms: Extracting the true neural correlates of consciousness. Trends Cogn. Sci. (Regul. Ed.) 19, 757–770 (2015).
11
C. Koch, M. Massimini, M. Boly, G. Tononi, Neural correlates of consciousness: Progress and problems. Nat. Rev. Neurosci. 17, 307–321 (2016).
12
M. Boly et al., Are the neural correlates of consciousness in the front or in the back of the cerebral cortex? Clinical and neuroimaging evidence. J. Neurosci. 37, 9603–9613 (2017).
13
N. Block, Consciousness, accessibility, and the mesh between psychology and neuroscience. Behav. Brain Sci. 30, 481–548 (2007).
14
V. A. F. Lamme, How neuroscience will change our view on consciousness. Cogn. Neurosci. 1, 204–220 (2010).
15
A. M. Haun, G. Tononi, C. Koch, N. Tsuchiya, Are we underestimating the richness of visual experience? Neurosci. Conscious. 2017, niw023 (2017).
16
H. Lau, D. Rosenthal, Empirical support for higher-order theories of conscious awareness. Trends Cogn. Sci. 15, 365–373 (2011).
17
M. A. Cohen, D. C. Dennett, Consciousness cannot be separated from function. Trends Cogn. Sci. 15, 358–364 (2011).
18
C. W. Tyler, Peripheral color demo. iPerception 6, 2041669515613671 (2015).
19
J. Gordon, Color vision in the peripheral retina. II. Hue and saturation. J. Opt. Soc. Am. 67, 202–207 (1924).
20
M. A. Webster, I. Juricevic, K. C. McDermott, Simulations of adaptation and color appearance in observers with varying spectral sensitivity. Ophthalmic Physiol. Opt. 30, 602–610 (2010).
21
T. Hansen, L. Pracejus, K. R. Gegenfurtner, Color perception in the intermediate periphery of the visual field. J. Vis. 9, 1–12 (2009).
22
C. Noorlander, J. J. Koenderink, R. J. den Ouden, B. W. Edens, Sensitivity to spatiotemporal colour contrast in the peripheral visual field. Vision Res. 23, 1–11 (1983).
23
K. T. Mullen, Colour vision as a post-receptoral specialization of the central visual field. Vision Res. 31, 119–130 (1991).
24
I. Abramov, J. Gordon, Color appearance: On seeing red—Or yellow, or green, or blue. Annu. Rev. Psychol. 45, 451–485 (1994).
25
M. Johnson, R. Massof, “The effect of stimulus size on chromatic thresholds in the peripheral retina” in Colour Vision Deficiencies VI, (Documenta Ophthalmologica Proceedings Series, 1982), Vol. 33, pp. 15–18.
26
M. A. Johnson, Color vision in the peripheral retina. Am. J. Optom. Physiol. Opt. 63, 97–103 (1986).
27
B. Tatler, M. Land, “Everyday visual attention” in The Handbook of Attention, Ed. A. Kingstone, J.M. Fawcett, E.F. Risko, Eds. (MIT Press, Cambridge, 2016).
28
J. M. Findlay, I. D. Gilchrist, “Visual attention: The active vision perspective” in Vision and Attention, (2001), pp. 83–103.
29
W. S. Geisler, Visual perception and the statistical properties of natural scenes. Annu. Rev. Psychol. 59, 167–192 (2008).
30
G. J. Burton, I. R. Moorhead, Color and spatial structure in natural scenes. Appl. Opt. 26, 157–170 (1987).
31
F. Long, Z. Yang, D. Purves, Spectral statistics in natural scenes predict hue, saturation, and brightness. Proc. Natl. Acad. Sci. U.S.A. 103, 6013–6018 (2006).
32
D. Kersten, Predictability and redundancy of natural images. J. Opt. Soc. Am. A 4, 2395–2400 (1987).
33
K. Rayner, The gaze-contingent moving window in reading: Development and review. Vis. Cogn. 22, 242–258 (2014).
34
C. A. Curcio, K. R. Sloan, O. Packer, A. E. Hendrickson, R. E. Kalina, Distribution of cones in human and monkey retina: Individual variability and radial asymmetry. Science 236, 579–582 (1987).
35
D. J. Simons, S. R. Mitroff, “The role of expectations in change detection and attentional capture” in Vision and Attention, (Springer New York, 2001), pp. 189–207.
36
R. Rosenholtz, Capabilities and limitations of peripheral vision. Annu. Rev. Vis. Sci. 2, 437–457 (2016).
37
A. J. Haskins, J. Mentch, T. L. Botch, C. E. Robertson, Active vision in immersive, 360 real-world environments. bioRxiv:10.1101/2020.03.05.976712 (06 March 2020).
38
J. M. Wolfe, “Inattentional amnesia” in Fleeting Memories: Cognition of Brief Visual Stimuli, V. Coltheart, Ed. (Bradford Books series in cognitive psychology, MIT Press, 1999), pp. 71–94.
39
R. Becklen, D. Cervone, Selective looking and the noticing of unexpected events. Mem. Cognit. 11, 601–608 (1983).
40
G. Rees, C. Russell, C. D. Frith, J. Driver, Inattentional blindness versus inattentional amnesia for fixated but ignored words. Science 286, 2504–2507 (1999).
41
E. J. Ward, B. J. Scholl, Inattentional blindness reflects limitations on perception, not memory: Evidence from repeated failures of awareness. Psychon. Bull. Rev. 22, 722–727 (2015).
42
B. Balas, P. Sinha, “Filling-in” colour in natural scenes. Vis. Cogn. 15, 765–778 (2007).
43
H. Komatsu, The neural mechanisms of perceptual filling-in. Nat. Rev. Neurosci. 7, 220–231 (2006).
44
D. C. Dennett, Consciousness Explained, (Little, Brown and Co., 1991).
45
L. C. Loschky, A. Nuthmann, F. C. Fortenbaugh, D. M. Levi, Scene perception from central to peripheral vision. J. Vis. 17, 6 (2017).
46
M. Cohen, J. Rubenstein, How much color do we see in the blink of an eye? PsyArXiv:10.31234/osf.io/enywt (28 July 2019).
47
L. C. Loschky, G. W. McConkie, J. Yang, M. E. Miller, The limits of visual resolution in natural scene viewing. Vis. Cogn. 12, 1057–1092 (2005).
48
C. E. Robertson, D. J. Kravitz, J. Freyberg, S. Baron-Cohen, C. I. Baker, Tunnel vision: Sharper gradient of spatial attention in autism. J. Neurosci. 33, 6776–6781 (2013).
49
J. Clark, The Ishihara test for color blindness. Am. J. Physiol. Opt. 5, 269–276 (1924).
50
M. A. Cohen, T. L. Botch, C. E. Robertson, The limits of color awareness during active, real-world vision. Open Science Framework. https://osf.io/uv8y9. Deposited 11 April 2020.
Information & Authors
Information
Published in
Classifications
Copyright
Copyright © 2020 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).
Submission history
Published online: June 8, 2020
Published in issue: June 16, 2020
Keywords
Acknowledgments
This study was supported by a National Science Foundation Collaborative Research Award (#1829470) (to M.A.C.), a grant from the Nancy Lurie Marks Family Foundation (to C.E.R.), and the donation of a Titan V GPU by the NVIDIA Corporation (to C.E.R.). We thank Kayla Hall and Ilyas Tezekbaev for assistance with data collection. We thank Chris Baker, Patrick Cavanagh, and Nancy Kanwisher for insightful comments on the manuscript. We also thank Jeff Mentch, Tim Brady, Sarah Cormiea, Daniel Dennett, Michael Pitts, Ken Nakayama, Brad Duchaine, Adam Steel, and A.J. Haskins for helpful discussion about the project.
Notes
This article is a PNAS Direct Submission.
Authors
Competing Interests
The authors declare no competing interest.
Metrics & Citations
Metrics
Citation statements
Altmetrics
Citations
Cite this article
117 (24) 13821-13827,
Export the article citation data by selecting a format from the list below and clicking Export.
Cited by
Loading...
View Options
View options
PDF format
Download this article as a PDF file
DOWNLOAD PDFLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Personal login Institutional LoginRecommend to a librarian
Recommend PNAS to a LibrarianPurchase options
Purchase this article to access the full text.