A different point of hue

Adistinguished line-up of scholars recently got together to stir up discussion about the physiological basis for color and have, with a simple manipulation of decades-old data, challenged one of the fundamental tenets of our current understanding of the neurobiology of color (1).

Understanding color is not easy. Newton made some headway, but his demonstration of color's physical basis provided only limited insight because, as Young (2) pointed out, there simply is not enough space for a receptor for each of the seven million or so perceivable colors at each retinal location. Young argued for a triplet color code, and we now know that such a code exists in the form of the three cone types (Fig. 1).

Young's idea made color a construction of the brain, not a physical attribute, and paved the way for opponent color theory (3) in which color is determined not by trichromacy but by three opponent processes: red-green, blue-yellow, and black-white. This theory gained ground because it accounted for the fact that we are unable to see a continuous mixture of “reddish-greens” and “bluish-yellows,” which should be perceivable if color were simply trichromatic.

In the 1960s, De Valois et al. (4) discovered that many cells in the lateral geniculate nucleus (LGN) (the thalamic relay from the retina to primary visual cortex) show chromatic opponency. LGN cells inherit this property from retinal ganglion cells. Some are excited by red and inhibited by green (R⁺/G^-); others are excited by blue and inhibited by yellow (B⁺/Y^-); others are excited by white and inhibited by black (W⁺/Blk^-). R^-/G⁺, Y⁺/B^-, and Blk⁺/W^- cells also exist. It was natural to suggest that these cells are the basis for opponent colors, and most neuroscientists today accept some version of this view: the three cone types embody trichromatic theory, and the chromatically opponent LGN cells, each receiving specific cone inputs, represent opponent theory, although the purity of the cone inputs is disputed (5–7).

In a recent issue of PNAS, Romney et al. (1) have done two straightforward things to shake up this interpretation. First, they plotted the spectral reflectance from the 1269 Munsell color chips§ in “cone space.” Cone space is often used to show the relative input of the three cone types to neurons, but Romney et al. used it to show how each Munsell color chip would activate the three cone types under standard illumination (Fig. 2A). Cells in the parvocellular layers of the LGN are clustered in cone space: they receive on average balanced opponent input from L versus M cones (8). As Romney et al. point out, the Munsell chips are not randomly distributed when plotted in cone space, but they cluster like LGN cells.

The second thing Romney et al. (1) did was to plot in color space the spectral sensitivity of the 147 LGN cells measured by De Valois et al. (4) (Fig. 2B). Color space uses perceptual coordinates of hue and saturation. Romney et al. treated the spectral sensitivities of the LGN cells as reflectance spectra, enabling a direct comparison with Munsell reflectance spectra. According to Munsell's original aim, the color chips populate the entire color space. Curiously, the LGN data seem to do so, too, despite being clustered in cone space (Fig. 2).

The clustering of LGN cells in cone space has been taken as proof that LGN cells represent the physiological implementation of opponent color theory. Romney et al. (1) ask how we can keep this conclusion if (i) the Munsell color chips also cluster in cone space and (ii) the LGN cells do not cluster when their spectral sensitivities are plotted in perceptual space. Well, they say, we cannot: “The LGN cells are more or less evenly distributed in perceptual space by ganglion cells that aggregate cone receptor responses in a large variety of combinations that represent all areas of the [color] space.”

This punchy conclusion is going to stir some hearts because it threatens to undermine the idea that parvocellular cells embody opponent-color axes and, more generally, aims once again to make color a physical attribute tied to reflectance spectra.

But should we be surprised that the Munsell color chip data cluster in the same pattern as the LGN cells in plots of cone space? Well, maybe not. Munsell's color scheme has successfully standardized color and is used in almost all jobs that require color identification. Munsell used his own visual system in developing his system, so it might not be a coincidence that the reflectance spectra of Munsell's color chips match the sensitivity of the LGN cells if LGN cells serve as the building blocks for color discrimination. But Romney et al. (1) assert that reflectance spectra of natural objects, not just the Munsell spectra, also cluster in cone space just like the Munsell data (A. K. Romney, personal communication). Does this mean that the nonrandom distribution of LGN responses in cone space is an adaptation to the same nonrandom distribution of reflectance spectra in the world and not the brain's implementation of opponent color theory?

We don't think so. For starters, the clustering of Munsell chips in cone space is not nearly as tight as that of LGN cells. In the original study of LGN responses in cone space, Derrington et al. (8) found that the LGN clustering was even tighter than the clustering in the resurrected LGN data of De Valois et al. (4). In addition, the blue-yellow cells in the reanalyzed data are not localized to the correct region of cone space: blue-yellow cells should be in the part of the plot where L and M cones are of the same sign, and they are not. But the puzzle remains: why would the Munsell chips be even coarsely clustered in cone space?

The answer lies in the fact that the cone absorption curves overlap considerably: the red and green cones differ in peak sensitivity by only 30 nm over their 400-nm range of sensitivity (Fig. 1 Bottom). This extensive overlap means that colors in neighboring regions of the spectrum elicit responses of very similar size from the three cone types (especially the red and green), despite pronounced differences in hue. In other words, the differences between cone responses are not nearly as large as the differences in perceived color. How, then, can we perceive different colors based on only subtle differences in cone responses? We can because the brain determines color by the relative, not absolute, activity of the different classes of cones. This comparison could be achieved by excitatory input from one cone type and suppressive input from a different cone type in single LGN cells, but even if it is not, the upshot is that, in cone space, the cone responses to a complete sample of colors will be clumped. And vice versa, small differences of cone responses will be expanded if plotted in perceptual space. This result means that slightly variable responses of LGN cells will masquerade as sensitivity to different hues, which is, in fact, exactly what Romney et al. (1) found. The spectral sensitivities of the 147 LGN cells of De Valois et al. (4) do not fall into discrete categories when plotted in perceptual coordinates (Fig. 2B): the “R⁺/G^-” cells, for example, include cells of varying strength of excitation to red and suppression to green.

Romney et al. (1) suggest that LGN cells seem to be discrete categories because the cells were categorized that way according to preconceived ideas of color processing. But the question boils down to the most basic one of empirical science: when is variability noise, and when does it represent functional specialization? Are different responses of different R⁺/G^- cells used by the nervous system to represent different hues, or is this variability simply biological sloppiness or experimental error?

Although provocative, the analysis of Romney et al. (1) doesn't clear this up for one critical reason: the LGN cells do not completely fill perceptual color space. The red, yellow, blue, and green cells form petals around the achromatic origin (Fig. 2B), not pie pieces like the ideal Munsell data; and, more importantly, the purple petal is missing. These holes necessitate a subsequent stage of color processing and lend support to the conclusion that the somewhat continuous representation of colors within the LGN is a misleading artifact produced when neural data are plotted in perceptual space.

To reconcile the pieces of the Romney et al. article (1), the critical clue is that perceptual space represents all colors with equal weighting, but the cones do not treat all wavelengths equally, so the same data will look compressed in plots of cone space and expanded in plots of perceptual space. Cone-isolating stimuli, which selectively modulate single cone classes at a time but do not sample the spectrum evenly, turn out to be useful in teasing apart the neural basis of color vision precisely because they measure responses in retinal coordinates rather than in perceptual coordinates, which probably arise in the brain after the early stages of color vision.

Given current neurophysiological data, we think that the best theories of color employ specialized double-opponent color neurons, which arise in primary visual cortex. These cells have receptive fields that respond well to chromatic boundaries, and chromatic boundaries are key features used by the visual system to construct color (9, 10). LGN receptive fields have receptive fields that are structured in exactly the wrong way for mediating spatial color contrast: R⁺/G^- cells have red-on centers and green-off surrounds (11) (Fig. 1). Indeed, some have argued that a small subset of LGN cells distinct from typical parvocellular cells is used to encode color (12–15), which is reasonable because color has relatively coarse resolution. Either way, it seems more likely to us that hue is determined after the LGN, in primary visual cortex, or perhaps beyond (16). But we'll have to rethink this if it is shown that LGN cells completely fill perceptual color space and that LGN cell receptive fields can encode specific hues despite their impoverished receptive fields.