Edited by Dale Purves, Duke University Medical Center, Durham, NC, and accepted by the Editorial Board August 15, 2008 (received for review March 27, 2008)
Several human and monkey studies have demonstrated a close relationship between motion perception and activation of area V5, leading to the general view that activity in this area correlates with the subjective experience of motion. In the present study, we investigate whether the responses of this area are still governed by the motion percept when the latter is in conflict with the reality of the physical visual stimulation. We simultaneously presented two different, specially designed random-dot kinematograms, one to each eye. These stimuli either both had a single direction of motion and worked in synergy, or had opposite motion directions and thus cancelled each other out perceptually. In this way, we were able to pit the visual stimulus (one vs. two stimulating directions) against the reported perception (directional motion vs. motion noise) of human volunteers during fMRI experiments. We found that a strong motion stimulus that is weakly perceived is more effective in activating V5 (as well as V3) than a weaker motion stimulus, which is nevertheless robustly perceived. Thus, contrary to the prevailing view of perception being the correlate of activity in higher visual areas, we show here that activity is instead dominated by the properties of the physical stimulus, raising the question of whether there is a subpopulation of cells in V5 whose activity is critical for generating the motion percept. In addition, our results provide the first robust evidence for the presence of directionally selective neuronal populations in human prestriate cortex.
Most studies attempting to dissect out brain activity that specifically correlates with visual experience have used paradigms that dissociate the visual percept from the physical stimulus that leads to it (1). Usually, the physical stimulus is kept constant and the perception varies, with the hope of pinpointing brain areas that modulate their activity with respect to what the subject is perceiving (2–13). The activity pattern in these areas is then considered to be the correlate of the subject's perceptual experience. Area V5, specialized for visual motion, is probably the most celebrated area of the primate brain. It is characterized by neurons selective for the direction of motion of the stimulus (14), with responses that follow the animal's performance in direction-discrimination tasks (15). The latter is impaired in monkeys after lesioning this area (16) or after inactivating its human homologue by transcranial stimulation in humans (17), whereas cortical microstimulation can alter the motion percept of the animal (18). Area V5 has thus been shown to play a primary role in the processing of visual motion. In addition, several studies suggest that activity in area V5 correlates directly with the perception of visual motion in human (3, 4, 9, 13, 19) and monkey (2, 5). The modulation of activity in this area as a function of the subjective percept of motion makes it a very plausible candidate for the neural correlate of motion perception. However, in all of the studies cited previously, the contribution of V5 to the motion percept has been investigated by keeping the physical stimulus constant and varying the motion perception. In a recent study (20), we have used the opposite method of keeping the motion percept constant and varying the (nonperceived) physical stimulus instead. We found that the responses of this area can also be modulated by changes in the properties of the physical stimulus that do not reach visual awareness. Thus, changes in both the stimulus and the percept seem to be able to independently influence the level of activity in V5. In the present study we take the next logical step and, instead of keeping the stimulus or the percept constant, we change both at the same time, so that they are opposed to each other. By pitting the physical stimulus against the subjective percept, we can directly test whether the strength of activity in V5 is dominated by the perceptual reality or the strength of the physical stimulus.
To dissociate the strength of the physical stimulus from the strength of the motion percept, we presented two different random-dot movies (15) separately to each eye. In one case, the movies were designed so that the two 50% coherence monocular signals had opposite directions of motion, perceptually canceling each other out (“Opposite” condition; see Methods). In the second case, two 50% coherence random-dot movies having the same direction of motion were presented to each eye (“Same” condition), resulting in a clear, directional motion percept. In a third and final condition, used as a baseline for the directional responses, a random-dot movie of 0% coherence was presented to each eye (“Noise” condition). The three types of stimuli were presented in randomly interleaved trials to each subject inside the scanner, immediately before scanning. In this way we could directly assess the impression that the stimuli produced during the scanning experiments. Subjects were asked to rate the amount of directional motion they perceived in each trial, using a 1–5 scale. The group results of this rating, shown in Fig. 1(Bottom), reveal a clear perceptual difference between the three conditions: Noise conditions were mostly rated as 1 and Same conditions were mostly rated as 5, with the Opposite conditions in between but more toward weak directional-motion perception, as reported previously (21). The effect was statistically significant (P < 0.05, ANOVA). Thus, the Same condition gave the strongest percept of directional motion, the Opposite condition resulted in a much weaker one, and the Noise condition in nondirectional motion. The weak percept of directional motion in the Opposite condition reveals that the cancellation between the two monocular stimuli was not perfect. Subjects reported an occasional sensation of weak, transparent motion. None of the subjects reported experiencing binocular rivalry. This is in agreement with the original psychophysical study (21) and is also evident in Fig. 1: complete suppression of one of the monocular stimuli would lead to high directional motion ratings, something that was not observed. Given these perceptual effects, our aim was to directly relate the strength of brain activation to the strength of the motion percept, as done previously (15, 22), but this time reversing the relationship between stimulus and percept (stronger stimulus–weaker percept, and weaker stimulus–stronger percept).
Subject performance in the fixation and directionality judgment tasks. (Top) The mean percentage of errors in counting how many times the fixation point flickered is presented individually for each subject. The SEM across scanning sessions is also shown. (Bottom) Histograms of the frequency of occurrence of the various directionality judgments (1: least directional) under the three different types of stimulation used (see text). Individual results are similar to the cumulative population result presented here.
For the main experiment, once the motion-rating task was completed, image acquisition began using the same stimuli but a different task: during scanning, subjects were asked to fixate vigorously and count the number of times that a fixation spot flickered. Because of the small size of the fixation spot, the small luminance change, and the unpredictability of the flicker events throughout the scanning session, this was a highly demanding task. Fig. 1 (Top) shows the percentage of errors (with respect to the total number of luminance changes) made by each of the five subjects. All performed above 95% correct, which means that they were maintaining good fixation throughout the session; because eye movements were not measured, this is an important point, making the fMRI results refractory to any possible eye movement confound. Furthermore, the fact that subject performance was good but not perfect suggests that a constant attentional load was present throughout the session, protecting against the possibility that the results could be due to differential attention between the different stimulus types. Additionally, the direction of motion was always vertical (up and down), so vergence could not have been induced by the binocularly inconsistent stimulation of the Opposite condition. The perceptually clear consistent stimulation in the Same condition would be a much better candidate for inducing eye movements, and thus such a confound would actually work against our main finding (see below).
The main question we asked is whether the strength of response in area V5 correlates with the strength of the (physical) motion stimulus or with that of motion percept. On the one hand, the Opposite condition activated twice as many neurons as the Same condition, because it consisted of movies with two directions of motion. On the other hand, the Same condition resulted in a much stronger motion percept because there was no perceptual cancellation of the two monocular inputs as in the Opposite condition (see Methods). We wanted to know which of the two conditions was more effective in activating this area.
The results for the statistical contrasts Same Noise, Opposite Noise, and Opposite Same for group as well as for individual subject analysis are shown in Tables 1–3. In all cases, two visual areas were selectively activated: an area we presume to be V3 (both dorsal and ventral subdivisions) and area V5. The areas were identified by means of their Talairach (41) coordinates, with reference to previous detailed retinotopic studies (23). In addition, for area V5, which was the main target of this study, functional localizers were also used (see following paragraph). The most significant activation was revealed by the Opposite Noise contrast: at the group level, both areas were bilaterally activated at the P < 0.05 corrected level. We therefore used this contrast to define the anatomical location of the areas, as shown in Fig. 2. Group results were also confirmed at the individual level, showing both areas to be activated in each subject at the uncorrected P < 0.001 level (and in many cases at the corrected level), usually bilaterally (Table 1). As expected from previous studies (22), area V5 was also significantly activated by the Same Noise contrast, whereas results for what we presume to be area V3 were less clear (Table 2). More importantly, the weaker effect of this contrast compared with the previous one suggests that the Opposite condition, which contains a stronger motion stimulus (activating both upward and downward directionally selective neuronal populations; see text following) compared with the Same condition is more effective in activating this area. This preference is directly evident in the Opposite Same contrast: both V5 and V3 are activated by this comparison at both the group and the individual level (Table 3). Therefore, despite the fact that motion perception is impaired under the Opposite condition, the stronger motion content of the stimulus results in a stronger activation of both of these areas. Similarly, despite the fact that it results in a much stronger motion percept, the Same condition is not as effective in activating either V5 or V3. It therefore seems that, contrary to the prevailing belief (especially for V5), it is the properties of the physical stimulus rather than the resulting motion percept that dominates the response of these areas.
Individual and group results (five subjects) for the contrast Same Noise, shown separately in each hemisphere for the three visual areas activated in the present study
Individual and group results (five subjects) for the contrast Opposite Noise, shown separately in each hemisphere for the three visual areas activated in the present study
Individual and group results (five subjects) for the contrast Opposite Same, shown separately in each hemisphere for the three visual areas activated in the present study
The location of brain areas activated in the present study. The group result of the Opposite Noise contrast, FEW corrected for multiple comparisons, is superimposed on a template anatomical scan. The Left column shows horizontal and the right column coronal sections through the region of interest. The name of each corresponding visual area is printed to the right. In the Top and Bottom horizontal sections, both V3d (posterior activation) and V5 (anterior activation) can be seen bilaterally. The Talairach coordinates (x–z) are as follows: lV5 (−44, −78, 4), rV5 (48, −72, 2), lV3d (−24, −96, 2), rV3d (26, −100, 8), lV3v (−30, −88, −14), and rV3v (28, −86, −14).
In addition to the whole-brain analysis described, we also analyzed our data in a region-of-interest (ROI) fashion. Area V5 was defined as responding more to the coherent motion (Same condition) than to the motion noise (Noise condition), as it has been shown previously by studies in monkey (15) and human (22). Fig. 3 shows the average time course of activation of this area for each of the three experimental conditions. The highest activation is observed for the Opposite condition and the lowest for the Noise condition, with the Same activation lying between the two. Exactly the opposite ranking is observed before the presentation of each condition, as expected given the different histories (absence of the Opposite condition before its own onset, etc.) and the normalization procedure (activation expressed as percent modulation of the average activation of each session). The fact that the highest activation was given by a condition different from the ones used to define V5 reassures us that the observed result is unbiased with respect to the method of selecting the ROI. Statistical comparisons between the three experimental conditions, corrected for the extent of the ROI in each individual, gave results identical to the ones described in the whole-brain analysis. Therefore, the main result reported here is statistically significant in each of the five subjects tested. In addition to the individual subjects analysis, the robustness of the result is also evident in across-subjects analysis, where we found that Opposite > Same in all five subjects (P < 0.05, binomial distribution). Furthermore, because visual areas are bilateral, we have a total of 10 independent V5 samples, in 9 of which we found that Opposite > Same (in 7 of 9 this was statistically significant), a highly significant result (P < 0.01, binomial distribution).
Event-related time courses for area V5. The percent signal change with respect to the average session signal is plotted as a function of time (stimulus onset at time 0) expressed in number of scans (TR = 3.48 s). Dotted line: Opposite condition; solid line: Same condition; dashed line: Noise condition (see text). Error bars represent SEM across the five subjects.
The relationship between motion perception and brain activation has been the main initiative of this study. However, a second, equally important conclusion can be drawn from the results described: the presence of different, directionally selective neuronal populations in human V5, as is the case with its monkey homologue (14). It is known from previous studies that nonperceived monocular information is not lost beyond the point of convergence of the two eyes in V1, but is still present in the higher visual areas of the brain (2, 6, 7, 24). If human V5 also contains directionally selective units, the two monocular inputs in the Opposite condition should activate two different neuronal populations in this area (i.e., twice as many neurons as activated by the Same condition). The presence of the nonpreferred direction is of course expected to reduce the strength of activation caused by the presence of the preferred direction. The resulting activation is therefore expected to be less than the linear sum of the two individual maximum responses (i.e., the response without inhibition). However, V5 neurons have been shown to still respond strongly to their preferred direction of motion, despite the presence of the opposite direction inside their receptive field (2, 25–27). If all this is true for human V5 as well, then the Opposite condition should result in a stronger BOLD response than the Same condition, as the response in this area is proportional to the power of directional energy present in the stimulus, both in monkey (15) as well as human (22).
This previous point is more clearly demonstrated in Fig. 4, which shows two possible models for the connections between V5 and a directionally selective input stage. In the first model (A), directional selectivity is present only at the input stage, units of which converge their signal upon the same units in V5. In the second model (B), directional selectivity is present in V5 as well, as neurons with different directional selectivities at the input stage send information to correspondingly separate neuronal populations in V5. In model B, a different directional input in each eye results in activating twice the number of V5 neurons, compared with having the same directional input in each eye—something which is not true for model A. Our results clearly favor model B, because they show that V5 is activated more in the Opposite than in the Same condition. These results therefore provide robust evidence for the presence of directionally selective neurons in human V5.
Two possible models for the excitatory input in area V5. Inhibitory interactions are omitted for simplicity. (A) Different directionally selective populations of the input stage converge in V5, resulting in units with no directional selectivity in this area. (B) The directionally selective input remains segregated in V5, resulting in the directional selectivity of units in this area as well. Large ellipses: V5 neurons; small ellipses: input neurons; arrows: directional preference; L and R: eye identity.
The present results strongly support the hypothesis that there must exist different, directionally selective neuronal populations in human V5 (model B), as is the case with the homologous area in monkey (14). Previous attempts to demonstrate this using fMRI in humans have exploited the phenomenon of motion adaptation and the aftereffects that it produces (4, 9, 10, 28). Aftereffects, however, might reflect activity in directionally selective neurons whose outputs are pooled at a nondirection-specific stage (Fig. 4A): observing the effects at this stage does not necessarily mean that it contains directional specificity, something long realized in the psychophysical literature of motion mechanisms. For example, the fact that after motion adaptation the contrast detection threshold for a moving grating is elevated in a direction-specific manner (29, 30) cannot be taken as evidence for the existence of separate detectors for opposite directions of movement (31). From Fig. 4 it is also evident that possible opponency effects previously described in human V5 (32) (but see also following text) might again reflect the presence of opponent mechanisms at the input stage, and are thus also inadequate in demonstrating the presence of such mechanisms in V5 itself. The presence of directional information in the responses of human visual cortex has also been recently demonstrated, using support vector machines (SVM) to analyze fMRI data (33). A disadvantage of this method, however, is that the nature of the information underlying the discrimination ability of the SVM is not always clear (34).
Motion opponency, as previously observed in V5 neurons using similar stimuli (25, 27), did not result in a weaker BOLD signal compared with the Same condition in the present study. This may seem to contradict a previous study, in which the BOLD response to a counterphase grating was found to be weaker than the one to a single moving grating (32). The fact that in our case the opposite directions were separated between the two eyes is one possible explanation. However, it is not clear whether a true counterphase or a simple contrast-reversing grating was used in this previous study; in the second case, any claims regarding opponency would not be valid because such a grating cannot be regarded as the result of two sinusoidal gratings moving in opposite directions (31). A simpler explanation would be that V5 responds more to a stimulus that carries directional motion energy (moving grating) than to one that does not (flicker). Although the transformation from physiology to fMRI is not yet straightforward (35), our findings are more consistent with studies that show that V5 neurons still respond strongly to stimuli moving in their preferred direction, despite the presence of motion in the opposite direction (2, 25–27). We show that, as far as the BOLD signal is concerned, it seems more efficient to activate twice the number of neurons less (Opposite condition) than to activate half the number of neurons more (Same condition). A similar result has been obtained in an fMRI study with transparent motion, where the activity in V5 was found to increase parametrically with the increase of the angle between the two directions of motion (36).
An alternative interpretation of the observed results is that the stronger BOLD signal during the Opposite condition reflects a possible greater demand for computational activity and thus a greater metabolic demand. This might be because in contrast to the more straightforward Same condition, the visual system needs to either resolve the conflict between the two monocular signals or to combine them in a single percept, or both. If true, this could result in a higher Opposite activation irrespective of whether there exists two separate directionally selective neuronal populations or whether the same neuronal population is activated by both directions of motion. Although we cannot rule out this possibility, considering the prominence of directional selectivity in the visual system of monkey, we favor our original interpretation. A possible heavier computational load during the Opposite condition could act synergistically, further increasing the difference between this and the Same condition. The issue remains highly hypothetical, first because we do not know for sure whether there is a difference in computational work between the two conditions, and second because we are not certain that any hypothetical conflict arising from the dichoptic presentation is solved inside area V5. Whatever the case, the main conclusion of the present study remains: activity in this area seems to be governed more by the physical properties of the stimulus (be it motion energy or computational complexity) than by the resulting percept.
Regarding the link between neuronal events and perception, even though the Same condition produces a much stronger perception of motion (Fig. 1), brain activation is stronger with stimulation in the Opposite condition (Table 3), in which directional motion is barely perceived. It is thus the stimulus strength rather than the strength of the motion percept that determines the magnitude of response in these brain areas. This is a surprising result, especially in the case of V5, whose responses have been supposed to accurately reflect the motion percept independent of the stimulus, both in monkey (2, 5) and in human (3, 4, 8, 9, 13). In these previous studies, however, the stimulus was always constant, and thus its role in the response could not be evaluated correctly. We have recently shown that if one reverses the experimental conditions by using a varying stimulus that results in a constant percept, stimulus changes can also modulate V5 responses independent of the percept (20). In the present study we go a step further to show that when the stimulus is pitted against the percept, it is the properties of the former that dominate the response of the area. This result is in agreement with an electrophysiological study in monkey, in which V5 neurons were found to signal the nonperceived, local motion direction of the stimulus rather than the perceived global motion direction (37). In a related study, it was shown that direction-selective neurons in V5 signal the local rather than the global motion inside their receptive field (38). These monkey results, together with our present and antecedent findings (39) in humans, call for a reevaluation of the way in which activity in V5 and other higher visual areas is considered to relate to subjective motion perception and raise in particular the question of whether it is the strength of the response that correlates with the experience of motion or the activation of particular groups of cells, whose presence may not be easily demonstrable in imaging experiments.
Five healthy right-handed volunteers (three male) with normal or corrected-to-normal visual acuity, aged 23–38, participated in this study. All gave informed consent in accordance with the Declaration of Helsinki. The Ethics Committee of the National Hospital for Neurology and Neurosurgery (London, U.K.) granted ethical approval for the study.
All stimuli were constructed on a power PC using COGENT 2000 Graphics (www.vislab.ucl.ac.uk) running in MATLAB (Mathworks). Stimuli were exact replicas of the ones described by Meng et al. (see figure 3 of ref. 21), with the difference that we used a condition with the same motion direction in each eye, instead of their unpaired condition. In brief, two different random-dot movies were dichopticaly presented to the volunteers. In the Noise condition, both movies contained noise of 0% coherence. In the Same condition, both movies contained 50% coherent motion of the same vertical direction. In the Opposite condition, both movies contained 50% coherent motion but of opposite vertical directions in the two eyes. In this condition, the new position of each dot that moved coherently in one eye was identical to the old position of a dot that moved coherently in the other eye, resulting in a cancellation of the overall motion at the binocular level. Once in the scanner and before the scanning began, each subject was presented with 60 intermixed trials equally divided between the three conditions, and was asked to rate the presence of vertical directional motion on a scale of 1 (min) to 5 (max). During scanning, subjects performed a demanding fixation task in which they had to count the number of times (across the whole scanning session) that the 1 pixel fixation spot flickered. Each scanning session (of a total of six per subject) consisted of four blocks of stimulation interposed between five 16 s fixation-only periods. Each stimulation block consisted of a 16 s presentation of each of the three conditions (Noise, Same, Opposite). The order of the three conditions within a block was different between blocks and between sessions to eliminate the possible effects of a different stimulation history.
Functional magnetic resonance data were acquired on a Siemens 3T Allegra scanner (Siemens) fitted with a head-coil. Subjects viewed a screen via an angled mirror onto which stimuli were projected using an LCD projector. The screen was split in the vertical meridian by a black cardboard and +3 lenses were used to help subject focus and fuse the two monocular images. BOLD contrast-weighted echoplanar images (EPIs) were acquired for all functional scans, using 38 slices to cover the whole brain with a TR of 3.48 s. The first six images of each session were discarded, to allow for T1 equilibration effects. A T1-weighted anatomical image was also acquired at the end of each experimental session. All data were analyzed using SPM2 (www.fil.ion.ucl.ac.uk/spm). Acquired images were all realigned to the first volume of the first session using sinc interpolation. Every image was then spatially normalized using a template that corresponds to the Montreal Neurological Institute reference brain in Talairach space, spatially smoothed with a Gaussian kernel of 6 mm (10 mm for the group analysis) full width at half maximum, and temporally filtered with a high-pass filter with a cutoff of 1/128 Hz. Serial autocorrelations were modeled using an AR(1) method. A single voxel-based analysis in which BOLD time courses from each individual voxel were regressed onto models of the stimuli was used. Each of the three conditions was modeled separately as a simple boxcar waveform to create regressors of interest. Fixation-only periods were not modeled. All regressors were convolved with a hemodynamic response function and entered into a general linear model (40). Visual areas were identified according to their Talairach coordinates, based on previous literature. Results were characterized as significant either at the P < 0.05 level after FWE correction or at the P < 0.001 uncorrected level. The location of both corrected and uncorrected results reported in Tables 1–3 all survived a small volume correction of radius 10 mm, centered on the Talairach coordinates given in Fig. 2. For the ROI analysis and the extraction of event-related time courses, MarsBar (http://marsbar.sourceforge.net) and custom software running in MATLAB (Mathworks) was used.
We thank the anonymous reviewers for their comments and suggestions. This work was supported by the Wellcome Trust (U.K.).
Author contributions: K.M. designed research; K.M. performed research; K.M. analyzed data; and K.M. and S.Z. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
