Several aspects of the proposal, which aims to expand open access, require serious discussion and, in some cases, a rethink.
Image credit: Dave Cutler (artist).
New Research In
Physical Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
Edited by Michael E. Goldberg, Columbia University College of Physicians and Surgeons and the New York State Psychiatric Institute, New York, NY, and approved December 12, 2014 (received for review August 6, 2014)
Significance
Perceiving the movements of people around us is critical for many daily skills (from detecting threats to social interactions) and involves both form and motion perception. Even though the “form” visual pathway is standardly activated by biological motion stimuli, it is unknown whether this pathway’s integrity is critical for the perception of biological motion. Here, we examined whether damage to different aspects of the form pathway affects biological motion perception. Individuals with lesions to the ventral aspects of this pathway evinced normal biological motion perception despite their impairments in form perception. Our counterintuitive findings indicate that biological motion can be perceived and processed normally even when the ability to perceive the form or the actor executing the movements is impaired.
Abstract
Identifying the movements of those around us is fundamental for many daily activities, such as recognizing actions, detecting predators, and interacting with others socially. A key question concerns the neurobiological substrates underlying biological motion perception. Although the ventral “form” visual cortex is standardly activated by biologically moving stimuli, whether these activations are functionally critical for biological motion perception or are epiphenomenal remains unknown. To address this question, we examined whether focal damage to regions of the ventral visual cortex, resulting in significant deficits in form perception, adversely affects biological motion perception. Six patients with damage to the ventral cortex were tested with sensitive point-light display paradigms. All patients were able to recognize unmasked point-light displays and their perceptual thresholds were not significantly different from those of three different control groups, one of which comprised brain-damaged patients with spared ventral cortex (n > 50). Importantly, these six patients performed significantly better than patients with damage to regions critical for biological motion perception. To assess the necessary contribution of different regions in the ventral pathway to biological motion perception, we complement the behavioral findings with a fine-grained comparison between the lesion location and extent, and the cortical regions standardly implicated in biological motion processing. This analysis revealed that the ventral aspects of the form pathway (e.g., fusiform regions, ventral extrastriate body area) are not critical for biological motion perception. We hypothesize that the role of these ventral regions is to provide enhanced multiview/posture representations of the moving person rather than to represent biological motion perception per se.
Perception of the movements of other peoples’ bodies is fundamental to our daily interactions (e.g., motor learning, social interactions, anticipating actions of others), and is sufficiently robust so as to succeed even under suboptimal conditions [e.g., poor illumination and even partial occlusion (1⇓⇓⇓⇓–6)]. A clear demonstration of the resilience of this ability is the ease with which people recognize biological motion from point-light displays (PLDs) that consist of only a small set of moving points that mark joints on the body (7), (Fig. 1A, Left). These stimuli appear to naïve observers as a set of incoherent dots when static, but evoke a vivid percept of a moving person when in motion. Observers are able to infer movement information, such as the motion or direction of the figure, in these impoverished PLDs even under conditions of masking, added noise (8⇓⇓–11), or night driving (3, 4, 12).
Fig. 1.
Biological motion perception: Exp. 1 paradigm and results. (A, Left) Static snapshots from the unmasked PLDs presented in the action recognition phase; participants were required to verbally describe the display. (Right) To determine perceptual thresholds, two masked PLDs embedded in noise points were presented simultaneously, one on the right and one on the left, one containing a biological movement (here on the left), and the other a spatially scrambled version of that movement. The task was to determine which side contained the moving human figure. The noise points were added adaptively according to individual performance (Methods). (B) Individual perceptual thresholds (y axis) of the ventral visual patients and the three control groups against age (x axis). Perceptual thresholds represent the number of noise points that can be tolerated while performing at 82% accuracy (more noise points correspond to better performance). Each ventral patient was not significantly different from any of the control groups, and performed similarly to or better than the brain damaged controls with spared ventral cortex (11) (Table 2). (C) Ventral visual patients’ performance was significantly better than that of brain-damaged controls (11) whose lesions invaded regions critical for biological motion perception (pSTS, vPMC). (D) Response times (y axis) as recorded during the experiment plotted against age (x axis). Response times are not very informative as instructions did not require speeded responses (participants were able to respond leisurely and take breaks). Response times of the brain-damaged control group were not available. Ventral visual patients’ response times (apart from EC) (Results) were not significantly slower than those of their age-matched controls.
Examination of the neural correlates of the perception of body movement reveals a widespread cortical network (13). Because biological motion perception, in natural vision or in PLDs, involves both form and motion perception (14), unsurprisingly, cortical regions associated with both form and motion perception are activated. It is unclear, however, whether all of these brain areas contribute causally to the perception of biological motion. Neuropsychological studies in patients and transcranial magnetic stimulation (TMS) studies in normal observers have identified several motion-sensitive areas as critical for biological motion perception, including the posterior superior temporal sulcus (pSTS) and ventral premotor cortex (vPMC) (11, 15⇓–17), given that a sustained or transient lesion to these regions impairs biological motion perception. However, whether form-sensitive regions in the ventral “form” visual pathway [for example, the extrastriate body area (EBA) (18⇓–20) in the lateral occipital cortex], which are consistently activated in response to biological motion in neuroimaging studies, play a critical role in biological motion perception remains unknown.
PLDs constitute ideal stimuli with which to explore whether the engagement of the ventral visual cortex is necessary for biological motion perception, as these displays permit the presentation of recognizable body movements while dissociating them from “classic ventral” form cues, such as contour, surface, shape, texture, and color. As such, PLDs are thought to depict dynamic body and action information solely via motion cues. To the extent that the ventral form pathway is involved in PLD perception, this cannot be attributed to processing classic ventral form cues. Even in the absence of classic form cues, however, moving PLDs convey coarse form information of the dynamic body, and investigating the structure of the articulated body that can be retrieved from the coherent movement of the dots has been a central driving motivation in biological motion research (e.g., refs. 21⇓⇓⇓⇓⇓–27). This approach has been true since the pioneering work of Johansson (7), and therefore, unsurprisingly, these displays have often been termed biological structure-from-motion or form-from-motion. In light of the above, whether the processes supporting biological motion perception—in the absence of classic form cues—critically depend on the form computations of the form visual pathway still remains unknown (20, 28⇓–30).
One way to address this issue is to study whether damage to the ventral “form” visual pathway affects the perception of biological motion. The predictions are straightforward: if ventral stream integrity and ventral form representations are necessary for the perception of biological motion, then individuals with form perception deficits following damage to the ventral visual cortex [including damage to specific areas implicated in biological motion processing or areas implicated in body form perception (e.g., the EBA) (18⇓–20)], should be impaired at perceiving biological motion.
To examine this hypothesis, we tested a group of six patients with form perception deficits following a circumscribed lesion to the ventral visual cortex in adulthood (Table 1). Using PLDs and paradigms that are successful in detecting biological motion perceptual deficits following brain damage (11, 17), we measured the patients’ recognition and perception of biological motion in two different experiments. We compared each patient’s performance to that of three different control groups: (i) a brain-damaged control group of 54 patients, whose cortical lesions fell outside of the ventral visual cortex; (ii) a group of healthy age-matched controls (patient-specific); and (iii) a group of 13 young control participants. The importance of the brain-damaged control group is twofold. First, comparing the ventral patients to patients with nonventral lesions allowed us to determine whether biological motion perception, if affected, is specifically a consequence of a ventral lesion or of brain damage, more generally. Second, because a subset of patients in the nonventral control group have lesions to brain areas known to significantly impair biological motion perception (pSTS and vPMC), we were able to compare directly the perceptual thresholds of the ventral patients with those of individuals with identified deficits in biological motion following damage to the pSTS or vPMC. Finally, given that the ventral cortex constitutes a large swath of the cortex and that patients’ lesions were not identical, we assessed the brain–behavior correspondences further by examining, at a finer grain, which—if any—affected subareas impair biological motion perception. To do so, for each patient we carefully delineated the lesion, assessed the magnitude of the damage, and situated the lesion relative to regions in the ventral, lateral, and middle temporal cortex that are standardly associated with biological motion processing, including body parts and visual motion-sensitive regions (e.g., refs. 13, 14, 20, 31⇓–33).
View this table:
Table 1.
Summary of the visual perceptual functions and impairments of the six ventral patients
To anticipate our findings, we show that biological motion recognition and perceptual thresholds of the ventral patients consistently fall within the normal range of all three control groups. Thus, our results indicate that the perception of biological motion (i) does not depend on the integrity of the ventral aspects of the form visual pathway or on the integrity of the ventral portion of the EBA (ventral to the human middle temporal V5 complex, MT+/V5), and (ii) can be dissociated from form perception. We conclude, therefore, that computations that suffice for the perception of biological motion are mediated by mechanisms independent of the form ventral cortex, and that such computations may be based on motion cues that represent movement kinematics rather than on form information per se.
Results
Experiment 1.
Recognition and perceptual thresholds for biological motion.
At the start of Exp. 1, participants were presented with unmasked PLDs (Fig. 1A, Left) and were asked to describe what they perceived. For these unmasked PLDs, all six ventral patients (as well as almost all individuals in three groups of controls) were able to name the movements effortlessly and immediately, even without prior knowledge of or training on PLDs. This observation is consistent with previous work showing that patients are generally able to recognize unmasked PLDs of biological motion (34⇓⇓⇓⇓–39). Only two of the control patients with brain-damage outside of the ventral cortex (from the “brain-damaged control group” in the present study) were unable to recognize unmasked PLDs (11).
After this recognition phase, perceptual thresholds for biological motion (number of noise points at which performance is 82% accurate; see Fig. 1A, Right, for illustration) were measured for all participants (Fig. 1B, and detailed in Table 2). Consistent with previous results showing that the perceptual thresholds of brain-damaged patients for biological motion are significantly lower than those of healthy age-matched controls (11), four of the ventral visual patients’ (CR, SM, EC, and SH) perceptual thresholds were at the lower end of their matched controls’ distribution (control group 1; light diamonds in Fig. 1B), but not statistically different (see Table 2 for statistical details). Moreover, the perceptual threshold of each ventral patient was also within the norm of the younger control group [control group 2; all |t(12)|s < 1.44, all Ps > 0.17 (40); Fig. 1B, light gray circles].
View this table:
Table 2.
Exp. 1: Biological motion perceptual thresholds of patients and controls
We then compared the thresholds of the ventral patients and those of 54 patients with unilateral nonventral brain damage [control group 3 (11); dark circles in Fig. 1B]. If the integrity of the ventral visual cortex is critical for biological motion perception, then the performance of the ventral visual patients should be significantly poorer than that of patients with brain damage elsewhere. In contrast, the ventral patients’ thresholds were trending to be significantly better than their brain-damaged controls [Wilcoxon nonparametric rank-sum test: ventral patients (median = 13.11, n = 6) vs. brain damaged controls (median = 9.82, n = 54): U = 258, P = 0.06]. In addition, in an individual case analysis, each of the ventral visual patients’ performance was better than the average performance of the right only (n = 11), left only (n = 43), or combined right and left hemisphere brain-damaged control patients (see Table 2 for full details). All of these comparisons indicate that the six ventral patients performed well within the range of other (nonventral) brain-damaged patients, thereby ruling out a specific role for the ventral cortex in biological motion perception.
The data from the control brain-damaged patients were taken from a previous study (11) that revealed that lesions to the left pSTS (L-pSTS) or to left vPMC (L-vPMC) had the greatest adverse effect on biological motion perception. The function and structure of these regions are associated with biological motion perception (11, 15, 16, 20, 41⇓⇓⇓–45) and their role in biological motion perception has been confirmed in several TMS studies (15, 16). In light of this finding, these data permit a stringent comparison between the performance of our ventral patients and that of the brain-damaged patients with lesions to L-pSTS or L-vPMC (the two “critical” lesion groups). As shown in Fig. 1C, the ventral visual patient group performed significantly better (had higher perceptual thresholds, meaning they could tolerate more noise points) than both of the critical lesion groups [Wilcoxon nonparametric rank-sum test: ventral patients (median = 13.11, n = 6) vs. lesioned L-pSTS (median = 7.1, n = 9): U = 73, P = 0.0016; ventral patients vs. lesioned L-vPMC (median = 7.6, n = 10): U = 77, P = 0.003]. Furthermore, in single-case comparisons [each ventral patient vs. the critical control groups (40)], four of the ventral visual patients performed significantly better than the critical control groups (Table 2). These results indicate that damage to the ventral visual cortex, unlike damage to the pSTS or vPMC, does not impair biological motion perception.
Response times for biological motion.
The results thus far indicate that biological motion perception does not rely on ventral stream integrity. To confirm this finding and ensure that the results were not a product of a speed-accuracy trade-off, we examined reaction times even though participants were informed that speeded responses were not required and participants were allowed to speak and take breaks during the experiment (Fig. 1D). The responses of the patients were not significantly slower than their age-matched controls [Wilcoxon nonparametric rank-sum test: ventral patients (median = 7.78 s, n = 6) vs. age-matched controls (median = 5.21 s, n = 33): U = 159, P = 0.134, z = 1.5]. This finding also held true for five ventral patients under single-case comparisons of patient vs. age-matched control group [two-tailed, |t|s < 0.5, Ps > 0.63 (40)]. EC was significantly slower than her age-matched controls [t(11) = 4.74, P = 0.0008]; however, this is almost certainly a result of the fact that she spoke during the experiment. These results confirm that reaction times were within the normal range for the ventral patients and that these normal perceptual thresholds did not result from elongated response times.
Experiment 2.
Recognition and perceptual thresholds for biological motion under a different paradigm.
To provide additional support for the findings from Exp. 1, we further examined the perceptual thresholds of the ventral patients using a modified biological motion experimental paradigm. This task included a larger set of biological motion animations, different presentation and task requirements, and provided feedback. In this experiment, each trial consisted of one centrally displayed PLD (Fig. 2A) and observers determined whether there was a moving human figure embedded in the display (compare Fig. 2A, Center and Right; see Methods). During the action-recognition phase (Fig. 2A, Left), all ventral patients and their controls effortlessly reported the actions present in the PLDs. Moreover, the ventral patients’ perceptual thresholds fell within the normal range of their age-matched controls (Fig. 2B and Table S1) [CR: t(14) = −1.42, P > 0.17; SM: t(14) = −0.88, P > 0.39; EC: t(14) = −0.76, P > 0.45; EL: t(10) = 0.30, P > 0.77; GB: t(10) = 1.56, P > 0.15; SH: t(10) = −0.65, P > 0.53); the performance of EL, GB, and SH also fell in the normal range of a bigger control group (n = 14), aged 60.2 ± 6.45 (SD): |t(13)|s < 0.5, Ps > 0.6.] Of great interest, the perceptual thresholds established here for each of the patients (and the relative rank ordering of the patients) were very similar to those obtained in Exp. 1, reflecting the reliability and consistency of these measures.
Fig. 2.
Biological motion perception: Exp. 2 paradigm and results. (A, Left) Static unmasked PLD snapshot presented in the action recognition phase, participants were required to verbally describe them. (Center and Right) To determine perceptual thresholds, each trial presented a masked PLD either containing a biologically moving figure (Center) or a spatially scrambled version of it (Right), and the task was to determine whether a moving figure was embedded in the display. The noise points were added adaptively according to individual performance with 75% accuracy (Methods). (B) Individual perceptual thresholds (y axis) of the ventral visual patients and the age-matched controls against age (x axis). Perceptual thresholds determined similar to Fig. 1 (Methods). Although patients commonly perform more poorly than healthy-matched controls, the performance of each ventral patient was not significantly different from that of their healthy age-matched controls (Results). (C) Response times are not very informative because instructions did not require speeded responses (participants were able to respond leisurely and take breaks). Ventral visual patients’ response times were not significantly slower than their age-matched controls (Results).
Response times for biological motion under a different paradigm.
The analysis of the reaction times of the patients versus the age-matched controls revealed no significant group differences (Fig. 2C) [all patients but EL: |t|s < 0.79, Ps >0.45, EL: t(10) = 1.51, P > 0.16], again confirming that the patients performed within the normal range.
Subjective Reports About Biological Motion Perception.
As a converging source of evidence, we obtained self-reports from the patients and controls in response to questions such as whether, on the basis of gait, they were able to recognize individuals and discriminate the age and sex of an individual and, for the patients, whether these abilities have changed postinjury. All patients, as well as controls, reported that they were able to comprehend movement patterns and actions even when they were unable to recognize the person doing it. Subjects also reported that they were able to discriminate sex and age based on gait, and that they could easily recognize atypical gait (e.g., limping). None of the patients reported that their abilities changed following their brain injury. Although these reports are subjective, they provide additional indications that biological motion perception may be dissociable from form perception.
Relationship of Perceptual Performance to Underlying Lesion.
The experimental findings reveal that the six ventral patients performed as well as or even better than (see comparison against patients with frank pSTS or vPMC lesions) the various control groups. We have proposed that the dissociation between the deficit in form perception and their intact biological motion perception rules out the functional contribution of the ventral cortex.
A possible alternative explanation, however, is that because the ventral cortex is so extensive and lesions are more circumscribed, there may be sparing of key ventral regions associated with biological motion processing (13), and it is these spared regions that account for the normal perceptual performance. To assess this possibility, we first carefully delineated the lesion of each patient and transformed each lesion into normalized Montreal Neurological Institute (MNI) space. Second, we superimposed the lesion onto the ventral cortex in which we identified areas that are consistently activated by biological motion; this included regions responsive to the perception of body movement (blue in Fig. 3), sensitive to static bodies (yellow in Fig. 3), to human movements (green in Fig. 3) as determined by a recent meta-analysis (13) (Methods), as well as motion-sensitive MT+/V5, biological motion-sensitive pSTS, vPMC, and static-body sensitive EBA (18⇓–20). For the EBA cluster, we further distinguished between three subregions: the ventral portion (v-EBA, ventral and not overlapping MT+/V5), the portion overlapping MT+/V5 (mt-EBA), and the portion anterior to MT+/V5 (a-EBA). Our analysis also took into consideration the functional anatomical organization of the ventral visual stream (e.g., ventral surface vs. lateral occipito-temporal aspects). Third, for each of these regions, we evaluated the extent of damage in each patient. The results of this fine-grained analysis are detailed in Table 3.
Fig. 3.
Situating the patients’ lesions with respect to visual regions standardly associated with biological motion processing, presented in rendered fashion. Lesion of each patient is delineated in red based on the structural images (Methods and Supporting Information). Regions consistently associated with biological motion processing are based on statistical maps of a meta-analysis (13): regions in blue are significantly activated to biological motion, regions in yellow are sensitive to static bodies, and regions in green are sensitive to human movement over nonhuman movements. As summarized in Table 3, ventral visual regions associated with all of the three types of biological motion perception were severely affected by brain damage in one or more of the ventral visual patients, including the ventral aspect of EBA (v-EBA, in EC, SM, and GB). This finding indicates that the spared perceptual thresholds for biological motion perception do not rely on the integrity of the ventral visual regions associated with biological motion processing. MT+/V5, middle temporal motion sensitive region; RH/LH, right/left hemisphere. See also Table 3.
View this table:
Table 3.
Summary of patients’ lesions
Our analysis revealed that regions situated on the ventral surface (e.g., fusiform gyrus) that are assumed to be engaged in some aspect of biological motion perception, including the regions that are sensitive to static body perception, body movements in general, and human body movements’ selectivity (13), are substantially damaged in three patients (60–100%, left regions in GB and SH, and the right regions in EC) (Table 3). Furthermore, the ventral portion of the EBA (46, 47), situated ventrally to but not overlapping MT+/V5, is also critically damaged in two right ventral patients (SM and EC) and partially damaged in the left ventral patient (GB). From these observations, we can conclude that regions situated on the ventral surface of the cortex in the ventral stream (20, 31, 48) and the ventral aspect of the EBA are not contributing critically to biological motion perception. Because other regions, such as those on the lateral occipito-temporal surface, including MT+/V5 and other parts of EBA, are only partially damaged in some of the patients (20–60%), we cannot rule out their possible role in biological motion perception.
Each of the regions associated with biological motion in the ventral aspect of the ventral visual cortex is significantly damaged in one or more of the patients (right fusiform in EC, left fusiform in GB and SH), including the right and left ventral portions of the EBA (R-v-EBA in EC and SM, L-v-EBA in GB), and also left MT+/V5 (EL); the superior and middle temporal regions are mostly spared (except for EL). Importantly, the two areas that are well-known to be sensitive and critical to human motion, the pSTS and vPMC (11, 13, 16, 28, 30, 32, 45, 49), are spared in all cases.
The ventral patients performed as well as healthy controls and brain-damaged patients, despite the fact that ventral regions associated with different aspects of biological motion processing were affected by their lesions. Importantly, the ventral-lesioned patients performed better than patients with brain damage to regions pSTS and vPMC, standardly considered the neural correlates of biological motion perception.
Discussion
In this study, we explored whether the integrity of form perception and of the ventral visual cortex are necessary for the perception of biological motion. This was achieved by examining the perceptual performance of patients with documented form deficits following brain damage to the ventral visual cortex. In two studies using animated PLDs (7) embedded in noise points, we derived a host of dependent measures (accuracy of unmasked displays, thresholds, reaction times, and self-report measures) There were no differences between the patients’ indices and those of healthy age-matched controls or of brain-damaged patients* with spared ventral cortex, and this was also true compared with healthy young controls. The fact that the performance of each of the patients was not significantly different from that of three different control groups, and that this result was replicated across different paradigms, suggests that we are not failing to detect a deficit. Moreover, the patients performed significantly better than patients with documented damage to regions critical for biological motion perception (pSTS and vPMC) (11). Furthermore, by showing that large swaths of cortex—assumed to be associated with biological motion perception—were lesioned in these patients, we were able to determine that the integrity of the ventral aspects of the ventral visual pathway is not critical for normal biological motion perception.
Hemispheric Contribution to Biological Motion Performance?
We consider it unlikely that the interindividual variability in the ventral patients’ biological motion performance is associated with the anatomical variations in brain damage. First, interindividual variability in biological motion perception is consistently found in healthy and nonhealthy populations (43, 50) (Figs. 1B and 2B). Interestingly, the severity of face or object perception deficits, which is influenced by the lesioned hemisphere (left with milder and right with severe deficits) did not correlate with biological motion performance either, because SH, who suffers mild face/object perception deficits, performed similarly to CR and SM, both of whom have severe face and object perception deficits. Second, performance following bilateral ventral damage (e.g., CR) was not worse than that of some patients with unilateral cortical damage (SM, EC, or SH). This result is in line with previous findings that individuals with bilateral damage to (WH) (38) or abnormal function of (LG) (51) the ventral visual cortex can successfully recognize biological motion displays. Third, lesion size was also not indicative of performance because CR, SM, and SH, all of whom have comparable performance (Figs. 1B and 2B), have very different lesion sizes (Table 3), whereas EL and GB, who have extensive lesions, performed best. On the other hand, spared bilateral ventral visual cortex does not suffice for normal biological motion performance when there is unilateral damage to regions critical to biological motion perception [sustained damage (11) or temporary lesions induced by TMS (16)]. Thus, it seems reasonable to conclude that the behavioral variability among our cohort of patients with ventral damage was a reflection of the variability in the general population, and that performance is not dependent on the specific anatomy of the ventral visual lesion.
The Role of the Ventral Visual Pathway in Biological Motion Processing.
Ventral and occipito-temporal regions, which comprise the ventral visual cortex, are associated with the processing of form information (“form pathway”). Perhaps surprisingly, these very regions, including the EBA, are activated in neuroimaging studies that focus on biological motion (refs. 13, 18, 19, 30; Figs. 3 and 4). A possible explanation for the engagement of these areas in biological motion processing is that form-based computations may be implicated in biological motion processing (7, 21⇓⇓⇓–25). Indeed, some have suggested that PLDs might be better characterized as “motion-from-form” (26), and recent models of biological motion recognition have proposed that biological motion recognition computations can be based on a sequence of static form snapshots derived from the movement itself (52, 53) (Fig. 4A). Some support for this comes from physiological recordings in nonhuman primates, where biological motion-sensitive neurons show sensitivity to body form in addition to, or instead of, body motion (54⇓–56).
Fig. 4.
Adaptation of the model for biological motion recognition based on Giese and Poggio (52). (A) The original model by Giese and Poggio with proposed distinctions: the form pathway’s main role involves snapshots for actor recognition, and the motion pathway’s main role involves kinematic patterns for human movement recognition. Brick color indicates how brain damage to the ventral cortex predominantly affects the form processing pathway that is involved in snapshot creation, thereby impairing actor but not action recognition. (B) Adaptation of the model to PLDs. Information flow in the case of PLDs resembles that of a damaged ventral visual cortex (A), because the information processed by the form pathway is insufficient, leading to abnormal actor recognition. At the same time, the information processed by the motion pathway is not significantly affected, so that the movement can be recognized. Based on our results, we speculate that the perception of human movement can be achieved based on motion kinematics alone. FBA, fusiform body area; ITS, inferior temporal sulcus; KO, kinetic occipital; MT+/V5, middle temporal motion-sensitive region; OF, optic flow; V1/V2/V4, visual retinotopic regions; v-EBA, ventral aspect of extrastriate body area.
The key question is whether the engagement of these cortical ventral areas is functionally relevant for the perception of biological motion. The answer cannot be reached based on findings from functional imaging and, thus, determining causality remains elusive. Our results indicate that intact ventral regions in the form visual pathway (e.g., along the fusiform gyrus) are not necessary for biological motion perception. Why are these areas activated then, as revealed in neuroimaging investigations? One possibility is that, following damage to the ventral visual cortex, cortical function is reorganized such that other regions become critical for biological motion perception, as perhaps the lesions may have altered the relative contributions of regions in the network supporting biological motion perception. However, notably, all of our patients performed within the normal range of other brain damaged, and notably, nonbrain-damaged control participants. This result indicates that if any such adaptive plasticity occurs, it is surprisingly effective. An alternative possibility, and the one we favor, is that these ventral regions play a role in the representation of the actor and his or her identity and, hence, are activated during biological motion processing. Specifically, a series of static posture snapshots may suffice for a whole-body viewpoint-based representation of the actor (Fig. 4A). Thus, whereas actor recognition and the form executing the motion may be computed by the ventral form pathway, the motion kinematics themselves may be computed by the motion pathway (Fig. 4A). When the ventral aspects of the ventral visual cortex are damaged (an example is conveyed in Fig. 4A by red-colored markings based on our ventral visual patients), snapshots leading to actor recognition might be disrupted. However, because the computations of kinematics mediated by the motion pathway are not significantly affected, movement perception is unaffected. Fig. 4B shows the prediction of this model for PLDs to illustrate our current findings. In addition to the supporting evidence from our patients, as well as from studies of individuals with developmental agnosia (51, 57), form-based and motion-based processing of body motion can be dissociated among healthy controls (50).
Parallel Processing Routes Supporting Biological Motion Perception?
The notion that biological motion perception might be computed in more than one way is also compatible with findings from a series of neuropsychological case studies. For example, LM, the “motion blind” patient with lesioned bilateral MT+/V5 (36), and AF, with severe damage to the dorsal cortex (35), are both able to recognize unmasked PLDs above chance. In addition, patients with brain damage or abnormal vision, such as patient MM (37), who recovered from long-term visual deprivation, or patient JW, who has widespread occipital damage following hypoxia (58), are both able to successfully recognize unmasked PLDs. Finally, the above-chance biological motion performance of patients with lesions that appear to invade early visual areas (35, 59) and are very different from the lesions of the patients examined in the present study, also seems to suggest multiple processing routes supporting biological motion perception.
Although all of these findings are consistent with an account in which biological motion perception may be achieved via multiple pathways, there is still some selectivity to the processing, and there are indications that biological motion perception is independent from other lower-level motion perception. For example, performance in biological motion and motion-coherence tasks are not correlated, as revealed in studies of patients (11), following congenital cataracts (60), or in healthy controls (43). Indeed, all of our ventral patients performed normally in the biological motion tasks but some have basic motion perception deficits [SM, CR, and EC are impaired in motion coherence and motion detection tasks (61)]. Similarly, patients AF and LM performed poorly on early motion tasks despite above-chance performance on biological motion (35, 36).
If biological motion recognition can be achieved via multiple pathways, this duplication might reflect the importance of this process to a multiplicity of abilities, such as social communication, motor learning, and theory of mind. Whether these pathways achieve movement recognition independently remains to be resolved. What is certain, though, is that the integrity of the ventral aspects of the ventral visual stream is not, in and of itself, critical for the normal perception of biological motion.
Conclusions
We have shown that biological motion perception can be achieved despite damage to the ventral aspect of the form visual pathway (e.g., fusiform body area, v-EBA). Although regions such as the pSTS and vPMC are critical for biological motion perception (11, 15, 16, 41, 62, 63), we speculate that the ventral regions of the form visual pathway are critical for recognizing the person performing the movement, but not for recognizing the motion being performed.
Methods
Patients with Ventral Visual Lesions.
Six premorbidly normal right-handed individuals who sustained brain damage to the right (n = 2), left (n = 3), or bilateral (n = 1) ventral visual cortex participated in the study. Following a lesion sustained in adulthood (except for CR, who was aged 16 y), all individuals reported visual perceptual problems and have well-established form-processing deficits. Table 1 summarizes the key demographics, neuropsychological descriptions, and detailed visual performance (including visual motion perception) of each patient; further details are available in Supporting Information and in earlier publications [SM (61, 64⇓⇓⇓⇓⇓–70), CR (61, 65⇓–67, 70), EL (61, 70⇓⇓⇓⇓–75), GB (61, 75), SH (70, 75), and EC (61)].
Experiment 1.
In this experiment, we used unmasked PLDs to assess recognition of biological motion and then measured perceptual thresholds using PLDs masked in noise points. To assess recognition of biological motion, unmasked PLD animations of actions (see below) were presented and participants were required to verbally describe the stimuli without having prior knowledge of what these would be. Each PLD animation looped until a coherent verbal description was given, after which the experimenter presented the next animation. To measure perceptual thresholds, on each trial two PLDs were presented simultaneously on the right and left sides of the screen, one containing a moving upright human figure performing one of seven actions (see Stimuli below) (Fig. 1A, Supporting Information, and Movies S1–S3) (11, 51, 76), and the other a spatially scrambled version of the same action. The side of the biological motion animation was randomly determined on each trial. Participants were instructed to identify which of the two displays contained the animation of the human movement [but did not have to identify the movement (i.e., jogging or walking), except in the action recognition phase; see below]. Both animations (intact and scrambled movements) were embedded in a number of noise points adaptively determined according to the participant’s performance (77). The task became more difficult as the number of noise points increased. Perceptual thresholds were determined based on the number of noise points with which a participant could perform at a predefined level of accuracy (82%). Stimuli and further procedures are fully described in Supporting Information and elsewhere (11, 51).
Participants.
All six ventral-lesion patients, tested in Pittsburgh, PA, and all healthy control participants (tested in Pittsburgh or in London) gave written informed consent to participate in the study and the experiments were approved by local ethics committees (Institutional Review Board, Carnegie Mellon University and University College London). All patients (except SM, who was tested at Carnegie Mellon University) and the older controls were tested at home for maximal convenience.
Procedures regarding the data collection from the nonventral brain-damaged patient control group (control group 3, see below) are provided elsewhere (11). Informed consent was obtained from these patients at the time of testing in accordance with guidelines of the University of California, San Diego and VA Northern California Health Care System Human Research Protections Programs. The findings from these patients have been published previously (11) and we simply adopted the de-identified data to serve as an additional benchmark against which to compare the ventral patients’ performance.
Healthy controls.
All healthy control participants had normal or corrected-to-normal vision, no history of neurological disorders, and were right-handed.
Control group 1 was the first neurologically normal control group that participated in this study and included 42 healthy adults, age-matched to the patients: 16 male control participants served as age-matched controls for CR (mean age 32.0 y ± 2.9 SD); 15 males served as age-matched controls for SM (mean age 35.2 y ± 3.3 SD, 11 of whom were also matched for CR); 13 females and 1 male (matched for SM as well) served as age-matched controls for EC (mean age 48.0 y ± 3.8 SD); and 14 females and 1 male served as age-matched controls for GB, EL, and SH (aged 50–70 y, mean 59.2 ± 6.1 SD, of whom 5 females were matched for EC as well).
Control group 2 was the second neurologically normal control group and included 13 healthy young controls (aged 20.4 y ± 1.1 SD).
Brain-damaged controls.
Control group 3 included 54 right-handed, brain-damaged patients (13 females, 41 males, aged 36.9–84.9 y) with focal, unilateral lesions (43 in the left hemisphere, 11 in the right hemisphere). From the 60 patients who had completed Exp. 1 in an earlier study (11), we selected for this control group only those for whom we could definitively determine that their ventral visual cortex was not affected by their lesion (as ascertained and confirmed by the lesion boundaries) as determined from computerized lesion reconstructions of the brain. The time between testing and patients’ cerebrovascular accident ranged from 6 mo to 22 y (mean of 6.5 y). Patients with diagnosed or suspected vision or hearing loss, dementia, head trauma, tumors, multiple infarcts, or prior psychiatric or neurological abnormalities were excluded from the sample. Motor and language impairments ranged from very mild to severe in the sample, but all patients were able to understand and carry out the task. None of the patients presented with spatial neglect or other attentional disorders.
Stimuli.
Briefly, biological motion PLD animations made of 12 white points on a black background depicting one of seven actions—walking, jogging, overarm throwing, underarm throwing (bowling), stepping up, high kicking into the air, and lower kicking—and lasting 0.8 s were presented and looped until a response was given.
For the perceptual threshold assessment, a matched spatially scrambled version was created for each animation so that the local motion of each point was preserved, without the global form (11, 51, 76).
In each trial of the perceptual threshold assessment, additional moving noise points were randomly superimposed on both PLDs (the biological motion and its scrambled counterpart) (9). The motion trajectory of each noise point that was added to the animations was equivalent to a motion trajectory of one of the animation’s points (randomly chosen), but starting at a random location.
Each animation subtended ∼4° × 6° (width × height) visual angle when viewed from 55 cm. The total area occupied by each PLD (comprising the animation plus the noise points) was ∼7° of visual angle in diameter. The two PLDs (biological motion and its scrambled counterpart) were displayed at ∼9° to the left and right of the center of the screen, their vertical centers horizontally aligned (Movie S1). Stimuli were presented and responses recorded using MATLAB (Mathworks) and the Psychophysics Toolbox v2.54 (78, 79).
Procedure.
In the first part of the experiment, we examined action recognition of unmasked PLDs. Participants were presented with seven different unmasked biological motion animations and were asked to verbally describe the stimuli on the screen. Each animation was displayed separately in the center of the screen without any masking noise points and looped until the verbal description given by the participant indicated that they were able to perceive the movement conveyed by the PLD animation. After that, the experimenter displayed the next animation.
Following the action-recognition phase, sensitivity to biological motion was assessed by measuring the number of noise dots that allowed successful discrimination (82%) of intact from scrambled animations when both are masked in noise points. On each trial, participants were required to report whether the intact PLD (“the person”) was on the right or left side of the screen by pressing the corresponding left or right key in a two-alternative forced-choice (2AFC) manner (see additional details below). The animations looped until a response was given. Although accuracy was the key dependent measure, response time of each trial was also recorded. EC, EL, the brain-damaged controls, and some of the older healthy controls, replied verbally or by pointing, after which the experimenter pressed the corresponding response button. Because the PLDs were presented on the two sides of the screen simultaneously, participants were not required to fixate at the center of the screen. After 16 practice trials with a predefined number of noise points, control participants completed 118 trials presented in two blocks separated by an optional rest period, whereas patients completed one block of 73 trials, with a rest period after 40 trials. Note that because the task was not timed, participants were able to take additional breaks at any time if needed.
To estimate perceptual thresholds, we varied the number of noise points in each trial to yield a psychometric measure of performance according to an efficient Bayesian adaptive procedure that uses the mean of the posterior probability density function [QUEST (77)]. The perceptual threshold was determined as the number of points at which a participant performed at 82% accuracy. For the healthy controls, who performed two blocks of trials, thresholds from the two blocks were averaged. Reaction time analysis was based on each participant’s average reaction time across the experiment.
In addition to between-group comparisons, which were based on Wilcoxon nonparametric rank-sum test (80), we also examined every patient’s performance individually. This was achieved by determining whether a patient’s performance (threshold or reaction time) was significantly different from that of a control group. The statistical evaluation was based on an established statistical procedure for comparing single cases to a control group (40), entailing a modified t test, significant performance differences roughly corresponding to more than two SDs from the controls’ mean performance.
Experiment 2.
The previous experiment assessed the participant’s ability to detect the PLD containing human movement when embedded in noise. This second experiment further examined the patients’ biological motion perception using a slightly different method, including a different and larger set of displays and a different criterion at which threshold is established.
Participants.
All six ventral-damaged patients, tested in Pittsburgh, PA, and 39 healthy neurologically normal controls (28 also participated in Exp. 1) gave informed consent to participate in the study and the experiment was approved by local ethics committees of Carnegie Mellon University (Institutional Review Board) and University College London. The controls were age-matched to each patient in the following way: 15 participants (14 male) served as age-matched controls for CR (mean age 32.5 y ± 4.1 SD); 15 participants (12 male) served as age-matched controls for SM (mean age 37.3 y ± 4.5 SD, 10 of whom were also matched for CR); 15 participants (12 female) served as age-matched controls for EC [mean age 46.5 y ± 4.5 SD, 5/1 controls also matched for SM/CR, respectively)]; and 11 participants (8 females) served as age-matched controls for GB, EL, and SH (mean 62.7 y ± 4.6 SD, 1 also matched for EC). All patients (but SM, who was tested at Carnegie Mellon University) and some of the controls were tested at home for maximal convenience.
Procedure.
We first assessed action recognition of 12 different PLDs (Supporting Information). Participants were asked to describe what they perceived so as to ensure that they were able to recognize the movements conveyed by PLDs. Thereafter, perceptual thresholds were established. A short practice comprising a few trials with a predefined number of noise points was completed, followed by the main experiment that measured the number of noise points a participant can tolerate and perform at 75% accuracy, using the same Bayesian estimation method as in Exp. 1 (77). In each trial, a single PLD (similar to those from Exp. 1) (Fig. 2A and Supporting Information) was presented at the center of the screen, either containing a movement of an upright human figure performing a movement (target), or a spatially scrambled version of it (nontarget). Animations (targets and nontargets) were masked in noise points (in the same manner as in Exp. 1), and the participant’s task was to decide whether the display contained a person (“target present”) or not (“target absent”) in a 2AFC manner using two predefined keys. Animations looped until a response was given. Visual feedback (green/red cross for correct/incorrect response, respectively) was provided after each response. As in Exp. 1, speeded responses were not required but we still recorded response times. Further experimental details are similar to those of Exp. 1; see details in Supporting Information.
Lesion and Anatomical Analysis.
Lesion delineation procedure.
For structural image-acquisition details, see Supporting Information. We followed a lesion delineation procedure that has been successfully used before (61). High-resolution anatomical images (EL, SM, and CR) were coregistered onto a T1 MNI canonical SPM image using SPM (www.fil.ion.ucl.ac.uk/spm), after which the lesions were traced manually in MRIcroN (www.cabiatl.com/mricro/mricro; see Supporting Information for tracing criteria) and saved as a binary image. For each patient, the coregistered anatomical images and the demarcated lesion were normalized into MNI space using the unified normalization segmentation of SPM. When only low-resolution anatomical images from clinical scans were available, lesions were traced manually onto the corresponding anatomical locations in an MNI canonical SPM image; to provide a consistent visualization of these patients’ lesions (Fig. 3), and to reach some approximate assessment of their lesion size (presented in Table 3), their noncontinuous traced lesions were then each manually interpolated to a continuous lesion in a conservative manner using MRIcroN (Fig. S1 and Supporting Information).
Comparing lesions to regions associated with biological motion processing.
To examine the lesion-behavior relationships, we sited each of our patients’ lesions relative to the anatomical locations of regions that are consistently activated across studies in response to biological motion stimuli, as determined by a recent meta-analysis (13). Specifically, we were interested in regions that are consistently activated in response to: (i) body movements [as reported in figure 1 and table 4 of Grosbras et al. (13)], (ii) static bodies [as reported in that study in figure 3 and table 7 of Grosbras et al. (13)], or (iii) human movements vs. nonhuman movements (as reported in table 8 of Grosbras et al. (13)]. Three image maps from that meta-analysis corresponding to these three contrasts-of-interest were included in our analysis. Each image map consisted of probability (P) values that corresponded to the activation likelihood estimation values reported in Grosbras et al. (13). These maps represent for each specific voxel the probability that a study will report significant activation in that voxel (for example with respect to one of our contrasts-of-interest: regions in which significant activation to human movements vs. nonhuman movements). The maps [as described in Grosbras et al. (13)] were thresholded at z = 3.2 (corresponding to P < 0.0005) and cluster size was 120 mm3 [see more details in Grosbras et al. (13)].
For each ventral patient, the three contrasts-of-interest from the meta-analysis along with the normalized brain and delineated lesion were loaded onto MRIcroN in four different colors as presented in Fig. 3 [lesion in red, body-movement regions (contrast 1) in blue, static-bodies regions (contrast 2) in yellow, and selective human movements (contrast 3) in green]. We then carefully examined whether the lesion invaded or overlapped any of the regions from each of the three contrasts-of-interest, according to anatomical location as well (e.g., ventral surface vs. lateral occipito-temporal regions). See also Table 3.
Acknowledgments
We thank all the patients and their families for their wonderful collaboration; Nina Dronkers and the VA Northern California Health Care System; Marie-Helene Grosbras for providing us the image maps from her study (13); and Christina Konen, Solmaz Shariat Torbaghan, Sabine Kastner, Tanja Kassuba, Adam Greenberg, Kate Fissell, Maxim Hammer, Mohamed Seghier, Ryan Egan, and John Pyles for their help. This study was supported by the Royal Society Grant TG102269 (to S.G.-D. and M.B.); Marie-Curie 236021 (to S.G.-D.); National Science Foundation Grant BCS0923763 and National Institute of Mental Health Grant 54246 (to M.B.); National Science Foundation BCS-CAREER-1151805 (to A.P.S.); and the Wellcome Trust (G.R.). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust 091593/Z/10/Z.
Footnotes
- ↵1To whom correspondence should be addressed. Email: shagido{at}gmail.com.
Author contributions: S.G.-D., A.P.S., G.R., and M.B. designed research; S.G.-D., A.P.S., and L.J.L. performed research; A.P.S. contributed new reagents/analytic tools; S.G.-D. and A.P.S. analyzed data; and S.G.-D., A.P.S., G.R., and M.B. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1414974112/-/DCSupplemental.
↵*Biological motion sensitivity following brain damage of any sort can be reduced relative to healthy controls (11), accounting for the findings that some ventral patients performed at the lower end of the healthy control distribution.
Freely available online through the PNAS open access option.
References
Ventral stream and biological motion perception
Sign up for Article Alerts
Jump to section
You May Also be Interested in
Several large or long-lived animals seem strangely resistant to developing cancer. Elucidating the reasons why could lead to promising cancer-fighting strategies in humans.
Image credit: Shutterstock.com/ronnybas frimages.
PNAS Profile of NAS member and biochemist Hao Wu
A study suggests a putative gene-expression hallmark common to monogamous male vertebrates of some species, namely cichlid fishes, dendrobatid frogs, passeroid songbirds, common voles, and deer mice, and identifies 24 candidate genes potentially associated with monogamy.
Image courtesy of Yusan Yang (University of Pittsburgh, Pittsburgh).
Physical and social well-being in old age are linked to self-assessments of life worth, and a spectrum of behavioral, economic, health, and social variables may influence whether aging individuals believe they are leading meaningful lives.
Image courtesy of Pixabay/MabelAmber.