Neural representations of kinematic laws of motion: Evidence for action-perception coupling

Dayan et al. 10.1073/pnas.0710033104.

Supporting Information

Files in this Data Supplement:

SI Figure 4
SI Figure 5
SI Figure 6
SI Figure 7
SI Figure 8
SI Figure 9
SI Figure 10
SI Figure 11
SI Figure 12
SI Table 2
SI Table 3
SI Table 4
SI Table 5
SI Table 6
SI Text

Fig. 4. Analysis of selected regions of interest (ROIs) identified by contrasting all conditions with "rest." Shown are ROIs that were similarly activated by all three types of motion or equally by the 0 and 1/3 conditions (a): ROIs in the left hemisphere (b): ROIs in left and right posterior cerebellum. Error bars represent SEMs. ROIs included superior temporal gyrus (STG; Talairach coordinates: -56,-41,18), inferior occipital gyrus (IOG; -33,-94,-14), fusiform gyrus (FG; -23,-92,-12), right posterior cerebellum (Declive; 17,-80,-22), and left posterior cerebellum (Pyramis; -16,-82,-32).

Fig. 5. Areas of significant activation during perception of different types of motion. Each condition is compared with the two other conditions. Orange indicates stronger activity in response to motion complying with the 2/3 power law (1/3 condition) than in the other two conditions (2/3 and 0 conditions). Purple indicates stronger activity in the 2/3 condition than in the two other conditions. CCZ, caudal cingulate zone; IPL, inferior parietal lobule; MFG, middle frontal gyrus; MTG, middle temporal gyrus; PcG, postcentral gyrus; SFG, superior frontal gyrus; PMd, dorsal premotor cortex; PMv, vental premotor cortex; preSMA, presupplementary motor area; SPL, superior parietal lobule; STG, superior temporal gyrus.

Fig. 6. Average percent signal increase (compared to rest) for each type of motion in the functionally localized left and right MT/V5. Results are shown separately for the data sets of main experiment (a) and the second control experiment (b). In the data set of the main experiment, Talairach coordinates for left and right MT/V5 were -46,-69,3 and 49,-62,-3, respectively. In the data set of the second control experiment, the coordinates were -45,-75, 3, and 45,-67,-1, respectively. Error bars represent SEMs. In both data sets, there were no significant differences among the three types of motion or between the two hemispheres.

Fig. 7. Velocity profiles of the different experimental conditions. Shown, are the profiles of the conditions of the main experiment (1/3,0, and -1/3) and those of the second control experiment (1/3, 0, and 2/3).

Fig. 8. Orientations of the elliptical trajectories

Fig. 9. Significant clusters of activation obtained in pair-wise comparisons among the different types of motion. Orange indicates stronger activity in the 1/3 condition than in the -1/3 condition. Blue indicates stronger activity in the 1/3 condition than in the 0 condition. CCZ, caudal cingulate zone; Cd, caudate; Cu, cuneus; IFG, inferior frontal gyrus; IPL, inferior parietal lobule; LgG, lingual gyrus; MFG, middle frontal gyrus; middle occipital gyrus; PMv, ventral premotor; SFG, superior frontal gyrus; RCZ, rostral cingulate zone, SMA, supplementary motor area; SMG, supramarginal gyrus; STG, superior temporal gyrus; STS, superior temporal sulcus; vmPFC, ventromedial prefrontal cortex.

Fig. 10. Analysis of selected ROIs, identified with an fMRI finger-tapping task. ROIs, contralateral to movement of the right fingers included left M1/S1 (-42,-25,57), PMd (-39,-15,54), PMv (-57,-2,37), SMA (-5,-6,52), and preSMA (-8,3,52). ROIs, contralateral to movement of the left fingers included right M1/S1 (40,-25,61), PMd (44,-14,51), PMv (53,-1,35), SMA (5,-9,56) and preSMA (6,1,49). In each ROI Average Percent Signal Change was compared across the conditions of the main experiment (middle panel). and the second control experiment (bottom panel). Error bars represent SEMs. M1, primary motor cortex; S1, primary sensory cortex; PMd, dorsal premotor cortex; PMv, ventral premotor cortex; SMA, supplementary motor area. I indicates statistically significant difference at P < 0.05; II P < 0.01; III P < 0.005; IV P < 0.001; V P < 0.0005; VI P < 0.0001; VII P < 0.00005; VIII P < 0.00001.

Fig. 11. Areas of significant activation during perception of different types of motion, while no button-pressing task was performed. Each condition is compared with the two other conditions. Orange indicates stronger activity in response to motion that complies with the 2/3 power law than in the other two conditions. Blue indicates stronger activity in the 0 condition than in the two other conditions. No activations were obtained when the -1/3 condition was compared with the two other conditions. CCZ, caudal cingulate zone; IPL, inferior parietal lobule; MTG, middle temporal gyrus; SMA, supplementary motor area.

Fig. 12. Mean distance from the fixation cross averaged across subjects and compared among the three conditions of the second control experiment (a) and the main experiment (b). Error bars represent SEMs. Each histogram is accompanied by a scatter plot showing the locations of horizontal and vertical gaze locations relative to the position of the fixation cross.

SI Text

Finger Tapping

An additional finger-tapping session was performed inside the scanner to functionally localize primary motor, premotor, and supplementary motor regions. Eleven subjects participated in this experiment, including all subjects who participated in the second control experiment and six subjects who participated in the main experiment. In this part of the study, subjects were instructed to tap, as fast as possible, their index, middle, and ring fingers (numbered 1 to 3, respectively, during practice) against their thumb in response to visually presented instructions. Before the scanning session, subjects took part in a practice session during which it was verified that they could easily perform the task. Inside the scanner, the instructions first specified the hand with which the tapping sequences should be performed and the sequences to be performed, which were easy and repetitive (e.g., "333," "111," "222"). In addition, the instructions specified the periods during which subjects were instructed not to move their fingers (defining the "rest" condition). The tapping sequences were presented in blocks (lasting 15 seconds each) interleaved with blocks of "rest" (lasting 12 seconds each). The whole session, which included six tapping blocks (three for each hand), took 234 seconds and was performed once by each subject.

Results

Pairwise comparisons among the different conditions of the main experiment

The privileged status of motion complying with the 2/3 power law was also evident when additional pair-wise contrasts were performed (SI Table 5; SI Fig. 9). First the "1/3" condition was contrasted with the "-1/3" condition. Clusters of voxels significantly more active in the 1/3 condition were found in a wide network of brain areas listed in SI Table 5. Activation mostly lateralized to the left was observed in inferior frontal and superior frontal areas, primary motor, ventral premotor (PMv) and supplementary motor areas (SMA), rostral and caudal cingulate zones, inferior parietal areas, middle and superior temporal areas and in posterior cerebellum. In contrast, the -1/3 condition was not significantly more active than the 1/3 condition in any brain area.

Contrasting the 1/3 condition with the 0 condition yielded significantly stronger activation for the 1/3 condition in bilateral inferior frontal gyrus, left ventromedial prefrontal cortex, and right superior frontal gyrus, left PMv, left IPL, right STG, left middle occipital gyrus, right fusiform gyrus, bilateral caudate and left posterior cerebellum (SI Table 5; SI Fig. 9). The right parahippocampal gyrus was significantly more activated in the 0 condition than in the 1/3 condition.

All these comparisons demonstrate that motion complying with the 2/3 power law is associated with activation in areas involved in different motor functions and in the perception of biological motion and actions. These results also confirm that different types of motion are processed, in part, by different brain networks. These networks obviously share core components (see Fig. 3), but the processing of each type of motion is associated with a different pattern of activated brain areas.

Analysis of ROIs based on a finger-tapping task

As evident from the different analyses reported in the main text, processing of visual motion complying with the 2/3 power law is associated with selective activity in motor areas. To further demonstrate the involvement of motor areas in the visual processing of this law of motion, we analyzed selected ROIs identified with the aid of an fMRI finger tapping task (see SI Methods). ROIs were localized in bilateral primary motor (M1, extending to the primary sensory cortex, S1), premotor (PMv; PMd), and supplementary motor (SMA; preSMA) regions. All ROIs were localized contralaterally to the moving fingers. Subsequently, mean percent signal change for each experimental condition (i.e., for each type of motion) was calculated within the data set of main experiment (n = 14) and also within the dataset of the second control experiment (n = 5). The results are shown in SI Fig. 10. In the dataset of main experiment, the 1/3 condition induced stronger activation than at least one of the other two conditions (0 or -1/3) in bilateral PMd, PMv, SMA, and preSMA. In the data set of second control experiment (comprising the 0, 1/3, and 2/3 types of motion), the 1/3 condition induced higher activation bilaterally in M1, PMd, SMA, preSMA, and in left PMv. These results provide further compelling evidence for the dominance of the 1/3 condition in the motor system over the other types of motion.

Ruling out confounds related to button pressing

The paradigm we have used in the main experiment included a color-monitoring task (performed inside the scanner with an MRI compatible response box) to ensure similar levels of attention across all conditions. To minimize task-related influences, the number of color changes, and accordingly the number of button presses, was small (eight instantaneous color changes in each condition, summing up to 24 color changes overall) and was counterbalanced across the various conditions. Overall, subjects were able to detect these color changes reasonably well (87.5%, 89.6% and 91.7% for the 1/3, 0, and -1/3 conditions, respectively), and there were no differences among the three conditions in this task [c² = 0.093, not significant (ns)]. However, since our results robustly indicated that motor and motor-related brain regions are selectively activated by motion that follows the 2/3 power law, we wanted to further verify that our results were not contaminated by the button-pressing task. We thus conducted an additional control experiment that was identical to the main experiment, but for which no button presses were required. Six subjects participated in this control experiment, none of which took part in any of the other experiments. As in the main experiment, we contrasted each condition with the two other conditions to isolate the selective activation patterns associated with each type of motion. First, motion that follows the 2/3 power law was contrasted with the two other conditions (i.e., 1/3-[-1/3 + 0]). As before, significant clusters of activation were obtained in a wide network of brain areas which was similar to the network obtained in the main experiment. This network again included primary motor, premotor and supplementary motor foci (for a full list see SI Table 6 and SI Fig. 11), as well as foci in the parietal lobe (inferior parietal lobule and precuneus), and several clusters in temporal and occipital regions. Prefrontal activations were also obtained in this contrast but under a slightly more liberal statistical threshold. Similarly to the results of the main experiment, contrasting the -1/3 condition, with the two other conditions did not result in any significant clusters of activation. Finally, contrasting the 0 condition with the two other conditions, yielded one significant cluster in the left middle temporal gyrus. As in the previous analyses reported in the main text, the 0 condition selectively activated the right parahippocampal gyrus (either when contrasted with the two other conditions, or during pair-wise comparisons), but this was evident only with a more liberal statistical threshold.

It should be noted that the lack of any button-pressing task made it difficult for some of the subjects to remain attentive throughout the experiment. In particular, one subject reported feeling drowsy during various stages of the stimulation. Yet, multisubject results were almost identical with or without this subject's data.

Overall, the results of this control experiment indicate that the motor and motor-related activations obtained for motion that followed the 2/3 power law cannot be accounted for by the button-pressing task performed by the subjects.

Eye Movements

The purpose of the eye-tracking procedure was to ensure that subjects were indeed able to maintain fixation throughout the experimental sessions. Following removal of eye-blink artifacts, mean distance from the fixation cross was calculated for each of the three experimental conditions. A one-way repeated measure ANOVA confirmed that the mean distance from the fixation cross did not differ significantly among the three conditions of the second control experiment (F = 2.167, ns; see SI Fig. 12a). This procedure was repeated (SI Fig. 12b) with the stimuli presented in the main experiment (i.e., b values of 1/3, 0, and -1/3). Again, a one-way repeated measure ANOVA confirmed that subjects were able to maintain fixation throughout the presentation of the different types of motion (F = 0.34, ns). Altogether these results indicate that the differential activation obtained for the various types of motion cannot be explained simply by differences between eye movements among the different conditions.

1. Saygin AP, Wilson SM, Hagler DJ, Jr, Bates E, Sereno MI (2004) J Neurosci 24:6181-6188.

2. Koski L, Iacoboni M, Dubeau MC, Woods RP, Mazziotta JC (2003) J Neurophysiol 89:460-471.

3. Grezes J, Costes N, Decety J (1998) Cognitive Neuropsychology 15:553-582.

4. Santi A, Servos P, Vatikiotis-Bateson E, Kuratate T, Munhall K (2003) J Cogn Neurosci 15:800-809.

5. Nishitani N, Hari R (2000) Proc Natl Acad Sci USA 97:913-918.

6. Gerardin E, Sirigu A, Lehericy S, Poline JB, Gaymard B, Marsault C, Agid Y, Le Bihan D (2000) Cereb Cortex 10:1093-1104.

7. Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ (2001) Eur J Neurosci 13:400-404.

8. Grezes J, Armony JL, Rowe J, Passingham RE (2003) Neuroimage 18:928-937.

9. Hanakawa T, Immisch I, Toma K, Dimyan MA, Van Gelderen P, Hallett M (2003) J Neurophysiol 89:989-1002.

10. Picard N, Strick PL (2001) Curr Opin Neurobiol 11:663-672.

11. Wheaton KJ, Thompson JC, Syngeniotis A, Abbott DF, Puce A (2004) Neuroimage 22:277-288.

12. Pelphrey KA, Morris JP, Michelich CR, Allison T, McCarthy G (2005) Cereb Cortex 15:1866-1876.

13. Janzen G, van Turennout M (2004) Nat Neurosci 7:673-677.

14. Bonda E, Petrides M, Ostry D, Evans A (1996) J Neurosci 16:3737-3744.

15. Vaina LM, Solomon J, Chowdhury S, Sinha P, Belliveau JW (2001) Proc Natl Acad Sci USA 98:11656-11661.

16. Grafton ST, Arbib MA, Fadiga L, Rizzolatti G (1996) Exp Brain Res 112:103-111.

17. Lewis SM, Jerde TA, Tzagarakis C, Georgopoulos MA, Tsekos N, Amirikian B, Kim SG, Ugurbil K, Georgopoulos AP (2003) J Neurophysiol 90:3874-3887.