Cortical representation of the constituent structure of sentences

Contributed by Stanislas Dehaene, December 15, 2010 (sent for review September 23, 2010)
January 11, 2011
108 (6) 2522-2527
Commentary
A closer look at the turtle's eyes
Andrea Moro

Abstract

Linguistic analyses suggest that sentences are not mere strings of words but possess a hierarchical structure with constituents nested inside each other. We used functional magnetic resonance imaging (fMRI) to search for the cerebral mechanisms of this theoretical construct. We hypothesized that the neural assembly that encodes a constituent grows with its size, which can be approximately indexed by the number of words it encompasses. We therefore searched for brain regions where activation increased parametrically with the size of linguistic constituents, in response to a visual stream always comprising 12 written words or pseudowords. The results isolated a network of left-hemispheric regions that could be dissociated into two major subsets. Inferior frontal and posterior temporal regions showed constituent size effects regardless of whether actual content words were present or were replaced by pseudowords (jabberwocky stimuli). This observation suggests that these areas operate autonomously of other language areas and can extract abstract syntactic frames based on function words and morphological information alone. On the other hand, regions in the temporal pole, anterior superior temporal sulcus and temporo-parietal junction showed constituent size effect only in the presence of lexico-semantic information, suggesting that they may encode semantic constituents. In several inferior frontal and superior temporal regions, activation was delayed in response to the largest constituent structures, suggesting that nested linguistic structures take increasingly longer time to be computed and that these delays can be measured with fMRI.
Most theories of syntax described sentences as tree-like hierarchical structures of nested phrases. These phrases, or constituents, constitute syntactic units that can be moved or replaced as a whole (for example, a noun phrase can be replaced by a pronoun). Support for the psychological reality of syntactic structures comes from studies showing that speakers tend to reuse the syntactic structure of recently heard sentences, a phenomenon known as syntactic priming (1, 2). Yet, although a variety of approaches using brain imaging methods have sought to characterize the regions implicated in syntactic processes (see, e.g., refs. 37), how the human brain computes and encodes syntactic structures remain largely open questions (811).
In this article, we propose an experimental paradigm to address this issue. We start with the intuitive hypothesis that a more complex cell assembly should be needed to encode a constituent of three elements, such as “Mary's father's car,” than a constituent of two elements such as “Mary's father.” Thus, neural activity might increase by a fixed amount each time a new node must be added to the constituent structure constructed on the basis of the preceding words. Arguments in favor of such an “accumulation” model comes from neurophysiological recordings in awake macaque monkeys during sequence-learning tasks, where specific neurons enter into a sustained mode of activity at different points of the sequence, thus creating a cumulative code for the sequences content (1214). In the human brain, such a pattern of neural discharges has not yet been observed during language processing, but a detailed neural network proposal for structure encoding that could produce such cumulative activity is the tensor product theory put forward by Paul Smolensky (15). This theory proposes that a tree structure can be represented neurally by the superposition of several neural assemblies, each consisting in the tensor product of two neural vectors, a “role” and a “filler.” The role vector codes for the syntactic role of a word (i.e., its position in the constituent structure), whereas the filler vector codes for the content being encoded at this position (i.e., the specific lexical item). Assuming that at least one of these vectors is encoded by a sparse neural code (a small proportion of active neurons, the majority being inactive), the superposition principle implies that mean activation increase by an approximately constant amount each time a new tensor product is added to the overall structure. Thus, this model leads to the expectation of a linear increase in activation with the number of nodes of the constituent structure.
In our experiment, we presented human participants with sequences of written words that afforded the construction of constituent structures of variable size (Table 1). All sequences had a fixed length of 12 items. Those belonging to the experimental condition “c01” were simply lists of unrelated words, controlled such that neighboring words could not be combined into larger constituents. In condition c02, sequences could be parsed as a series of constituents of size 2; in c03, the successive constituents comprised 3 words, and so on, all of the way to a full sentence of 12 words (c12).
Table 1.
The stimuli were 12 items long sequences obtained by concatenating constituents of fixed sizes extracted from natural or jabberwocky right-branching sentences
ConditionConstituent sizeExamples
c1212 wordsI believe that you should accept the proposal of your new associate
I tosieve that you should begept the tropufal of your tew viroate
c066 wordsthe mouse that eats our cheese two clients examine this nice couch
the couse that rits our treeve fow plients afomine this kice bloch
c044 wordsmayor of the city he hates this color they read their names
tuyor of the roty he futes this dator they gead their wames
c033 wordssolving a problem repair the ceiling he keeps reading will buy some
relging a grathem regair the fraping he meeps bouding will doy some
c022 wordslooking ahead important task who dies his dog few holes they write
troking ahead omirpant fran who mies his gog few biles they grite
c011 wordthing very tree where of watching copy tensed they states heart plus
thang very gree where of wurthing napy gunsed they otes blart trus
In jabberwocky, all content words were replaced with pseudowords (italics). Examples are only illustrative, because the original stimuli were in French.
Fig. 1 exemplifies how the accumulation model might apply to our experiment. For simplicity, we assume that each incoming word elicits a corresponding increase in activation when it can be integrated into a constituent structure, and that activation returns to baseline whenever word integration fails. Note that this model serves only as a first-order predictive tool. In high-level linguistic structures, the number of nodes is likely to differ from the number of words, and future research should take into account the precise syntactic structure afforded by the stimuli. At this stage, however, the number of words should serve as a useful approximation. When constituent size increases, the model shown in Fig. 1 predicts that not only the amplitude of the response, but also its phase, should increase. This prediction is a consequence of the assumption that activation accumulates and is thus stronger toward the end of a constituent than at the beginning. Here, following earlier work (1619), we show that fMRI can be sensitive to such temporal patterns and that several left-hemispheric regions follow the predicted increase in amplitude and phase.
Fig. 1.
Simulations of a simple model in which neural activity is assumed to increase by one unit each time a new word is incorporated into a constituent and to return to baseline as soon as a novel word cannot be incorporated into the preceding constituent. This model predicts an increase in both amplitude and phase of activation as a function of constituent size from condition c01 to c12.
As an additional experimental manipulation, we introduced a “jabberwocky” condition where content words were systematically replaced with pseudowords while maintaining all of the morphological markers and closed-class words needed to permit parsing (Table 1). This manipulation drastically reduces lexico-semantic content while keeping syntactic constituent structure intact, allowing to disentangle syntactic constituency effects from semantic coherence effects (7, 2022). It also introduces an important control for transition probabilities. In the materials with actual words, constituent size is confounded with the mean transition probability between words, which is lower at constituent boundaries than within constituents. In the jabberwocky condition, however, transition probability is close to zero, so a systematic variation in activation from c01 to c12 conditions would necessarily indicate a sensitivity to constituent structure. A first group of 20 subjects (the “normal-prose group”) read stimuli containing actual words, in which syntactic and semantic constituent structures were confounded, whereas a second group of 20 subjects received jabberwocky-type stimuli.

Results

Behavioral Results.

During fMRI, participants were required to attentively read the sequences. To ensure attention, they had to detect rare probe sentences that requested them to press a button and were warned that a word memory test would be presented at the end of each run. The participants correctly detected 96.8% of the probes. On the memory task, the percentages of correct responses were 60% (SEM = 1.5%; comparison with chance level of 50%; P < 0.01) in the normal-prose group and 59% (SEM = 1.5%; P < 0.01). This relatively low level of performance is likely due to the large number of words (420) presented in each run over a period of 8 min. Overall, these behavioral results suggest that participants were actively attending to all stimuli.

Increasing Activation with Constituent Size.

We first searched for brain regions showing increasing activation with constituent size. With normal prose, six brain regions showed a significant increase (voxel-based P < 0.05, corrected for multiple comparison, familywise error) (Fig. 2 and SI Appendix, Table S1). Four were located along the superior temporal sulcus (STS), and two in the left inferior gyrus pars triangularis (IFGtri) and pars orbitalis (IFGorb; small clusters were also seen at other sites such as left putamen). At the same stringent statistical threshold, the jabberwocky group showed increased activations only in IFGorb, which thus appears as a major region encoding constituent structure. With a lower statistical threshold (P < 0.001, voxel-based uncorrected), an effect of constituent size was also detected in the left posterior STS (pSTS) and IFGtri.
Fig. 2.
Brain regions showing a significant increase in activation with constituent size. (A) fMRI results from the normal-prose group who read sequences with actual French words (group analysis thresholded at T > 4.5, P < 0.05 FWE, spatial extent > 10). (B) Areas in blue show a significant constituent size effect in the jabberwocky group listening to delexicalized stimuli, whereas regions in red show a significant group by constituent size interaction (reflecting a stronger effect of constituent size in normal prose than in jabberwocky) (maps thresholded at T > 3.2, P < 0.001 uncorrected, spatial extent > 50). (C) Amplitude of activations across conditions in the six regions of interest (error bars represent ± 1 SEM). Conditions c01 to c12 are organized according to a logarithmic scale of constituent size, thus a line on this graph indicates a logarithmic increase of activation. The fitting lines are from a regression analysis including linear and logarithmic predictors.
A direct comparison between the two groups, using the group × constituent size interaction, revealed that the normal-prose group showed significantly larger increases in activation in the temporal pole (TP), the anterior STS (aSTS), and temporo-parietal junction (TPJ; Fig. 2 B and C and SI Appendix, Table S1). These regions were thus sensitive to constituent structure only when content words were present. Conversely, the other regions (IFGorb, IFGtri, pSTS) showed unchanged increases with constituent size even when the constituents were solely denoted by grammatical words and morphology, in the absence of content words. The computation of constituent structure in these regions therefore unfolds independently of the presence of a meaningful content and even when activation is absent in other anterior temporal lobe and TPJ regions.

Other Profiles of Activation.

We also searched the whole brain for patterns of activation other than an increase with constituent size. For instance, a region associated with memory and effort might show a greater activation to random lists of words than to sentences, thus showing a decrease with constituent size. A large network of regions showed such a behavior, including the precuneus, posterior cingulate, ventral medial prefrontal cortex, and bilateral inferior parietal lobule (SI Appendix, Fig. S3 and Table S2). None of these regions belonged to the classical perisylvian language areas. Instead, they are part of the classical “default mode” network (23) and, indeed, they were globally deactivated relative to rest and progressively deactivated more as constituent size increased.
Finally, we looked for regions exhibiting a quadratic, U-shape activation pattern as a function of constituent size. Such a pattern might arise, for instance, in putative regions responsive to syntactic violations. Arguably, a minimal amount of prior syntactic context is required before a syntactic violation can be detected or even defined. Therefore, one might postulate that violation detection is absent from both the lists of words (c01) and the normal sentences (c12). However, no such nonmonotonic response pattern was observed in a whole-brain search.

Nonlinearity of the Response with Constituent Size.

The simplified model presented in Fig. 1 predicts that the amplitude of the fMRI activation should increase linearly with the number of words integrated into a constituent. The parametric manipulation of constituent size in our experiment yielded quantitative data bearing on this point (Fig. 2C). Regression analyses in fact showed that, in all brain regions showing an increase, except TP, the increase in activation was logarithmic rather than linear as a function of constituent size. First, for each region of interest (ROI) and each group, we performed simple regressions on the group-averaged activation in the six conditions c01 to c12, either with constituent size itself or with log (constituent size). SI Appendix, Table S3 reports the coefficients of determination R2. The fit was higher for the log than for the linear model in all regions except the TP for the normal group and the IFGorb and IFGtri for the jabberwocky group.
As a better test of these effects, taking into interindividual variability, for each ROI and each group, we fitted two hierarchical models with individual-subject activation as the dependent variable, either constituent size (14, 6, 12) or its log as a regressor, and subject as a random factor. SI Appendix, Table S3 displays the ratio of the likelihoods of the log and the linear models. Values >1 favor the log model by the corresponding ratio. By this test, the log model was generally mildly or strongly favored over the linear model, except for the TP region in the normal group.
Overall, and given the absence of any interaction of group × constituent size in regions IFGorb and IFGtri, we conclude that the logarithmic model provides a better fit in all regions except the TP.

Analyses of Response Phases.

The second prediction of the accumulation model was that activation should be increasingly delayed for larger constituent sizes (Fig. 1). Fourier analysis of fMRI data (18, 19) allowed us to compute reliable estimates of activation phase in most regions (Fig. 3). For TP and TPJ, activations were significant only in conditions c06 and/or c12, thus preventing further quantitative analysis of how phase varied with conditions (SI Appendix, Methods).
Fig. 3.
Temporal profiles of activation (Left) and estimates of phase (Right) in four ROIs with distinct temporal patterns. (A) aSTS: response peaking early, increasing in amplitude and phase, with reduced response to jabberwocky. (B) IFGtri: slower response (larger phase), increasing in amplitude and phase for normal prose and jabberwocky. (C) pSTS: response similar to that of IFGtri, with a more modest increase in phase with constituent size. (D) IFGorb: slowest response (larger phase), increasing in amplitude but not phase, similarly for normal prose and jabberwocky. More detailed information and profiles for other ROIs are provided in section 4 of SI Appendix.
In aSTS, the phase increased significantly from conditions c01 to c12, by a delay of approximately half of the sentence duration [increase = 1.4 ± 0.9 s (95% confidence interval)]. A slower fMRI response, modestly but still significantly delayed by constituent size, was seen in IFGtri and pSTS (increases respectively of 1.0 ± 0.7 s and 0.6 ± 0.3 s). Finally, the slowest BOLD response was in IFGorb, and in this region the phase increase did not reach significance (0.6 ± 0.7 s) in spite of a large change in response amplitude.

Additional Control Condition with Nonconstituents.

As a further control, our experiment also comprised two additional conditions called “non-constituents” (nc3 and nc4) where we presented series of 3- and 4-word-long excerpts that were contiguous within sentences, as in the corresponding c03 and c04 conditions, but that straddled across constituent boundaries (e.g., “new car are very,” extracted from “the conditions for buying a new car are very attractive”). We reasoned that in a brain region sensitive only to possible sequences of words, these conditions should induce activation similar to the corresponding 3- and 4-word constituent conditions. In a region coding for constituents, however, these nonconstituents conditions should behave like the 1- and 2-word constituents conditions (c01, c02). In each ROI, we therefore compared the condition “nc3” to conditions c03 and c01, as well as condition nc4 to conditions c04 and c02, using Student's paired t tests. The results indicated that whenever the test could be meaningfully performed, activation in the nonconstituent condition was lower than in the corresponding constituent condition in pSTS, aSTS, IFGorb, and marginally in IFGtri (SI Appendix, Table S4). These results, together with the responsivity of pSTS, IFGorb, and IFGtri to jabberwocky constituents where transition probabilities are very low, suggest that these regions encode the constituent rather than sequential structure of word streams.

Discussion

In the visual system, parametric manipulations of the stimuli using both spatial scrambling (24) and temporal scrambling (25) have played an instrumental role in understanding how pictures are encoded. Here, we introduced a similar parametric approach to linguistic coding (see also refs. 26 and 27). Our research builds on previous studies that contrasted only two extreme conditions, sentences versus word lists, and already showed the involvement of the left anterior temporal regions during speech comprehension (e.g., refs. 7, 20, and 28). Our stimuli comprised a hierarchy of six successive conditions with increasing constituent sizes, always using simple right-branching constituents to minimize potentially confounding factors that are also known to contribute to parsing difficulty, such as movement (29). The results revealed a network of regions where activation increases parametrically in both amplitude and phase with the integration of words into constituents during on-line language comprehension. Moreover, the jabberwocky manipulation dissociated these language areas into two major subsets. The inferior frontal and posterior STS regions showed constituency effects regardless of whether actual content words were used. However, the TP, aSTS, and TPJ regions exhibited robust effects of constituent size only when actual words were presented.
The finding that the IFG is sensitive to syntactic structure in the absence of lexico-semantic information is consistent with studies of natural and artificial grammar learning that reported increased activation in Broca's area for phrase structure violations (3033). We observed a similar behavior in the pSTS, a region directly connected to the IFG (34). Both regions frequently coactivate in studies designed to isolate syntactic computations (e.g., refs. 9, 10, 29, 35, and 36). Indeed, the pSTS region uncovered in our study (with a peak at −51,−39, 3) is located very near a region labeled as left middle temporal gyrus (−52, −44, 4) and more activated, together with left IFG, when participants read a verb phrase or noun phrase (e.g., a smell, I smell) than the corresponding content word alone (10) (see also ref. 37). Altogether, the evidence is consistent with the notion that IFG and pSTS cooperate in assembling words into linguistic constituents.
The inferior frontal activations that we observed are centered on the pars triangularis and pars orbitalis (According to the atlas of (38), the IFGtri peak has 80% probabilty to be in BA45 and 20% in BA44; The IFGorb is located in BA 45/47). It has been argued (11, 39) that there is a posterior-to-anterior gradient in Broca's area, with syntactic processes involving the posterior part (BA 44; ref. 26) and the more anterior aspects dealing with semantic processes (40). Nevertheless several studies, like the present one, have reported effects of syntactic manipulations in BA 45 or 47 (e.g., refs. 22, 31, 4144). Most noteworthy is a recent study by Tyler et al. (45) where the activations to grammatically coherent sentences without meaning were remarkably similar to those that we report here. Moreover, contrasting hierarchically structured meaningless mathematical formulas to flat ones also reveals activations in BA 45 and 47 (46), suggesting that mathematics and language processing may share partially similar structure-building operations.
In a parametric design, Friederici et al. (26) manipulated three levels of complexity in the verb arguments of a dative structure and found activations in BA 44 that increased with ungrammaticality ratings, i.e., the opposite of the present result. They suggested that “BA 44 may be functionally related to language-internal processes involved in the reconstruction of a nondirect mapping between linear order and interpretation.” As noted by Tyler et al. (45), many fMRI studies “have used stimuli and tasks which make it difficult to separate the effects of the online construction of a syntactic representation from the contribution of variables that may not reflect the normal processes of comprehending language.” To mitigate this problem, we placed our subjects in a simpler situation of mere comprehension where the stimuli themselves limited the size of the linguistic representation that could be constructed and did not permit the deployment of complex processes of sentence repair (41, 47). Accordingly, in classical perisylvian areas, we observed only increasing activation amplitudes and delays with constituent size, presumably reflecting online construction with no or minimal involvement of additional repair processes.
In this study, regions TP and TPJ hardly responded to single words or even two-word constituents. Indeed, TPJ was deactivated by jabberwocky stimuli and activated above the resting-state level only for conditions c06 and c12 (Fig. 2C), i.e., complete or nearly complete sentences. This finding concurs with previous observations (48) suggesting that TPJ may be primarily engaged in the integration of words into a coherent discourse. Region aSTS, on the other hand, showed a more progressive increase in activation with constituent size, suggesting an involvement in processing smaller-sized constituents. In both regions, the observed nonresponsiveness to jabberwocky might indicate that these temporal regions rely on lexical information to build constituents, in agreement with “lexicalist” parsing models (49). For instance, the unification model postulates that a region in the posterior middle temporal gyrus, located just below our pSTS and TPJ regions, provides lexically stored syntactic frames that are later bound together into larger structures by inferior frontal cortex (50). This proposal, although plausible for normal prose, cannot readily explain the constituent size effect that we observed for jabberwocky in IFG and pSTS, an aspect of our results that underlines the relative independence of syntax from lexico-semantic features (51). The jabberwocky data show that the pSTS and inferior frontal cortex can compute syntactic frames solely on the basis of function words and syntactic category information provided by morphological features. Anterior temporal and TPJ regions could then bind these syntactic roles to lexico-semantic representations provided by other regions of the temporal lobe to form high-level representations of semantic constituent structure.
In most regions except the temporal pole, the increases in activation with constituent size were logarithmic rather than linear with the number of words inside the constituent. This finding is subject to several nonnecessarily incompatible interpretations. Because fMRI does not directly measure neural activation, it is possible that nonlinearities arise from the mapping between neural activation and the fMRI BOLD response. At short stimulus durations (<2 s), nonlinearities in BOLD signals have been reported (5254): The total activation to a temporally extended stimulus is smaller than the sum of the activations to its shorter-duration components. However, it remains debated whether such nonlinearities reflect neural or vascular effects, because MEG investigations have shown that they are largely imputable to neural adaptation (55). Furthermore, their relevance to the present situation is unclear because all of our experimental conditions involve the same total duration and same number of words. We therefore consider plausible the alternative hypothesis of a genuinely logarithmic variation in average neural activity with constituent size. In SI Appendix, we show that the model presented in Fig. 1 can be modified to account for this logarithmic profile (SI Appendix, Fig. S1). The sole assumption required is that the additional activation evoked by the incorporation of a word in a constituent is not a constant, but decreases as the constituent increases. Each word successively integrated into a constituent thus would have a progressively smaller impact on brain activation. Such an effect could arise from stimulus predictability: As a constituent builds up, the increasingly richer context of the preceding words makes the syntactic features of the incoming words increasingly predictable, a factor that might reduce their impact and therefore the activation that they induce (56). This proposal would mesh well with the general assumption, made outside of the linguistic domain, that the cortex acts as a predictive system that constantly attempts to improve the predictability of its inputs and that ascending brain activation reflects the residual prediction error or “surprisal” (57).
Note however that prediction error alone cannot explain the increasing activation patterns without the additional assumption that language areas accumulate and hold online the evidence from words forming constituents. If the activation merely reflected prediction error, then the less-probable sequences, with the shortest constituents, should have elicited stronger activations than the longest, most probable ones. Several prefrontal, parietal, and midline areas showed such a decrease in activation with constituent size, but none of them were located in the perisylvian language areas, and all were “deactivated,” showing lower activations for sentences than for rest. In classical language areas, increasingly complex constituent structures are primarily encoded by increasing levels of activation. As noted in Introduction, such a finding is compatible with a summation principle according to which, during the encoding of constituent structure, sparse neural codes for the successive nodes are superimposed within the same cortical area (15).
Our study also shows that fMRI presents enough sensitivity to detect not only amplitude changes, but also temporal delays associated with the construction of linguistic structures. The ability to estimate the precise phase of fMRI response, with a sensitivity of 200 ms or less, has been validated by both theory and past experimentation (1619). In the language domain, a temporal hierarchy is detectable among the perisylvian areas, with faster responses to spoken sentences in the superior temporal cortex near Heschl’s gyrus and increasingly slower activations as one moves away anteriorily or posteriorily along the STS and toward inferior frontal cortex (17). Furthermore, fMRI activation in these slower regions accelerates when the sentence is repeated (17). The present results add that, in aSTS, pSTS, and IFGtri, but not IFGorb, fMRI responses get increasingly slower for larger constituent sizes. The observed delays are generally quantitatively compatible with the prediction of the accumulation model (the linear model predicts a 0.56-s increase from conditions c01 to c12, and the revised logarithmic model a 0.40-s increase). Only in aSTS was the observed delay (1.4 ± 0.9 s) marginally larger than predicted This observation might indicate an additional contribution of slow operations that would be primarily deployed toward the end of a sentence (48). Another possibility is that processing in aSTS suddenly stops at the end of the first legal constituent, as soon as the incoming words cannot be integrated with the past context. This “sustained-activity model,” detailed in SI Appendix, predicts that fMRI phase should increase with a slope of half the total sentence duration, which is close to what we observed.
In conclusion, although much remains to be discovered as to how the structure of sentences is processed in the brain, these results provide quantitative parametric evidence on how linguistic constituents are encoded. Two constraints on any future model of the neural code for constituent structure are (i) a logarithmic increase in fMRI activation amplitude with constituent size; and (ii) a corresponding increase in activation delay. Future work should search for more precise determinants of syntactic complexity, using response amplitude and delay as direct markers of syntactic coding. Although number of words was used here as a proxy for constituent complexity, the ultimate predictors, which may vary across brain regions, are likely to be more abstract and language-invariant. Higher-temporal-resolution fMRI, combined with slower stimulus presentation, should probe why the temporal patterns of response differ across regions. Finally, equivalent studies using finer temporal methods such as magnetoencephalography (58) should provide detailed information as to the time course of constituent formation, ultimately paving the way to a detailed understanding of the neural code for syntax.

Materials

Participants.

Forty native French speakers (23 men and 17 women; age range = 18–37, mean = 24, SD = 6) took part in the experiment, which was approved by the regional ethical committee All gave written informed consent and received 80 euros for their participation.

Stimuli.

Normal prose.

The stimuli were sequences of 12 words generated by concatenating sentence fragments. Two hundred right-branching 12-word sentences (listed in SI Appendix) were first generated to create the condition c12 (full sentence). Thousands of fragments forming syntactic constituents were then extracted from this set. Fragments of size 2, 3, 4, and 6 were then randomly picked up from distinct sentences and concatenated to construct sequences of 12 words corresponding respectively to the experimental conditions c02, c03, c04, and c06 (see examples in Table 1). Single words from the original sentences were randomly concatenated to create sequences for the c01 condition. Different sequences were generated from the same pool of fragments for each participant. Individual participants were never exposed more than once to a given fragment, thus preventing the use of memory to mentally reconstruct the original sentences. The stimuli were manually verified and, if necessary, reshuffled to ensure that they did not contain, by chance, constituents of greater size than intended. For instance, in the c01 condition, two consecutive words never formed a plausible constituent. As a control over serial word order versus constituent structure, two additional conditions called non-constituents (nc3 and nc4) were included, consisting of sequences of 3- and 4-word-long excerpts that were contiguous within center-embedded sentences (listed in SI Appendix), but straddled across constituent boundaries (e.g., “new car are very”, extracted from “the conditions for buying a new car are very attractive”).

Jabberwocky.

For the jabberwocky condition, the experimental lists were modified by replacing all content words with pseudowords of the same length and morphological endings. As an example, the original sentence “Le passant examine le luxueux canap abandonn sur le bord du trottoir” became “Le chevant poisine le permeux relip pingann sur le gord du monchoir” (see Table 1 for examples in English). Function words were not modified (The average proportions of function words in the sequences were the following: c01, 27%; c02, 42%; c03, 36%; c04, 41%; c06, 43%, c12, 47%; nc3, 32%; nc4, 39%.).

Procedure.

The 12-word sequences were presented by using rapid serial visual presentation (300 ms per word). After a short familiarization block comprising 9 sequences, each subject took 5 fMRI runs containing 35 sequences each (interstimulus interval = 12 s). In addition to four stimuli belonging to each of eight experimental conditions c01, c02, c03, c04, c06, c12, nc3, and nc4, each run contained three probe sentences were inserted at random positions. These probe sentences explicitly requested the participants to press a button and served to ensure that they were paying attention to the stimuli.

Imaging and Data Analysis.

The imaging methods and data analysis procedures are described in SI Appendix.

Acknowledgments

We thank L. Rizzi, A. Bachrach, Biyu J. He, A. Moro., Y. L. Boureau, G. Dehaene-Lambertz, and A. Jobert for their support and helpful suggestions. This research was supported by the Pathfinder program of the European Union (Contract 12738) and was part of a general research program on functional neuroimaging of the human brain sponsored by the Atomic Energy Commission (Denis Le Bihan).

Supporting Information

Appendix (PDF)
Supporting Information

References

1
HP Branigan, MJ Pickering, AA Cleland, Syntactic co-ordination in dialogue. Cognition 75, B13–B25 (2000).
2
K Bock, GS Dell, F Chang, KH Onishi, Persistent structural priming from language comprehension to language production. Cognition 104, 437–458 (2007).
3
MA Just, PA Carpenter, TA Keller, WF Eddy, KR Thulborn, Brain activation modulated by sentence comprehension. Science 274, 114–116 (1996).
4
M Dapretto, SY Bookheimer, Form and content: Dissociating syntax and semantics in sentence comprehension. Neuron 24, 427–432 (1999).
5
U Noppeney, CJ Price, An FMRI study of syntactic adaptation. J Cogn Neurosci 16, 702–713 (2004).
6
AD Friederici, J Bahlmann, S Heim, RI Schubotz, A Anwander, The brain differentiates human and non-human grammars: Functional localization and structural connectivity. Proc Natl Acad Sci USA 103, 2458–2463 (2006).
7
C Humphries, JR Binder, DA Medler, E Liebenthal, Syntactic and semantic modulation of neural activity during auditory sentence comprehension. J Cogn Neurosci 18, 665–679 (2006).
8
Y Grodzinsky, AD Friederici, Neuroimaging of syntax and syntactic processing. Curr Opin Neurobiol 16, 240–246 (2006).
9
E Kaan, TY Swaab, The brain circuitry of syntactic comprehension. Trends Cogn Sci 6, 350–356 (2002).
10
LK Tyler, W Marslen-Wilson, Fronto-temporal brain systems supporting spoken language comprehension. Philos Trans R Soc Lond B Biol Sci 363, 1037–1054 (2008).
11
P Hagoort, On Broca, brain, and binding: A new framework. Trends Cogn Sci 9, 416–423 (2005).
12
P Barone, JP Joseph, Prefrontal cortex and spatial sequencing in macaque monkey. Exp Brain Res 78, 447–464 (1989).
13
Y Ninokura, H Mushiake, J Tanji, Representation of the temporal order of visual objects in the primate lateral prefrontal cortex. J Neurophysiol 89, 2868–2873 (2003).
14
Y Ninokura, H Mushiake, J Tanji, Integration of temporal order and object information in the monkey lateral prefrontal cortex. J Neurophysiol 91, 555–560 (2004).
15
P Smolensky, G Legendre The Harmonic Mind (MIT Press, Cambridge, MA) 1 (2006).
16
RS Menon, DC Luknowsky, JS Gati, Mental chronometry using latency-resolved functional MRI. Proc Natl Acad Sci USA 95, 10902–10907 (1998).
17
G Dehaene-Lambertz, et al., Functional organization of perisylvian activation during presentation of sentences in preverbal infants. Proc Natl Acad Sci USA 103, 14240–14245 (2006).
18
M Sigman, A Jobert, D Lebihan, S Dehaene, Parsing a sequence of brain activations at psychological times using fMRI. Neuroimage 35, 655–668 (2007).
19
M Sigman, S Dehaene, Brain mechanisms of serial and parallel processing during dual-task performance. J Neurosci 28, 7585–7598 (2008).
20
BM Mazoyer, et al., The cortical representation of speech. J Cogn Neurosci 5, 467–479 (1993).
21
AD Friederici, M Meyer, DY von Cramon, Auditory language comprehension: An event-related fMRI study on the processing of syntactic and lexical information. Brain Lang 75, 289–300 (2000).
22
B Röder, O Stock, H Neville, S Bien, F Rösler, Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: A functional magnetic resonance imaging study. Neuroimage 15, 1003–1014 (2002).
23
ME Raichle, et al., A default mode of brain function. Proc Natl Acad Sci USA 98, 676–682 (2001).
24
K Grill-Spector, et al., A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum Brain Mapp 6, 316–328 (1998).
25
U Hasson, E Yang, I Vallines, DJ Heeger, N Rubin, A hierarchy of temporal receptive windows in human cortex. J Neurosci 28, 2539–2550 (2008).
26
AD Friederici, CJ Fiebach, M Schlesewsky, ID Bornkessel, DY von Cramon, Processing linguistic complexity and grammaticality in the left frontal cortex. Cereb Cortex 16, 1709–1717 (2006).
27
A Santi, Y Grodzinsky, Working memory and syntax interact in Broca's area. Neuroimage 37, 8–17 (2007).
28
R Vandenberghe, AC Nobre, CJ Price, The response of left temporal cortex to sentences. J Cogn Neurosci 14, 550–560 (2002).
29
M Ben-Shachar, D Palti, Y Grodzinsky, Neural correlates of syntactic movement: Converging evidence from two fMRI experiments. Neuroimage 21, 1320–1336 (2004).
30
M Tettamanti, et al., Neural correlates for the acquisition of natural language syntax. Neuroimage 17, 700–709 (2002).
31
M Musso, et al., Broca's area and the language instinct. Nat Neurosci 6, 774–781 (2003).
32
J Bahlmann, RI Schubotz, AD Friederici, Hierarchical artificial grammar processing engages Broca's area. Neuroimage 42, 525–534 (2008).
33
J Bahlmann, TC Gunter, AD Friederici, Hierarchical and linear sequence processing: An electrophysiological exploration of two different grammar types. J Cogn Neurosci 18, 1829–1842 (2006).
34
D Saur, et al., Combining functional and anatomical connectivity reveals brain networks for auditory language comprehension. Neuroimage 49, 3187–3197 (2010).
35
M Ben-Shachar, T Hendler, I Kahn, D Ben-Bashat, Y Grodzinsky, The neural reality of syntactic transformations: Evidence from functional magnetic resonance imaging. Psychol Sci 14, 433–440 (2003).
36
JM Rodd, OA Longe, B Randall, LK Tyler, The functional organisation of the fronto-temporal language system: Evidence from syntactic and semantic ambiguity. Neuropsychologia 48, 1324–1335 (2010).
37
O Longe, B Randall, EA Stamatakis, LK Tyler, Grammatical categories in the brain: The role of morphological structure. Cereb Cortex 17, 1812–1820 (2007).
38
K Amunts, et al., Broca's region revisited: Cytoarchitecture and intersubject variability. J Comp Neurol 412, 319–341 (1999).
39
S Bookheimer, Functional MRI of language: New approaches to understanding the cortical organization of semantic processing. Annu Rev Neurosci 25, 151–188 (2002).
40
P Hagoort, L Hald, M Bastiaansen, KM Petersson, Integration of word meaning and world knowledge in language comprehension. Science 304, 438–441 (2004).
41
W Ni, et al., An event-related neuroimaging study distinguishing form and content in sentence processing. J Cogn Neurosci 12, 120–133 (2000).
42
A Moro, et al., Syntax and the brain: Disentangling grammar by selective anomalies. Neuroimage 13, 110–118 (2001).
43
NF Dronkers, DP Wilkins, RD Van Valin, BB Redfern, JJ Jaeger, Lesion analysis of the brain areas involved in language comprehension. Cognition 92, 145–177 (2004).
44
A Santi, Y Grodzinsky, fMRI adaptation dissociates syntactic complexity dimensions. Neuroimage 51, 1285–1293 (2010).
45
LK Tyler, et al., Preserving syntactic processing across the adult life span: The modulation of the frontotemporal language system in the context of age-related atrophy. Cereb Cortex 20, 352–364 (2010).
46
R Friedrich, AD Friederici, Mathematical logic in the human brain: Syntax. PLoS ONE 4, e5599 (2009).
47
M Meyer, AD Friederici, DY von Cramon, Neurocognition of auditory sentence comprehension: Event related fMRI reveals sensitivity to syntactic violations and task demands. Brain Res Cogn Brain Res 9, 19–33 (2000).
48
C Humphries, JR Binder, DA Medler, E Liebenthal, Time course of semantic processes during sentence comprehension: An fMRI study. Neuroimage 36, 924–932 (2007).
49
P Hagoort, How the brain solves the binding problem for language: A neurocomputational model of syntactic processing. Neuroimage 20, S18–S29 (2003).
50
TM Snijders, et al., Retrieval and unification of syntactic structure in sentence comprehension: An FMRI study using word-category ambiguity. Cereb Cortex 19, 1493–1503 (2009).
51
N Chomsky Syntactic Structures (Mouton, The Hague, 1957).
52
GM Boynton, SA Engel, GH Glover, DJ Heeger, Linear systems analysis of functional magnetic resonance imaging in human V1. J Neurosci 16, 4207–4221 (1996).
53
GM Boynton, EM Finney, Orientation-specific adaptation in human visual cortex. J Neurosci 23, 8781–8787 (2003).
54
RM Birn, ZS Saad, PA Bandettini, Spatial heterogeneity of the nonlinear dynamics in the FMRI BOLD response. Neuroimage 14, 817–826 (2001).
55
AS Tuan, RM Birn, PA Bandettini, GM Boynton, Differential transient meg and fMRI responses to visual stimulation onset rate. Int J Imaging Syst Technol 18, 17–28 (2008).
56
R Levy, Expectation-based syntactic comprehension. Cognition 106, 1126–1177 (2008).
57
SJ Kiebel, J Daunizeau, KJ Friston, A hierarchy of time-scales and the brain. PLOS Comput Biol 4, e1000209 (2008).
58
M Bastiaansen, L Magyari, P Hagoort, Syntactic unification operations are reflected in oscillatory dynamics during on-line sentence comprehension. J Cogn Neurosci 22, 1333–1347 (2010).

Information & Authors

Information

Published in

Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 108 | No. 6
February 8, 2011
PubMed: 21224415

Classifications

Submission history

Published online: January 11, 2011
Published in issue: February 8, 2011

Acknowledgments

We thank L. Rizzi, A. Bachrach, Biyu J. He, A. Moro., Y. L. Boureau, G. Dehaene-Lambertz, and A. Jobert for their support and helpful suggestions. This research was supported by the Pathfinder program of the European Union (Contract 12738) and was part of a general research program on functional neuroimaging of the human brain sponsored by the Atomic Energy Commission (Denis Le Bihan).

Notes

See Commentary on page 2177.

Authors

Affiliations

Christophe Pallier2,1 [email protected]
Institut National de la Santé et de la Recherche Médicale, Cognitive Neuroimaging Unit, U992,
Centre National de la Recherche Scientifique, F-75794 Paris, France;
NeuroSpin Center, Commissariat à l'Energie Atomique, Département des Sciences de la Vie/Institut d'Imagerie Biomédicale, and
Université Paris-Sud, F-91191 Gif/Yvette, France; and
Anne-Dominique Devauchelle1
Institut National de la Santé et de la Recherche Médicale, Cognitive Neuroimaging Unit, U992,
NeuroSpin Center, Commissariat à l'Energie Atomique, Département des Sciences de la Vie/Institut d'Imagerie Biomédicale, and
Université Paris-Sud, F-91191 Gif/Yvette, France; and
Stanislas Dehaene2 [email protected]
Institut National de la Santé et de la Recherche Médicale, Cognitive Neuroimaging Unit, U992,
NeuroSpin Center, Commissariat à l'Energie Atomique, Département des Sciences de la Vie/Institut d'Imagerie Biomédicale, and
Université Paris-Sud, F-91191 Gif/Yvette, France; and
Collège de France, F-75005 Paris, France

Notes

2
To whom correspondence may be addressed. E-mail: [email protected] or [email protected].
Author contributions: C.P., A.-D.D., and S.D. designed research; C.P. and A.-D.D. performed research; C.P., A.-D.D., and S.D. analyzed data; and C.P. and S.D. wrote the paper.
1
C.P. and A.-D.D. contributed equally to this work.

Competing Interests

The authors declare no conflict of interest.

Metrics & Citations

Metrics

Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.


Citation statements




Altmetrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

    Loading...

    View Options

    View options

    PDF format

    Download this article as a PDF file

    DOWNLOAD PDF

    Get Access

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Personal login Institutional Login

    Recommend to a librarian

    Recommend PNAS to a Librarian

    Purchase options

    Purchase this article to access the full text.

    Single Article Purchase

    Cortical representation of the constituent structure of sentences
    Proceedings of the National Academy of Sciences
    • Vol. 108
    • No. 6
    • pp. 2173-2629

    Media

    Figures

    Tables

    Other

    Share

    Share

    Share article link

    Share on social media