Imaging cortical dynamics of language processing with the event-related optical signal

  1. Chun-Yu Tse,
  2. Chia-Lin Lee,
  3. Jason Sullivan,
  4. Susan M. Garnsey,
  5. Gary S. Dell,
  6. Monica Fabiani, and
  7. Gabriele Gratton*
  1. Beckman Institute, University of Illinois at Urbana–Champaign, 405 North Mathews Avenue, Urbana, IL 61801-2325

  1. Communicated by William T. Greenough, University of Illinois at Urbana–Champaign, Urbana, IL, August 23, 2007 (received for review April 8, 2007)

 
Next Section

Abstract

Language processing involves the rapid interaction of multiple brain regions. The study of its neurophysiological bases would therefore benefit from neuroimaging techniques combining both good spatial and good temporal resolution. Here we use the event-related optical signal (EROS), a recently developed imaging method, to reveal rapid interactions between left superior/middle temporal cortices (S/MTC) and inferior frontal cortices (IFC) during the processing of semantically or syntactically anomalous sentences. Participants were presented with sentences of these types intermixed with nonanomalous control sentences and were required to judge their acceptability. ERPs were recorded simultaneously with EROS and showed the typical activities that are elicited when processing anomalous stimuli: the N400 and the P600 for semantic and syntactic anomalies, respectively. The EROS response to semantically anomalous words showed increased activity in the S/MTC (corresponding in time with the N400), followed by IFC activity. Syntactically anomalous words evoked a similar sequence, with a temporal-lobe EROS response (corresponding in time with the P600), followed by frontal activity. However, the S/MTC activity corresponding to a semantic anomaly was more ventral than that corresponding to a syntactic anomaly. These data suggest that activation related to anomaly processing in sentences proceeds from temporal to frontal brain regions for both semantic and syntactic anomalies. This first EROS study investigating language processing shows that EROS can be used to image rapid interactions across cortical areas.

Language processing evolves rapidly over time and involves multiple brain regions. Lesion studies have long identified both superior/middle temporal cortices (S/MTC) and inferior frontal cortices (IFC) in the left hemisphere (often referred to as Broca's and Wernicke's areas) as critical regions for language processing (1). However, the way in which these areas interact is subject to debate (2–4). Neuroimaging methods may provide information about the dynamics of these interactions.

Because most brain-imaging techniques have either good temporal or good spatial resolution, but not both, different aspects of language processing have been investigated with different methods. Because of their exquisite temporal resolution, event-related brain potentials (ERPs) have been the method of choice for studying the temporal aspects of language-related processes. The N400 and P600 components of the ERP have been found to be related to the processing of semantic (5) and syntactic (6) anomalies, respectively. However, because of the relatively low spatial resolution of ERPs, it is difficult to link the electrophysiological responses to brain regions, which is particularly problematic for language because the relevant areas may be in close proximity.

In contrast, functional magnetic resonance imaging (fMRI) provides more precise spatial information, and studies using this method also have implicated both the left IFC and S/MTC in both semantic and syntactic processing (4). A recent review of such studies by Hagoort (7) led to the proposal that the lexical properties of each perceived word are retrieved in S/MTC and then sent to the IFC, which integrates information across multiple words over time. Hagoort's model emphasizes a functional specialization within frontal regions, with more posterior and dorsal areas related to the integration of lexical-syntactic information and more anterior and ventral areas related to lexical-semantic integration. Hagoort further speculated that a similar pattern might extend to temporal regions where the syntactic and semantic properties of words are initially retrieved, although the evidence so far is sparse. However, fMRI alone does not have sufficient temporal resolution to determine the relative timing of different activities occurring during language processing, such as the flow of information from temporal to frontal regions, which typically unfolds over just fractions of seconds.

Analysis of the neural basis of sentence processing requires separating, in both space and time, the activity observed in each of the relevant regions. Several methods have been used to address this need, including magnetoencephalography (MEG) (8) and intracranial recordings (9). In the current study, we used a recently developed functional brain-imaging method, the event-related optical signal (EROS) (10). EROS identifies changes in the light-scattering properties of cortical tissue related to neuronal activity (i.e., fast optical signals, as opposed to optical correlates of hemodynamic signals) (11) by using near infrared light. With high spatial and temporal sampling, it is possible to obtain EROS images with spatial and temporal resolutions on the order of a few millimeters and milliseconds, respectively.

EROS has been used to study basic sensory processes in the visual (12), auditory (13), and somatosensory (14) modalities, as well as higher cognitive functions, such as preattentive change detection (13, 15, 16), attention modulation, and target detection (17). However, no EROS study of language processing has been conducted thus far. In this study, we show how EROS can identify two adjacent, but distinct, areas in S/MTC that are involved in the processing of semantic and syntactic anomalies and are associated with scalp-recorded ERP activity (N400 and P600, respectively). We also study the relative timing of frontal and temporal EROS activity associated with these anomalies.

Previous SectionNext Section

Results

Subjects classified most sentences correctly: semantically acceptable (94%), semantically unacceptable (95%), syntactically acceptable (88%), and syntactically unacceptable (86%). Only correct trials were included in the following analyses.

To test for the presence of N400 and P600 effects in the semantic and syntactic conditions, mean ERP amplitudes were computed on the difference waveforms in time windows from 200 to 500 ms (N400) and from 500 to 1,500 ms (P600). The mean values were tested against baseline by using one-sample t tests (one-tailed) on the Pz electrode. A significant increase in N400 (Fig. 1 Upper) (mean = −6.06 μV, SD = 2.46, t(15) = −9.88, P < 0.001, peaking at 420 ms) was found in the semantic condition, and a significant increase in P600 (Fig. 1 Lower), peaking at 860 ms, was found for the syntactic condition (mean = 5.58 μV, SD = 3.07, t(15) = 7.26, P < 0.001). There were no differences in either the P600 time window in the semantic conditions (mean = 0.73 μV, SD = 2.73, t(15) = 1.07, ns) or the N400 time window in the syntactic conditions (mean = 0.36 μV, SD = 1.40, t(15) = 1.02, ns).

Fig. 1.
View larger version:
Fig. 1.

Grand average ERP difference waveforms (unacceptable minus acceptable) at the Pz electrode, time-locked to critical word onset. (Upper) For the semantic condition, the N400 effect is shown in orange. (Lower) For the syntactic condition, the P600 effect is shown in green.


Statistical maps of EROS showed significant effects (i.e., increased phase delay for anomalous critical words) in left S/MTC and IFC for both the semantic and syntactic conditions. Table 1 summarizes the latency†, Talairach coordinates, peak Z scores, critical Z (with P < 0.05 corrected for multiple comparisons) (18), corresponding brain region, and Brodmann's area (BA) of the largest statistically significant peak EROS activity in each time interval. Activity at similar locations and adjacent time intervals was regarded as belonging to the same temporospatial cluster of activity. Fig. 2 a and b shows the most significant peak EROS response of each cluster for the semantic and syntactic conditions, respectively. The relative position of each peak response is shown on a left lateral view of the brain in Fig. 2 c (Upper, semantic condition; Lower, syntactic condition).

View this table:
Table 1.

Statistically significant peak EROS responses


Fig. 2.
View larger version:
Fig. 2.

Statistical maps of EROS data. (a and b) Axial slices with significant increases in optical signal for semantic and syntactic anomalies, respectively. Green boxes indicate the locations of the 4-cm cubes defining each ROI. The white cross within the ROI indicates the location of peak activity. The dark gray shading shows the area of cortex interrogated by the montage. Significant EROS activities are typically found at similar locations across more than five axial slices (1.25-mm thickness for each slice) and across adjacent time windows (25.6 ms). EROS activities at similar locations and time windows were regarded as temporospatial clusters (white labels above each slice). Only the slices with the most significant peak activity for each temporospatial cluster are shown here, except for the 819-ms syntactic response (whose cluster peaks at 844 ms). The latencies of the peak optical activities are indicated in yellow below each slice. S/MTC 384 ms in the semantic condition and S/MTC 819 ms in the syntactic condition (corresponding to the N400 and P600 ERP effects, respectively) are highlighted. (c) Relative positions of the peak activities of the temporospatial clusters plotted over left brain views. The semantic and syntactic conditions are shown in Upper and Lower, respectively. The latency of the EROS response is color coded.


As shown in Table 1 and Fig. 2, a pattern with S/MTC activation followed by IFC activation was elicited in both the syntactic and semantic conditions, beginning at 179 ms in the semantic condition and at 256 ms in the syntactic condition. In the semantic condition, this activation pattern recurred multiple times, suggesting oscillatory activity. Note that the two regions of interest (ROIs) were analyzed independently, and that the statistically significant peak for each individual ROI at each time point was selected, rather than the highest value across both ROIs.

Fig. 3 presents a direct comparison of the EROS responses in the semantic (small circles) and syntactic (small squares) conditions. This figure indicates that different areas in the temporal and frontal lobes were involved in the semantic and syntactic conditions up to ≈665 ms, whereas similar areas were involved in the semantic and syntactic conditions after 665 ms. A more ventral anterior/middle temporal (dark ellipse) region was involved in early semantic processing, whereas a more dorsal posterior temporal region (light ellipse) was involved in early processing in the syntactic condition. A more extensive area of the frontal cortex was involved in the semantic (dark square) than in the syntactic (light square) condition across time‡. In addition, two regions were activated in both conditions at similar times: a dorsal temporoparietal area (BA 40/41) activated between 665 and 844 ms (yellow rectangle) and an inferior/middle frontal area (BA46) activated ≈927 ms (red rectangle). These data suggest that, early on, distinct areas respond to semantic and syntactic anomalies, but that at longer latencies some regions are involved in responding to both anomaly types. A separate analysis conducted on midsentence semantic anomalies [see supporting information (SI) Text and SI Figs. 5 and 6] showed a similar, albeit weaker, temporospatial pattern of activity elicited by semantic violations, suggesting that these differences are not due to word position in the sentence.

Fig. 3.
View larger version:
Fig. 3.

Semantic (small circles) and syntactic (small squares) peak responses for each temporospatial cluster overlaid on the same template brain. The latency of the response is color-coded. Regions activated by semantic anomalies are dark-gray-shaded; those activated by syntactic anomalies are light-gray-shaded (both ellipses, temporal; rectangle, frontal). Convergence regions are indicated by the dashed rectangles.


To specifically investigate the relationship between EROS and ERP signals, stepwise multiple regressions§ were conducted to determine which of the observed optical effects, if any, predicted the amplitudes of the N400 and P600 ERP components across subjects. EROS activity elicited in the semantic condition in both S/MTC (at 179 ms, β = 0.76, P < 0.001; at 384 ms, β = −2.27, P < 0.005) and IFC (at 512 ms, β = 1.37, P < 0.05) predicted the N400 ERP effect [R 2 = 0.884, F(3, 7) = 17.88, P < 0.001] (Table 2). The P600 effect was predicted by the EROS in S/MTC (at 819 ms, β = 0.46, P < 0.005; at 914 ms, β = −0.52, P < 0.05) elicited in the syntactic condition [R 2 = 0.608, F(2, 12) = 9.31, P < 0.005] (Table 2). No statistically significant relations were found by using optical signals from the syntactic conditions to predict the N400 effect or from the semantic condition to predict the P600 effect (Table 2). Taken together, these analyses suggest a double dissociation in the EROS–ERP prediction pattern. Only EROS signals elicited in the semantic condition predict the N400 effect, and only EROS signals elicited in the syntactic condition predict the P600 effect.

Among all significant predictors, the EROS activity elicited in the semantic condition in S/MTC at 384 ms was the only predictor that increased with larger N400s (the sign for this predictor is negative because the N400 becomes more negative as its amplitude increases), whereas the EROS activity elicited in the syntactic condition in S/MTC at 819 ms was the only predictor that increased with larger P600s. Thus, once the effects of other significant predictors were removed, the S/MTC EROS activity at 384 ms increased as the size of the N400 effect (peaking 36 ms later at 420 ms) increased, and the S/MTC EROS activity at 819 ms increased as the size of the P600 effect (peaking 41 ms later at 860 ms) increased. Interestingly, the other significant predictors, which were not as close in time to the N400 or P600 peaks, were opposite in sign from the ones just described (i.e., positive for the N400 and negative for the P600). Thus, when the predictors increased, N400 or P600 amplitude decreased, suggesting that they may reflect neural activities modulating the N400 or P600 effects.

View this table:
Table 2.

Stepwise multiple regression analyses showing a double-dissociation pattern in the relationship between EROS and ERP effects


To determine whether the location of the optical activity in S/MTC differed between semantic and syntactic conditions, the location of the peak EROS activity corresponding to the N400 effect (semantic S/MTC at 384 ms) was compared with the location of the peak EROS activity corresponding to the P600 effect (syntactic S/MTC at 819 ms) (Fig. 4 a and b) by using a jackknife procedure and multivariate t test. The peak locations for the two conditions differed (Hotelling's T 2 = 11.02, P < 0.05). The peak of the S/MTC optical signal obtained in the semantic condition was 17 mm inferior to that obtained in the syntactic condition [t(15) = 3.21, P < 0.005], whereas there was no difference along the anterior–posterior dimension [t(15) = 0.59, ns]. Thus, the combined temporal and spatial resolutions of EROS allowed us to separate the optical signals corresponding to the N400 and P600 in the S/MTC region, although they were <2 cm apart. The relative location of these signals in the semantic and syntactic conditions is consistent with Hagoort's (7) speculation that regions handling lexical-syntactic information may be more dorsal than those handling lexical-semantic information in S/MTC.

Fig. 4.
View larger version:
Fig. 4.

Test of location of ERP-predicting EROS responses. (a) Statistical maps of EROS results corresponding to the N400 and P600 ERP effects. The green boxes indicate the ROI for the STC, and the red boxes indicate the region shown in b. (b) Enlargement of the region within the red box in a, in which the areas showing the effects for semantic and syntactic conditions are overlaid. Blue, EROS activity in the semantic condition; red, EROS activity in the syntactic condition.


At latencies exceeding 665 ms, some common areas were activated by syntactic and semantic anomalies. Within both the superior temporal gyrus (latency, 844 ms) and inferior frontal gyrus (IFG) (latency, 972 ms), there were no reliable differences between the locations of activity for the syntactic and semantic contrasts. Some of this late activity is probably related to the behavioral response, which is the same for the two kinds of anomalies.

Previous SectionNext Section

Discussion

EROS revealed the dynamic activation of partially overlapping distributed networks associated with the processing of semantic and syntactic anomalies. The location of these temporal-frontal networks is consistent with previous fMRI studies (19, 20–25), which suggest that frontal regions (BA44, BA45, and BA47), temporal regions (BA37, BA38, BA40, BA41, BA21, and BA22), and the precentral gyrus (BA6) are involved in language processing. This posterior–anterior flow of activity for semantic anomaly processing also has been observed by using MEG (8).

We interpret these activation patterns in terms of a model proposed by Hagoort (7), in which comprehension involves the retrieval of lexical information in temporal regions and the subsequent integration of that information across words in frontal regions. In addition, we bring in a proposal by Federmeier (26) that left-hemisphere language areas work in a top–down predictive mode. Specifically, we suggest that the integration process in IFC includes the generation of predictions about upcoming words, which are communicated to temporal memory areas. These predictions concern both semantic features (e.g., that the upcoming word will designate something edible) and syntactic features (e.g., that it will be a plural verb). We speculate that it is the failure of the input to match these predictions that produces the earliest EROS effects in temporal regions (200–400 ms) in different S/MTC locations for semantic and syntactic information. Further, our syntactic anomalies, being simple number or case mismatches, may be easier to integrate in frontal regions because they require correcting only a single feature, so there is little differential frontal response to the syntactic anomalies, aside from the late frontal activation we ascribed to the behavioral response (972 ms). In contrast, the attempt to integrate the semantic anomalies leads to interpretations that cannot be easily corrected (e.g., children eating floors), and hence there is an anomaly-sensitive response in IFC (e.g., 512 ms), as well as a need to further consult lexical memory in temporal cortex (e.g., S/MTC activation at 665 ms) possibly to check for accuracy of the retrieval process. The result is oscillation between the anterior and posterior language areas in response to semantic anomaly.

Our data suggest spatial segregation of lexical-semantic and lexical-syntactic processing in the temporal lobes, just as previous fMRI studies (25, 27) have found spatial segregation of semantic and syntactic processing in the frontal lobes. Interestingly, the pattern in temporal regions, with the response to syntactic anomaly more posterior and dorsal than that to syntactic anomaly, is consistent with Hagoort's (7) speculation that the pattern observed in frontal regions may extend to temporal regions as well.

With regard to frontal cortex, the same region of IFG is activated between 972 and 998 ms by the two anomaly types, possibly reflecting participants' ultimate judgment that the sentence is incorrect in some way. However, a similar (although slightly more inferior) (25, 27) region is already activated at a latency of 563 ms in the semantic condition, suggesting that the frontal cortex responds to a semantic anomaly earlier than it does to syntactic anomaly, at least for the kinds of sentences we used. The current study also suggests that the more extensive involvement of frontal regions in the semantic conditions observed in fMRI studies (24) may reflect the superimposition of adjacent frontal activations across time.

In addition to IFG, however, the data show other regions involved in the processing of both syntactic and semantic anomaly: Dorsal superior temporal gyrus, which is proposed to be involved in syntactic processing, also is involved in processing semantic anomalies at 665 and 844 ms poststimulus (i.e., within the range of the ERP P600 effect). The hypothesis outlined earlier about IFC sending predictions about semantic and syntactic features of upcoming words to S/MTC is one possible explanation for this region's involvement in both the semantic and syntactic conditions.

In this study, we established a correspondence between the N400 and P600 ERP effects and EROS activity in S/MTC. Previous EROS studies (13, 15, 17) have shown temporal correspondence between optical and ERP signals, but in the current study we also showed that EROS amplitude is correlated with ERP amplitude. The optical data accounted for a high proportion of variance in the ERP components (88% for the N400, 61% for the P600). Further, the ERP–EROS correspondence was specific to each condition, as revealed by the double dissociation in the prediction pattern. This finding rules out accidental factors, such as differences across subjects in skull thickness, which may lead to a spurious relationship between ERP and EROS, and points instead to stimulus-specific neuronal factors. However, we are not attempting to locate the generators of N400 and P600 by using EROS, which would require the use of full-head montages for both measures and computation of forward models for the ERP data. Further, there may not be complete correspondence of the ERP and EROS effects because they may measure only partially overlapping brain activity.

The syntactic and semantic anomalies occurred in different positions in our sentences, with the semantic anomalies and their corresponding controls always in final position and the syntactic anomalies earlier. In this first EROS study of language processing, manipulations were chosen to maximize the N400 and P600 ERP effects as well as the EROS effects. Software constraints dictated an eight-word maximum sentence length, so we placed semantically anomalous words in sentence-final position to build up sufficient semantic constraint. The syntactic violations, however, did not lend themselves to sentence-final position. To estimate how much of the difference between conditions was due to word position, we analyzed the EROS elicited by midsentence semantic anomalies in 48 filler sentences. The same temporospatial pattern of S/MTC and IFC activity was found for both sentence-final and midsentence anomalies, but both the EROS and corresponding ERP effects were considerably weaker in midsentence positions. This finding suggests that the differences between semantic and syntactic anomalies are not entirely due to word position (see SI Text). However, the more prominent temporal-frontal oscillatory pattern observed for the sentence-final semantic anomalies, which we interpret as the activity of an integration-prediction network in response to an anomaly, may be partly due to their position, perhaps because of sentence wrap-up processes, the need to make the decision leading to the behavioral response, or both. Similarly, we cannot make any claims about the relative speed of semantic and syntactic processing. The apparent difference in the earliest significant effect of semantic versus syntactic deviance may be due to factors such as the degree of contextual constraint or magnitude of the deviance.

We recorded activity only from the left hemisphere. However, other investigators (21) have suggested that the right S/MTC and IFC also are involved in sentence comprehension. The unexplained variance in the EROS–ERP prediction pattern may be related in part to right-hemisphere contributions to language processing, as well as to other brain regions not measured in this study.

In summary, the current study used EROS to observe the temporal and spatial dynamics of activity in S/MTC and IFC in processing semantic and syntactic anomalies, confirming previous observations of the importance of these regions in sentence processing. The data suggest a flow of activation from posterior (temporal) to anterior (frontal) regions, as well as the involvement of distinct areas in the temporal cortex in the processing of semantic and syntactic anomaly and overlapping areas in the frontal cortex. Importantly, spatially and temporally distinct components of EROS were related to the N400 and P600 ERP components elicited by semantic and syntactic anomalies. Finally, this study demonstrates the feasibility of using EROS to study language processing.

Previous SectionNext Section

Methods

Sixteen right-handed native English speakers (11 females, ages 18–30) participated in this study after providing informed consent. Procedures were approved by the campus Institutional Review Board. Participants were presented with sentences of five to eight words, word by word at the center of a computer screen, and instructed to make a yes/no decision about whether each sentence was both well formed and sensible (henceforth called “acceptable”) after its completion (see SI Fig. 7). Each participant saw 864 sentences (528 acceptable, 336 unacceptable). Among the 336 unacceptable sentences, 144 became semantically anomalous at the final (critical) word (e.g., “The hungry child ate the floor”), 96 exhibited grammatical violations of subject–verb agreement (e.g., “If work isn't done, it pile…”), and 48 had grammatically incorrect pronoun case (e.g., “My mother promised to buy I…”). For the ungrammatical sentences, the critical words were at positions 4–8. To prevent expectations that semantic anomalies could only occur in the final position, 48 filler sentences contained semantically anomalous words in positions 3–7. Of the 528 acceptable sentences, 288 were the matched controls for the semantic (144), syntactic subject–verb agreement (96), and pronoun case (48) anomaly conditions. The remaining 240 were included so that participants would expect that most sentences would be correct, which they generally are in normal comprehension.

In the analyses presented here, ungrammatical sentences with subject–verb and pronoun–case disagreement were combined to maximize statistical power, and only those semantically anomalous sentences whose critical word was the final one were included because they were the only semantic anomalies with matched control versions (see SI Text for an analysis of midsentence semantic anomalies). There were two complementary lists of stimuli (i.e., acceptable sentences in list 1 were the correct versions of unacceptable sentences in list 2, and vice versa), and each participant saw only one list. The mean lengths and frequencies of the critical words in acceptable and unacceptable sentences did not differ. EROS and ERPs were recorded simultaneously. The semantic and syntactic conditions described throughout this paper refer to the brain activity elicited by critical words in the acceptable sentences subtracted from that for the corresponding words in the unacceptable sentences, time-locked to the eliciting word onset.

ERP Recordings.

The EEG was recorded with gold electrodes at four scalp locations based on the 10/20 system (Fz, Cz, Pz, and right mastoid) referenced to the left mastoid (with an average mastoid reference computed offline). Four electrodes, one above and one below the right eye and two at the outer canthi of each eye, were used for vertical and horizontal EOG recording. Electrode impedance was <5 kΩ. The EEG was filtered online by using a 0.01- to 30-Hz bandpass sampled at 100 Hz, filtered offline with a 0.1- to 20-Hz bandpass, segmented by using 1,500-ms epochs time-locked to the onsets of critical words (with 200-ms prestimulus baselines), and averaged according to the stimulus condition. Ocular artifacts were corrected (28), and epochs containing other EEG artifacts were removed from the analysis.

EROS Recordings.

EROS data were recorded using a frequency-domain oximeter (Imagent; ISS, Inc., Champaign, IL). Near-infrared light (830 nm) from laser diodes modulated at 220 MHz was conducted to the participant's head by optical fibers (sources). Light that scattered through the head was carried by detector optical fibers to photo-multiplier tubes used for the measurements. The photo-multiplier tubes were modulated at 220.00625 MHz generating a 6,250-Hz heterodyning frequency (i.e., cross-correlation frequency). The output current was Fast Fourier-transformed, and relative phase-delay measures (in degrees) were computed.

The light source and detector fibers were held in place by using a custom-built head-mount system (15, 16). Two montages (i.e., two sets of light source/detector configurations) (SI Fig. 8) were used to interrogate the left S/MTC and IFC. Each montage comprised 128 source-detector pairs (8 detectors, each multiplexed with 16 of 24 light sources). The sampling frequency was 39.0625 Hz (i.e., it took 25.6 ms to cycle through the 16 multiplexed channels). Montage order was counterbalanced across subjects.

For coregistration with individual subjects' anatomy, the locations of sources and detectors, the nasion and preauricular points, and another 185 random points over the face and scalp of each person were digitized with a Fastrak 3Space 3D digitizer (Colchester, VT). A T1-weighted brain anatomical (MPRAGE) MRI was obtained for each participant by using a 3T Siemens Magnetom Allegra Headscanner (New York, NY). The nasion and preauricular points were marked by vitamin E pills in the MRI scan for coregistration of the functional optical data with the structural MRI. The coregistration was based on a successive application of fiducial and scalp-fitting methods that have been shown to reduce the coregistration error to <5 mm (29). The individually coregistered data were then Talairach-transformed (30) before statistical analyses.

Optical data were preprocessed by correcting for phase wrapping, normalizing, pulse correcting (31), filtering with a 0.01- to 8-Hz bandpass filter, and then averaging for each channel, time point, condition, and subject by using a 200-ms prestimulus baseline. Channels with phase standard deviations >210 ps and/or a source-detector distance outside the 15- to 75-mm range (≈16.5%) were excluded from the analysis. The same criteria were applied to all conditions.

Special-purpose software (Opt-3D) (32) was used to analyze the averaged data. With the assumption that the light path in the brain is similar to that obtained in a homogenous medium, the optical signal for a given voxel was calculated as the mean value of the channels overlapping at that particular voxel (33). The montage was designed to generate a high degree of overlap between the volumes interrogated by different channels (>50% overlap between adjacent channels). This approach was useful for the spatial reconstruction process, resulting in an improved signal-to-noise ratio and spatial resolution, allowing us to examine focal EROS responses. T statistics for the phase data were calculated at the group level for each voxel and then converted to Z scores. Statistical maps of the optical signal for each data point were generated by 3D reconstruction of the Z scores on a template brain in Talairach space, with an 8-mm spatial filter according to the location information from the coregistration procedure. The ROIs for the statistical analysis of brain activation in S/MTC and IFC were based on previous brain-imaging studies (34–39). The S/MTC and IFC ROIs were tested independently in the analyses, and only the statistically significant peak activities are reported here (more details regarding EROS recording and analysis methods can be found in refs. 10 and 40).

Previous SectionNext Section

Acknowledgments

We thank K. Low and E. Maclin for assistance in data collection and analysis, S. Coulson and K. McConnell for help with sentence materials, T. B. Penney and A. Schirmer for helpful comments. This work was supported by National Institute of Mental Health Grant R01 MH080182 (to G.G.) and Beckman Graduate Fellowships (to C.-Y.T. and J.S.).

Previous SectionNext Section

Footnotes

  • *To whom correspondence should be addressed. E-mail: grattong{at}uiuc.edu

  • Author contributions: C.-Y.T., J.S., S.M.G., G.S.D., M.F., and G.G. designed research; C.-Y.T. and C.-L.L. performed research; M.F. and G.G. contributed new reagents/analytic tools; C.-Y.T. analyzed data; and C.-Y.T. wrote the paper.

  • The authors declare no conflict of interest.

  • This article contains supporting information online at www.pnas.org/cgi/content/full/0707901104/DC1.

  • ↵ † The latencies given in Table 1 refer to the beginning of each 25.6-ms sampling interval.

  • ↵ ‡ Alternatively, the IFG regions activated in the semantic and syntactic conditions may be of similar extent. However, because the semantic activity is more extended in time, it is more likely for different points to become active at different times.

  • ↵ § The statically significant optical effects reported in Table 1 were entered into the model as candidate predictors. Because EROS responses extended across time, responses at adjacent time points are highly correlated with both each other and the ERP effect. If all EROS responses were entered simultaneously into the regression model, there would be a high degree of multicollinearity resulting in unstable parameter estimation. This result was avoided by using a stepwise procedure, in which only the sets of EROS effects that best correlated with the ERP effects were selected. The results were reported in the final models (Table 2). The predictors selected into the final model by the stepwise procedure encompassed the duration of the N400 and P600 effects.

  • Abbreviations:
    BA,
    Broadmann's area;
    EROS,
    event-related optical signal;
    ERP,
    event-related potential;
    fMRI,
    functional magnetic resonance imaging;
    IFC,
    inferior frontal cortex;
    IFG,
    inferior frontal gyrus;
    MEG,
    magnetoencephalography;
    ROI,
    region of interest;
    S/MTC,
    superior/middle temporal cortex.

  • Freely available online through the PNAS open access option.

Previous Section
 

References

  1. ↵
    1. Goodglass H
    (1993) Understanding Aphasia (Academic, San Diego).
  2. ↵
    1. Bushell CM
    (1996) Brain Lang 55:264–288.
    CrossRefMedlineISI
  3. ↵
    1. Caramazza A,
    2. Capitani E,
    3. Rey A,
    4. Berndt RS
    (2001) Brain Lang 76:158–184.
    CrossRefMedlineISI
  4. ↵
    1. Kaan E,
    2. Swaab TY
    (2002) Trends Cognit Sci 6:350–356.
    CrossRefMedlineISI
  5. ↵
    1. Kutas M,
    2. Hillyard SA
    (1980) Science 207:203–205.
    Abstract/FREE Full Text
  6. ↵
    1. Osterhout L,
    2. Holcomb PJ
    (1992) J Mem Lang 31:785–806.
    CrossRefISI
  7. ↵
    1. Hagoort P
    (2005) Trends Cognit Sci 9:416–423.
    MedlineISI
  8. ↵
    1. Halgren E,
    2. Dhond RP,
    3. Christensen N,
    4. Van Petten C,
    5. Marinkovic K,
    6. Lewine JD,
    7. Dale AM
    (2002) NeuroImage 17:1101–1116.
    CrossRefMedlineISI
  9. ↵
    1. McCarthy G,
    2. Nobre AC,
    3. Bentin S,
    4. Spencer DD
    (1995) J Neurosci 15:1080–1089.
    Abstract
  10. ↵
    1. Gratton G,
    2. Fabiani M
    1. Parasuraman R,
    2. Rizzo M
    (2007) in Neuroergonomics: The Brain at Work, eds Parasuraman R, Rizzo M (Oxford Univ Press, Cambridge, MA), pp 65–81.
  11. ↵
    1. Rector DM,
    2. Carter KM,
    3. Volegov PL,
    4. George JS
    (2005) NeuroImage 26:619–627.
    CrossRefMedlineISI
  12. ↵
    1. Gratton G,
    2. Corballis PM,
    3. Cho E,
    4. Fabiani M,
    5. Hood DC
    (1995) Psychophysiology 32:505–509.
    CrossRefMedlineISI
  13. ↵
    1. Rinne T,
    2. Gratton G,
    3. Fabiani M,
    4. Cowan N,
    5. Maclin E,
    6. Stinard A,
    7. Sinkkonen J,
    8. Alho K,
    9. Naatanen R
    (1999) NeuroImage 10:620–624.
    CrossRefMedlineISI
  14. ↵
    1. Maclin EL,
    2. Low KA,
    3. Sable JJ,
    4. Fabiani M,
    5. Gratton G
    (2004) NeuroImage 21:1798–1804.
    CrossRefMedlineISI
  15. ↵
    1. Tse C-Y,
    2. Tien KR,
    3. Penney TB
    (2006) NeuroImage 29:314–320.
    MedlineISI
  16. ↵
    1. Tse C-Y,
    2. Penney TB
    (2007) IEEE Eng Med Biol Mag 26(4):52–58.
    CrossRefMedlineISI
  17. ↵
    1. Low KA,
    2. Leaver E,
    3. Kramer AF,
    4. Fabiani M,
    5. Gratton G
    (2006) Psychophysiology 43:127–136.
    CrossRefMedlineISI
  18. ↵
    1. Poline JB,
    2. Worsley KJ,
    3. Evans AC,
    4. Friston KJ
    (1997) NeuroImage 5:83–96.
    CrossRefMedlineISI
  19. ↵
    1. Ruschemeyer SA,
    2. Fiebach CJ,
    3. Kempe V,
    4. Friederici AD
    (2005) Hum Brain Mapp 25:266–286.
    CrossRefMedlineISI
  20. ↵
    1. Friederici AD,
    2. Ruschemeyer SA,
    3. Hahne A,
    4. Fiebach CJ
    (2003) Cereb Cortex 13:170–177.
    Abstract/FREE Full Text
  21. ↵
    1. Just MA,
    2. Carpenter PA,
    3. Keller TA,
    4. Eddy WF,
    5. Thulborn KR
    (1996) Science 274:114–116.
    Abstract/FREE Full Text
  22. ↵
    1. Friederici AD
    (2002) Trends Cognit Sci 6:78–84.
    CrossRefMedlineISI
  23. ↵
    1. Bookheimer SY
    1. Bigler ED
    (1996) in Neuroimaging 1: Basic Science, ed Bigler ED (Plenum, New York), pp 97–117.
  24. ↵
    1. Bookheimer S
    (2002) Annu Rev Neurosci 25:151–188.
    CrossRefMedlineISI
  25. ↵
    1. Hagoort P,
    2. Hald L,
    3. Bastiaansen M,
    4. Petersson KM
    (2004) Science 304:438–441.
    Abstract/FREE Full Text
  26. ↵
    1. Federmeier KD
    (2007) Psychophysiology 44:491–505.
    CrossRefMedlineISI
  27. ↵
    1. Petersson KM,
    2. Forkstam C,
    3. Ingvar M
    (2004) Cognit Sci 28:383–407.
    CrossRef
  28. ↵
    1. Gratton G,
    2. Coles MGH,
    3. Donchin E
    (1983) Electroencephal Clin Neurophysiol 55:468–484.
    CrossRefMedlineISI
  29. ↵
    1. Whalen C,
    2. Maclin EL,
    3. Fabiani M,
    4. Gratton G
    (2007) Hum Brain Mapp, in press.
  30. ↵
    1. Talairach J,
    2. Tournoux P
    (1988) A Co-Planar Stereotaxic Atlas of a Human Brain (Thieme, Stuttgart).
  31. ↵
    1. Gratton G,
    2. Corballis PM
    (1995) Psychophysiology 32:292–299.
    CrossRefMedlineISI
  32. ↵
    1. Gratton G
    (2000) Psychophysiology 37:S44–S44.
  33. ↵
    1. Wolf U,
    2. Wolf M,
    3. Toronov V,
    4. Michalos A,
    5. Paunescu LA,
    6. Gratton E
    (2000) OSA Meeting in Optical Spectroscopy and Imaging and Photon Migration (Miami).
  34. ↵
    1. Newman AJ,
    2. Pancheva R,
    3. Ozawa K,
    4. Neville HJ,
    5. Ullman MT
    (2001) J Psycholing Res 30:339–364.
    CrossRefMedlineISI
  35. ↵
    1. Kiehl KA,
    2. Laurens KR,
    3. Liddle PF
    (2002) NeuroImage 17:842–850.
    CrossRefMedlineISI
  36. ↵
    1. Caplan D,
    2. Alpert N,
    3. Waters G