Cross-language differences in the brain network subserving intelligible speech

Subjects.

Thirty native Chinese speakers and 26 native English speakers participated in the current research. The Chinese and English groups were matched on subject sex, age, and handedness. Native Chinese speakers participated in this study as paid volunteers (15 males and 15 females; aged between 21 and 28 y; mean age, 24.2 y). Native English speakers participated in an experiment that was reported in a previous study by Leff et al. (21), and the brain imaging data were reanalyzed in the current study. All participants were right-handed, with normal hearing and normal or corrected-to-normal vision, with Mandarin Chinese or English as their first language, and had no neurological or psychiatric history. Written informed consent was obtained from each participant before scanning, and the study was conducted under the approval of the Institutional Review Board of Beijing MRI Center for Brain Research.

Stimuli.

The experimental paradigm was adopted from a previous study about the cortical dynamics of intelligible English speech (21), whereas both intelligible and unintelligible auditory stimuli were presented to subjects to make a gender judgment of the speakers. Half of the intelligible stimuli in English were idiomatic word pairs such as “cloud nine,” and the other half were reordered word pairs of idioms (e.g., “mint nine”). To create their unintelligible counterparts, the stimuli of intelligible stimuli were time-reversed because this method removed the intelligibility of the forward speech while preserving the acoustic and voice identity information (5, 21). The Chinese stimuli were designed with consideration of matching the acoustic and psycholinguistic characteristics of the idiomatic word pairs between Chinese and English. Words in the English stimuli were mainly disyllabic. We selected Chinese intelligible stimuli with a three-, four-, or five-syllable length that matched in duration with the English word pairs. Half of the intelligible stimuli in Chinese were idiomatic words with three to five characters such as 和事佬 (he2 shi4 lao3, means “peacemaker,” the letters represent the official Romanization of standard Chinese, that is, Pinyin, whereas the number indicates the corresponding tone), 画龙点睛 (hua4 long2 dian3 jing1, means “finishing touch”), and the other half consisted of words from two unrelated idioms [e.g., 鸿门户 (hong2 men2 hu4), 恶贯好龙 (e4 guan4 hao4 long2); 恶贯好龙 was the combination reordered from the first two words of the idiom 恶贯满盈, (e4 guan4 mang3 ying2) and the last two words of the idiom 叶公好龙 (ye4 gong1 hao4 long2)]. All intelligible stimuli were recorded digitally in a soundproof studio using Adobe Audition CS4 software. A male and a female speaker (both native speakers) produced all of the stimuli twice. As for Chinese unintelligible stimuli, the unintelligible counterparts of Chinese stimuli were also time-reversed of the intelligible stimuli that removed the intelligibility but preserved the acoustic and voice identity information. There were 84 auditory stimuli (half by the male speaker and half by the female speaker) for each of the three stimulus types, and no stimulus was repeated. All stimuli were edited for quality and length and amplified so that there was no difference of loudness between speakers or between the idioms and reordered idioms.

Comparison and Psycholinguistic Analysis of Stimuli Between Groups.

We first calculated the duration of all auditory stimuli in the English and Chinese groups (English: 677–1,080 ms; Chinese: 730–1,007 ms). A two-sample t test revealed no significant difference on the duration between English and Chinese groups. A similar procedure was also conducted to examine the stimulus loudness across languages and types of stimuli, and no significant difference was found. Both the Chinese and English idioms were phrases established by use and referred to a certain meaning. The reordered idioms were created by combining two unrelated idiom. Therefore, the reordered idioms in both English and Chinese were meaningless but still recognizable syllable by syllable.

Because there are no comprehensive databases recording key psycholinguistic variables for either English or Chinese idioms, two psycholinguists who use English or Chinese as their native language independently rated the idiomatic stimuli in their native language (either English or Chinese) on a binary scale, guided by the Cronk criteria (33), splitting the stimuli into the following four uneven groups: (i) high familiarity, high literalness; (ii) high familiarity, low literalness; (iii) low familiarity, high literalness; and (iv) low familiarity, low literalness. Percentage proportions of stimuli that fell into each group were then calculated (Chinese: English): (i) 35%:33%; (ii) 39%:31%; (iii) 8%:11%; and (iv) 18%:35%. Nonparametric statistical tests of all four groups (independent samples Mann–Whitney U test) revealed no significant differences across the two languages (all P = 1.0).

Procedure.

In the experiment, subjects were required to make a gender judgment of the speaker for each stimulus, and they were not informed of the types and contents of the stimuli in advance. Therefore, the task was orthogonal to the effect of interest. The auditory stimuli were arranged in blocks by stimulus types with an alterable ratio of male and female speakers (2:5, 3:4, 4:3, or 5:2). All auditory stimuli only were presented once. There were 12 blocks for each type of stimuli, and they were evenly distributed in four scanning sessions. Each block was preceded by a preparation cue for 3,150 ms, which was followed by seven trials. On each trial, an auditory stimulus was presented for 1,180 ms, followed by a response cue for 2,420 ms and then a fixation cross for 450 ms. The blocks were separated by a symbol “∼” of 9,450 ms presented at the center of the screen. Participants pressed one of the two buttons with their right index or middle finger to indicate their gender judgment of the speaker after the response cue immediately. The order of the blocks and the assignment of response buttons were counterbalanced across participants.

MRI Data Acquisition.

Scanning of native Chinese speakers was performed on a 3-T Siemens MRI scanner in our laboratory in Beijing, China. Scanning of native English speakers was performed by one of our coauthors in London, UK, and was reported in a previous study (21). In both scannings, functional images were acquired using a gradient-echo echo-planar imaging (EPI) pulse sequence (TR/TE/θ = 2.08 s/30 ms/90°, 64 × 64 × 35 matrix with 3 × 3 × 3-mm³ spatial resolution). The visual displays were presented through a Sinorad LCD projector (Shenzhen Sinorad Medical Electronics) onto a rear-projection screen located over the subject’s head, viewed with an angled mirror positioning on the head coil. The auditory stimuli were presented binaurally using a pair of home-made MRI compatible headphones that provided 25–30 dB/SPL attenuation of the scanner noise. In consideration of the scanner noise generated by EPI sequence, participants were required to adjust sound volume of the sample auditory stimuli to a clear-and-comfortable level during a pilot EPI scan before the experiment. After the volume adjustment, participants were instructed to pay attention to the experimental task. Four sessions of functional task scanning were acquired while participants performed the gender judgment of the auditory stimuli. Each session started with a blank screen for 10 s and was followed by nine blocks of auditory stimuli, which lasted 378.56 s in total. After the functional scanning, T1-weighted 3D structural images were also obtained (TR/TE = 2.6 s/3.02 ms, 1 × 1 × 1-mm³ spatial resolution).

Behavioral Data Analysis.

Response accuracy and reaction time were recorded for each type of stimuli. Because our interests in this research were focused on the processing of speech intelligibility, paired t tests were conducted to compare the differences in behavioral performance between intelligible speech (averaging across idioms and rearranged idioms) and unintelligible speech (time-reversed idioms). The results of behavioral data in English participant were reported in a previous paper (21). The mean response accuracy of Chinese participants for gender judgment across all auditory stimuli was 98.5%. A paired t test of response accuracy showed that subjects made significantly more accurate gender judgments for intelligible speech than for time-reversed speech (t = 3.612, P < 0.01; intelligible: 99.0 ± 1.8%; time-reversed: 98.0 ± 2.3%), whereas there was no significant difference in reaction time between the two speech types (overall reaction time, 1,731 ± 176 ms).

fMRI Data Analysis.

Statistical parametric mapping software (SPM8; Wellcome Trust Centre for Neuroimaging) was used for imaging data processing and analysis. Native English speakers participated in the experiment in a previous study, and the fMRI data were reanalyzed in the current research with an identical procedure. The EPI images were realigned to the first scan to correct head motion. Then, the mean image produced during the process of realignment, and the realigned images were coregistered to the high-resolution T1 anatomical image. All images were spatially normalized to standard MNI space. The normalized EPI images were then spatially smoothed using an isotropic Gaussian kernel with a full-width at half maximum (FWHM) parameter of 8 mm. The functional imaging data were modeled using a boxcar function with head motion parameters as unrelated regressors. Parameter estimates for each condition (three types of stimuli) were calculated from a general linear model (GLM) based on the hemodynamic response function with overall grand mean scaling. Whole-brain statistical parametric mapping analyses were performed, and contrasts were then defined to reveal brain areas specifically involved in processing intelligible stimuli (idioms and reordered idioms) and that of unintelligible stimuli (time-reversed idioms). The t-contrast images were generated for comparison at each voxel. Statistical tests were first assessed in individual subjects, and random effect analyses were then conducted based on statistical parameter maps from each individual subject to allow population inference. A one-sample t test was applied to determine group-level activation for intelligibility effect. Moreover, parameter estimates of signal intensity for processing intelligible and unintelligible speech were extracted from regions of interests (ROIs) and compared between the Chinese group and the English group using ANOVA. The ROIs in the left anterior temporal cortex (region A), left posterior temporal cortex (region P), and left inferior frontal cortex (region F) were defined as spheres with 6 mm diameter centered at the local maxima voxel around the center of ROI landmarks (group-level peak voxel) observed in the intelligibility effect in both groups. Because no significant activation could be found in the right anterior temporal cortex for the English group even with a more liberal threshold (P < 0.01 voxel-level uncorrected), the landmark ROI definition of this region (R) in the English group was identical to the Chinese group.

DCM Analysis.

After we identified the involvement of several brain regions in the processing of speech intelligibility, we conducted a DCM analysis (20) to examine the effective connectivity among these brain regions. In DCM analysis, differential equations, dx/dt=(A+uB)x+Cu, were used to model the cortical dynamics of the neuronal populations in brain ROIs, which describe how the current state of one neuronal populations causes dynamics in another through synaptic connections that are intrinsic and fixed and how these interactions change under the external influence of experimental manipulations or the influence of endogenous brain activity (34). Here, x is a vector representing the neural state of all brain regions that are in consideration, and u is a vector representing all external input. Matrix A represents the strength of fixed connection between the brain regions that are in consideration; in other words, the strength of connection when no external input exists. Because it has been shown that an anatomical connection exists between each pair of the three brain regions (7, 18), it was assumed that a reciprocal fixed connection between each pair of the brain regions existed: in other words, all parameters in matrix A would be set to a nonzero value (34). Matrix B represents the strength of modulation of the connections by external inputs, i.e., the experimental manipulation (in the current research, this was the processing of the speech varying in intelligibility). Matrix C represents the direct influence to the brain regions by the external inputs that were generally considered to be sensory stimuli, such as the auditory input in this experiment. Furthermore, it is also widely assumed that modulatory stimuli, such as intelligibility of speech in this experiment, can only indirectly influence brain regions, i.e., only some related parameters in matrix B would be nonzero, whereas all relative parameters in matrix C would be zero. In our model-space design, the supposition above was applied and all nonzero parameters in the matrixes were assumed Gaussian.

ROIs Selection and Time Series Extraction.

In the current research, we were particularly interested in the brain regions in the temporal and frontal cortex that showed significant involvement in the processing of speech intelligibility. The coordinates of the peak voxel in the clusters identified in the group-level random effect analysis of the intelligibility effect (comparing the neural activity during listening to intelligible speech with unintelligible speech, i.e., the time-reversed speech) with P < 0.05 family-wise error (FWE) corrected threshold were used to serve as a landmark for the individual ROIs. For each Chinese subject, ROIs were defined as 6-mm spherical volumes centered at the peak activation voxel of intelligibility effect in the regions of our interests by searching voxels that survived at least the P < 0.05 threshold around the landmarks (revealed in group-level analysis) within the 8-mm radius distance and within the same anatomical regions. Because the exact location of activation varied for each subject, this procedure ensured comparability of models and the extracted time series across subjects by applying both functional and anatomical constraints (35, 36). Given these criteria, we were able to define ROIs and extract time series for a three-region model (P-A-F) in 22 of 30 Chinese subjects and for a four-region model (P-A-F-R) in 18 of 30 Chinese subjects, and the remaining subjects were excluded from the respective DCM analysis in whom the activation of at least one brain region failed to meet the criteria. An identical procedure was performed on English subjects and was reported in the previous study (21). For each ROI in an individual subject, the time series was extracted and computed as the first eigenvector across all suprathreshold voxels.

DCM Specification.

We formed a specific model space that contained the whole set of alternative models that were anatomically and functionally plausible and used Bayesian methods to estimate the model parameters. In model selection, considering the large number of the models to compare, we used the family-level inference and Bayesian model averaging within families on the model space (22) instead of the “single best model” selection strategy. This procedure provided inference about model parameters that could minimize the assumption bias on the model structure.

To establish the basic neural dynamic network of processing intelligible speech in Chinese and English, we first specified the model space that consisted of three shared left hemisphere brain regions: A, P, and F. For input “auditory”, treated as a sensory input, seven alternate ways of how input auditory could enter the system were contained into the model space: via pSTG only, via aSTG only, via pSTG and aSTG, via IFG only, via pSTG and IFG, via aSTG and IFG, and via all three brain regions. Intelligibility was treated as a modulatory input; it may modulate any combination of six directed connections among the three regions, resulting in 63 (i.e., 2⁶ − 1) different model structures with the null model excluded from the analysis. Therefore, 63 different structures of modulatory input intelligibility crossed with the 7 different structures of sensory input auditory resulted in a total of 441 models to be compared in the three-region modeling analysis.

To investigate the specific neural dynamic for the tonal language (Chinese), we then specified the four-region model space that included the right STP into the DCM analysis together with the previously identified left pSTG, aSTG, and IFG. Based on our prior results, we assumed that inputs were into the left pSTG only. Modulatory input intelligibility was assumed to modulate any combination of the 12 directed connections among the four regions, with the null model excluded from the analysis. Thus, a total of 4,095 (i.e., 2¹² – 1) different models were estimated and compared.

Model Estimation and BMA.

After the model space including all of the candidate models was specified, all candidate models of all subjects were estimated using expectation maximization algorithm, calculating the parameters in each model and the free energy F as a good estimation of the log-evidence of each model. Then the model estimation was compared among families with different input sites or different modulated connections.

Our first question was where sensory inputs entering the network. For the three-region model space, seven families with different auditory inputs were compared by the random effect method (RFX) to determine the most possible input of the network, and for the four-region model, the input was assumed to be the same as that in three-region model. After the comparison of the model-input families, we then tested the existence of modulation on each connection. For modulation on each connection, all the candidate models could be categorized into two families according to whether such connection is included (i.e., modulation parameter is assumed to be nonzero) in this model. Then the paired families were compared by the RFX method on each connection. The winning families were determined according to the exceedance probability of the RFX result with high confidence and entered into the BMA to generate a model summary that combined the likely parameters values for each family of good model fitness (22). The comparison for seven families of input sites for three-region model space showed that the input auditory entered the system most likely via only pSTG (with the highest exceedance possibility of 0.53), thus indicating the auditory input likely entered the neural system in the Chinese group exclusively from the posterior part of left temporal cortex, which was similar to the data from the English speakers (21, 22). The family-wise comparison analysis for the four-region model space in the Chinese group showed that the modulations from pSTG to IFG, from IFG to aSTG, from IFG to rSTP, and from aSTG to rSTP were probably nonexistent (all with exceedance possibility < 0.01). Thus, there were 255 models that survived the above restrictions, and these were entered into BMA to calculate the group-level parameters (means and SEs).

Second-Level Analysis of Model Parameters.

A one-sample t test was used to examine the statistical significance of modulation (nonzero parameter in matrix B) across each group of subjects based on individual BMA results with a threshold of P < 0.05 (FDR corrected). Two-sample t tests were then used to compare the modulation parameters on these connections between the Chinese and English groups.

Behavioral Dictation in Chinese.

Chinese subjects who participated in the MRI experiment and were eligible in the DCM analysis were contacted 4 mo after their brain scanning and asked if they would participate in a surprise dictation test. Eight subjects (4 males and 4 females; mean age, 24.0 y) agreed to participate in the dictation study. They were presented the same intelligible stimuli of Chinese idioms they had been exposed to in the scanner and were required to write down the Chinese characters in the idioms they heard. The dictation results were classified into three categories: (i) correct (90.6 ± 2.5%, mean ± SD); (ii) phonologically correct (8.7 ± 2.3%, where pronunciation of the words were shown to be correct, but the words were not correctly identified, either the Chinese characters chosen were homophones or Pinyin transcription were given, rather than the correct Chinese character); and (iii) wrong (0.7 ± 0.5%, both the word identity and the pronunciation were wrong). The performance of the dictation was calculated as the percentage of correctness of their writing according to the above three categories. We then conducted correlation analysis to investigate the correlations between the dictation results and the modulation strength of brain network connections from the DCM analysis (Fig. 3B and Fig. S3).