Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
  • Submit
  • About
    • Editorial Board
    • PNAS Staff
    • FAQ
    • Accessibility Statement
    • Rights and Permissions
    • Site Map
  • Contact
  • Journal Club
  • Subscribe
    • Subscription Rates
    • Subscriptions FAQ
    • Open Access
    • Recommend PNAS to Your Librarian

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Home
Home
  • Log in
  • My Cart

Advanced Search

  • Home
  • Articles
    • Current
    • Special Feature Articles - Most Recent
    • Special Features
    • Colloquia
    • Collected Articles
    • PNAS Classics
    • List of Issues
  • Front Matter
    • Front Matter Portal
    • Journal Club
  • News
    • For the Press
    • This Week In PNAS
    • PNAS in the News
  • Podcasts
  • Authors
    • Information for Authors
    • Editorial and Journal Policies
    • Submission Procedures
    • Fees and Licenses
  • Submit
Research Article

Time-varying functional network information extracted from brief instances of spontaneous brain activity

Xiao Liu and Jeff H. Duyn
  1. Advanced Magnetic Resonance Imaging Section, Laboratory of Functional and Molecular Imaging, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD 20892

See allHide authors and affiliations

PNAS March 12, 2013 110 (11) 4392-4397; https://doi.org/10.1073/pnas.1216856110
Xiao Liu
Advanced Magnetic Resonance Imaging Section, Laboratory of Functional and Molecular Imaging, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD 20892
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: liux15@ninds.nih.gov
Jeff H. Duyn
Advanced Magnetic Resonance Imaging Section, Laboratory of Functional and Molecular Imaging, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD 20892
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  1. Edited by Marcus E. Raichle, Washington University in St. Louis, St. Louis, MO, and approved January 29, 2013 (received for review September 27, 2012)

  • Article
  • Figures & SI
  • Info & Metrics
  • PDF
Loading

Abstract

Recent functional magnetic resonance imaging studies have shown that the brain is remarkably active even in the absence of overt behavior, and this activity occurs in spatial patterns that are reproducible across subjects and follow the brain’s established functional subdivision. Investigating the distribution of these spatial patterns is an active area of research with the goal of obtaining a better understanding of the neural networks underlying brain function. One intriguing aspect of spontaneous activity is an apparent nonstationarity, or variability of interaction between brain regions. It was recently proposed that spontaneous brain activity may be dominated by brief traces of activity, possibly originating from a neuronal avalanching phenomenon. Such traces may involve different subregions in a network at different times, potentially reflecting functionally relevant relationships that are not captured with conventional data analysis. To investigate this, we examined publicly available functional magnetic resonance imaging data with a dedicated analysis method and found indications that functional networks inferred from conventional correlation analysis may indeed be driven by activity at only a few critical time points. Subsequent analysis of the activity at these critical time points revealed multiple spatial patterns, each distinctly different from the established functional networks. The spatial distribution of these patterns suggests a potential functional relevance.

  • instantaneous coactivation
  • network dynamics
  • nonstationary connectivity
  • resting-state network
  • clustering analysis

In recent years, a large number of studies have investigated the brain’s functional connectivity by monitoring its activity at rest using the functional magnetic resonance imaging (fMRI) technique. The premise of these studies is that fMRI blood oxygenation level-dependent (BOLD) signals (1⇓⇓–4), acquired without explicit stimulation or task, are temporally correlated within functionally related brain regions (5, 6). In fact, the spatial patterns of the temporal BOLD correlations show strong resemblance with many established brain networks, e.g., the motor, visual, language, default mode, and attention networks (5, 7⇓⇓–10), and for this reason are generally referred to as “resting-state networks” (RSNs) (6, 11). Even though RSNs measured across subject are remarkably consistent (12), substantial modulation can be seen throughout development (13, 14), at different levels of consciousness (15⇓⇓–18), and in a variety of brain diseases (19). RSNs therefore may provide critical information regarding the brain’s dynamic organization that is complementary to structural information.

Until now, much of the fMRI research on RSNs has concentrated on characterizing differences in RSNs across conditions or populations, and little effort has gone toward investigating potential network changes within the same condition: seed-based correlation maps (5) or independent components (ICs) (11) are often estimated based on analysis of an entire scan session and further averaged across sessions or even subjects. Such analysis, however, may not fully capture the potential variation in correlational patterns over time, and as a result, some of more subtle information about network interactions may be lost. In fact, analysis of power correlations of resting-state magnetoencephalography signals has shown a time-varying relationship among major nodes of the default mode network (DMN) and the dorsal attention network (DAN) (20), a phenomenon often referred to as “nonstationarity.” Similarly, gamma-band local field potentials (LFPs) power recorded from cats also showed nonstationary correlations ranging from positive to negative between the regions that activate and deactivate with behavioral tasks (referred to as “task-positive” and “task-negative” regions, respectively), with anticorrelation appearing more frequently in waking and paradoxical sleep than in slow-wave sleep (21).

Likewise, temporal variability in fMRI signal correlations has also been examined over the typical duration (a few minutes) of a resting-state scan of human brain (22⇓⇓⇓–26). It was found that interregional BOLD correlations showed considerable variation over the timescale of tens of seconds to minutes, confirming nonstationary nature of functional connectivity (22). Similar observations in unconscious, anesthetized macaques suggest that some of this nonstationarity cannot simply be attributed to unconstrained mental activity (24), and more likely reflects a more basic property of complex systems.

The nonstationary correlational structure of fMRI data provides an opportunity to increase the network information extracted during the analysis process. Recent approaches toward this goal have relied mainly on examining temporal variability in the fMRI signal correlations (or coherences) using the time–frequency analysis or sliding window method (22, 24, 25, 27). This analysis is often limited to a few brain regions of interest and their first-order (pairwise) relationship. Another drawback of these time domain methods is the confounding contribution of nonneurogenic signals such as motion, instrumental drift, and thermal noise, which is difficult to distinguish from the signals of interest, especially when using shorter (sliding) time windows (22). These limitations severely hamper the analysis, interpretation, and use of nonstationary RSN dynamics.

In the present study, we first test the hypothesis that correlational patterns in resting-state fMRI data arise from relatively brief (a few seconds long) periods of coactivation or codeactivation of regions, and then present a dedicated analysis method for extracting these patterns. The hypothesis is motivated by recent introduction of the point process analysis (PPA) approach (28). Using a similar approach, we demonstrate RSN patterns extracted with conventional correlation analysis are a temporal average of multiple distinct coactivations or codeactivation patterns at different time points, which can be resolved with a temporal decomposition procedure.

Results

Replication of RSN Patterns Through Averaging of Selected fMRI Time Frames.

To test our hypothesis, we analyzed a 247-subject resting-state fMRI data set from a publicly accessible database known as the “1000 Functional Connectomes Project (FCP)” (29) (www.nitrc.org/projects/fcon_1000/). To illustrate the phenomenon of nonstationary correlation, the fMRI BOLD signals were extracted from the posterior cingulate cortex (PCC) and medial prefrontal cortex (mPFC) of a representative participant (Fig. 1A). Although these signals have some common features reflected in their substantial correlation (r = 0.40), there appear to be some brief periods of large deviations (Fig. 1A, red arrows). This phenomenon resembles the previous observation based on gamma-band LFP power in cat brain, which exhibited alternating positive and negative (∼20% of the total time) correlations between regions activated and deactivated during a behavioral task (21). Given that PCC and mPFC are major nodes of DMN (generally deactivated by behavioral tasks in human), nonstationary correlation may be a common phenomenon, even for brain regions belonging to the same brain network.

Fig. 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig. 1.

RSN patterns found from correlation analysis can be accurately replicated by averaging only a small portion of fMRI time frames showing high signal level at the seed region. (A) FMRI signals from PCC (green) and mPFC (black) regions of a representative participant show nonstationary correlations. Clear but distinct patterns are seen in two fMRI frames at which the PCC and mPFC signals strongly deviate (frame 1) or are similar (frame 2) phase (slice location: Z = 27 and −6). (B) Similarity (spatial correlation) between the average of selected time frames (gray points) and the PCC-seeded correlation map using all time frames increases quickly when including more frames by lowering the threshold. The gray background represents regions within ±1 SD across participants. (C) A single (Left, Top) or the average of 18 (Left, Middle) frames from the representative participant closely resemble (r = 0.669 and 0.820, respectively) the correlation map calculated with all 123 frames (Right, Top). At the group level, the average of a 15% selection of time frames (Left, Bottom) is almost identical (r = 0.995) to the group correlation map derived from the entire dataset (Right, Bottom). The color scale of each map was automatically adjusted based on the 98th percentile of map values. Only three slices at Z = 27, 39, and 51 are shown. (D) The selected fMRI frames can be further classified into different groups according to their spatial pattern and then averaged within groups to generate spontaneous coactivation patterns (CAPs), which is equivalent to a temporal decomposition of the overall mean of all frames. All axial images are shown in the radiologic convention (Left in image is Right in brain).

To illustrate the brain’s instantaneous coactivation at single time points, two fMRI time frames (frames 1 and 2; Fig. 1A) were extracted from the representative participant at time points where fMRI signal in the PCC was high. Notably, these single fMRI frames already show coarse resemblance to the DMN pattern (Fig. 1A). We then compared the average of the fMRI time frames whose PCC signal exceeded a predefined threshold (Fig. 1B, Left) to the PCC-seeded correlation map calculated using all frames. The average of even one or two frames (corresponding to the threshold at 99th percentile; Fig. 1B, Right), which are not necessarily the most correlated ones, is already surprisingly similar (spatial correlation r = 0.51 ± 0.12 across subjects, mean ± SD, same hereinafter) to the PCC-seeded correlation map (Fig. 1B, Right). In comparison, the most correlated single frame is even closer to the correlation map (r = 0.59 ± 0.07). The similarity increases rapidly when lowering the selection threshold and thus including more frames, and approaches a plateau after including ∼15% of all frames (Fig. 1B, Right). Further lowering the threshold does not substantially improve similarity, and even substantially reduces similarity when most or all frames are included.

The effect of averaging frames is further illustrated in Fig. 1C with a specific example. The correlation between a single time frame (Fig. 1A, frame 2) and the correlation map of the entire run (123 time frames) is 0.669, and this value goes up to 0.830 if a selection of 18 frames (∼15% of 123) is averaged. Particularly, if the analysis is extended to the entire group, 15% of data from all 247 participants will generate a pattern almost perfectly matching the group PCC-seeded correlation map derived from all data (r = 0.995). These results confirm the previous finding that a specific RSN pattern can be accurately replicated with only a small portion of the dataset (28).

Temporal Decomposition of RSNs into Multiple Coactivation Patterns.

The two exemplary frames (frames 1 and 2) shown in Fig. 1A were selected at times when mPFC signal level was distinctly different. Apparently, their spatial coactivation patterns are also quite different: apart from differences in mPFC, the bilateral parietal components in frame 1 are more posterior than those in frame 2. Close inspection of the patterns suggests that this difference may not solely originate from thermal (spatially uncorrelated) noise. It is, therefore, possible that the fMRI frames actually represent distinct coactivation patterns, and that much of this information is lost when simply averaging time frames.

With the goal of extracting further information, the selected 15% fMRI frames were classified into different groups based on their spatial similarity (Fig. 1D) (see Materials and Methods for more details) and then averaged within each group. These group averages are in the following referred to as spontaneous coactivation patterns (CAPs).

Using this method, the DMN pattern was decomposed into eight PCC-related CAPs (PCC-CAPs), which were sorted based on their consistency values (the average spatial similarity of each fMRI frame to the group mean) (see Materials and Methods for details) in Fig. 2A. The first four PCC-CAPs with relatively high within-group similarity cover almost the entire PCC/precuneus region, as well as some parts of the frontal and parietal cortex. Comparison of the PCC-CAP 1−4 by spatial overlay (Fig. 2C) indicates that PCC-CAP 1 and 2 more closely resemble DMN than PCC-CAP 3 and 4, with PCC-CAP 1 extending more dorsally and PCC-CAP 2 more ventrally. Moreover, PCC-CAP 1 shows clear structure at the caudate nucleus (CN), whereas PCC-CAP 2 has high values specifically at the hippocampus (HC). PCC-CAP 3 shows relatively strong activity in the middle frontal gyrus (MFG), and its parietal components are more superior than those of other CAPs. PCC-CAP 4 is characterized by very specific patterns at the superior frontal gyrus (SFG) and the parahippocampus gyrus (PHG), and its bilateral parietal components extend much more posteriorly compared with the other PCC-CAPs. All four PCC-CAPs show large negative areas covering the insula, intraparietal sulcus (IPS), and other brain regions preferentially active during cognitive tasks (“task-positive” regions) (30), although with different magnitudes and spatial distributions. In contrast, the PCC-CAPs 5−8, showed relatively low within-group similarity, less resemblance to the DMN pattern, and mainly covered the visual (PCC-CAP 5 and 6) and motor cortex (PCC-CAP 8).

Fig. 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig. 2.

Temporal decomposition of the default mode network (DMN) pattern into multiple PCC-related spontaneous coactivation patterns (PCC-CAPs). (A) The overall mean of the 15% fMRI time frames having highest values at the PCC seed region (the green square) generates a DMN pattern very similar (r = 0.995) to that from the PCC-seeded correlation map. This overall pattern can be decomposed into eight distinct PCC-CAPs, which were ranked by their within-group consistency values. Both consistency and fraction values are shown on the right side of the corresponding CAPs. The slices shown in the maps are at Z = −21, −9, 3, 15, 27, 39, 51, respectively. (B) The same decomposition procedure was applied to a simulated dataset assuming stationary statistics (slice location: Z = 27). (C) Spatial features of the first four PCC-CAPs are compared by overlaying their binarized Z maps (Z > 6). Although they have significant overlap (yellow color), the individual CAPs show distinct differences at multiple brain regions. The six slices shown in the maps are at Z = −9, 3, 15, 27, 39, 51, respectively.

To test whether the multiple CAPs indeed reflect nonstationary property of their associated RSN or solely resulted from the classification method, a surrogate dataset was synthesized based on the statistics of the actual data while assuming stationarity. In this case, the decomposition procedure failed to reveal any distinct patterns other than those similar to the overall DMN pattern (Fig. 2B).

The extraction of CAPs was further explored with the seed region selected in the left IPS. Using the same strategy, the DAN pattern associated with this seed was decomposed into 12 IPS-CAPs (Fig. 3). The first six IPS-CAPs have relatively distinct patterns with bilateral coactivations at IPS regions. Three of these overlap substantially over the visual cortex but with distinct preference for areas along the ventral visual stream (IPS-CAP 1), involved in peripheral vision (IPS-CAP 5), or high-order visual association (IPS-CAP 2). IPS-CAP 3 preferentially overlaps the middle temporal(MT)/V5+ complex and the frontal eye field (FEF). In contrast, IPS-CAP 4 showed significant activity at insular, operculum, and the posterior paracingulate gyrus (PCG). Despite high values at the seed region, IPS-CAP 7−9 are similar to PCC-CAP 8, 4, and 1, respectively, suggesting they may originate from the same underlying networks. IPS-CAP 10 resembled the salience network (31), whereas the last two IPS-CAPs were not recognized as matching any specific functional regions.

Fig. 3.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig. 3.

Temporal decomposition of the dorsal attention network (DAN) pattern into 12 IPS-related spontaneous coactivation patterns (IPS-CAPs). The six slices shown in the maps are at Z = −9, 3, 15, 27, 39, 51, respectively.

A number of the CAPs showed coactivations at small but very specific thalamic or cerebellar structures, indicating a possibly specific functional relevance. For example, IPS-CAP 2, 4, and 5 included distinct thalamic nuclei including the pulvinar, ventral lateral nucleus (VL), and lateral geniculate nucleus (LGN) (Fig. 4A), and the thalamic structures in different CAPs were clearly separated from each other, even for two vision-related CAPs (IPS-CAP 2 and 5). Similarly, distinct cerebellar structures appearing in IPS-CAP 2, 3, 5, and 6 also covered very specific locations without significant overlap (Fig. 4B).

Fig. 4.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig. 4.

Spontaneous coactivation patterns (CAPs) covering specific areas in the thalamus and cerebellum. (A) The Z maps for IPS-CAP 2, 4, and 5 show distinct nucleus-like structures in the thalamus (the top three rows) covering the parts of the pulvinar (IPS-CAP 2 and 5), the VL (IPS-CAP 4), and the LGN (IPS-CAP 5). The composite map (the bottom row) indicates that these structures are spatially distinct (slice locations: Z = 3, 6, 9, 12, 15, and 18, respectively, from Left to Right). (B) The Z maps for IPS-CAP 2, 3, 5, and 6 show specific coactivations in the cerebellum (the left four columns). There is almost no overlap between these cerebellar regions as shown in the corresponding composite map (the rightmost column). The seven slices shown in the maps are at Z = −51, −48, −45, −42, −30, −27, and −24, respectively, from bottom to top.

Comparison with ICs and Temporal Functional Modes.

For comparison, we also performed a more conventional network extraction based on spatial independent component analysis (ICA). We compared the 30 extracted ICs (shown in Fig. S1) with four and six CAPs based on PCC and IPS seeds, respectively. For this purpose, four and six ICs were selected with highest z scores at the PCC region (Fig. 5) or IPS region (Fig. S2A), respectively. Only subsets of these ICs (IC 13, 16, and IC 27, 29, respectively) appeared to resemble the DMN and DAN patterns, respectively. On close comparison, substantial differences were seen between the selected ICs and CAPs in detailed structures, strength of negative statistics, and spatial cross-correlations (Fig. S2 B and C). Increasing the dimensionality of the ICA (to up to 100) did not reveal additional IC patterns that resembled the CAPs. Thus, the analysis proposed here identified multiple CAPs, with spatial distributions deviating from ICs and suggestive of a functional relevance.

Fig. 5.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig. 5.

Comparison of the first four PCC-CAPs (Left) to four ICs (Center) and four TFMs (Right) with highest statistics at the PCC seed region (green square). The ICs and TFMs are arranged according to a descending order of their PCC statistics. The six slices shown in the maps are at Z = −9, 3, 15, 27, 39, and 51, respectively.

Recently, an alternative to spatial ICA was proposed that combines spatial and temporal ICA to extract temporally independent components from resting-state data (32). These components, called temporally independent functional modes (TFMs), may have some similarity to CAPs, which originate from distinct time points and therefore also may have high temporal independence. Preliminary comparison of TFMs and CAPs (Fig. 5 and Fig. S2) reiterates the finding from comparing CAPs and ICs in that they have both similarities and differences, suggesting they may highlight different aspects of the underlying covariations in neuronal activity.

Effects of Global Signal Regression.

To verify that global signal regression (GSR) did not affect the CAP extraction procedure, the analysis was repeated without GSR. Similar to the changes observed with correlation analysis (33), GSR generally reduced activity values, however without affecting the spatial pattern of the CAPs (Fig. S3).

Reproducibility.

Reproducibility of CAPs was evaluated by analyzing a second, 198-subject set of resting-state data from the database that originated from a different fMRI site (SI Text and Table S1). Very similar results were obtained, as exemplified by the comparison of the four major PCC-derived CAPs shown in Fig. S4.

Discussion

General Findings.

In this study, we presented a modified version of a point-process method for the analysis of resting-state fMRI data, which allows extraction of network information with only a fraction of the measurement data. We demonstrated the ability of this method to extract different networks based on either emphasizing stationary network properties or nonstationary network properties. Emphasizing the stationary properties led to RSNs that resembled seed correlation or ICA-derived RSNs, whereas emphasizing nonstationary properties led to more fine-grained networks (here called CAPs).

CAPs Account for Nonstationary Functional Connectivity.

Resting-state fMRI connectivity is conventionally assessed by interregional correlations (5), and distinct RSNs (6) can be extracted by ICA or seed-based correlation analysis. In this study, we found that seed-based RSNs can be replicated by averaging a few fMRI time frames selected based on the signal level at the seed region. This finding confirms previous findings with PPA-type analysis and suggests that RSNs may be driven primarily by activity at a few discrete events (point processes) (28). Thus, it is possible that interregional BOLD correlations result from instantaneous coactivations (or codeactivations) of multiple brain regions at some critical time points rather than from continuous, sustained interregional neuronal interactions. Such instantaneous coactivations may originate from brief neuronal events, separated by relatively long time intervals (on average, 33.5 ± 24.5 s; Fig. S5A), which determine the timescale of resting-state fMRI signal fluctuations (<0.1 Hz). Confirming this notion and establishing the precise neuronal origin and functional relevance of the apparent coactivations will likely require electrophysiological data to be acquired concurrently with the fMRI experiment.

One possible explanation for these putative neuronal events is the presence of large-scale neuronal avalanching activity (28), defined as spontaneous activity initiating at a specific location in the form of a brief burst, and taking with it, like an avalanche, other connected regions that are near activation threshold (34⇓–36). Such neuronal avalanches may or may not be consciously perceived, and in this regard, it would be interesting to investigate CAPs during conditions of reduced consciousness such as sleep and anesthesia. This may reveal the relationship between CAPs and conscious processes such as mind wandering (37).

Even though the average of the critical time frames over extended measurement periods tends to converge to the spatial distribution of RSNs derived from conventional analysis, activity distributions of individual frames may be distinctly different. This was confirmed by the classification of spatial patterns of critical frames, which identified multiple groups of the frames associated with distinct CAPs. Judging from their distribution, the CAPs appeared to have some relationship to the underlying functional connectivity. Some CAPs showed coactivations at very specific thalamic and cerebellar regions that are spatially distinct among CAPs, even for those showing large overlaps in cortical regions (IPS-CAP 2 and 5; Fig. 4A). These thalamic and cerebellar patterns most likely are not an artifact from the classification procedure, as their extent is too limited to substantially affect spatial correlations between frames and therefore classification outcomes. Furthermore, the observation that a single subject usually shows multiple CAPs (Fig. S5B) also excludes the possibility that the finding of multiple CAPs is a result of the anatomical variation across subjects. In summary, not only can the resting brain be decomposed spatially into many RSNs, but the RSNs can also be further decomposed temporally into multiple CAPs that have apparent functional significance.

The BOLD correlations (or coherences) between different brain regions have been found to vary notably over time using the time–frequency analysis or sliding-window method, and this was regarded as an important piece of evidence for a nonstationarity of functional connectivity (22, 24, 25). More strikingly, using ICA on a 60-frame subselection of resting-state fMRI data that shifts progressively over time by as little as a single frame, a recent study found dramatic variability of the DMN pattern, suggesting disproportional contributions of specific frames (26). The findings in this study illuminate the previous observations from two different aspects. First, we found that the information regarding a specific RSN indeed is not distributed evenly in time domain, but highly condensed into a few critical time points. Therefore, the temporal variation in the BOLD correlation could result from the change of the relative signal contents within the time window of analysis, e.g., whether or not including a critical time point may substantially affect interregional correlations observed within a sliding window. In part, this variability results from underlying noise sources rather than neuronal activity. Second, and more importantly, analysis of only these critical time points suggested the presence of multiple, spatially distinct coactivation patterns. Therefore, the variability in patterns across these critical time points will also affect interregional BOLD correlations within the analysis window. For example, the MFG-PCC correlation is expected to be relatively high during a window that includes the PCC-CAP 3 type of time frames compared with inclusion of other types of CAPs. This second mechanism for variation in the BOLD correlations is likely to be physiologically relevant and to reflect the underlying system dynamics.

As is the case with the interpretation of ICs, TFMs, and correlation patterns derived from fMRI data, the interpretation of CAPs in terms of neuronal activity is to some extent confounded by the fact that the BOLD signal is a rather indirect measure of neuronal activity. It is likely that strength and appearance of CAPs is affected by regional differences in cortical vascularization, neurovascular control, and the level and nature of neural activity. A recent study suggests that region-specific intersubject variations may occur in some of these parameters (38), and this will need to be taken into account when interpreting fMRI-derived network information.

Relation to Stationary Functional Connectivity.

Although correlation analysis may not be an optimal approach to subdivide RSNs into multiple distinct CAPs, it does preserve some of their information because the existence of CAPs can bias the BOLD correlations in a way that reflects their spatial patterns. Correlation analysis has been used previously for subpartitioning brain regions into multiple functional sections (39⇓–41). For example, the IPS has been subdivided into regions with different functional specialization based on the location of the largest correlation with other brain areas (39). Specifically, visual regions show strongest correlations with the posterior medial portion of IPS, auditory and insular cortex have strongest correlations with the anterior lateral IPS, sensorimotor-correlated IPS locates also anteriorly but toward medial direction, and the DMN brain regions are maximally correlated with posterior lateral IPS. Such an organization coincides well with the spatial patterns of IPS-CAP 5, IPS-CAP 4, IPS-CAP 7, and PCC-CAP 3, respectively. The correspondence suggests a close relationship between the fine-scale connectivity found with the type of correlation analysis described above and the CAPs found with proposed method.

One aspect of CAPs not analyzed here is their frequency of occurrence, which, together with their spatial patterns, can affect the level of fMRI signal correlations. These two types of contributions to RSNs cannot be distinguished with conventional correlation analysis and likely have different physiological bases. This remains a topic for future investigations.

Implication for Future Resting-State fMRI Studies.

The notion that the interaction between brain regions is dominated by short, intermittent events rather than the result of a continuous, sustained process has important implications for future studies. For example, the search of neuronal correlates of functional connectivity may benefit from a focus on transient neuronal events rather than continuous neuronal modulations. Additionally, separating of and focusing on critical time points may increase sensitivity and specificity of resting-state fMRI analysis. Further improvement can be achieved through the differentiation of the critical points, exemplified by more specific cortico-thalamic and cortico-cerebellar connectivity in CAPs (Fig. 4).

The existence of multiple CAPs for a RSN suggests strong but complex dependency among brain regions; therefore, the fMRI signal from single brain region is probably not adequate for quantifying ongoing brain dynamics. Instead, multivariate approaches involving multiple regions or even the whole brain are preferred for certain types of studies, e.g., those aiming to understand interaction between ongoing and evoked brain activity (42⇓–44).

Methodological Perspectives.

Seed region-based linear correlation analysis has several limitations for analyzing resting-state fMRI signals. Although this bivariate statistic is a reliable measurement of coactivation level of two regions, it provides little information about high-level correlation, i.e., coactivation of more than two regions. For example, even when all pairwise correlations between three voxels are known, we cannot tell how frequently these three voxels are activated at the same time. Furthermore, unlike electrophysiological data, fMRI signals are more densely sampled in the spatial domain than in the temporal domain. Consequently, the whole-brain fMRI temporal correlation matrix is usually highly rank deficient with much more elements than actual measurements. This redundancy implies strong dependency among the correlation values and may significantly complicate our understanding of interregional relations of the brain. For the above reasons, spatial-domain methods may have advantages for analyzing fMRI signals.

ICA is usually applied in spatial domain to identify components representing RSNs, and it also provides time courses for each IC reflecting temporal dynamics of the RSNs (45). However, while generally regarded as a data-driven method, ICA actually has a very specific model assumption: independence between components. This strong assumption of independence leads to near-zero correlations (both linear and nonlinear) between ICs (Fig. S2B), and make them semiorthogonal to each other (inner product or spatial correlation is close to zero). This explains why many of the CAPs found with our approach were not extracted by ICA (Fig. 5), because most of the CAPs have considerable overlap (Fig. 2C).

The analysis method used in this study differs from conventional methods in the way it regards fMRI frames as the most basic unit for analysis. It has few model assumptions and performs few data transformations. The outcomes, in the form of averages of multiple fMRI frames, are close representations of the original data and thus possibly of the underlying neuronal processes. Moreover, the approach is also flexible. Even though the current study focused only on seed-based analyses, the method is also applicable without preselection of a seed region. The latter is expected to generate many more CAPs, including those related to distinct RSNs, and as such constitutes a data-driven technique for analyzing resting-state fMRI signals.

Technical Limitations.

The fMRI time frames showed significant variation in their signal-to-noise ratio, and the activity in some frames did not significantly exceed noise level. Although such noisy frames are not expected to substantially affect final CAPs that are generated from many time frames, they may affect the stability of classification. Therefore, an optimized selection procedure is desired to automatically separate these frames from more informative ones.

The CAPs can be generally divided into two categories: one group resembles the corresponding RSN pattern (e.g., PCC-CAP 1−4) and the other does not (e.g., PCC-CAP 5−8). As a result, the indices quantifying the cluster structure [e.g., Silhouette (46)] tend to give the highest score for the k-means clustering with k, the number of CAPs, equal to 2. We therefore did not use these indices to determine k. Instead, we repeated a series of k values and arbitrarily chose the cases presented here, i.e., k = 8 for PCC-CAPs and k = 12 for IPS-CAPs. We believe that these two cases represent a good balance between richness and redundancy. It also worth noting that the final patterns of the CAPs are not very sensitive to k, and the majority of the CAPs presented herein were also found in the cases with different k values.

Materials and Methods

Our analysis was performed on resting-state fMRI datasets from 247 study participants (mean age, 22.72 ± 4.61; range, 18−44; sex, 151 females) selected from the FCP (29). Data collection was approved by the institutional review board of the individual site, and informed consent was obtained from each subject.

The fMRI data were first preprocessed using FCP scripts (29) (version 1.1-beta) with small modifications. The scripts performed the typical preprocessing steps for functional connectivity analysis (see SI Materials and Methods for specific steps). The fMRI data of each subject was first spatially coregistered to high-resolution anatomical images and then to the 152-brain Montreal Neurological Institute (MNI) space. Additionally, the signal of each voxel was de-meaned and then normalized by its temporal SD. Given that GSR during preprocessing may have undesirable consequences (33, 47), a separate set of data were preprocessed identically except for omission of the GSR step.

The previous study with the PPA-type analysis (28) proposed a method to calculate conditional rate maps replicating RSN patterns. Here, we modified this method by simply extracting fMRI frames at time points with suprathreshold signal of a seed region. These frames can then be simply averaged, or further analyzed with a clustering algorithm. The modified approach was demonstrated on two seed regions, constituting 6 × 6 × 6 mm3 cubes centered at the PCC [[0, −53, 26] (48) in MNI coordinates, same hereinafter] and the left IPS ([−24, −58, 52]), respectively.

We then classified extracted time points (fMRI time frames) based on their spatial similarity using the k-means clustering method (see SI Materials and Methods for details), which has been frequently used in resting-state fMRI data analysis (39, 40, 49). After clustering, the frames assigned to the same cluster are simply averaged to generate CAP maps, from which Z-statistical maps were derived by dividing by the SE (within-cluster and across fMRI time frames) map. For each cluster, the spatial correlation between individual fMRI frames and their mean was calculated, and the results were averaged to generate a measure of consistency. The number of frames in each cluster was also divided by the total number of extracted frames to quantify the fraction of time each CAP occurs.

The k-means clustering was repeated with k = 2, …, 20, but only the results corresponding to specific k values (k = 8 for PCC-related CAPs and k = 12 for IPS-related CAPs) are reported herein.

A surrogate dataset was simulated under the stationarity assumption (stable correlational patterns over time) and with statistical properties (mean, variance, number of frames) identical to those of the real dataset.

The group ICA using temporal concatenation was implemented with the Multivariate Exploratory Linear Optimized Decomposition into Independent Components (MELODIC) program in the FSL software (45), with setting the number of components to be extracted specified to 30 and 100.

More details are available in SI Materials and Methods.

Acknowledgments

This research was supported by the Intramural Research Program of the National Institutes of Health, National Institute of Neurological Disorders and Stroke.

Footnotes

  • ↵1To whom correspondence should be addressed. E-mail: liux15{at}ninds.nih.gov.
  • Author contributions: X.L. and J.H.D. designed research; X.L. and J.H.D. performed research; X.L. contributed new reagents/analytic tools; X.L. and J.H.D. analyzed data; and X.L. and J.H.D. wrote the paper.

  • The authors declare no conflict of interest.

  • This article is a PNAS Direct Submission.

  • This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1216856110/-/DCSupplemental.

Freely available online through the PNAS open access option.

References

  1. ↵
    1. Ogawa S,
    2. Lee T-M,
    3. Kay AR,
    4. Tank DW
    (1990) Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc Natl Acad Sci USA 87(24):9868–9872.
    OpenUrlAbstract/FREE Full Text
  2. ↵
    1. Bandettini PA,
    2. Wong EC,
    3. Hinks RS,
    4. Tikofsky RS,
    5. Hyde JS
    (1992) Time course EPI of human brain function during task activation. Magn Reson Med 25(2):390–397.
    OpenUrlPubMed
  3. ↵
    1. Kwong KK,
    2. et al.
    (1992) Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. Proc Natl Acad Sci USA 89(12):5675–5679.
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Ogawa S,
    2. et al.
    (1992) Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci USA 89(13):5951–5955.
    OpenUrlAbstract/FREE Full Text
  5. ↵
    1. Biswal B,
    2. Yetkin FZ,
    3. Haughton VM,
    4. Hyde JS
    (1995) Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magn Reson Med 34(4):537–541.
    OpenUrlPubMed
  6. ↵
    1. Fox MD,
    2. Raichle ME
    (2007) Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci 8(9):700–711.
    OpenUrlCrossRefPubMed
  7. ↵
    1. Cordes D,
    2. et al.
    (2000) Mapping functionally related regions of brain with functional connectivity MR imaging. AJNR Am J Neuroradiol 21(9):1636–1644.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    1. Hampson M,
    2. Peterson BS,
    3. Skudlarski P,
    4. Gatenby JC,
    5. Gore JC
    (2002) Detection of functional connectivity using temporal correlations in MR images. Hum Brain Mapp 15(4):247–262.
    OpenUrlCrossRefPubMed
  9. ↵
    1. Greicius MD,
    2. Krasnow B,
    3. Reiss AL,
    4. Menon V
    (2003) Functional connectivity in the resting brain: A network analysis of the default mode hypothesis. Proc Natl Acad Sci USA 100(1):253–258.
    OpenUrlAbstract/FREE Full Text
  10. ↵
    1. Fox MD,
    2. Corbetta M,
    3. Snyder AZ,
    4. Vincent JL,
    5. Raichle ME
    (2006) Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proc Natl Acad Sci USA 103(26):10046–10051.
    OpenUrlAbstract/FREE Full Text
  11. ↵
    1. Damoiseaux JS,
    2. et al.
    (2006) Consistent resting-state networks across healthy subjects. Proc Natl Acad Sci USA 103(37):13848–13853.
    OpenUrlAbstract/FREE Full Text
  12. ↵
    1. Vincent JL,
    2. et al.
    (2007) Intrinsic functional architecture in the anaesthetized monkey brain. Nature 447(7140):83–86.
    OpenUrlCrossRefPubMed
  13. ↵
    1. Fair DA,
    2. et al.
    (2007) Development of distinct control networks through segregation and integration. Proc Natl Acad Sci USA 104(33):13507–13512.
    OpenUrlAbstract/FREE Full Text
  14. ↵
    1. Fair DA,
    2. et al.
    (2008) The maturing architecture of the brain’s default network. Proc Natl Acad Sci USA 105(10):4028–4032.
    OpenUrlAbstract/FREE Full Text
  15. ↵
    1. Fukunaga M,
    2. et al.
    (2006) Large-amplitude, spatially correlated fluctuations in BOLD fMRI signals during extended rest and early sleep stages. Magn Reson Imaging 24(8):979–992.
    OpenUrlCrossRefPubMed
  16. ↵
    1. Boly M,
    2. et al.
    (2009) Functional connectivity in the default network during resting state is preserved in a vegetative but not in a brain dead patient. Hum Brain Mapp 30(8):2393–2400.
    OpenUrlCrossRefPubMed
  17. ↵
    1. Horovitz SG,
    2. et al.
    (2009) Decoupling of the brain’s default mode network during deep sleep. Proc Natl Acad Sci USA 106(27):11376–11381.
    OpenUrlAbstract/FREE Full Text
  18. ↵
    1. Liu X,
    2. Zhu XH,
    3. Zhang Y,
    4. Chen W
    (2011) Neural origin of spontaneous hemodynamic fluctuations in rats under burst-suppression anesthesia condition. Cereb Cortex 21(2):374–384.
    OpenUrlAbstract/FREE Full Text
  19. ↵
    1. Zhang D,
    2. Raichle ME
    (2010) Disease and the brain’s dark energy. Nat Rev Neurol 6(1):15–28.
    OpenUrlCrossRefPubMed
  20. ↵
    1. de Pasquale F,
    2. et al.
    (2010) Temporal dynamics of spontaneous MEG activity in brain networks. Proc Natl Acad Sci USA 107(13):6040–6045.
    OpenUrlAbstract/FREE Full Text
  21. ↵
    1. Popa D,
    2. Popescu AT,
    3. Paré D
    (2009) Contrasting activity profile of two distributed cortical networks as a function of attentional demands. J Neurosci 29(4):1191–1201.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    1. Chang C,
    2. Glover GH
    (2010) Time-frequency dynamics of resting-state brain connectivity measured with fMRI. Neuroimage 50(1):81–98.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Kang J,
    2. et al.
    (2011) Characterizing dynamic functional connectivity in the resting brain using variable parameter regression and Kalman filtering approaches. Neuroimage 56(3):1222–1234.
    OpenUrlCrossRefPubMed
  24. ↵
    1. Hutchison RM,
    2. Womelsdorf T,
    3. Gati JS,
    4. Everling S,
    5. Menon RS
    (2012) Resting-state networks show dynamic functional connectivity in awake humans and anesthetized macaques. Hum Brain Mapp, in press.
  25. ↵
    1. Rack-Gomer AL,
    2. Liu TT
    (2012) Caffeine increases the temporal variability of resting-state BOLD connectivity in the motor cortex. Neuroimage 59(3):2994–3002.
    OpenUrlCrossRefPubMed
  26. ↵
    1. Kiviniemi V,
    2. et al.
    (2011) A sliding time-window ICA reveals spatial variability of the default mode network in time. Brain Connect 1(4):339–347.
    OpenUrlCrossRefPubMed
  27. ↵
    1. Majeed W,
    2. et al.
    (2011) Spatiotemporal dynamics of low frequency BOLD fluctuations in rats and humans. Neuroimage 54(2):1140–1150.
    OpenUrlCrossRefPubMed
  28. ↵
    1. Tagliazucchi E,
    2. Balenzuela P,
    3. Fraiman D,
    4. Chialvo DR
    (2012) Criticality in large-scale brain FMRI dynamics unveiled by a novel point process analysis. Front Physiol 3:15.
    OpenUrlPubMed
  29. ↵
    1. Biswal BB,
    2. et al.
    (2010) Toward discovery science of human brain function. Proc Natl Acad Sci USA 107(10):4734–4739.
    OpenUrlAbstract/FREE Full Text
  30. ↵
    1. Fox MD,
    2. et al.
    (2005) The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci USA 102(27):9673–9678.
    OpenUrlAbstract/FREE Full Text
  31. ↵
    1. Seeley WW,
    2. et al.
    (2007) Dissociable intrinsic connectivity networks for salience processing and executive control. J Neurosci 27(9):2349–2356.
    OpenUrlAbstract/FREE Full Text
  32. ↵
    1. Smith SM,
    2. et al.
    (2012) Temporally-independent functional modes of spontaneous brain activity. Proc Natl Acad Sci USA 109(8):3131–3136.
    OpenUrlAbstract/FREE Full Text
  33. ↵
    1. Murphy K,
    2. Birn RM,
    3. Handwerker DA,
    4. Jones TB,
    5. Bandettini PA
    (2009) The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced? Neuroimage 44(3):893–905.
    OpenUrlCrossRefPubMed
  34. ↵
    1. Beggs JM,
    2. Plenz D
    (2003) Neuronal avalanches in neocortical circuits. J Neurosci 23(35):11167–11177.
    OpenUrlAbstract/FREE Full Text
  35. ↵
    1. Chialvo DR
    (2010) Emergent complex neural dynamics. Nat Phys 6(10):744–750.
    OpenUrlCrossRef
  36. ↵
    1. de Arcangelis L,
    2. Herrmann HJ
    (2012) Activity-dependent neuronal model on complex networks. Front Physiol 3:62.
    OpenUrlPubMed
  37. ↵
    1. Mason MF,
    2. et al.
    (2007) Wandering minds: The default network and stimulus-independent thought. Science 315(5810):393–395.
    OpenUrlAbstract/FREE Full Text
  38. ↵
    1. Taylor PA,
    2. Gohel S,
    3. Di X,
    4. Walter M,
    5. Biswal BB
    (2012) Functional covariance networks: Obtaining resting-state networks from intersubject variability. Brain Connect 2(4):203–217.
    OpenUrlCrossRefPubMed
  39. ↵
    1. Anderson JS,
    2. Ferguson MA,
    3. Lopez-Larson M,
    4. Yurgelun-Todd D
    (2010) Topographic maps of multisensory attention. Proc Natl Acad Sci USA 107(46):20110–20114.
    OpenUrlAbstract/FREE Full Text
  40. ↵
    1. Kelly C,
    2. et al.
    (2012) A convergent functional architecture of the insula emerges across imaging modalities. Neuroimage 61(4):1129–1142.
    OpenUrlCrossRefPubMed
  41. ↵
    1. Zhang D,
    2. et al.
    (2008) Intrinsic functional relations between human cerebral cortex and thalamus. J Neurophysiol 100(4):1740–1748.
    OpenUrlAbstract/FREE Full Text
  42. ↵
    1. Liu X,
    2. Zhu XH,
    3. Chen W
    (2011) Baseline BOLD correlation predicts individuals’ stimulus-evoked BOLD responses. Neuroimage 54(3):2278–2286.
    OpenUrlCrossRefPubMed
  43. ↵
    1. Hesselmann G,
    2. Kell CA,
    3. Eger E,
    4. Kleinschmidt A
    (2008) Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proc Natl Acad Sci USA 105(31):10984–10989.
    OpenUrlAbstract/FREE Full Text
  44. ↵
    1. Fox MD,
    2. Snyder AZ,
    3. Vincent JL,
    4. Raichle ME
    (2007) Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior. Neuron 56(1):171–184.
    OpenUrlCrossRefPubMed
  45. ↵
    1. Beckmann CF,
    2. DeLuca M,
    3. Devlin JT,
    4. Smith SM
    (2005) Investigations into resting-state connectivity using independent component analysis. Philos Trans R Soc Lond B Biol Sci 360(1457):1001–1013.
    OpenUrlAbstract/FREE Full Text
  46. ↵
    1. Rousseeuw PJ
    (1987) Silhouettes—a graphical aid to the interpretation and validation of cluster-analysis. J Comput Appl Math 20:53–65.
    OpenUrlCrossRef
  47. ↵
    1. Fox MD,
    2. Zhang D,
    3. Snyder AZ,
    4. Raichle ME
    (2009) The global signal and observed anticorrelated resting state brain networks. J Neurophysiol 101(6):3270–3283.
    OpenUrlAbstract/FREE Full Text
  48. ↵
    1. Van Dijk KR,
    2. et al.
    (2010) Intrinsic functional connectivity as a tool for human connectomics: Theory, properties, and optimization. J Neurophysiol 103(1):297–321.
    OpenUrlAbstract/FREE Full Text
  49. ↵
    1. Mezer A,
    2. Yovel Y,
    3. Pasternak O,
    4. Gorfine T,
    5. Assaf Y
    (2009) Cluster analysis of resting-state fMRI time series. Neuroimage 45(4):1117–1125.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top
Article Alerts
Email Article

Thank you for your interest in spreading the word on PNAS.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Time-varying functional network information extracted from brief instances of spontaneous brain activity
(Your Name) has sent you a message from PNAS
(Your Name) thought you would like to see the PNAS web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Time-varying functional networks
Xiao Liu, Jeff H. Duyn
Proceedings of the National Academy of Sciences Mar 2013, 110 (11) 4392-4397; DOI: 10.1073/pnas.1216856110

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
Time-varying functional networks
Xiao Liu, Jeff H. Duyn
Proceedings of the National Academy of Sciences Mar 2013, 110 (11) 4392-4397; DOI: 10.1073/pnas.1216856110
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Mendeley logo Mendeley

Article Classifications

  • Biological Sciences
  • Neuroscience
Proceedings of the National Academy of Sciences: 110 (11)
Table of Contents

Submit

Sign up for Article Alerts

Jump to section

  • Article
    • Abstract
    • Results
    • Discussion
    • Materials and Methods
    • Acknowledgments
    • Footnotes
    • References
  • Figures & SI
  • Info & Metrics
  • PDF

You May Also be Interested in

Reflection of clouds in the still waters of Mono Lake in California.
Inner Workings: Making headway with the mysteries of life’s origins
Recent experiments and simulations are starting to answer some fundamental questions about how life came to be.
Image credit: Shutterstock/Radoslaw Lecyk.
Depiction of the sun's heliosphere with Voyager spacecraft at its edge.
News Feature: Voyager still breaking barriers decades after launch
Launched in 1977, Voyagers 1 and 2 are still helping to resolve past controversies even as they help spark a new one: the true shape of the heliosphere.
Image credit: NASA/JPL-Caltech.
Drop of water creates splash in a puddle.
Journal Club: Heavy water tastes sweeter
Heavy hydrogen makes heavy water more dense and raises its boiling point. It also appears to affect another characteristic long rumored: taste.
Image credit: Shutterstock/sl_photo.
Mouse fibroblast cells. Electron bifurcation reactions keep mammalian cells alive.
Exploring electron bifurcation
Jonathon Yuly, David Beratan, and Peng Zhang investigate how electron bifurcation reactions work.
Listen
Past PodcastsSubscribe
Panda bear hanging in a tree
How horse manure helps giant pandas tolerate cold
A study finds that giant pandas roll in horse manure to increase their cold tolerance.
Image credit: Fuwen Wei.

Similar Articles

Site Logo
Powered by HighWire
  • Submit Manuscript
  • Twitter
  • Facebook
  • RSS Feeds
  • Email Alerts

Articles

  • Current Issue
  • Special Feature Articles – Most Recent
  • List of Issues

PNAS Portals

  • Anthropology
  • Chemistry
  • Classics
  • Front Matter
  • Physics
  • Sustainability Science
  • Teaching Resources

Information

  • Authors
  • Editorial Board
  • Reviewers
  • Subscribers
  • Librarians
  • Press
  • Cozzarelli Prize
  • Site Map
  • PNAS Updates
  • FAQs
  • Accessibility Statement
  • Rights & Permissions
  • About
  • Contact

Feedback    Privacy/Legal

Copyright © 2021 National Academy of Sciences. Online ISSN 1091-6490