The videos, shown with minimal information and often without sound or music, are meant to provide a sort of scientific cinéma vérité.
Image courtesy of Nipam Patel (University of California, Berkeley, CA).
New Research In
Physical Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
-
Edited by Ranulfo Romo, Universidad Nacional Autónoma de México, Mexico, D.F., Mexico, and approved January 14, 2009
-
↵2M.S.B. and D.Y. contributed equally to this work. (received for review May 22, 2008)
Abstract
We studied whether detectable percepts could be produced by electrical stimulation of intracranial electrodes placed over human visual areas identified with fMRI. Identification of areas was confirmed by recording local-field potentials from the electrode, such as face-selective electrical responses from electrodes over the fusiform face area (FFA). The probability of detecting electrical stimulation of a visual area varied with the position of the area in the visual cortical hierarchy. Stimulation of early visual areas including V1, V2, and V3 was almost always detected, whereas stimulation of late visual areas such as FFA was rarely detected. When percepts were elicited from late areas, subjects reported that they were simple shapes and colors, similar to the descriptions of percepts from early areas. There were no reports of elaborate percepts, such as faces, even in areas like FFA, where neurons have complex response properties. For sites eliciting percepts, the detection threshold was determined by varying the stimulation current as subjects performed a forced-choice detection task. Current thresholds were similar for late and early areas. The similarity between both percept quality and threshold across early and late areas suggests the presence of functional microcircuits that link electrical stimulation with perception.
Electrical stimulation in the visual cortex of nonhuman primates has revealed a great deal about visual perception (1–4). Performing electrical stimulation in humans provides a unique opportunity to study the qualitative properties of stimulation-induced percepts, which can offer insights about the functional organization of visual cortex (5, 6), and may advance efforts to restore sight with cortical prosthetics for retinal blindness (7–11). Previous electrical stimulation studies have not taken advantage of functional neuroimaging to identify specific visual areas. By combining electrical stimulation with functional magnetic resonance imaging, we studied the percepts produced by activating a limited number of neurons in different identified visual areas.
First, we addressed the ability of different identified visual areas to support perception. Some have proposed that early visual areas or late visual areas are privileged in supporting perception (12, 13). Others have suggested, instead, that ventral areas support perception and dorsal areas do not (14). Penfield and Rasmussen (5) showed that stimulating the cortical surface of the occipital lobe produced the perception of a “phosphene” (a bright spot of light) in most subjects, whereas stimulation of the temporal lobe usually resulted in no detectable percept. More recently, Lee et al. (15) reported that most sites in the occipital, temporal, and parietal lobes did not produce any percept when stimulated. However, without functional identification of the stimulated visual areas, it remains unclear how the ability to produce a percept varies across areas.
Second, we explored the quality of percepts created by stimulation in different identified visual areas. Although complex percepts have been reported during stimulation of human cortex [e.g., Penfield and Rasmussen (5) reported the recall of memory episodes after stimulation of the ventral temporal lobe, and Puce et al. (16) reported the perception of faces and face parts after stimulation of the fusiform gyrus], it is unknown whether the complexity of stimulation-induced percepts changes systematically across visual areas.
The third question we hoped to address dealt with the intensity of electrical stimulation required to produce percepts in different identified visual areas. If some areas are functionally more distant from perception and not as well-suited to support it, they might require stronger electrical currents to produce a detectable percept than other areas. Previous electrical stimulation studies in humans have made only informal assessments of detection thresholds. In the current study, a psychophysical forced-choice task (17) was used to generate unbiased measures of detection thresholds (18).
Results
Visual cortex was stimulated by using 50 electrodes in 10 subjects. Before electrode implantation, functional MRI was used to identify individual visual areas in each subject by using phasic retinotopic stimuli and functional localizers. Postimplantation computed tomography (CT) scans were merged with the presurgical structural and functional MRI to locate electrodes relative to specific visual areas [Fig. 1 and supporting information (SI) Figs. S1–S8]. When possible, local-field potential (LFP) responses to visual stimulation were recorded from the electrodes to help confirm the identity of the visual areas. Fig. 2A shows a sample electrode identified as being in fusiform face area (FFA) with fMRI. This identification was confirmed with LFP responses that were selective for faces (Fig. 2B and Figs. S9–S12).
Fig. 1.
Identification of an implanted electrode over retinotopic visual cortex. (A) Posterior-lateral view of the gray-white matter boundary of the left hemisphere of a single subject. A strip containing 10 electrodes was implanted subdurally. The location of each electrode on the strip is shown as a black disc, corresponding to the actual size of the electrode. (B) Magnified view of 6 electrodes in Fig. 1A. The patch of cortex closest to each electrode is colored in blue. The circle and arrow identify the cortex under the electrode of interest. (C) Posterior view of a partially inflated cortical surface (at the gray-white matter boundary) with retinotopic fMRI data (same subject as A and B, arrow indicates electrode of interest). Surface coloring corresponds to the location in the visual field that evoked maximal activity from each cortical node. Dorsal visual areas are labeled with dotted lines, indicating visual area boundaries. The electrode is in the area V3A, consistent with previous anatomical and functional studies (28).
Fig. 2.
Identification of an implanted electrode over the FFA. (A) Ventral view of the pial surface of a subject's right hemisphere. The fusiform gyrus sits between the collateral sulcus (upper white dashed line) and the inferior temporal sulcus (lower white dashed line). The electrode (black disc indicated with arrow) sits over the FFA, seen as orange surface nodes with significantly greater (P < 10−6) BOLD fMRI response to faces than to places (nodes with stronger responses to places are colored blue). (B) LFPs recorded from the electrode in A. Average response to images of faces (orange) and houses (blue). The gray bar indicates the 125-ms period when each stimulus was presented.
The 50 electrodes were distributed over 11 different visual areas identified with fMRI. To provide a quantitative measure of the position of each electrode, we measured the distance of each electrode from the occipital pole along the 2D cortical surface. Cortical surface distance corresponds approximately to position in the visual hierarchy (Fig. 3C and Table S1) with early visual areas (defined as V1, V2, V3, V3a, and V4) close to the pole, and later visual areas [defined as middle temporal (MT), lateral occipital (LO), V8, FFA, and parahippocampal place area (PPA)] further from the pole on the lateral and ventral surface of the hemisphere.
Fig. 3.
Summary data across all electrodes. (A) The location of 50 electrodes across 10 subjects are plotted as spheres on a single, inflated left hemisphere. The left hemisphere is shown from posterior, medial, and ventral views. Green color indicates that electrical stimulation of the electrode produced a percept. Red color indicates that it did not. (B) In each subject, the distance between the occipital pole and the electrode along the cortical surface was measured. For each distance, the probability of evoking a percept was computed. Each blue point shows the average cortical surface distance and probability for 10 electrodes, calculated with a moving-window average. The black curve shows the best-fit Weibull function. (C) Electrodes were also classified depending on their position in the visual hierarchy as either early (electrodes located in areas V1, V2, V3, V3a, and V4) or late (all other areas). Electrodes that produced a percept are shown as a black O, electrodes that did not are shown as a red X. Most early electrodes (symbols at the bottom) produced a percept, most late electrodes (symbols at the top) did not. There was a rough correspondence between early and late classification and distance from the occipital pole (shown on the x axis).
Probability of Detection.
To visualize the location of stimulation sites that did or did not produce a percept, we plotted all 50 sites on a single brain in standard space (Fig. 3A). We found a striking relationship between the position of an electrode and its ability to produce a detectable percept when stimulated. Plotting the probability against cortical surface distance (Fig. 3B) showed a sharp decline with increasing distance from the occipital pole, falling to 50% at 4 cm from the occipital pole. This decreasing probability with cortical surface distance corresponded to a sharp drop in detectable percepts from early to late areas, with 100% of V1 sites producing a percept and 11% of FFA sites producing a percept (Fig. 3C and Table S1). Even some late sites that showed strong and selective LFP responses, such as the FFA electrode in Fig. 2, failed to produce a percept when stimulated.
Percept Quality.
For those sites that produced a detectable percept, subjects were asked to report on the qualitative properties of the percept (Table 1). Stimulation of many sites in both early and late visual areas elicited descriptions consistent with simple phosphenes (small flashes of light) such as “small, white plus sign” (V2), “silver flash” (V1), and “little (visual) explosion” (PPA). Stimulation of some sites elicited reports of slightly more complex percepts, such as “Chinese checkers” (V1/V2), “triangle/green/aqua's” (V2), or “projecting light cone” (V4/V8). However, no elaborate percepts were described at the 24 sites where electrical stimulation was detected and the quality was reported. Also, no systematic difference was noted between percept complexity for early and late sites. Subjects usually reported a consistent percept with repeated stimulation of a single electrode.
View this table:
Table 1.
Perceptual quality summary, 24 electrodes, 10 subjects
For 5 early electrodes, we were able to examine percept reliability using size and location of the evoked percept. For each of at least 10 trials per electrode, subjects used custom software to adjust the size and location of a bright white circle on a black background. There was no statistically significant correlation between detection threshold and reported percept size (r = −0.42, P = 0.49) or eccentricity (r = 0.27, P = 0.66).
Detection Thresholds.
To examine how thresholds varied across visual cortex, analysis was restricted to the 26 electrodes that produced a detectable percept. For each electrode, stimulus trains of different strengths were delivered pseudorandomly in one of two intervgals on different trials of the 2-interval forced choice (IFC) task (Fig. 4 A and B). The resulting behavioral performance was fit with a sigmoid function (Fig. 4C) to find the threshold for detection. Thresholds ranged from a low of 0.49 mA in V3 to a high of 2.65 mA in LO. Across electrodes, the mean detection threshold was 1.21 mA (SD = 0.68 mA). To determine whether there was a relationship between position in the visual cortical hierarchy and detection threshold, the threshold for each electrode was plotted against the cortical surface distance (Fig. 4D). With increasing distance from the occipital pole, there was a slight increase in threshold (0.01 mA/mm, r = 0.48, P = 0.03), corresponding to a trend toward higher thresholds for late visual areas (1.9 vs. 1.1 mA, P = 0.08; Table S1). For sites where no percept was produced, Table S1 shows the maximum current level tested (range 2–7 mA).
Fig. 4.
Determining thresholds for detecting electrical stimulation. (A) Each trial contained 2 300-ms epochs, marked by the words “One” and “Two.” The current amplitude (and the epoch containing the stimulus) varied from trial to trial. The subject's task was to detect the epoch in which the stimulation was delivered. A shows a sample trial in which a high amplitude current train was delivered in the second epoch, the subject responded by pressing mouse button Two, and received positive feedback. (B) A sample trial in which a low amplitude current was delivered in the first epoch, the subject responded by pressing mouse button Two, and received negative feedback. (C) Behavioral performance at a single V2 electrode. Each point shows the performance at different stimulation currents (error bars, 95% confidence intervals). The black curve is the best-fit psychometric function. The dashed line shows the threshold of 2.53 mA (95% CI 2.36–2.66 uA). (D) Correlation between threshold and distance from the occipital pole across electrodes (error bars, 95% confidence interval for detection threshold). Dashed line shows linear fit (r = 0.48, P = 0.03).
Discussion
We used electrical stimulation to compare the ability of specific human visual areas to contribute to perception. The probability of evoking a percept dropped for later visual areas. However, the ability to produce a percept was not restricted to early areas, suggesting that there is no sharp dichotomy between early and late visual areas in their ability to support perception (12, 13). Neither did we find systematic differences in the ability of electrical stimulation to produce percepts in dorsal vs. ventral areas (14).
The decreasing probability of eliciting a percept in early versus late visual areas may arise from differences in the functional organization of these areas. If perception of a stimulus requires activity in a network of brain areas, electrical stimulation in early areas may more often propagate to this network because of greater extrinsic connectivity in early areas. Consistent with this idea, electrical stimulation of monkey V1 produces activity in V2, V3, and MT, demonstrating the strength of extrinsic connections in V1 (19). Conversely, stimulation of late areas with fewer extrinsic connections may be less likely to propagate to this network, reducing the likelihood of evoking a percept.
In contrast to the results reported here, nonhuman primates were able to reliably detect stimulation of every site tested throughout visual cortex, including V1, inferotemporal cortex, and the frontal eye fields (20, 21). What could explain this major discrepancy? Technical differences between electrical stimulation methods in monkeys and in humans are an obvious possibility. The nonhuman primate studies used penetrating microelectrodes at microampere currents, activating on the order of 100 neurons in a small region around the electrode tip (4). The human studies used superficial disc electrodes at milliampere currents and activated a larger population of neurons surrounding the electrode. Future experiments with smaller electrodes in humans will be able to address the issue of whether electrode size contributes to the difference in results between humans and monkeys. Although differences in electrodes no doubt contributed to absolute differences in threshold (μA vs. mA), the critical difference between human and nonhuman studies may be training rather than electrode size: the animals were trained extensively on detecting the electrical stimulus, whereas the humans were not. Improved performance on a forced-choice detection task over hundreds or thousands of training trials may reflect reorganization of cortical circuitry. Repeatedly stimulating neurons may strengthen connections within the stimulated area or between the stimulated area and other visual areas, enabling the detection of stimulation. If humans received extended training, they might also reliably detect electrical stimulation at sites where no percepts were evoked in the current study.
When stimulation of visual cortex produced a percept, the qualitative property of the percept was typically a small flash or pattern that became visible when the stimulation began and disappeared when the stimulation ended. Even in later areas, subjects reported relatively simple percepts [e.g., “a little (visual) explosion” for a PPA site]. These later-area percepts were much simpler than the formed visual hallucinations of animals, people, places (15), visual scenes (22), and faces or face parts (16) reported in previous studies. The discrepancy may reflect selection from a large set of tested sites in previous studies. For example, the results of Lee et al. (15) were drawn from 1,196 stimulated electrodes. We found that most percepts produced by stimulation of later areas were relatively simple, leading us to conclude that complex percepts are a rare exception when stimulating later visual areas. One difficulty with interpreting self-reports is that subjects might use different words to describe very similar percepts (or the reverse), emphasizing the importance of combining self-reports with more objective measures of perception.
An important dichotomy in our results is the difference between the probability of evoking a percept, which varied greatly from early to late areas, and the threshold if a percept was evoked, which varied little. In one later area that responsed selectively to colored stimuli, stimulation produced a percept of color that matched the receptive field preference (23). In late areas with more complex stimulus preferences, stimulation might produce neuronal activity that is uninterpretable and hence is not perceived by the subject. When we stimulated late areas and consistently failed to evoke complex percepts, our subjects had their eyes closed and covered with a blindfold. If electrical stimulation was instead delivered while subjects viewed an actual stimulus, their perception might be influenced. In monkeys, stumulation of MT or MST can bias the perceived direction of motion (1, 2) and stimulating monkey inferotemporal cortex can bias the perception of a face in an ambiguous stimulus (3). Therefore, in future experiments it will be important to study the effect of electrical stimulation while human subjects view real stimuli. A better understanding of the effects of electrical stimulation of visual cortex on perception enhances the likelihood of developing a cortical visual prosthesis that can provide a rich visual experience.
Experimental Procedures
Subject Information.
Informed consent was obtained from 10 subjects with medically intractable epilepsy [4 females (F), mean age 40 years, age range 19–67]. The Baylor College of Medicine Institutional Review Board and the Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston approved the experimental protocols. Subdural electrodes were implanted to determine the location of the seizure focus, with placement guided solely by clinical criteria. Clinical neurophysiologists identified epileptogenic regions of cortex based on the intracranial recordings. Only data from electrodes that did not exhibit interictal epileptiform activity and that were not found to be sites of seizure onset were used for the analyses reported here.
Neuroimaging.
Before electrode implantation, structural and functional MR scans were obtained on a whole-body 3 tesla scanner (Phillips Medical Systems) in the University of Texas Health Science Center at Houston Magnetic Resonance Imaging Center. Were reconstructed 3D surface models of the subject's brain by using FreeSurfer (24). To identify early visual areas in the 8 subjects eligible to undergo high-field research scanning, BOLD fMRI scan series were collected while the subject viewed visual stimuli. Details can be found in Table S1. Visual stimuli were back-projected from an LCD projector onto a Lucite screen and viewed through a mirror attached to the MR head coil. An MR-compatible infrared eye-tracking system (Applied Science Laboratories) was used to monitor the subjects' behavioral state.
Electrophysiology.
Electrode implantation and localization.
After implantation surgery, subjects underwent whole-head CT. The center of each electrode of interest was located, and the CT scan was aligned to the presurgical structural MRI. Were tested 2 algorithms, a proprietary fusion algorithm in a commercial neurosurgical workstation (StealthStation, Medtronic), and a mutual information algorithm in the “3dAllineate” program in the AFNI package. Both alignment methods gave similar results, and adequately accounted for any shifts in brain location resulting from electrode implantation. All final electrode locations were observed to be directly adjacent to the cortical surface visible on MRI.
Standard subdural recording electrodes were used (AdTech). Each electrode consisted of a platinum alloy disc embedded in silastic with a 2.2-mm diameter recording surface. To locate each electrode relative to fMRI activity on the cortical surface, the AFNI program SurfaceMetrics was used, which assigned each electrode to the closest surface node. Based on the MRI data, the cortical surface distance from the occipital pole was measured for each electrode. Based on the fMRI data, visual electrodes were assigned either to an identified visual area (see above) or to “unnamed visually responsive” cortex. For 8 electrodes in 2 subjects who could not participate in high-field research scans and 6 electrodes that could not otherwise be classified in functionally scanned subjects, we relied on a combination of group average retinotopy (25), and recorded LFP to visual stimulation to assign electrodes to specific visual areas (26). Fig. 1 summarizes the location of all 50 electrodes we stimulated on a standardized cortical surface (27).
Electrical recording.
When possible, visual area identification by fMRI was supplemented by visually evoked LFP recordings from the stimulated electrodes (23); 30 sites showed robust LFP responses to visual stimulation. During recording of LFPs, subjects were seated in a hospital bed facing a 19-inches LCD monitor at a viewing distance of 57 cm, resulting in a display size of 38 × 30° of visual angle. Checkerboards were presented at different visual field locations to map the spatial receptive field of each electrode (26). Responses were also recorded to images presented at fixation (screen center) with size 5 × 5°. To ensure attention to the stimuli, subjects performed a target detection task.
Electrical stimulation.
300 ms of a 200 Hz biphasic pulse (200 μs) train was delivered through the electrode of interest to evoke neuronal activity. We performed monopolar stimulation with a large surgical grounding pad placed on the subject's thigh. In addition to depolarization of neuronal cell bodies, depolarization of nearby axons followed by orthodromic or antidromic action potentials is also a possible consequence of electrical stimulation.
Threshold estimation.
Subjects were trained on a 2-IFC visual contrast detection task during a brief session the week before electrode implantation. During data collection, electrical stimulation replaced the visual stimulus (23). The subjects' eyes were closed in a dimly lit room. In most cases, we used preliminary testing to find a range of currents that spanned detection thresholds. If we were unable to elicit a detectable percept with stimulation at 2.0 or 2.5 mA (12 sites) or 6 or 7 mA (12 sites), we collected data at that current level to document chance performance. For each electrode, we tested 4 to 10 current levels, typically 10 or more times each, delivered in a random order. When testing >1 electrode in a subject, we usually tested electrodes in sequence from early to later visual cortex. Clinical recording continued during these experiments, and no after-discharge potentials or other clinically relevant side effects were observed after the electrical stimulation.
Determining the ability of a stimulation site to produce a percept.
To convert the binary measure of perception obtained from each site to a probability, a moving boxcar procedure averaged the cortical surface distances for 10 sites and determined the probability of detection from those 10 sites. The results of this moving average were plotted in Fig. 3B, with cortical surface distance on the abscissa and probability of detection on the ordinate.
Percept quality determination.
After determining detection thresholds by using the 2-IFC task, an interview was performed to determine the subjective quality of the percepts produced by stimulation. Electrical stimulation was delivered alone at a current that was typically 1.25 times the detection threshold, and subjects had no task other than to describe the size, location, color, and complexity of evoked percepts. Repeated stimulation of the electrode usually produced the same subjective percept. For 5 sites, the subjects adjusted a computerized simulated phosphene to match the real phosphene, allowing better quantitative estimates of phosphene size and location. The size of phosphenes at different sites ranged from 0.46 ± 0.23 to 1.28 ± 0.3328° of visual angle, and the eccentricity of phosphenes at different sites ranged from 0.71 ± 0.24 to 7.0 ± 2°.
Acknowledgments
We thank the subjects and their families, whose cooperation made these experiments possible; the clinical staff at St. Luke's Episcopal Hospital, including Lisa Rhodes, Rodney Hall, Ian Goldsmith, and Eli Mizrahi; Ping Sun for critical programming support; Ziad S. Saad for continued development of the SUMA surface modeling package; and Robert W. Cox for the AFNI software suite. This work was supported by National Institutes of Health Grants NS045053 (to D.Y.) and EY005911 (to J.H.R.M.), and National Institutes of Health Grant S10 RR19186 provided partial funding for the purchase of the 3T scanner.
Footnotes
- 1To whom correspondence may be addressed. E-mail: dmkim{at}post.harvard.edu, dyoshor{at}bcm.tmc.edu, or michael.s.beauchamp{at}uth.tmc.edu
-
Author contributions: D.K.M., J.H.R.M., M.S.B., and D.Y. designed research; D.K.M., M.S.B., and D.Y. performed research; D.K.M. and M.S.B. analyzed data; and D.K.M., J.H.R.M., M.S.B., and D.Y. wrote the paper.
-
The authors declare no conflict of interest.
-
This article is a PNAS Direct Submission.
-
This article contains supporting information online at www.pnas.org/cgi/content/full/0804998106/DCSupplemental.
References
Perceiving electrical stimulation of identified human visual areas
More Articles of This Classification
Biological Sciences
Related Content
Cited by...
- Facephenes and rainbows: Causal evidence for functional and anatomical specificity of face and color processing in the human brain
- Conversion of object identity to object-general semantic value in the primate temporal cortex
- Saturation in Phosphene Size with Increasing Current Levels Delivered to Human Visual Cortex
- Making sense: Determining the parameter space of electrical brain stimulation
- Playing the electric light orchestra--how electrical stimulation of visual cortex elucidates the neural basis of perception
- Electrical Stimulation of the Left and Right Human Fusiform Gyrus Causes Different Effects in Conscious Face Perception
- Seeing Scenes: Topographic Visual Hallucinations Evoked by Direct Electrical Stimulation of the Parahippocampal Place Area
- Spatial and Object-Based Attention Modulates Broadband High-Frequency Responses across the Human Visual Cortical Hierarchy
- New methods devised specify the size and color of the spots monkeys see when striate cortex (area V1) is electrically stimulated
Similar Articles
You May Also be Interested in
Victoria Orphan and Elizabeth Trembath-Reichert discuss microbial life in the deep subseafloor.
PNAS Profile with NAS member and anthropologist Michael Tomasello
Researchers report an early monumental burial site near Lake Turkana in Kenya that may have served as a stable landmark for mobile herders in a changing physical environment and as a social anchor point to foster communal identity and interaction among mobile herders.
A study reports evidence of water ice on the moon’s surface, discerned via a signature in the near-infrared reflectance spectra that suggests the ice was formed by slow condensation due to impact or water migration through the lunar exosphere.
Image courtesy of Pixabay/Ponciano.
