Asymmetries in behavioral and neural responses to spectral cues demonstrate the generality of auditory looming bias

Baumgartner, Robert; Reed, Darrin K.; Tóth, Brigitta; Best, Virginia; Majdak, Piotr; Colburn, H. Steven; Shinn-Cunningham, Barbara

doi:10.1073/pnas.1703247114

Asymmetries in behavioral and neural responses to spectral cues demonstrate the generality of auditory looming bias

Robert Baumgartner https://orcid.org/0000-0003-0899-4903 [email protected], Darrin K. Reed, Brigitta Tóth, +3 , Virginia Best, Piotr Majdak, H. Steven Colburn, and Barbara Shinn-CunninghamAuthors Info & Affiliations

Edited by Dale Purves, Duke University, Durham, NC, and approved July 28, 2017 (received for review February 25, 2017)

August 21, 2017

114 (36) 9743-9748

https://doi.org/10.1073/pnas.1703247114

PDF/EPUB

Significance

Previous studies demonstrated “auditory looming bias” exclusively by manipulating overall sound intensity. Hence, it is not clear whether this bias truly reflects perceptual differences in sensitivity to motion direction rather than changes in intensity. We manipulated individualized spectral cues to create stimuli that were perceived as either approaching or receding, while controlling loudness. We assessed discrimination of motion direction and analyzed simultaneously recorded neural responses using electroencephalography. Our results show both behavioral and neural evidence of looming bias. Therefore, our study demonstrates that the bias is truly about perceived motion in distance, not intensity changes.

Abstract

Studies of auditory looming bias have shown that sources increasing in intensity are more salient than sources decreasing in intensity. Researchers have argued that listeners are more sensitive to approaching sounds compared with receding sounds, reflecting an evolutionary pressure. However, these studies only manipulated overall sound intensity; therefore, it is unclear whether looming bias is truly a perceptual bias for changes in source distance, or only in sound intensity. Here we demonstrate both behavioral and neural correlates of looming bias without manipulating overall sound intensity. In natural environments, the pinnae induce spectral cues that give rise to a sense of externalization; when spectral cues are unnatural, sounds are perceived as closer to the listener. We manipulated the contrast of individually tailored spectral cues to create sounds of similar intensity but different naturalness. We confirmed that sounds were perceived as approaching when spectral contrast decreased, and perceived as receding when spectral contrast increased. We measured behavior and electroencephalography while listeners judged motion direction. Behavioral responses showed a looming bias in that responses were more consistent for sounds perceived as approaching than for sounds perceived as receding. In a control experiment, looming bias disappeared when spectral contrast changes were discontinuous, suggesting that perceived motion in distance and not distance itself was driving the bias. Neurally, looming bias was reflected in an asymmetry of late event-related potentials associated with motion evaluation. Hence, both our behavioral and neural findings support a generalization of the auditory looming bias, representing a perceptual preference for approaching auditory objects.

Imagine yourself alone in the wilderness. Suddenly, a threatening sound permeates the darkness. Is it approaching? This is a critical question when it comes to your survival because approaching objects usually pose a greater threat than receding objects (1). The phenomenon that approaching sounds are more salient than receding sounds is commonly termed “auditory looming bias.” Looming bias is reflected in a broad variety of psychophysical tasks related to salience and alertness: bias in loudness-change estimates (2–4) and judgments of duration (5), improved discriminability of motion speed (6), underestimated distances for egocentrically moving (4) or bypassing sounds (7, 8), and reduced reaction time for auditory (3, 9) and visual (3) targets preceded by looming sounds. In animals, looming biases result in faster learning speed during associative conditioning (10) and longer duration of attention (11). This list shows that looming bias triggers a variety of percepts across a wide range of psychoacoustic tasks. Despite its broad behavioral significance, the mechanisms underlying auditory looming bias are still poorly understood.

A universal problem of previous research on auditory looming bias is that source distance was manipulated using overall sound intensity (sounds increasing in intensity perceived as approaching). Hence, it is unclear whether looming bias actually reflects a bias in the perception of intensity changes or distance motion. Moreover, it is hard to disentangle the contribution of simple neural nonlinearities in response to intensity gradients (12, 13) from higher-level perceptual asymmetries of motion in distance. Previous studies addressed this issue by comparing sounds of different spectral structure: complex tones elicited stronger behavioral looming bias (2, 11) and larger neural differences (14–16) than did noise with equal intensity increase, indicating that intensity changes did not directly cause looming bias. However, studies with a different spatial task (17) or species (10) found the opposite effect (i.e., stronger looming bias for noise than tonal sounds), calling into question these results.

An early fMRI study that contrasted static sounds with sounds increasing or decreasing in intensity found activity in the right planum temporale (4), which is associated with processing of auditory motion in any spatial direction (18, 19), consistent with the idea that sound intensity changes can be perceived as motion in distance. The contrast between sounds increasing vs. decreasing in intensity, however, revealed a different network, including superior temporal sulcus and amygdala, which may reflect a warning process for approaching objects (4, 9, 20). A more recent fMRI study (21) and neural recordings in animals (10, 14) reported differences related to looming bias in low-level auditory cortical areas. Whether these differences are caused by lateral or top-down processes is unclear. The processing latencies reported in previous studies appear to be inconsistent: some found multisensory enhancements in visual cortical areas at early sensory levels (80-ms postsound onset) (15, 22), whereas others found no significant differences in global field strength before 600 ms (16). A more instantaneous change of distance cues (rather than gradual changes of sound intensity) might be better suited to identify the time course of auditory looming bias.

Auditory event-related potential (ERP) studies of horizontal and vertical motion perception support a stereotypical “motion onset response” that consists of two consecutive stages associated with different functions (23–26): a negative deflection occurring roughly 100 ms after the evoking event, denoted by N1, represents an early, presumably automatic, sensory detection of spatial changes, and a positive deflection occurring roughly 200 ms after the evoking event, denoted by P2, likely represents evaluation of motion direction. Looming bias arguably relies on the evaluation of motion direction. Hence, if the motion onset response can be generalized to motion in distance, then looming bias may be reflected in P2.

Here, we simultaneously measured behavior and neural responses to explore whether (i) changes in overall sound intensity and (ii) stimulus continuity are required to activate looming bias, and (iii) whether auditory looming bias is linked to early sensory or the later stages of neural processing. Based on the hypothesis that looming bias reflects a gain in perceptual detectability of approaching objects relative to receding objects, we measured looming bias in a motion direction discrimination task. Hence, we aimed to assess the origin of looming bias more directly than previous studies that measured consequential effects of the bias related to stimulus salience and alertness. We used spectral changes to create sounds perceived as approaching or receding between external, internal, and intermediate positions (colored red, blue, and green, respectively, in Fig. 1 A and B) by reducing the contrast of measured high-frequency spectral cues from the natural acoustics of each individual listener. Spectral cues are particularly suitable for this investigation because (i) in contrast to gradual intensity changes, spectral cues can evoke instantaneous changes of distance percepts (27–30) and thus allow precise analysis of processing latencies; (ii) spectral cue manipulations can create transitions to internal (infinitely close) auditory percepts within peripersonal space where looming bias is most prominent (31); and (iii) spectral cues can be manipulated independently of overall stimulus intensity.

Fig. 1.

Contrast of spectral cues naturally induced by the pinnae and torso was manipulated to create approaching and receding sounds. (A) Spatial configuration of hypothesized spatial percepts with number of listeners tested per source angle (left, front, and right) in Exps. I and II. (B) Effect of spectral contrast manipulation according to factor C on magnitude responses of listener-specific stimuli of Exp. I (B, *Top*) as well as their frequency-specific (*B, Bottom Left*) and overall (*B, Bottom Right*) loudness changes relative to C = 1. Shaded areas denote ±1 SEM (n = 15). Note that changes in overall loudness oppose the intended effect of contrast switch. (C) Schematic representation of an experimental trial in Exp. I and Exp. II: starting with noise filtered according to C₁, then cross-faded to C₂, followed by an open response period, and a jittered intertrial interval. The left–right arrow in Exp. I represents a temporal jitter of ±50 ms. Listeners were instructed to report whether the sound was perceived as approaching, receding, or static in Exp. I and only approaching or receding in Exp. II.

Results

Loudness Predictions.

We used predictions of a loudness model (32) to assure that loudness changes cannot explain the looming bias caused by spectral contrast switches. Fig. 1B shows the effect of our contrast manipulation technique on the stimulus magnitude spectrum (Top) and on the corresponding predictions of relative loudness changes (Bottom) for all stimuli of experiment (Exp.) I (individual data provided in Interindividual Comparison of Frequency-Specific Loudness Changes and Fig. S1). Flattening the reference magnitude spectrum (C = 1, red) decreased frequency-specific loudness in the lower two to three octaves of the stimulus (1–6 kHz) and increased it only in the highest octave (8–16 kHz). On average across frequencies (Lower Right), flattening the reference magnitude spectrum decreased predicted overall loudness. Hence, changes in loudness perception, if they occurred, would be expected to oppose the typical loudness-induced distance percept.

Fig. S1.

Model predictions of frequency-specific loudness for every individual listener of Exp. I. For frontal source angles the left ear is arbitrarily shown as ipsilateral.

Interaural level differences (ILDs) are also known to act as cues for distance perception (18, 29). Relative changes in frequency-specific ILDs can be easily extracted from loudness predictions (Fig. 1B, Bottom) as the difference between ipsilateral (solid lines) and contralateral (dashed lines) loudness. The predictions show that a decrease in spectral contrast was accompanied by an ILD increase at the lower two octaves and an ILD decrease at the higher two octaves.

Behavioral Results.

In two experiments, listeners judged switches between spectral contrasts (C₁ and C₂) in forced choice paradigms (Fig. 1C). Experiment I allowed subjects three response alternatives [3-alternative forced-choice (AFC): “approaching,” “receding,” or “static”] and tested instantaneous spectral contrast switches [using an interstimulus interval (ISI) of zero] while neural responses were measured using electroencephalography (EEG). Experiment II tested speeded responses for two alternatives (2-AFC: approaching or receding) and presented trials with either instantaneous but continuous (ISI of 0 ms) or discontinuous (ISI of 100 ms) changes in spectral shape across stimulus pairs. Experiment II served as a control to test whether looming bias is truly linked to the perception of sounds moving in distance rather than the mere percept of source proximity. Some listeners perceived spectral contrast manipulations as elevation changes at some source angles. Therefore, for each individual, we selected the source angle that led to the most consistent percept of a change in distance, based on a short initial run of the discrimination task (pretest) using three source angles (left, front, and right). Fig. 1A shows the source angles that were selected for the subsequent looming experiments.

Although listeners never received feedback about the hypothesized distance change, decreasing spectral contrasts (C₁ > C₂, solid orange lines in Fig. 2) were more likely perceived as approaching (filled orange triangles) and increasing spectral contrasts (C₁ < C₂, dashed green lines) as receding (open green triangles). These dominant associations were used for subsequent statistical analyses and were also observed in the pretest results (Behavioral Results of Pretest for Individual Source Angle Selection and Fig. S2). Trials without spectral changes (C₁ = C₂) in Exp. I were almost always perceived as “static,” confirming that the listeners remained vigilant and performed the task reliably.

Fig. 2.

Behavioral responses were more consistent for sounds perceived as approaching compared with sounds perceived as receding if instantaneous spectral changes were presented in continuous stimuli. Mean behavioral responses in the 3–AFC motion discrimination task of Exp. I (*Left*; n = 15) and the 2–AFC motion discrimination task of Exp. II (*Middle* and *Right*; n = 13). Results of Exp. II are separated between trials presenting instantaneous but continuous (*Middle*; as in Exp. I) and discontinuous (*Right*) spectral contrast switches. Decreasing spectral contrast switches (orange lines) were predominantly perceived as approaching (orange triangles), increasing spectral contrast switches (green lines) as receding (green triangles), and constant spectral contrasts (no lines) as static (gray squares). Statistical analyses focused on these predominant response associations. Behavioral looming bias was observed in terms of significantly higher consistency in approaching responses compared with receding responses only if spectral switches occurred within a continuous stimulus. Feedback was provided only on the detection accuracy of constant stimuli (C₁ = C₂) after blocks in Exp. I. Values reflect mean ±1 SEM. Levels of significance: *P < 0.05, **P < 0.01, ***P < 0.001.

Fig. S2.

Mean behavioral responses (n = 15) of the pretest (3–AFC motion discrimination task) for source angle selection. Decreasing spectral contrast switches (orange lines) were predominantly perceived as approaching (orange triangles), increasing spectral contrast switches (blue lines) as receding (blue triangles), and constant spectral contrasts (no lines) as static (gray squares). Statistical analyses focused on these predominant response associations. Behavioral looming bias was observed in terms of significantly higher consistency in approaching responses compared with receding responses. Feedback was provided only on the detection accuracy of constant stimuli (C1 = C2) after blocks. Values reflect mean ±1 SEM. Levels of significance: *P < 0.05, **P < 0.01, ***P < 0.001.

In Exp. I (Fig. 2, Left), response consistency (percentage of responses for the same percept category), used here as a measure of cue salience, differed significantly between contrast pairs, i.e., the combination of spectral contrasts independent of presentation order (0 ↔ 1 vs. 0 ↔ 0.5 vs. 0.5 ↔ 1) (F_1.4,20 = 14, P < 0.01,

η_{p}^{2}

= 0.51). Listeners were most consistent for the largest contrast switches (C: 0 ↔ 1) and least consistent for pairs consisting of the two higher contrast values (C: 0.5 ↔ 1; Fig. 2 shows significance levels of paired comparisons). Importantly, response consistency also differed significantly between switch directions, i.e., the presentation order within contrast pairs (C₁ < C₂ vs. C₁ > C₂) (F_1,14 = 6.9, P = 0.020,

η_{p}^{2}

= 0.33). Listeners were more consistent in labeling decreasing spectral contrasts than increasing spectral contrasts; this pattern is consistent with looming bias in that approaching sounds elicited more consistent identification responses than did receding sounds.

Response consistency in Exp. II also differed significantly between contrast pairs (F_1.9,22 = 40, P < 0.001,

η_{p}^{2}

= 0.77). As in Exp. I, judgments were least consistent for pairs consisting of the two higher contrast values (C: 0.5 ↔ 1). Overall response consistency was not affected by stimulus continuity (ISI = 0 ms vs. ISI = 100 ms) (F_1,12 = 0.55, P = 0.47,

η_{p}^{2}

= 0.044), but there were tendencies of a main effect of switch direction (F_1,12 = 4.7, P = 0.052,

η_{p}^{2}

= 0.28) and an interaction between stimulus continuity and switch direction (F_1,12 = 3.9, P = 0.072,

η_{p}^{2}

= 0.25). While interindividual differences in the amount of looming bias were generally large [continuity (cont.): M = 15%, SD = 19%; discontinuity (discont.): M = −0.32%, SD = 19%], only 1 of 13 listeners showed a markedly larger looming bias for discontinuous stimuli compared with continuous stimuli (data provided in Outlier Evaluation for Experiment II and Fig. S3). The interaction became highly significant after removing this one outlier from the analysis (F_1,12 = 15, P < 0.01,

η_{p}^{2}

= 0.57). Removal of this outlier had no significant effect on the other statistics.

Fig. S3.

Comparison of behavioral looming bias between the continuous and discontinuous stimulus conditions of Exp. II. Note that listener S09 responded very differently compared with others.

Response times were measured with respect to the onset of spectral contrast switches and were evaluated only for the dominant response associations (data provided in Response Times in Experiments I and II and Fig. S4). In Exp. I, early response times (25% percentiles) were slightly faster for decreasing compared with increasing contrasts (F_1,14 = 5.2, P = 0.039,

η_{p}^{2}

= 0.27) and for larger contrast switches (F_1.5,21 = 6.0, P = 0.014,

η_{p}^{2}

= 0.30). Differences in response times for decreasing vs. increasing contrasts were smaller for later responses (30–90% percentiles). In Exp. II with speeded responses, variance in response times dramatically increased and neither early nor late response times differed significantly.

Fig. S4.

Response times (relative to the onset of spectral switch) of judgements consistent with spectral contrast switch in Exp. I (*Left*) and Exp. II (*Middle* and *Right*). Only the fastest quarter (25% percentiles) of responses were evaluated per listener. Results of Exp. II are separated between listening conditions with continuous stimuli (*Middle*) and discontinuous stimuli (*Right*).

Event-Related Potentials.

We investigated the neural underpinnings of auditory looming bias by means of ERPs. Fig. 3 shows the ERPs elicited by stimulus onset and spectral switch. The stimulus onset (Fig. 3A) was accompanied by a frontocentral negativity (onset-N1) followed by a central positivity (onset-P2). Average onset-N1 and onset-P2 amplitudes were measured for each spectral contrast (C₁) at the Cz site within the intervals of 80–140 ms and 140–280 ms, respectively, as determined by zero-crossings of the grand average. Onset-N1 amplitudes were not significantly different across spectral contrasts (F_1.6,23 = 1.9, P = 0.18,

η_{p}^{2}

= 0.12). Onset-P2 amplitudes were significantly larger for the original contrast (C₁ = 1) compared with the flattened contrast (C₁ = 0) but not to the intermediate contrast (C₁ = 0.5; see Fig. 3 for significance levels) (F_1.9,26 = 6.7, P < 0.01,

η_{p}^{2}

= 0.32).

Fig. 3.

Event-related potentials (n = 15) evoked by stimulus onset (A) and stimulus switch (B and C). (A and B) Event-related potential waveforms (A and B, *Left*) and extracted N1 and P2 amplitudes (A and B, *Middle*) at vertex electrode Cz. Note that central P2 activity reflects distance perception at stimulus onset and looming bias at stimulus switch. Error bars reflect SEM. Levels of significance: *P < 0.05, **P < 0.01, ***P < 0.001. (A and B, *Right*) Grand average topographies for N1 and P2 latency windows. (C) Significant cluster in time and space (sites marked by asterisks) that is distinctive between decreasing and increasing spectral contrasts overlaid on the corresponding difference topography.

The latencies of ERP components elicited by switches in spectral contrast were similar to the stimulus onset and were chosen at 90–150 ms for switch-N1 and 150–290 ms for switch-P2 (Fig. 3B). As for the onset-ERPs, these time ranges were determined on the basis of zero-crossings of the grand average; the only exception was the P2 end point, which had to be extrapolated because no final zero-crossing occurred. Switch trials (C₁ ≠ C₂) all elicited N1–P2 complexes. Constant trials with no contrast switch (C₁ = C₂) elicited neither N1 nor P2 activity (dash-dotted line in Fig. 3B, Lower Left); these trials were therefore excluded from statistical analyses. Switch-N1 amplitudes differed significantly across contrast pairs (F_1.9,26 = 4.6, P = 0.022,

η_{p}^{2}

= 0.25) but not switch direction (F_1,14 = 1.8, P = 0.20,

η_{p}^{2}

= 0.12). After multiple comparison correction of post hoc paired comparisons, none of the switch-N1 amplitudes were significantly different across contrast pairs, but their magnitudes tended to increase with increases in contrast change. Because the scalp topography of the switch-N1 showed a right-hemispheric dominance, we confirmed that switch-N1 amplitudes evaluated at the more frontolateral electrode site F4 yield similar statistical results (contrast pairs: F_1.7,24 = 7.2, P = 0.005,

η_{p}^{2}

= 0.34; switch direction: n.s.).

In contrast to switch-N1, switch-P2 amplitudes differed significantly, not only across contrast pairs (F_1.7,24 = 16, P < 0.001,

η_{p}^{2}

= 0.54) but also switch direction (F_1,14 = 12, P < 0.01,

η_{p}^{2}

= 0.46) [interaction not significant (F_1.6,23 = 0.96, P = 0.38,

η_{p}^{2}

= 0.064)]. Switch-P2 amplitudes for the most salient contrast pair (0 ↔ 1) were significantly larger than for both intermediate pairs (0 ↔ 0.5 and 0.5 ↔ 1). Regarding switch direction, switch-P2 amplitudes were larger for decreasing contrasts (predominantly perceived as approaching) than for increasing contrasts (predominantly perceived as receding). In other words, the amplitude of the switch-P2 was greater for approaching sounds and thus constituted a neural correlate of auditory looming bias.

A post hoc cluster-based permutation test was performed to quantify statistical differences in the scalp distribution and the timing of the elicited neural responses for increasing vs. decreasing spectral contrast. This test revealed that switch direction was best represented by a single spatiotemporal cluster with a broad central topography and latencies between 120 ms and 200 ms. Fig. 3C shows the spatiotemporal coverage on top of a topographic representation of potential differences between switch directions averaged across listeners and contrast pairs. Potential differences of up to 1 μV occur around the central electrode Cz and around latencies of 160 ms in favor of switch directions with decreasing spectral contrasts (approaching sounds). This potential difference was ≈1 μV in magnitude, or about half the size of the maximum grand-average peak amplitude (switch-P2 at Cz).

Discussion

We investigated the mechanisms underlying auditory looming bias with combined EEG and psychoacoustic measurements in an individualized virtual auditory environment. Our results show that looming bias can be elicited by changes of spectral cues, while maintaining overall sound intensity. Interestingly, the bias only arises if the stimulus is temporally continuous, leading to perceived motion; it is not present when there is a temporal gap between presentations of sounds at different perceived distances. Looming bias was observed both in increased response consistency and increased central cortical activity in the early P2 time range.

Spectral Contrast Manipulation Affected Auditory Distance Perception.

Consistent with previous studies on distance perception and sound externalization (27–29), our individualized spectral cue manipulations affected perceived distance. Natural spectral cues (C = 1, measured at a distance of 1.5 m) created externalized auditory percepts that were perceived as farthest from the listener, whereas flattened spectra (C = 0) created percepts of sounds that were very close, often “internal” to the head (29). In everyday life, internalized sources are often perceived when listening via head- or earphones, where natural spectral cues are not present. The intermediate spectral contrast (C = 0.5) in our experiment is not necessarily physically plausible, but it shows that perceived distance gradually decreases with reduced salience of spectral cues.

A recent psychoacoustic study showed gradual degradations of elevation localization based on very similar spectral manipulations (33), which suggests that spectral cues affect localization performance similarly in both elevation and distance. In fact, some of our listeners associated spectral cue manipulations at some source angles with changes in elevation; this is why we selected the source angles used in subsequent testing separately for every individual. The presence of a looming bias in our motion discrimination task confirms that perceptual changes mainly occurred in the intended distance dimension and not in elevation.

The nature of cues that lead to changes in distance perception can only be debated. Initially we intended to manipulate the salience of monaural spectral shape cues that are arguably most important for sound localization within a sagittal plane (34). Monaural loudness predictions, however, revealed that smaller spectral contrasts were accompanied by a decrease in loudness at low and mid frequencies and an increase in loudness at high frequencies. Hence, systematic changes of sound intensity at high frequencies could help explain the apparent looming bias. For lateral source directions, the frequency-dependent loudness changes also differed between ipsi- and contralateral ears such that low-frequency ILDs systematically increased for smaller spectral contrasts. This frequency-dependent ILD change is in the same direction as naturally occurring, distance-dependent changes of ILDs (35), which is arguably a prominent auditory distance cue (18, 29, 36). Further experiments and/or modeling analyses will be required to clarify the perceptual weighting of these potential distance cues.

Perception of Motion in Distance but Not Changes in Overall Loudness Are Required to Induce Looming Bias.

As summarized earlier, looming bias was previously represented in a broad variety of psychoacoustic measures that are either only remotely (e.g., ratings of loudness change or duration) or indirectly (e.g., absolute distance estimation) related to motion perception in distance. Instead of using more inferential measures, here, we directly measured response consistency in a forced-choice motion discrimination task because it was not necessarily clear that perceived distance, and thus induced looming biases, would be consistently affected using our spectral manipulation technique.

Our analysis shows that there was a small concomitant change in predicted overall loudness that opposed the dominant percept of motion direction. Thus, our data demonstrate that changes in overall sound intensity are not required to elicit looming bias. Moreover, in previous looming studies, overall sound intensity was usually changed slowly over time to simulate motion in distance. Here, we elicited looming bias by a near instantaneous (cross-fade of 10 ms) change of spectral cues in an ongoing stimulus. If the stimulus was interrupted by a short gap (100 ms) during which spectral cues changed, looming bias did not occur; these stimuli were likely perceived as two different auditory objects at different distances, rather than a single auditory object approaching or receding the listener. Distance perception itself was, however, negligibly affected by stimulus dis/continuity as indicated by similar overall response consistencies.

Previous studies also argued that noise stimuli are less or not at all effective in inducing looming bias compared with tone complexes, because noise stimuli might be less likely perceived as a single auditory object (2, 11). The fact that we found looming bias based on noise stimuli suggests that this stimulus dependence might be specific to intensity-based distance simulations.

Taken together, our behavioral findings suggest that looming bias does not require specific tasks, distance cues, or stimuli for its activation; it rather constitutes a general increase of salience whenever sounds are perceived as moving toward vs. away from a listener.

Cortical Activity After 120 ms Reflected Salience of Spectral Cues and Perceptual Looming Bias.

At stimulus onset, the similarity of onset-N1 amplitudes across conditions underscores that sound intensities were similar across spectral contrast conditions (37). Unlike onset-N1, central onset-P2 amplitudes increased with spectral contrast, i.e., salience of spectral cues. For sound localization, spectral cues are more important in the median plane compared with the horizontal plane. Consistent with our findings, a previous magnetoencephalography (MEG) study found that spatial deviants in the median plane were associated with increased neural response amplitudes within a similar latency range (200–250 ms) as our onset-P2, whereas spatial deviants in the frontal horizontal plane elicited earlier differences (100–150 ms) (38). Moreover, a previous ERP study tested spectral changes comparable to ours in a left- vs. rightward motion discrimination task and did not find any differences in onset-P2 amplitudes (25). For this task, however, spectral cue evaluation was not essential, whereas in our experiment, listeners had to evaluate changes in distance based on spectral cues. This suggests that in our study, the onset-P2 reflects early distance-evaluation processes based on spectral cues that may lead the percept of sound source distance.

At the onset of the spectral switch, larger spectral changes elicited larger switch-N1 magnitudes, independent of motion direction. Conversely, the switch-P2 magnitude depended on motion direction and reflected auditory looming bias. These results are in line with the current functional understanding of how vertical and horizontal motion is reflected in ERPs. Specifically, switch-N1 responses are associated with the process of spectral change detection and switch-P2 responses with motion direction evaluation (23–26). Hence, our ERP results suggest a generalization of the motion onset response to the distance dimension and further support the conclusion that motion perception is required to elicit auditory looming bias.

Regarding the timing of looming perception, it has been shown that harmonic sounds increasing in intensity lead to enlarged visual cortical activity as early as 80 ms from stimulus onset (15, 22). Our results find looming bias reflected after 120 ms from the switch, primarily at central sites associated with auditory processing, which suggests that activation of looming bias requires higher-order processing of auditory motion. This apparent discrepancy may reflect the fact that processing of increments in intensity is less demanding and faster than sound motion inferences based on spectral analysis. Another possibility is that, at early processing stages, sound intensity increments are not associated with changes in distance but are simply salient events. For instance, interaural spatial cues occurring at intensity increments are weighted strongly when computing left–right location (39, 40). A strong effect of left–right congruence between increasing-intensity sounds and visual targets (41) further supports this reasoning.

Listeners showed large interindividual differences in the amount of behavioral looming bias they exhibited. Preliminary analyses showed that switch-P2 amplitudes and measures of global field power might be able to explain these interindividual differences with a potential specificity in gender (Correlation Between Interindividual Neural and Behavioral Looming Biases and Fig. S5). While our sample size was sufficient to manifest the strong effect of looming bias within subjects (

η_{p}^{2}

≥ 0.33), future investigations with larger and gender-balanced sample sizes are needed to explore the marked interindividual differences in the amount of behavioral looming bias.

Fig. S5.

Gender-specific correlation analyses between different EEG measures and individual behavioral looming bias. (*Left*) Amplitude difference in switch-P2 at Cz averaged across time (150–290 ms). (*Middle*) Amplitude difference in spatiotemporal cluster averaged across time (120–200 ms) and space (centered around Cz). (*Right*) Difference in global field power averaged across time (120–200 ms). Pearson’s correlation coefficients for all data (black), only female data (pink), and only male data (cyan) are shown within each scatterplot. Levels of significance: *P < 0.05, **P < 0.01, ***P < 0.001.

In summary, our results demonstrate that distance cues other than overall sound intensity can elicit auditory looming bias and that its activation requires higher-level auditory processing related to motion perception. Previous studies on auditory looming bias were exclusively based on sound intensity as a distance cue and might have overinterpreted the effect of this particular stimulus feature. Future studies should review the cortical origins of auditory looming bias based on different modes of activation.

Materials and Methods

Subjects.

We tested 15 paid volunteers within the age of 20–29 y (M = 24, SD = 3.7; 10 females, 5 males) in Exp. I and 12 paid volunteers plus one coauthor within the age of 20–42 y (M = 29, SD = 5.7; 7 females, 6 males) in Exp. II. A subset of five listeners participated in both experiments. None of the listeners had any indication of a hearing loss greater than 20 dB relative to the threshold of the normal-hearing population, as confirmed by audiometric thresholds for frequencies from 500 Hz to 8 kHz. All subjects gave informed consent as overseen by the Boston University Institutional Review Board.

Stimuli and Procedure.

Sound source locations were simulated based on individualized measurements of head-related transfer functions (HRTFs) (measurement procedure described in Measurements of Head-Related Transfer Functions). Stimuli consisted of two consecutively presented Gaussian white noises filtered by the band-limited HRTFs with a specific spectral contrast factor,

C \in {0, 0.51}

. The magnitude spectrum,

M_{1} (f)

, in decibels of the measured HRTFs was manipulated within a frequency range between 1 and 16 kHz (

N_{f}

frequency bins) according to

C

as:

M_{c} (f) = C M_{1} (f) + (1 - C) \frac{1}{N_{f}} \sum_{k \in f} w^{'} (k) M_{1} (k),

with a frequency weighting function,

w^{'}

, that approximates auditory frequency resolution by the across-frequency derivative of equivalent rectangular bandwidths (ERBs) (42). The measured phase spectrum remained unmodified. Filtered noise stimuli were band limited between 1 kHz and 16 kHz by a fourth-order Butterworth filter. Two filtered noise representations were cross-faded by 10-ms sine ramps to form a stimulus pair. This stimulus pair was faded in and out by 50-ms squared sine ramps. Details on stimulus delivery are provided in Stimulus Presentation and Experimental Procedure.

In Exp. I, 840 trials were presented per listener. The spectral contrast switched in 86% of all trials (14% constant trials). In Exp. II, 192 trials were presented and spectral contrast always switched (no constant trials). The presentation order of trials was completely randomized in both experiments. Listeners received block-based feedback only on their detection accuracy of constant trials in Exp. I (no feedback in Exp. II). Feedback was never provided on switch trials to prevent listeners from learning to use cues other than perceived distance (e.g., timbre) for discrimination. Further details on the experimental procedure are also provided in Stimulus Presentation and Experimental Procedure.

EEG Recordings.

The EEG of 32 scalp electrodes (Activetwo system with Activeview acquisition software, Biosemi B.V.) was recorded in Exp. I with standard 10/20 montage. In addition, one vertical and two lateral eye electrodes were recorded to better capture eye blinks and saccades, and external reference electrodes were placed at the mastoids (not used) and ear lobes. Real-time processing hardware (RP2.1, Tucker Davis Technologies, Inc.) marked timing of critical experimental events, which were recorded to an additional data channel alongside the EEG data. The postprocessing procedure for EEG data analysis is described in Processing of EEG Data.

Statistics.

For statistical analyses, we used the Matlab Statistics Toolbox. We conducted repeated-measures analyses of variance with Greenhouse–Geisser correction (functions: fitrm, ranova). Tukey’s honest significant difference tests were conducted for post hoc paired comparisons (multcompare). Effects were considered significant at P < 0.05.

Data Availability

Code for stimulus creation (sig_baumgartner2017looming), HRTFs, and experimental results (both data_baumgartner2017looming) are integrated in the Auditory Modeling Toolbox (43). EEG recordings and analysis scripts are provided via Zenodo (44).

Measurements of Head-Related Transfer Functions.

To measure HRTFs, listeners were seated at the radial center of a half circle created by seven loudspeakers (Acoustic Research 215 PS; amplified by Crown 1002 XTi) in a semianechoic booth (IAC Acoustics, single-walled, 12 ft × 13 ft). The radius was 1.5 m and the loudspeakers were elevated approximately at ear level and provided an angular sampling of 30° ranging from −90° at the right of the listener to 90° at the left of the listener. Absorbing foam wedges were placed on the floor and the wall to the back of the listener to reduce early reflections. Binaural miniature microphones (AuPMC002; Ausim, Inc.) were inserted into the listener’s ear canals, blocked by the microphone and foam seals. Transfer characteristics were identified by a maximum length sequence (MLS) of order 15, repeated 20 times. Recorded signals were amplified (Motu Ultralite) and analog-to-digital converted (Motu 24I/O) with a sampling rate of 44.1 kHz. Extracted impulse responses were windowed to 3 ms with 0.5-ms cosine ramps to remove reflections and equalized by reference measurements for each loudspeaker with the microphones placed at the center of the loudspeaker setup.

Stimulus Presentation and Experimental Procedure.

Stimulus presentation was timed with Psychtoolbox (45) as illustrated in Fig. 1C. Stimuli were delivered via headphone preamps (HB7; Tucker Davis Technologies, Inc.) and tubephones (ER-2; Etymotic Research, Inc.). Transfer properties were measured by an artificial ear coupler to calibrate the delivered sound pressure level for the frontal stimulus to be around 75 dB and to assure spectral magnitude deviations to be within ±5 dB as stated in the tubephones’ spec sheet. Further equalization of the headphone transfer functions was not applied because an informal listening test suggested that a perfect equalization does not seem to be critical to create externalized sounds with the headphones we used.

Before starting the EEG-monitored Exp. I, listeners completed a stimulus selection procedure (pretest), which consisted of two steps. First, listeners familiarized themselves with the sounds by listening to examples of the 1→0 contrast switch condition at all three different source angles and reported whether they associated the spectral contrast switch with a change in distance (alternative response options: no spatial change, change in elevation). The contrast switches were always associated with spatial changes, but for some listeners and angles, the manipulation only caused changes in perceived elevation. In total for the 15 listeners, one right source, five frontal sources, and zero left sources were excluded from further testing based on the listeners’ reports. Second, listeners were tested on their ability to discriminate between approaching, receding, and static auditory percepts in a 3-AFC task to select the source angle (one out of three) for which each individual demonstrated best discriminability. All switch trials were tested six times, and all constant trials were tested two times. Contrast combinations were randomized and source angles were blocked in randomized order. Fig. 1A shows the selected distribution of source angles across listeners finally tested in the following EEG-monitored experiment. The whole stimulus selection procedure lasted up to 30 min.

In the EEG-monitored Exp. I, listeners performed the same discrimination task for the individually selected source angle with 120 repetitions of every switch trial and 40 repetitions each for the three constant trials. The presentation order of these 840 trials was completely randomized and grouped in 40 blocks of about 1 min. Listeners were able to take breaks between blocks and had to take breaks of at least 1 min after every 10th block. The whole recording session lasted about 1 h.

Before starting Exp. II, listeners performed only the first step of the pretest of Exp. I, which now also included discontinuous example stimuli with an ISI of 100 ms. If listeners reported that they associated example stimuli with changes in distance at multiple source directions, we randomly chose between left and right. Following this procedure, the left direction was selected for eight listeners and the right direction for five. The stimuli and motion discrimination task of Exp. II was identical to Exp I, with the exceptions of including the condition of discontinuous stimuli with an ISI of 100 ms, allowing immediate responses, and removing the constant/static stimulus/response alternative. After two short blocks of 12 trials for training, listeners were tested on 12 blocks of 16 trials. Subjects completed the entire protocol for Exp. II in about 30 min.

Processing of EEG Data.

Processing of EEG data was conducted with the Eeglab toolbox (version 13.6.5b) (46) in Matlab (Mathworks, Inc.). EEG signals were referenced to the two ear lobe electrodes. Three bad channels were identified manually throughout the recording sessions. A finite-impulse-response filter with Kaiser window design (

β = 7.2

,

n = 462

) limited the bandwidth of signals between 0.5 Hz and 20 Hz. Bandlimited signals were epoched and then resampled to 100 Hz. Epochs were baseline corrected to the 200-ms long time window preceding the epoching event. Epochs exceeding thresholds of −200 μV and 800 μV for brain channels and −800 μV and 200 μV for eye channels were removed. The extended Infomax approach was used to analyze independent signal components (option: binica). Up to three independent components per subject were manually selected to correct for eye artifacts. After this correction of eye artifacts, bad channels were interpolated and brain channel epochs that exceeded a final threshold of ±70 μV were removed. For every listener and condition, at least two thirds of the trials (minimum 83 trials per condition) did not exceed the final threshold for rejection. The number of trials per condition was equalized within listeners by removing trials selected to be equally distributed across the recording session.

We used the Matlab Fieldtrip toolbox (FTP release 20160526) (47) to perform a cluster-based permutation test (function: ft_timelockstatistics) (48). The significance probability of a cluster was calculated as the Monte Carlo estimate of the P value under a distribution of 500 permutations. Dependent single-sample T statistics (two tailed) were thresholded at an alpha level of 0.05 and summed up to form cluster-level statistics. The maximum of the cluster-level statistics was then evaluated as the actual test statistic (alpha level of the permutation test: 0.025).

Interindividual Comparison of Frequency-Specific Loudness Changes.

Fig. S1 shows that the frequency-specific loudness changes relative to the reference spectral contrast C = 1 are similar across listeners in Exp. I. Decreasing spectral contrast generally caused loudness to decrease at low to mid frequencies and to increase at high frequencies, and caused interaural level differences to increase at low frequencies and decrease at high frequencies. Only the two listeners—those tested for frontal sources—are exceptional because interaural differences are very small for positions in the median plane.

Behavioral Results of Pretest for Individual Source Angle Selection.

Fig. S2 shows the results of the pretest for source angle selection. The experimental results were very similar to those from the EEG experiment (Exp. I) despite a generally smaller response consistency. Response consistency improved in the EEG experiment possibly because of the individual selection of the source angle and accumulated task experience.

Outlier Evaluation for Experiment II.

One listener (S09) of Exp. II showed a very strong looming bias in response consistency for the discontinuous stimuli, but a negative looming bias in response consistency for the continuous stimuli (Fig. S3). This behavior was very different from the other listeners of Exp. II and also from her own results from Exp. I, where she showed a small but positive consistency bias. For this reason, we excluded her results from statistical analyses of Exp. II.

Response Times in Experiments I and II.

In contrast to the clear representation of looming bias in response consistency, its representation in response times was not clear. On the one hand, the fastest response times (25% percentiles) in Exp. I reflected the auditory looming bias (Fig. S4, Left), even though the listeners were not instructed to respond as fast as possible and they had to withhold their response until stimulus offset. On the other hand, later response times (30–90% percentiles) in Exp. I as well as speeded responses in Exp. II (Fig. S4, Middle and Right) were not significantly different between increasing and decreasing contrast conditions. Late responses are probably less distinctive because response certainty is better reflected in early responses. The large variance of response times in Exp. II suggests that at least some listeners had difficulties in rapidly discriminating motion direction and providing their response. More training and/or a different response method may be required to reveal differences in response times.

Correlation Between Interindividual Neural and Behavioral Looming Biases.

According to our ERP analyses differences in (i) central switch-P2 amplitudes (“P2 amplitude bias”) and (ii) cluster-average potentials of a broad spatiotemporal cluster centered around 160 ms and central sites (“cluster amplitude bias”) were potential candidates for predictors of individual behavioral looming bias. We used multivariate linear regression models (Matlab Statistics Toolbox: fitlme) to assess these predictive relationships. To account for interindividual differences in overall ERP intensity, we standardized (z score) potentials within subjects and quantified the individual neural bias as the z-score difference between increasing and decreasing contrasts, averaged across contrast pairs. Similarly, individual behavioral looming biases were quantified as the standardized percentage difference between consistent approaching and receding responses, also averaged across contrast pairs.

We first investigated predictability based on the P2 amplitude biases. A simple linear regression between P2 amplitude bias and behavioral bias would have suggested that the biases were not related to each other (r = 0.22, P = 0.37) (Fig. S5, Left). However, in line with a previous study based on skin conductance (3), we observed gender differences in the distribution of the P2 amplitude bias. Individual P2 amplitude biases were positive for every single female listener, whereas the biases of the males were widely distributed. Including gender in the regression model revealed a relationship between P2 amplitude bias and behavioral bias (β = 33, t₁₁ = 6.3, P < 0.001) that depended on gender (male coded as 1: β = 29, t₁₁ = 5.6, P < 0.001; interaction term: β = −38, t₁₁ = 6.1, P < 0.001). Pearson’s correlation coefficients further suggested that correlation between P2 amplitude bias and behavioral looming bias was strong for females (r = 0.95, P < 0.001) but not statistically significant for males (r = −0.28, P = 0.45).

In contrast to this strong correlation between P2 amplitude bias and behavioral bias for female listeners, similar regression analyses based on the cluster amplitude biases revealed no significant relationships (Fig. S5, Middle).

A previous looming study showed that enhancements in reaction times due to audiovisual looming congruency were correlated with enhancements in global field power (22). Hence, we also evaluated differences in global field power within the time range of the cluster and analyzed its correlation with the behavioral looming bias (Fig. S5, Right). The power differences were not biased as much as the P2 amplitude or cluster amplitude differences, but revealed surprisingly high correlation with behavioral looming biases, even when both genders were pooled together.

It is unclear why the less-biased differences in global field power better reflect interindividual differences in behavioral looming bias than do the maximally biased differences in cluster amplitude. Future investigations with a larger sample size will be necessary to clarify these relationships also with respect to gender differences.

Data Availability

Data deposition: Code for stimulus creation (sig_baumgartner2017looming), HRTFs, and experimental results (both data_baumgartner2017looming) are integrated in the Auditory Modeling Toolbox (amtoolbox.sourceforge.net). EEG recordings and analysis scripts are provided via Zenodo (https://doi.org/10.5281/zenodo.832899).

Acknowledgments

We thank Gerald Kidd Jr. and Norbert Kopčo for fruitful discussions and Tim Streeter and Leonard Varghese for technical support. This work was mainly funded by R.B.’s individual fellowship from the Austrian Science Fund (Project SpExCue, Grant J3803-N30). Additional funding was provided by the European Commission (Project ALT, Grant 691229) and the Boston University Hearing Research Center.

Supporting Information

Supporting Information (PDF)

Download
1.04 MB

References

1

JH Riskind, EM Kleiman, E Seifritz, J Neuhoff, Influence of anxiety, depression and looming cognitive style on auditory looming perception. J Anxiety Disord 28, 45–50 (2014).

Featured Topics

Articles By Topic

Featured Topics

Articles By Topic

Featured Topic

Articles By Topic

Significance

Abstract

Sign up for PNAS alerts.

Results

Loudness Predictions.

Behavioral Results.

Event-Related Potentials.

Discussion

Spectral Contrast Manipulation Affected Auditory Distance Perception.

Perception of Motion in Distance but Not Changes in Overall Loudness Are Required to Induce Looming Bias.

Cortical Activity After 120 ms Reflected Salience of Spectral Cues and Perceptual Looming Bias.

Materials and Methods

Subjects.

Stimuli and Procedure.

EEG Recordings.

Statistics.

Data Availability

Measurements of Head-Related Transfer Functions.

Stimulus Presentation and Experimental Procedure.

Processing of EEG Data.

Interindividual Comparison of Frequency-Specific Loudness Changes.

Behavioral Results of Pretest for Individual Source Angle Selection.

Outlier Evaluation for Experiment II.

Response Times in Experiments I and II.

Correlation Between Interindividual Neural and Behavioral Looming Biases.

Data Availability

Acknowledgments

Supporting Information

References

Information

Published in

Classifications

Copyright

Data Availability

Submission history

Keywords

Acknowledgments

Notes

Authors

Affiliations

Notes

Competing Interests

Metrics

Citation statements

Altmetrics

Citations

Cite this article

Cited by

View options

PDF format

Login options

Recommend to a librarian

Purchase options

Restore content access

Share

Share article link

Share on social media

Further reading in this issue

Insulin-like growth factor 1 receptor regulates hypothermia during calorie restriction

Emerging role of wetland methane emissions in driving 21st century climate change

The 4E-BP growth pathway regulates the effect of ambient temperature on Drosophila metabolism and lifespan

Postdoc publications and citations link to academic retention and faculty success

Bodily maps of emotions

Gut dysbiosis: Ecological causes and causative effects on human disease

Sign up for thePNAS Highlights newsletter