Somatosensory function in speech perception

We examined whether we could change the perceptual boundary between 2 words by using speech-patterned somatosensory input that was produced by stretching the facial skin as subjects listened to the stimulus words. We used a robotic device (Fig. 1) to create patterns of facial skin deformation that would be similar to those involved in the production of head and had. We tested 3 directions of skin stretch (up, down, backward) with 3 different groups of subjects. We also carried out 2 control tests in which we assessed the effects on speech perception of patterns of skin stretch that would not be experienced during normal facial motion in speech. One involved a twitch-like pattern of skin stretch; the other involved static skin deformation.

We found that the perceptual boundary between head and had was altered by the skin stretch perturbation. Fig. 2 shows, for a single subject, a representative example of the perceptual modulation using a single cycle of 3-Hz skin stretch in an upward direction. The figure shows the probability that the subject judged the stimulus word as had for each of the 10 stimulus levels on the head to had continuum. The fitted lines give the estimated psychometric function, with the control condition shown in blue and the skin stretch condition shown in red. As can be seen, the value corresponding to the 50th percentile of the estimated psychometric function shifted to the right in the perturbation condition. This means that the upward skin stretch resulted in an increase in the probability that a word was identified as head. It can also be seen that the skin stretch perturbation had different effects at different points along the stimulus continuum. The perceptual effect of the skin stretch was greatest for intermediate values of head and had and had little effect near the ends of the continuum, where the stimuli were clearly identified as one word or the other.

We found that speech sound classification changed in a predictable way depending on the direction in which we stretched the skin. When the skin was stretched upward, the stimulus was more often judged as head. When the skin was stretched downward, the stimulus sounded more like had. When the stretch was in a backward direction, there was no perceptual effect. Fig. 3 summarizes the mean difference between the control and the perturbation conditions for the 3 directions of 3-Hz skin stretch by using mean probability (Fig. 3A) and identification cross-over (Fig. 3B) as dependent measures. The error bars give the standard error of the difference between means (i.e., between perturbation and control conditions). We assessed perceptual differences due to skin stretch and skin stretch direction by using a 2-way ANOVA, with one repeated factor (perturbation versus control trials) and one between the subject's factor (skin stretch direction). Bonferonni-corrected comparisons showed that when we stretched the facial skin upward, the identification cross-over value increased and the mean probability of responding had decreased (P < 0.01 in both cases). When we stretched the facial skin downward, we observed the opposite pattern. Specifically, the identification cross-over value decreased and the mean probability of responding had increased (P < 0.025 for both post hoc tests). Finally, when we stretched the skin backward there was no effect on identification performance (P > 0.85). These systematic changes in speech sound classification as a function of skin stretch direction indicate that somatosensory information associated with the facial skin stretch can play a role in the perception of speech sounds. The modulation suggests a close tie between the neural processes of speech perception and production.

We assessed the extent to which the perceptual classification of speech sounds was dependent on the specific temporal pattern of facial skin stretch. For the data described above, we used 3-Hz sinusoidal patterns of skin stretch to approximate the temporal pattern of facial skin deformation that would normally occur in conjunction with the jaw lowering and raising movements in the production of head and had. Here we examined whether the skin stretch pattern had to be speech-like if changes in auditory classification were to be observed. We compared the previously obtained responses by using 3-Hz skin stretch with 2 new patterns: a single cycle of 9-Hz stretch and a pattern involving static skin stretch (see Methods). We focused on the effects of skin stretch in an upward direction because this had previously produced the largest perceptual effects.

Fig. 4 summarizes the difference between control and perturbation conditions in terms of mean probability (Fig. 4A) and identification cross-over value (Fig. 4B). The error bars show the standard error of the difference between means across subjects. Data for the 3-Hz condition are repeated from the previous section. Two-way ANOVA followed by Bonferroni-corrected comparisons was used to test for differences between skin stretch and control conditions for the 3 temporal patterns of stretch. In contrast to the reliable perceptual modulation observed in response to the 3-Hz pattern (P < 0.02), there was no perceptual change associated with either the 9-Hz twitch-like stretch pattern (P > 0.75) or the static stretch pattern (P > 0.10). This result suggests that perceptual modulation of speech sounds by somatosensory information is dependent on a speech-like temporal pattern of somatosensory change. To the extent that this can be extended to speech production and perception more generally, it would suggest that the effects of production on speech perception are movement specific.

The principal finding of the present study is that the perception of speech sounds is modified by stretching the facial skin and that the perceptual change depends on the specific pattern of deformation. Moreover, the presence of any perceptual change depended on the temporal pattern of the stretch such that perceptual change was present only when the timing of skin stretch was comparable to that which occurs during speech production.

Evidence to date for the idea that speech production and perception systems have a common neural substrate has come entirely from work on motor function and has been motivated by the motor theory of speech perception (17, 18). For example, studies using transcranial magnetic stimulation (TMS) have recently documented the involvement of the premotor and motor cortex in the neural processing of speech (19–21). These studies have shown that the evoked electromyographic response to TMS to the lip area of motor cortex is facilitated by watching speech movements and listening to speech sounds (20), and that repetitive TMS to premotor cortex affects performance in a speech perception task (21). In contrast, the possible effects of somatosensory function on speech perception have been unexplored despite extensive evidence that somatosensory inputs are important in speech production as well as in motor function more generally. Present findings show that speech perception is linked not only to cortical motor areas involved in speech production but importantly is also affected by the kinds of somatosensory inputs that would normally arise in conjunction with speech production.

The modulation of speech perception observed in the present study may arise as a consequence of facial somatosensory input to facial motor and premotor areas, a pattern that would be consistent with the motor theory of speech perception (17, 18). However, somatosensory inputs may also affect auditory processing more directly. A number of studies have reported bidirectional effects linking somatosensory and auditory cortices (2–5). Indeed, activity due to somatosensory inputs has been observed within a region of the human auditory cortex that is traditionally considered unisensory (3). It is also possible that the observed perceptual effects arise at the subcortical level, in particular, in the superior colliculus, which is considered a site of multisensory integration, including auditory-somatosensory interaction (1).

The role of somatosensory inputs in speech perception was investigated by using a different approach in the facial somatosensory system (i.e., by stretching the facial skin). Somatosensory function associated with facial skin deformation is a little-recognized source of orofacial kinesthesia. However, the facial skin is rich in cutaneous mechanoreceptors (15) and is systematically deformed in the context of normal speech production (16). Somatosensory inputs associated with facial skin deformation are a primary source of sensory support for speech motor function owing to the lack of muscle proprioceptors in many orofacial muscles (8–11).

The neural responses of cutaneous mechanoreceptors are dependent on the direction of sensory input both in the orofacial system and in the hands (22). Moreover, in recent studies of somatosensory function in the limbs (12–14) it has been reported that stretching the skin results in a sensation of movement if the skin is stretched in a manner corresponding to normal movement. Different patterns of skin stretch at the side of the mouth may similarly bias the perception in the associated direction. Stretching the skin lateral to the oral angle, which is the area that we focused on in the present study, induced a cortical reflex that was associated with a modification of lip position in response to a sudden change in the position of the jaw (23). Because the cutaneous mechanoreceptors lateral to the oral angle are activated during speech movements and especially jaw motion (15), stretching the facial skin lateral to the oral angle could provide kinesthetic information concerning articulatory motion. This information may both complement and shift the perception of the speech sounds.

Seventy-five native speakers of American English participated in the experiment. The subjects were all young adults, had normal hearing, and had no neurological deficits. There were 5 separate experimental conditions, and 15 different subjects were tested in each condition. All subjects signed the approved Yale University Human Investigation Committee informed consent form.

We thank Douglas N. Honorof, Rafael Laboissière, Andrew A. G. Matter, David W. Purcell, and Bruno Repp for advice and assistance. This work was supported by National Institute on Deafness and Other Communication Disorders Grant DC-04669.