First author

That humans use visual information in processing speech is well known, as is the fact that people produce tiny puffs of air when making certain speech sounds. But until now nobody had succeeded in determining whether listeners use the tactile information produced by those puffs in speech perception. Now, phonetician Bryan Gick and his colleague Donald Derrick at the University of British Columbia in Vancouver, Canada, have shown that humans do combine tactile information with auditory information to help them perceive human speech (see page 502). Gick tells Nature more.

What inspired your study?

For the past 30 years or so, tight coupling has been assumed to exist between what people see and hear because of shared experience between these two senses. If there's a discrepancy, most people go with what the eyes perceive, because the eyes usually are a more reliable source than the ears. However, many in the field suspected that there might be a looser coupling between what people feel and what they hear. We wanted to find out whether a listener could integrate tactile information to help them hear.

Why did you use puffs of air as the tactile input?

We wanted to test whether a listener would use their tactile sense without being aware of it. When you make certain speech sounds in English, such as 'pa', you release a puff of air. When you say 'ba', you don't. The question was, if a listener were to hear someone saying 'ba' but was also touched by a puff of air — so lightly that they were not conscious of it — would their brain trick them into thinking they had heard 'pa'?

How did you generate the air puffs?

We started out with a pilot test, using a turkey baster to produce puffs of air. For the main experiment, we used a standard air compressor and tubing to deliver air puffs with about half the amount of pressure produced in regular speech, so that a lot of volunteers weren't consciously aware of any air puffs at all. Even in the initial pilot, which wasn't very accurate, we saw evidence of auditory override. This is indicative of the robustness of the tactile effect.

Why is this finding important?

Instead of the conventional model, which states that we perceive sound with our ears and eyes alone, we now have a model based on the integration of several senses. The implication is that we're able to take any information that our body s receives from any sense and use it to tell us what's going on in the world.