the black and white of me: Synaesthesia: a world of wonders

People with synaesthesia can't help but get two sensory perceptions for the price of one. Some perceive colours when they hear words or musical notes, or read numbers; rarer individuals can even get tastes from shapes.

Neuroscientists have now reported¹ another variant, in which flashes and moving images trigger the perception of sounds. The finding could help to identify the precise neural causes of the phenomenon, reportedly experienced by at least one in every hundred people, and suggests that at least some types of synaesthesia are closely related to ordinary perception.

"This [study] will make a big impact," says synaesthesia expert Edward Hubbard of France's biomedical research organization INSERM. "It will affect not just the synaesthesia community, but also researchers interested in how the brain handles information from multiple senses."

Neuroscientist Melissa Saenz of the California Institute of Technology (Caltech) in Pasadena stumbled across the variant last year while giving a group of undergraduate students a tour of her perception research lab. In front of a silent display (see picture) designed to evoke activity in the motion processing centre of the visual cortex, one of the students asked: "Does anybody else hear something?"

The student, Johannes Pulst-Korenberg, reported hearing a distinct whooshing sound when he watched the display. "Everybody was looking at me, like, 'Are you crazy?'" he remembers.

Saenz could find no description of this variant of synaesthesia in the scientific literature. But to her surprise, after screening several hundred people in the Caltech community, she found three more who reported a similar experience.

"They're generally soft sounds, but they can't be ignored, even when they're distracting," says Saenz. "One of the synaesthetes told me that the moving images on computer screen savers are terribly annoying to her. She can't do anything about it but look away."

Pulst-Korenberg, who is pursuing a doctorate in neuroscience and economics at Caltech, says that he has experienced the same effect watching a butterfly fly. "For some reason, the jerky motion generates little clicks," he says.

The sound of change

Saenz and her lab director, Christof Koch, confirmed the four cases with a test that gave true 'hearing–motion' synaesthetes a distinct advantage. They asked 14 people, including the 4 with synaesthesia, to watch two quick sequences of Morse-code type flashes, and then determine whether the sequences were the same or subtly different. As they were able to 'hear' the sequences too, the synaesthetes, could distinguish them much more accurately than 10 people without synaesthesia. When the sequences instead comprised auditory beeps, the synaesthetes again performed well, but having lost their advantage they scored no better than the non-synaesthete controls.

Some neuroscientists attribute synaesthesia to remnant cross-links between closely-spaced cortical areas — links that develop in the early stages of life but are usually pruned away during childhood. In letter-to-colour synaesthesia, for example, the relevant cortical area for recognizing letters turns out to be immediately adjacent to the one for perceiving colour. Another leading theory explains the condition as an excess of feedback signals from multi-sensory regions, where perceptions are usually integrated, down to single-sensory areas.

Brain-imaging studies have provided evidence for both theories, says Hubbard. "But I don't think the type of synaesthesia that Melissa has discovered really fits neatly into either one."

Instead, it might be an enhanced form of the fast, cross-cortical correspondences the brain makes all the time, he suggests. "For example, we find it easier to understand what someone is saying if we can also see their mouth move. So the brain is constantly integrating audition and vision." Delineating how the brain performs this ordinary integration, says Hubbard, "is probably a part of what will be required if we're to explain the type of synaesthesia that Melissa has discovered here."

If hearing–motion synaesthesia is closely related to ordinary cross-sensory correspondence, it might also explain why it has taken so long to discover. "In real life," says Saenz, "things that move or flash usually do make a sound, so that association is more logical than, say, numbers-to-colours."