Visually representing bodies with sound
‘Seeing’ bodies with sound (no sight required)
People born unable to see are readily capable of learning to perceive the shape of the human body through soundscapes that translate images into sound, according to researchers who report their findings in the Cell Press journal Current Biology on March 6. With a little training, soundscapes representing the outlines and silhouettes of bodies cause the brain’s visual cortex — and specifically an area dedicated in normally sighted people to processing body shapes — to light up with activity.
The researchers first taught people to perceive simple dots and lines. Then those individuals learned to connect the lines with junctions or curves, gradually working up to more and more complex images.
The success of the sensory substitution approach suggests great potential for its use in therapy more broadly, the researchers say.
“We’re beginning to understand [that] the brain is more than a pure sensory machine,” Amedi says. “It is a highly flexible task machine. The time has come to revive the focus on practical visual rehabilitation with sensory substitution devices.”
In the current study, the researchers used an algorithm they call vOICe, which translates images in black and white. But they’ve since developed a newer algorithm, EyeMusic, which incorporates color information through the use of different musical instruments. The EyeMusic app is freely available on iTunes, where the researchers hope it can be useful to the blind and the general public alike.
Read the study: Ella Striem-Amit, Amir Amedi. Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People “Seeing” by Using Sounds. Current Biology, 06 March 2014 DOI: 10.1016/j.cub.2014.02.010