A new device that links spy glasses, a webcam and a smart phone could make it easier for blind people to "see" shapes by converting visual signals to auditory ones and sending them to another part of the brain.
Its developers also hope that the same device could be used to give a new twist to infrared vision for seeing at night or take sonar to a different level for navigating underwater.
Michael Proulx, a neuroscientist at Queen Mary’s College in London, will be demonstrating the device, know as "vOICe," at the American Psychological Association meeting this week in Washington.
vOICe works by mapping visual images to sound and then providing blindfolded users with a sense of what an object is and where it is located.
"The most user-friendly way is to have a pair of spy glasses or sunglasses with a tiny webcam in the bridge of the nose to provide visual input," Proulx told Discovery News. "Then I use a small pocket PC, which runs the software which takes input and provides auditory input through headphones."
Differences in sound, including pitch and loudness, provide clues to the map. For example, for vertical location, "up" is represented by high frequencies and "down" by low frequencies. Timed-stereo panning, which is much like panning with a video camera, keeps an object within a picture while giving a panoramic view. Horizontal location is indicated by the time it takes for a left-to-right scan of each image. Bright white is heard at maximum volume and dark is silent.
"The program takes visual input from the camera, then scans the image from left to right," Proulx said. "Then you hear this soundscape where the changes in frequency and volume correspond to pixels in the image."
The basic software and hardware to convert visual signals to audio ones has been around for more than 15 years, but advances in computer technology and communications now make it easier for people to actually use a more portable version in their daily lives. That's according Dutch software engineer Peter Meijer, original developer of the vOICe.
One of the biggest challenges is that it takes people three months of training to use it. He also says that he's had trouble getting a big enough demand to make it commercially viable, so he gives away the device software for free and developed an Android mobile application.
Meijer is also hopeful that new kinds of "augmented reality" glasses may be adapted to use his imaging and sound set-up. While these bulky goggles may be find for gamers, many blind people want to look a bit more normal, so more work is also needed in that area.
"The main thing is to work out how to make the brain and the technology meet in the middle," Meijer said. "The technology is mature, but we don't know how the brain deals with complex sounds."
Meijer said neuroscientists like Proulx and others are learning that parts of the brain may be more plastic than we originally thought, and that areas where we process visual images (up to 30 percent of our brain) may receive other sound and touch signals as well.
The two had a small NSF grant to develop a portable device that will be demonstrated this week.
"You could use this to bring in ultraviolet or infrared as well," Meijer said. "If you want to see like a snake sees or if police are looking for someone in a forest, they could use a device like this to augment their vision with infrared information."