For blind students, surfing the Web or taking detailed notes in physics and calculus classes can require cumbersome and expensive learning aids. Two new technologies, however, promise to greatly expand access to text and graphics for the blind community without breaking the bank.
The Pulse Smartpen, which debuts in retail stores this month, records and synchronizes nearly everything a student hears and writes. And WebAnywhere, a free Internet-based service released last month, works as a screen reader by converting Web site text to electronic speech.
Beyond meeting an immediate need in the classroom or office, some researchers believe the products could help dramatically expand career options for the blind by opening new doors to information.
“Many of the solutions that are provided to this community are pretty expensive because the community is small,” said Andy Van Schaack, a lecturer in Vanderbilt University’s Peabody College of Education and Human Development in Nashville, Tenn. But by focusing on a consumer product marketed to the general public, he and colleague Joshua Miele, an associate scientist at the Smith-Kettlewell Eye Research Institute in San Francisco, have found an economical way to close the accessibility gap.
The audio-tactile graphic system resulting from their research centers on the Pulse Smartpen by Oakland start-up Livescribe, for which Van Schaack serves as a senior science advisor.
The pen works just like a regular one if it’s not turned on, he said “But when you turn it on, it’s now actually a computer.”
The smartpen recognizes handwriting through a small infrared camera at its tip that snaps 72 pictures per second, focusing on millions of tiny dots printed on every page of a special notebook accompanying the pen.
“It actually takes a picture of the page and what’s printed onto each page, which is a very fine, almost invisible dot pattern that’s almost like a barcode pattern,” Van Schaack said. The notebook’s unique microdot patterns can help the pen pinpoint the location of a word on the top of page 11, say, instead of the bottom of page 12.
When a note-taker taps the image of a record button on a control strip along the bottom of every notebook page, twin microphones on the pen synchronize the notes to whatever is being said at the time. Tapping on the notes later — a calculus equation, for example — recalls the relevant digital audio recording, such as the teacher’s explanation for each symbol in the equation.
The smartpen has enough memory to capture up to 100 hours of audio or 30,000 pages of notes. Both audio and visual elements can be uploaded to a computer via a USB cradle that also recharges the pen, and the pen can play back recorded audio either through its tiny speaker or through a headset attached to an audio jack.
Putting a finger on problem graphics
Miele said he immediately recognized the pen’s potential for helping blind people, and in particular, blind students taking science, technology, engineering or math classes.
“Spatial information like charts and maps are heavily used in technical fields,” he said. “Blind people are historically underrepresented in those fields because it’s hard to represent all of the graphical information for blind students.”
As a blind scientist himself, he said, “it’s important to push those numbers in the right direction.”
Braille, roughly equivalent to 24-point type, has traditionally permitted only a few words on complex figures and tables due to size constraints. Furthermore, Miele said only about 10 percent of blind people are literate in Braille. With its ability to produce audio labels, the pen system could be used not only by people who don’t yet know Braille, but could also help them to learn it. “If you don’t know what a character is, you could tap it with the pen and it could tell you,” he said.
The blind can now publish professional-quality, pen-enabled materials such as maps, periodic tables, economic tables, bar charts and organizational charts on a Braille embosser — or printer — all embedded with audio tags based on the printed microdot patterns instead of the much larger dots required for Braille, Miele said.
“You can annotate stuff to your heart’s content,” he said. “You can layer stuff. Tap it once, and it tells you what it is. Tap it twice, and it may give you supplemental information. You can keep on layering information that would be absolutely impossible if you were to do it in Braille.”
Aided by a $300,000 grant from the National Science Foundation, the researchers are also using the smartpen in conjunction with what’s known as a Sewell Raised Line Drawing Kit. “Think of it as a clipboard that has a rubberized surface — your thumbnail would leave an impression,” Van Schaack said.
A clear plastic transparency over the clipboard allows users to draw an image that will leave a physical impression. “The [teaching assistant] can say something like, ‘The teacher just drew a triangle,’ while drawing it on the clipboard, and you can feel it,” he said. “So it’s a tactile figure.”
The smartpen, used in conjunction with the Sewell kit, records not only the drawing’s location but also what was said when it was drawn. Later, a student can tap on the raised triangle and hear the associated classroom commentary as a site-specific auditory label.
The researchers will eventually bring in students on an individual basis and gauge how well they can use the combination of tools as learning aids. Later on, Van Schaack hopes to train students and teaching assistants and let them use the system in real classroom settings, “so we can figure out how what we’ve learned in the lab translates to the real world.”
A smartpen with one gigabyte of memory sells for $149, while a pen with two gigabytes goes for $199. A four-pack of 100-page notebooks sells for $19.95, though Van Schaack said users eventually will be able to print out their own notebook paper.
Free-range surfing
WebAnywhere, an Internet-based service released last month, boasts an even better price tag: free. The program’s innovation isn’t so much about what it does — no more than existing Web readers that convert written text to digital speech — as it is about its availability on almost any computer.
Screen readers normally require users to download software that connects to the Web server and converts text into speech from an electronically generated voice, compressing it into an MP3 file and playing it back to the person.
WebAnywhere instead processes the text on an external server before sending the audio file back to whichever Web browser is in use, thus allowing blind people to access unfamiliar computers in libraries, schools and other public places.
“All the individual parts had been designed, but hadn’t been linked up together before to come up with this,” said Jeffrey Bigham, a Ph.D. candidate in computer science and engineering at the University of Washington and WebAnywhere’s creator.
Once a user connects to the service, WebAnywhere can act as a search engine or as a conduit for a specific URL. By default, the system processes all of the text on a Web page. Alternatively, the software can anticipate what the user may want to read, retrieve it ahead of time, and cache it on the computer browser to minimize the time needed for the text-to-speech conversion.
This algorithm-driven shortcut, known as pre-fetching, can waste computer time if it retrieves the wrong text block. But by taking account of user preferences during the session, the program can get better at anticipating upcoming text blocks to minimize the latency, or the delay between a user’s request and the actual playback.
“You hit a button and expect it to play text to you. Any delay is really noticeable and really annoying,” Bigham said.
Reducing the lag time, in fact, is key to making the system fully functional, and Richard Ladner, Bigham’s thesis advisor, said other computer scientists were initially skeptical that a Web-based system could get around the problem. By incorporating pre-fetching, caching and other known concepts into the system’s design, however, he said Bigham was able to neatly sidestep the issue. “No one had put them together in this unique way,” Ladner said.
WebAnywhere’s current server can support 100 to 200 people simultaneously (based on current Web server limitations), though Bigham said he’s commandeered other computers in the department to help support the system. So far, early reviews have been generally positive, and speed hasn’t been an issue.
“People are really positive about this, they really see a need for this,” Bigham said.
In initial tests, blind Web surfers have successfully used the program for reading e-mail, retrieving bus schedule information and performing Internet searches. Ladner said he dreams of the program eventually being added to Google’s roster of free products as its first accessibility tool, an arrangement that would greatly expand WebAnywhere’s reach.
Already, inquiries have poured in from China, Portugal, India and other countries. With the release of the program’s HTML and Javascript source code, the researchers are hopeful that programmers can add some of the many extras requested by users as well as translate the software into multiple languages — further helping to break down many of the long-standing barriers for the blind community.