Engineers create touchscreen Braille writer

Some of the undergraduates are gathered into teams. Some work alone. All are assigned mentors and tasked with a challenge. They compete, American Idol-style, for top honors at the end of the summer.

The competition is made possible in part by a collaboration between the U.S. Army and several university and industry partners that makes up the AHPCRC.

Adam Duran is one such undergraduate, a student both lucky and good. He is now in his senior year at New Mexico State University. Last June, he came to Stanford at the suggestion of one of his professors. His mentors were Adrian Lew, an assistant professor of mechanical engineering, and Sohan Dharmaraja, a doctoral candidate at Stanford studying computational mathematics.

"Originally, our assignment was to create a character-recognition application that would use the camera on a mobile device -- a phone or tablet -- to transform pages of Braille into readable text," said Duran. "It was a cool challenge, but not exactly where we ended up."

Bigger fish

Even before Duran arrived for the summer, Lew and Dharmaraja began to talk to the Stanford Office of Accessible Education, people whose profession is helping blind and visually impaired students negotiate the world of higher learning. It became clear that there were bigger fish to fry.

While a Braille character reader would be helpful to the blind, Lew and Dharmaraja learned, there were logistics that were hard to get around.

"How does a blind person orient a printed page so that the computer knows which side is up? How does a blind person ensure proper lighting of the paper?" said Duran. "Plus, the technology, while definitely helpful, would be limited in day-to-day application."

"It was a nice-to-have, not a must-have," said Dharmaraja.

So, the three began to ask questions. That is when they stumbled upon a sweet spot.

"The killer app was not a reader, but a writer," said Dharmaraja.

"Imagine being blind in a classroom, how would you take notes?" said Lew. "What if you were on the street and needed to copy down a phone number? These are real challenges the blind grapple with every day."

There are devices that help the blind write Braille, to send email and so forth, but they are essentially specialized laptops that cost, in some cases, $6,000 or more. All for a device of limited functionality, beyond typing Braille, of course.

"Your standard tablet has more capability at a tenth the price," said Duran.

"So, we put two and two together. We developed a tablet Braille writer," said Dharmaraja, "A touchscreen for people who can't see."

First, however, the student-mentor team had to learn Braille. Originally developed for the French military, Braille is a relatively simple code with each character made up of variations of six dots -- or bumps, really -- arranged in a 2-by-3 matrix. The blind read by feeling the bumps with their fingertips.

As any computational mathematician will tell you, such a matrix yields two-to-the-sixth minus one variations, or 63 possible characters. These 63 characters are enough for a Western alphabet plus 10 numerical digits, with several left over for punctuation and some special characters.

Over the years, however, those 63 characters got quickly gobbled up -- through the addition of character-modification keystrokes, the total grew and now includes chemical, mathematical and other symbols.

Challenge

A modern Braille writer looks like a laptop with no monitor and an eight-key keyboard -- six to create the character, plus a carriage return and a delete key.

Duplicating the Braille keypad on a touch-based tablet seemed simple enough, but there was at least one significant challenge: How does a blind person find the keys on a flat, uniformly smooth glass panel?

Dharmaraja and Duran mulled their options before arriving at a clever and simple solution. They did not create virtual keys that the fingertips must find; they made keys that find the fingertips. The user simply touches eight fingertips to the glass, and the keys orient themselves to the fingers. If the user becomes disoriented, a reset is as easy as lifting all eight fingers off the glass and putting them down again.

"Elegant, no?" said Lew. "The solution is so simple, so beautiful. It was fun to see."

Beyond the price difference, touchscreens offer at least one other significant advantage over standard Braille writers: "They're customizable," Dharmaraja noted. "They can accommodate users whose fingers are small or large, those who type with fingers close together or far apart, even to allow a user to type on a tablet hanging around the neck with hands opposed as if playing a clarinet."

"No standard Braille writer can do this," said Professor Charbel Farhat, the chair of the Aeronautics and Astronautics Department and executive director of the summer program. "This is a real step forward for the blind."

Showing off

In a demo, Duran donned a blindfold and readied himself before the touchscreen. He typed out an email address and a simple subject line. Then he typed one of the best-known mathematical formulas in the world, the Burgers Equation, and followed with the chemical equation for photosynthesis -- complex stuff -- all as if writing a note to his mother.

For Duran, who has an uncle who is blind, the greatest joy was in seeing a blind person using his creation for the first time. "That was so awesome," he said. "I can't describe the feeling. It was the best."

In the immediate future, there are technical and legal hurdles to address, but someday, perhaps soon, the blind and visually impaired may find themselves with a more cost-effective Braille writer that is both portable and blessed with greater functionality than any device that went before.

"AHPCRC is an excellent model for outreach, which not only trains undergraduate students in computational sciences but also exposes students to real-world research applications," said Raju Namburu, the cooperative agreement manager for AHPCRC.

The center addresses the Army's most difficult scientific and engineering challenges using high-performance computing. Stanford University is the AHPCRC lead organization with oversight from the Army Research Laboratory.

As for his summer courses, Farhat is optimistic. "Let's remember," he points out, "This was a two-month summer project that evolved because a few smart people asked some good questions. I'm always amazed by what the students accomplish in these courses, but this was something special. Each year it seems to get better and more impressive."