A Smartphone that Sees For You

A new set of software that works with relatively inexpensive smart phones could help visually disabled people and the elderly. Using a technology called "augmented reality", the software can help people by recognizing text, objects and places.

Augmented Reality, or AR, uses location and visual information from a camera to supply a smartphone user with extra information about their surroundings. For an introduction to AR, you can take a look at my hands-on with Spyglass, the augmented reality navigation application.

LookTel takes AR technology and applies it to a very real problem. It was developed by Ipplex, in partnership with the National Institutes of Health, the National Eye Institute,  the Department of Veterans Affairs and a research organization for aides for the blind, to help people who can't see well or may become disoriented.

The Windows Mobile application uses character recognition technology and a voice synthesizer to read labels to the smartphone user.  All that processing can't be done on the smartphone, so character and object recognition software are part of LookTel's "base station" software, which runs on a Windows-based home computer. You can also add coded labels to objects , as this YouTube demonstration shows, and associate those labels with voice descriptions of objects you put them on.

The app also can connect a user to a real person for assistance by connecting back to the application's Windows PC-based "base station" software:

For more complicated visual tasks such as locating a specific street corner or bus stop, LookTel enables you to remotely connect to another person for assistance. Not only can you talk to them, but you can also transmit live video from the cell phone's camera so that they can see your surroundings and help you with orientation, recognizing objects and media, and more. Your position will also be shown to them on Google Maps using the GPS services available on smart phones. These services aim to help users to independently go about daily activities with the confidence that they will have the assistance they need anytime, anywhere.

LookTel is currently in testing, and is available on pre-configured HTC smartphones and mobile Internet devices. Ipplex hasn't published the pricing for the technology, but is offering it at a discount during testing.