Aiding the Blind: Kinecthesia - Oct 2011 - YouTube

Two Students from the University of Pennsylvania have modified a Kinect to help the vision impaired to become more aware of their environment. The project is called Kinecthesia and functions as a radar-like device, but instead of using sound, the device uses the Kinect's cameras to map out the environment and translate it into a sensation the blind can understand.

Eric Berdinis and Jeff Kiske, juniors of the University of Pennsylvania majoring in Computer Engineering, has garnered attention in the 2011 Google's Zeitergeist Young Minds Conference with their application of the gaming peripheral.

The project was born from both student's final project for Rahul Mangharam's embedded systems class. The resulting device can be likened to a high-tech walking cane.

The Kinect is a motion detecting gaming peripheral produced by Microsoft. The device is used for the Xbox 360 which can interpret specific gestures and track movement of individuals or objects.

The device is composed of an RGB camera, an infrared sensor acting as a depth sensor and multiple-array microphone. It can perform facial recognition, voice recognition and gesture recognition.

Using the Kinect's multiple cameras, Berdinis and Kiske have made a harness that produces vibrations to notify the wearer of coming obstacles.

The Kinect is connected to a BeagleBoard, a whole computer in a single board, and fitted in a waistband or belt. The device is ran in Linux and 16 AAA batteries. Since it was still in development the device is not made to run for hours. There are six vibrating motors to guide the user and weighs 1349g as of now.

There are already a number of people hacking their Kinect devices for other applications besides gaming like 3D modeling, controlling robots and using motion capture to automate devices, augment reality, and even produce a low-budget CGIs for amateur movies.


It is great that people are experimenting and applying on technologies readily available for them. Will this open doors on new innovators? Does Microsoft see a new market for their devices?

The project's development can be tracked on their own website at kinecthesia.com. The website also contains the project source code, parts list and assembly instructions.