Microsoft Turns Jazz Hands Into Gesture Commands Using Sound Waves

07.05.2012
Most gesture-based control systems we use today rely on either motion-capture cameras--like the --or a . But researchers from Microsoft Research and the University of Washington are developing a system that can detect object with sound waves, like how a bat does with .

With the , the researchers aim to bring gesture controls to any computer that has a set of speakers and microphone. The program uses the Doppler Effect of sound waves for to detect objects and recognize motion.

The , as you might remember from so many years of science classes, is a principle of physics where the frequency of waves (sound waves, light waves) shifts depending on whether the source of that sound (or light, etc...) is moving toward or away from you. The most common example is a passing train, where the sound coming from the train seems to get deeper as it passes you.r.

SoundWave uses your computer's speakers to emit an imperceptible, ultrasonic audio signal at 20-22 Kilohertz--and you can still play music while it's doing this. If you wave your hand in the vicinity of your computer, the sound waves that would have traveled through the air bounce off of your hand instead.

The computers microphone can pick up the audio signal and the SoundWave program can interpret any changes in the frequency as a gesture command. So far the team has been able to detect five different variables involved with a change of frequency, including the speed, direction, proximity, size, and number of the objects, as well as the frequency of the motion.

From this, the Soundwave program can perform a number of the usual computer commands such as scrolling, clicking and double clicking, rotating objects, or automatically locking the computer while you're away. The researchers say that SoundWave is 90 to100 percent accurate, even in a loud crowded room, and can even be used to play a round of Tetris.