Lipreading mobile phone

18 03 2010

A prototype lip reading mobile phone promises to end noisy phone calls. The technology allows people to have silent phone conversations by measuring the tiny electrical signals produced by muscles used when someone speaks.

The phone can record these pulses even when a person does not audibly utter any words and use them to generate synthesised speech in another handset.

The device was on show at the Cebit electronics fair (2-6 March) in Germany. It relies on a technique called electromyography which detects the electrical signals from muscles. It is commonly used to diagnose certain diseases, including those that involve nerve damage.

The prototype that was on display in Germany uses nine electrodes that are stuck to a user’s face. These capture the electrical potentials that result from moving the articulatory muscles used to produce speech. The electrical pulses are then passed to a device which records and amplifies them before transmitting the signal via Bluetooth to a laptop. There, software translates the signals into text, which can then be spoken by a synthesiser. In the future, the technology could be packed in a mobile phone for instantaneous communication. It could also form the basis of an instant translation system.

It is not the first time that electromyography has been explored for silent communication. The US space agency Nasa has investigated the technique for communicating in noisy environments such as the Space Station. It has also used the technique to explore advanced flight control systems that do away with joysticks and other interfaces. Nasa explored the technique to understand simple commands. The difference with this prototype  is that continually spoken sentences can be recorded and recognised.

Source BBC News

Exciting, eh? More electrodes for me! Although how Text Relay would cope with this, poses an interesting question …

Advertisements

Actions

Information

6 responses

18 03 2010
Dianrez

Wonder how this device performs when we all know that 1/3 to 1/2 of all speech sounds in English are not visible on the lips and that lipreading has an extremely high error rate?

Or how does this compare to voice recognition systems that transliterate speech to computer text? That also has a high error rate and requires training the program to recognize individual patterns properly.

Just curious….

19 03 2010
Speak Up Librarian

I’m with Dianrez on this. My first thought was what about two different words that look the same on the lips? It’s also hard to imagine anyone willing to put on the nine electrodes in everyday life – on a space station maybe, but here on Earth, I’m not so sure.
Very interesting post.

19 03 2010
Bluesky

I don’t think this technology will work. It’s fraught with too many challenges to overcome.

Furthermore, even if it succeeded in 85% speech recognition it would simply be too expensive to make for a viable return.

19 03 2010
Jeremy Freeman

Can’t imagine this working very well…

23 03 2010
kim

What amazes me most is that someone would spend so much time developing this technology without actually researching the accuracy and usefulness of lip reading.

23 03 2010
funnyoldlife

I am surprised they haven’t appeared to have consulted any professional lipreaders before embarking on this, expensive I am sure, project.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s