Virtual interpreter translates web content into sign language 'for the first time'

New software gives easy access, translates text to Arabic sign language for UAE deaf community

Mahmoud Darawsheh is the founder of Mind Rockets, a company that has developed an app translating from voice recording or typing into sign language in English, Arabic and Korean. Anna Nielsen for The Show more

A friendly animation figure in a dishdasha pops up on a web page and translates text into sign language for the hearing impaired. This software to help the deaf community better understand online content could soon be popular in the UAE.

A Jordanian company has developed a programme being used on travel, telecommunication, restaurant, beauty, furniture and health care websites to guide the deaf and hard of hearing in real time.

“This helps a deaf person access websites in sign language for the first time ever. An avatar we have created appears on the page to translate whatever text they need,” said Mahmoud Darawsheh, founder of Mind Rockets, a startup company developing technology to integrate the deaf community.

In the UAE, the application has recently been launched in Arabic sign language on the Ministry of Community Development and the Dubai Health Authority websites.

When a user clicks on the symbol indicating hearing loss at the top of the page, an avatar appears on the bottom left corner of the page.

The animated character in white national robes then begins translating the text into sign language when the user hovers over the words they want translated.

For a large section of the hearing impaired, the internet is shut off. Many did not attend regular school and are not adept at reading. The technology aims to bridge this gap for the deaf.

Animated web interpreters will be available in American Sign Language soon. Both Arabic and American forms of sign are popular in this region.

“We wanted to help them access information just like everyone else but in the mother tongue they are comfortable with. Our belief is that the deaf should not have to pay for this technology. The idea is for corporations to use this to create deaf-friendly content, make their businesses accessible,” Mr Darawsheh said.

“Many deaf people cannot read and we set up this interface so they do not get further isolated.”

Not being proficient in reading limits the world of the hearing impaired, experts said.

“This is a very important technology because the written word is not their natural language. Literacy rates are not very high among this community,” said Bedour Al Raqbani, director and founder of Kalimati Communication and Rehabilitation Centre for the hearing impaired in Dubai.

“They are very visual and so this could be something that will be beneficial because it will be faster, instantaneous and help them comprehend quicker because it’s in sign language. This gives them a real choice as it respects their natural language.”

The aim is to build innovative technology to fully engage the hearing impaired.

Zain Telecom, one of the early adopters, have hired special teams in call centres in Jordan to provide further technical support via video interaction.

_______________

Read more:

Bridging the communication gap

App provides interpretation services to hearing impaired - video

Call for inclusion at school and work

_______________

“We believe no group or segment should be excluded so Zain has chosen to partner with Mind Rockets to harness its technology. This will provide the best user support for the deaf and hard of hearing when they browse Zain Jordan’s website,” a spokesman said.

Mind Rockets first began two years ago with mobile applications that translated spoken or written words into American or Arabic Sign Language acted out by an animated figure.

The website interpreter is its latest offering.

There are plans to tie-up with movie websites to load a version of the interpreter that will translate film dialogue into sign language.

Testing is also ongoing in Saudi Arabia to use it on tablets to communicate between staff and customers in pharmacies.

A poster at the entrance or a sign on the website will let the hearing impaired know that an instant interpretation service is available at specific branches.

Staff are equipped with handheld tablets with the programme interpreting typed out questions or answers from staff into sign language for the customer and repeating the process in reverse as well.

Embedded blurbs will have pre-prepared queries about allergies, patient’s age and gender to ensure the correct drug is prescribed.

“We got this idea from the deaf community because they complained they could not communicate in a pharmacy without help from a third party,” Mr Darawsheh said.

“Specific medication is required for people who are older or for a pregnant woman.  Questions asked directly to the deaf person will ensure the medicine is safe.”

If the pilot succeeds it could also be used as an interface in hotels and government offices.

There are plans to add Korean and British Sign Language, said Mohammed Kilany, co-founder of Mind Rockets.

“We would like to unify the community so there are fewer limitations. Our plan this year is to launch new products and services to enable further inclusion. It will help services become deaf-friendly and further their participation in society,” he said.

Some UAE websites featuring Arabic Sign Language interpreters are the Ministry of Community Development: https://www.mocd.gov.ae/ and Dubai Health Authority: https://www.dha.gov.ae/en/pages/dhahome.aspx.

Other inclusive travel, food, beauty and home décor websites are www.thetravelguideonline.com, www.yamyumi.com, www.zafafi.me and www.albait.me.

_______________

How does the app work?

Website Interpreter: Web plug-ins allow the hearing impaired to read website content in sign language. When the hearing aid symbol on the top of the web page is clicked, a figure or avatar appears on the bottom left corner of the page and uses hand symbols to communicate any text on which the cursor is placed.

The same concept is being planned to interpret movie dialogue into sign language.

A separate software is being tested in pharmacies on tablets used by staff to allow questions or text messages to be interpreted in sign language with real time replies.

The interpreter is currently in Arabic Sign Language. Plans for American Sign Language soon. Both are popular in the Gulf region.

Mobile applications: Using the software, a person can speak or type a message and an animated figure signs in real-time for a person with a hearing impairment. A person with a hearing impairment uses a sign language keyboard that translates sign language symbols into written text.

Why different sign languages:

One word has different hand signs because sign like spoken language developed when groups interacted with each other.

No one unified system of sign language is used across the community.

While English is the common thread, American Sign Language differs from British Sign Language. This is because sign language is independent of spoken language and has developed within deaf communities.

Gestures regularly used by a group could be improvised.

For instance, the word ‘where’ in British Sign Language is indicated with both palms upturned, while in American Sign Language it is demonstrated with one raised finger.