DUBAI // Games consoles are usually used to blast alien invaders into cyberspace or win virtual World Cups, but a group of students have used the technology inside an Xbox to create a way of teaching sign language.
A team of computer science students at the University of Wollongong in Dubai have developed software that enhances the Microsoft Xbox’s Kinect system, which uses sensors to detect a person’s movement.
Until now, the software was only able to recognise a gamer’s head, body and limbs, not individual fingers. This posed a fundamental problem in teaching sign language, as fingers are an essential part, representing different words and meanings.
However, the students were able to upgrade the technology to identify individual digits.
“Getting Kinect to recognise the fingertips is epic because Microsoft hasn’t been able to do this, and to bring that to a platform which helps the community is fantastic,” said Dr Zeinath Khan, who is overseeing the project. The software, called Sama, meaning “hearing” in Arabic, can be updated continuously, meaning it can store a database of different types of sign language.
American is currently the dominant form in online learning methods, meaning other sign languages such as Arabic are often excluded. “There’s a lot of CDs and things out there to learn sign language, but this [one] is dynamic,” Dr Khan said. “This is something anyone can use, those with hearing impairment or those living or working with those with hearing impairment.”
The interactive system gives feedback, picks up mistakes from the user and teaches progressively with online lessons and tests.
The students spent three months carrying out research at a number of institutions to find the best possible ways to teach sign language to staff to make it easier for the hearing-impaired to be part of the team.
“From our research, [we realised that] very few of the deaf community are employed,” said team leader Shuji Shafiq, 24.
“This kind of technology would help them and companies to get them more integrated into the community and the workplace, where they could work and they could develop.”
World Health Organisation figures state that more than 5 per cent of the world’s population has disabling hearing loss. Though no statistics are available for the UAE, it is believed there are about 8 million deaf people in the region, many of whom are either out of work or in low-paid, low-status jobs.
Hassan Belo, 21, a member of the team, said the aim was to “bridge the gap between those with hearing impairment and the hearing community”.
“It will be like the Google translation of sign language as it won’t be limited to one kind. This has video lessons, progress tracking, and is much more comprehensive than anything out there now.”
“For organisations like the Red Cross, this could be vital for times of crisis,” said Mr Belo, from Nigeria. “If you can’t hear each other but you can all understand sign, it could save lives.”
mswan@thenational.ae
