Artificial intelligence is frequently presented as a revolutionary technology that will make the world smaller by breaking down barriers to communication and mutual understanding. Yet many of the systems now in place have inherited some of modern society’s most persistent biases.
The information age, of which AI is a key driver, was supposed to be characterised by connectivity. In many ways it is, but much of the information through which the world is understood is being filtered through the English language and western cultural assumptions. According to the Internet Society Foundation, a non-profit group, content in English dominates more than half of all online written material.
This is not a niche issue. AI – a technology that now touches all our lives – is being trained on such material and being asked to explain the world back to us. If the underlying information is skewed, the technology risks creating a system that confidently explains entire cultures while misunderstanding their references, humour and customs.
Where that leaves tens of millions of non-English speakers is an open question. This is why the work being done to diversify the digital realm and make sure as many global languages as possible are represented is so important.
This week, it was announced that researchers from the Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi developed ArabCulture-Dialogue, the first benchmark to measure AI models’ ability to understand and engage with Arab culture across 13 national dialects. Studies done so far reveal a striking gap between many models’ ability to understand what Arabs say and their ability to speak the way Arabs do in everyday life.
Such research joins other UAE-led initiatives to make the internet more culturally representative, such as the Falcon Arabic and Jais language models. These are projects with transnational import. Arabic is spoken across more than 20 countries as an official or primary language, is used by millions of additional non-native speakers globally for reading and prayer and is one of the UN’s official tongues. Embedding Arabic in cutting-edge AI research is vital.
This is not an argument against English, nor against the benefits that familiarity with the language have brought to the UAE. The country's ability to operate comfortably across languages and cultures has been one of the foundations of its success. Instead of building technological walls around individual cultures, the digital world should contain enough cultural knowledge to represent them accurately.
There is still much work to do. Research published in September last year by Saudi and Qatari academics in the Communications of the ACM journal found that Arabic online “lacks abundant, well-annotated resources, particularly for regional dialects and informal language use”. In addition, they claimed that “a significant portion of Arabic knowledge remains undigitised, making it inaccessible for training large-scale models”.
Those remaining gaps are on their way to being filled. This is an issue to be watched closely for, if machines are going to help us understand the world, they must first learn that humanity does not speak with one voice.
