Facebook trains AI algorithms to 'see' and 'hear'

New technologies will train virtual assistants to navigate routes, build memories and perform tasks by listening to sounds

AI is predicted to contribute $320 billion to the Middle East’s GDP by 2030, according to KPMG. Reuters

Facebook is developing new artificial intelligence capabilities that could help virtual assistants such as Siri, Alexa, Bixby and other chatbots to understand the physical world better and deal with its complexities just as the human brain does.

These new technologies could help the virtual assistants understand and remember 3D environments, navigate effectively and perform tasks by listening to sounds even from a good distance.

"We are making rapid progress in training [AI] agents to accomplish a wide range of challenging tasks in photorealistic 3D simulators featuring complex physical dynamics," Facebook research scientists Kristen Grauman and Dhruv Batra wrote in a blog post.

“These efforts are part of Facebook AI’s long-term goal of building intelligent systems that can intuitively think, plan and reason about the real world … where even routine conditions are highly complex and unpredictable,” they added.

An AI assistant, or a virtual assistant, is an application that understands voice commands and can execute tasks for the user.

Last week, Facebook unveiled its first audio-visual simulator that will add realistic simulations of any sound source. It will help an AI assistant to move through an unmapped environment to find a sound-emitting object such as a ringing phone.

This means an assistant can now act upon a command of “go find the ringing phone” rather than “go to the phone that is 25 feet southwest of your current position”. It can discover the position of the phone on its own using sensors.

This is the first attempt to train AI assistants to both see and hear by mapping surroundings and finding sound-emitting targets.

With this approach, “we achieved faster training and higher accuracy in navigation”, the company said.

The AI market is booming globally and regionally as governments invest in technology to drive efficiency and cost savings. The UAE, the Arab world's second-largest economy, is projected to benefit most in the region from AI adoption.

The technology is expected to contribute up to 14 per cent to the country’s gross domestic product – equivalent to Dh352.5 billion – by 2030, according to a report by consultancy PwC. In Saudi Arabia, AI is forecast to add 12.4 per cent to GDP.

Facebook has also announced an indoor mapping tool that will allow AI assistants to better understand, remember and recall details about interior spaces, such as how many chairs are in the second-floor conference room or whether the kitchen is next to the laundry room.

“To build robust and capable AI assistants that can perform these sorts of tasks well, we need to teach machines to explore, observe and remember a space from their first-person point of view and then create a third-person top-down semantic map of that 3D environment,” Ms Grauman and Mr Batra said.

Facebook's latest AI tools build upon an existing open-source environment simulator, AI Habitat, which the company launched last year. It allows researchers to train AI assistants in life like environments.

Facebook said it aims to “accelerate progress in building AI assistants that can help people accomplish a wide range of complex tasks in the physical world”.

As investments in AI grow, many tech companies have also taken steps to address ethical issues in AI adoption. For instance, Alphabet-owned Google is giving away $25 million (Dh91.83m) to projects that propose ways to use AI in building a more humane society. Microsoft has also committed $115m to an 'AI for Good' initiative.