Google's AI researchers propose a new kind of search engine

The world’s most popular outlet for answers has an idea for what comes next

While Google has improved its ranking of useful information over the years, its basic foundation remains the same. Reuters

A team at Google has laid out a vision for a new search engine, arguing artificial intelligence is advancing fast enough to one day completely replace the index-and-retrieve system the company has relied upon since 1998.

In a draft paper titled "Rethinking Search", AI researchers describe a new kind of search engine that provides a succinct expert answer with the help of algorithms, as opposed to generating a list of web pages that lead to possible answers.

“Given the significant recent progress developing information retrieval, question answering and pre-trained language modelling capabilities, now is an opportune time to take a step back to try to envision what possibilities the future might hold,” Donald Metzler, a software engineer at Google, and his co-authors write. “What if the distinction between retrieval and ranking went away and instead there was a single response?”

In the 23 years since it was founded in Menlo Park, California, Google has continued to index the vast quantities of information constantly updating and changing on the Internet, retrieve the appropriate searched-for information when queried and rank those web pages by the relevance to the person searching.

Over the last couple of decades, and with the help of machine learning, Google has improved its ranking of useful information, but the basic foundation remains the same.

Advertising on its search platform also generates the bulk of parent company Alphabet’s revenue. Google made $31.9 billion in search ad sales last quarter, up from $24.5bn in the first quarter of 2020.

While it is unclear if the company wants to upend its business model, AI researchers say that machine learning, a type of AI, has advanced enough in the last few years to begin thinking about replacing the Google search platform wholesale. The Google team envisions a new search engine that is able to synthesise the top relevant information to provide a single answer.

A number of interesting and difficult research and engineering challenges ... must be solved before the envisioned system can be realised.

They argue that people prefer to ask an expert for questions that might require nuance or information. Instead of asking an expert, however, over 70 per cent of Internet users Google it, according to a 2017 analysis by Search Engine Journal. This often works well, but the responsibility is with the searcher to sift through and identify the useful information and come up with an answer.

At the moment, pre-trained question-and-answer programmes are “dilettantes rather than experts”, according to the researchers. “They do not have a true understanding of the world.”

Ask a popular voice-assistant programme like Siri, Alexa or Google Assistant for an answer to a question and the chances are the response will be, “I’m sorry, I don’t understand” or worse, an entirely incorrect answer.

Crowd-sourced platforms like Quora and Ask are popular because of this preference to ask an expert, but are limited by the quality of the source providing the answer and incomplete subject matter.

Jason Dorrier, managing editor at tech publisher SingularityHub, compared the plan to the beloved conversational computer in Star Trek, but questioned how a change to Google's search engine might alter the way people contribute to the Internet.

“If we primarily consume information by reading prose-y responses synthesised by algorithms—as opposed to opening and reading the individual pages themselves—would creators publish as much work? And how would Google and other search engine makers compensate creators who, in essence, are making the information that trains the algorithms themselves?”

In addition to these questions, several major hurdles familiar to the AI community remain, including training language models to be unbiased, to always select the most authoritative source and to learn from a diverse range of viewpoints, the authors said. There are also open questions about whether the model can scale and be trained in multiple languages.

"A number of interesting and difficult research and engineering challenges ... must be solved before the envisioned system can be realised," the Google team wrote. Still, they conclude that it is a new and "ambitious research direction" worth pursuing.