Power of AI has limits in fight against Covid-19, experts caution

Artificial intelligence forecasts useful for allocating healthcare resources, but less so for pinpointing end of pandemic

Medics work at an intensive care unit at Moulay Abdellah Hospital in Sale, Morocco, where Covid-19 patients are being treated. AP

Nearly two months into the global pandemic, people around the world are united by one question: When will it end?

Recently, a team at the Singapore University of Technology and Design sought to come up with an answer using artificial intelligence. Their algorithm predicts the end of the Covid-19 pandemic in different countries as well as for the world - and the charts have understandably been making the rounds on Twitter and picked up by media.

But experts warn this type of certainty is - certainly - “too good to be true”, and an example of what to watch out for amid a cacophony of research.

"Normally, we look for peer-reviewed papers, studies and clinical trials that help decide whether a system meets the requirements of evidence-based medicine," Dr Bertalan Mesko, director of The Medical Futurist Institute, told The National.

But amid a rapidly changing pandemic, “thresholds are definitely lower”, he added, and “large data sets and good algorithms” that power AI are proving helpful to public health experts and decision makers.

There are numerous examples of these "good" algorithms: from the AI company in Toronto that put out an early warning of the virus in January, to the supercomputers being used to help surface vaccine and treatment options, to Abu Dhabi's use of short-term data forecasting to allocate adequate health care resources. These use cases are using complete data sets and reasonable time horizons, which make them actionable.

“There is no laser-precise line between what's useful and what is not,” Dr Mesko said, making it tough to tell when a certain solution is overpromising an answer.

Indeed, according to the World Economic Forum, which has been monitoring AI’s role in the pandemic response, addressing the question of trustworthiness is “particularly difficult” in part because the AI community currently has no set standards. Compared to the software industry, for example, where companies rely on design documents, rigorous testing and security protocols, AI practitioners “are still trying to figure out what these processes should be” for their field.

This lack of protocol is a reminder of just how nascent the field of AI is - even as the technology is being called upon in these unprecedented times.

"There is an ongoing hope that we can turn to AI to solve our problems," Kay Firth-Butterfield, the head of artificial intelligence and machine learning at the World Economic Forum, told The National. "Covid-19 shows us that may not be possible."

The researchers from the Data-Driven Innovation Lab in Singapore are updating confirmed cases nearly every day and feeding them into a machine learning model to predict the spread of the disease in order to come up with their predictions.

“I think their disclaimer says it all,” Ms Firth-Butterfield said, a reminder to always read the fine print as research emerges at a breakneck pace.

The disclaimer reads, in part, that it "is STRICTLY ONLY for educational and research purposes and may contain errors. The model and data are inaccurate to the complex, evolving and heterogeneous realities of different countries".

“In other words, whilst some algorithms can be trained with small amounts of data, the more data the more accurate the predictions,” she said. “Predicting the end of Covid with the data we have available would be too good to be true. However, the [researchers] seem to add all available data every day so they are getting better predictions daily."

But she cautioned against relying on the predictions since the data was coming from a broad range of countries with diverse healthcare systems and different outcomes from treatment, which together make it difficult to avoid bias or to come up with a general data set to make this type of prediction.

At a press conference over the weekend, Kenneth Mak, a director at the Ministry of Health in Singapore, where the research originated, fielded a question regarding the projections from SUTD on when the pandemic would end, Mothership, a Singapore media company, reported.

While the projections might not give "a full and accurate picture", Mr Mak acknowledged that "these projections are useful in terms of depicting that, in fact, there's still a long way to go".