Evidence-based medicine is the cornerstone of modern health care, but what if the system of proof was wrong and  most published research is false? That is the question being posed at NYU Abu Dhabi. Getty Images
Evidence-based medicine is the cornerstone of modern health care, but what if the system of proof was wrong and most published research is false? That is the question being posed at NYU Abu Dhabi. GeShow more

Is medical evidence accurate?



Modern computing, data generation, scientific software, email, the internet, and social media have all placed strain on the scientific publication infrastructure.

Evidence-based medicine is the cornerstone of modern health care. In the bad old days, treatments were based on a doctor's personal clinical experience or traditional, time-honoured practice. Today, treatment decisions are based on evidence-based medicine, better known in the medical profession as simply EBM. The phrase had its first public outing as recently as the spring of 1991, in an essay by a young Canadian doctor in the Journal of the American College of Physicians.

Evidence-based medicine, wrote Gordon Guyatt, head of the intern programme at McMaster University, Ontario, was “the way of the future”.

Assisted by the introduction of what were still being called desktop “microcomputers”, linked by telephone line to Medline, the newly searchable database of medical literature run by the United States national library of medicine, doctors could quickly track down studies relevant to their case, and determine “the optimal management of the individual patient” rather than consult a textbook, expert or senior physician.

It was, without doubt, a revolution in health care, which has led to an avalanche of published research, estimated to generate about 2.5 million papers a year, in print and online in an ever-escalating number of journals.

It all sounds good. But what if most published research is false?

That is the startling premise that will be explored in Abu Dhabi on Tuesday by Jeff Leeks, associate professor of biostatistics at Johns Hopkins Bloomberg School of Public Health, in a provocative talk at the NYU Abu Dhabi Institute.

As Prof Leeks wrote in a paper in April, the suggestion that most published research findings are false “seems absurd on the first reading”. After all, scientific research is conducted by skilled scientists, vetted through peer review, and publicly scrutinised.

What’s more, the entire scientific publishing infrastructure “was originally conceived to prevent the publication of incorrect results and provide a forum for correcting false discoveries”.

But “modern computing, data generation, scientific software, email, the internet, and social media have all placed strain on the scientific publication infrastructure”.

It sounds counterintuitive – surely all these innovations can only assist researchers in reaching more accurate and reliable conclusions?

Well, yes and no, says Prof Leek. “In many ways these innovations really help researchers,” he says. “But the landscape of research changed so quickly that training had a hard time keeping up.”

The volume of data generated makes analysing and storing it hard to do and not every scientist is trained in the statistics that play a major role in every study. What’s more, new outlets for research, such as online-only journals and “preprints” – previews of work – that have sped up the publishing rate, are not vetted or peer reviewed. And the limits of social media, Prof Leek says, can pressure researchers to oversimplify their results.

Conventional media also often misrepresent findings. If a story based on a small study that has not been vetted or confirmed is reported as fact, rather than an as interesting idea – think about health claims or warnings about salt, coffee, red wine and so on – when it is later disproved “people will think scientists can’t make up their minds”.

All of these things, he says, “apply unexpected pressures to researchers that make it more difficult to execute dispassionate science”.

The alarm was first sounded in 2005 by John Ioannidis, a medical statistician at Tufts University School of Medicine, Boston, who is now professor of health research and statistics at Stanford University.

There was, he wrote in a paper published in PLoS Medicine in August 2005, "increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims".

What followed was an exhaustive and highly technical analysis of the statistical flaws and assumptions which, Prof Ioannidis said, undermined so many research conclusions.

Since then, “there has been major progress on many fronts”, Prof Ioannidis says.

Increasingly, clinical trials are registered with central bodies, meaning that results both positive and disappointing are reported, leading to a fuller picture, and more data that supports conclusions is being shared, often online.

“But I fully agree with Jeff that much work needs to be done,” says Prof Ioannidis. “I cannot say that research has become less reliable over the 12 years [since his 2005 paper was published].

“Some fields have become more reliable, and a few fields that were extremely unreliable – for example, genetic associations – have become extremely reliable.

“But at the same time, we have an explosion of data, often poor data, and thus many more results that get derived from working with poor data.”

Inevitably, there could be ramifications for those on the receiving end of medical research – the patients.

“For health care, we clearly need robust inferences,” Prof Ioannidis says, but in most medical applications “we have less than optimal evidence.

“This doesn’t mean that we have no evidence, but we can clearly do better”.

A stark example of the potentially deadly consequences of flawed research surfaced in 2014, when the British Medical Journal was forced to retract an article that reached a false conclusion on a drug affecting millions of people.

In October 2013 John D Abramson, a healthcare policy expert at Harvard Medical School, and three co-authors, published a peer-reviewed article in which they stated that 20 per cent of patients who took statins, which are widely prescribed to lower cholesterol, suffered side-effects including diabetes and muscle pain.

Sir Rory Collins, professor of medicine and epidemiology at the University of Oxford, revealed that they had overestimated the side effects of statins by more than 20 times and this “may have meant people stopped taking them or high-risk patients didn’t start taking them”.

It was months before The BMJ retracted the article and Dr Abramson and colleagues admitted they had made an "error of interpretation".

Overstating the dangers of a drug is one thing, but well-designed and correctly interpreted trials can also fail to reveal negative side effects.

When it comes to new drugs that come on the market, says Prof Leeks, there are two main types of research, exploratory and confirmatory.

“In exploratory research, scientists are looking for new ideas and trying them out,” he says.

“Sometimes they work really well, sometimes a little bit, and sometimes not at all. But we wouldn’t go out and start offering a new treatment to the public based on a single exploratory study.”

That’s where confirmatory research comes in. This involves “carefully vetting pre-specified hypotheses – like that a certain drug is a safe and effective treatment for a particular disease”.

Regulatory agencies, such as the FDA in the US, provide oversight to make sure any resulting drugs are safe and effective.

But, Prof Leeks cautions, while “the system is set up to be very conservative about approving drugs, that doesn’t mean there will never be mistakes”.

Before being approved by regulatory bodies for adoption by the medical profession, proposed new drugs or other treatments are first tested in human trials, which are carried out according to a set of stringent, standardised protocols.

The “gold standard” is the so-called randomised double-blind placebo study, or control trial, in which patients with the same diagnosis are divided at random into two groups, one of which is given the new treatment for the condition, and the other a placebo. Neither side knows who is in which arm of the trial.

The results – a range of predetermined outcomes, up to and including death – are then subjected to sophisticated statistical analysis, which reveals whether the treatment is better or worse than existing treatments.

But this system is far from infallible.

Unnervingly for patients, it remains a sobering fact that drugs can be approved and in use for years before statistical evidence emerges to show that, long-term, they do more harm than good.

• Prof Leek's talk on Tuesday from 6.30pm-8pm at the NYU Abu Dhabi Conference Centre (A6) is free and open to the public. Online registration is required.

______________________________________________________

Four cases where side effects were greater than the drug’s benefits

• In 2011, drug company Eli Lilly and Co withdrew a product called Xigris, designed to treat severe cases of septic shock, in which the body’s organs start to shut down in an -overreaction to infection. The decision, a -decade after the drug’s -approval in Europe and the United States, was based on findings that the chance of death was greater in those who took the drug than in those who took the placebo.

• In 2014, ProCon.org, a US non-profit educational charity, identified 35 prescription drugs that had been approved by the FDA but were later withdrawn from the market, often only after many years, because they had been found to do more harm than good.

• Accutane, a treatment for acne, one of the 35, above, had been on the market for 27 years before it was withdrawn in 2009 for causing side effects including birth defects, miscarriages and premature births when used by pregnant women.

• Darvon, an opioid painkiller, was in use for 55 years before being banned in Britain in 2005 and the US in 2010. Toxic to the heart, it was thought to have caused thousands of deaths.

newsdesk@thenational.ae

The specs

Engine: Four electric motors, one at each wheel

Power: 579hp

Torque: 859Nm

Transmission: Single-speed automatic

Price: From Dh825,900

On sale: Now

Formula Middle East Calendar (Formula Regional and Formula 4)
Round 1: January 17-19, Yas Marina Circuit – Abu Dhabi
 
Round 2: January 22-23, Yas Marina Circuit – Abu Dhabi
 
Round 3: February 7-9, Dubai Autodrome – Dubai
 
Round 4: February 14-16, Yas Marina Circuit – Abu Dhabi
 
Round 5: February 25-27, Jeddah Corniche Circuit – Saudi Arabia
Country-size land deals

US interest in purchasing territory is not as outlandish as it sounds. Here's a look at some big land transactions between nations:

Louisiana Purchase

If Donald Trump is one who aims to broker "a deal of the century", then this was the "deal of the 19th Century". In 1803, the US nearly doubled in size when it bought 2,140,000 square kilometres from France for $15 million.

Florida Purchase Treaty

The US courted Spain for Florida for years. Spain eventually realised its burden in holding on to the territory and in 1819 effectively ceded it to America in a wider border treaty. 

Alaska purchase

America's spending spree continued in 1867 when it acquired 1,518,800 km2 of  Alaskan land from Russia for $7.2m. Critics panned the government for buying "useless land".

The Philippines

At the end of the Spanish-American War, a provision in the 1898 Treaty of Paris saw Spain surrender the Philippines for a payment of $20 million. 

US Virgin Islands

It's not like a US president has never reached a deal with Denmark before. In 1917 the US purchased the Danish West Indies for $25m and renamed them the US Virgin Islands.

Gwadar

The most recent sovereign land purchase was in 1958 when Pakistan bought the southwestern port of Gwadar from Oman for 5.5bn Pakistan rupees. 

THE SPECS

Engine: 6.75-litre twin-turbocharged V12 petrol engine 

Power: 420kW

Torque: 780Nm

Transmission: 8-speed automatic

Price: From Dh1,350,000

On sale: Available for preorder now

Emergency

Director: Kangana Ranaut

Stars: Kangana Ranaut, Anupam Kher, Shreyas Talpade, Milind Soman, Mahima Chaudhry 

Rating: 2/5

Dubai Bling season three

Cast: Loujain Adada, Zeina Khoury, Farhana Bodi, Ebraheem Al Samadi, Mona Kattan, and couples Safa & Fahad Siddiqui and DJ Bliss & Danya Mohammed 

Rating: 1/5

Mercer, the investment consulting arm of US services company Marsh & McLennan, expects its wealth division to at least double its assets under management (AUM) in the Middle East as wealth in the region continues to grow despite economic headwinds, a company official said.

Mercer Wealth, which globally has $160 billion in AUM, plans to boost its AUM in the region to $2-$3bn in the next 2-3 years from the present $1bn, said Yasir AbuShaban, a Dubai-based principal with Mercer Wealth.

Within the next two to three years, we are looking at reaching $2 to $3 billion as a conservative estimate and we do see an opportunity to do so,” said Mr AbuShaban.

Mercer does not directly make investments, but allocates clients’ money they have discretion to, to professional asset managers. They also provide advice to clients.

“We have buying power. We can negotiate on their (client’s) behalf with asset managers to provide them lower fees than they otherwise would have to get on their own,” he added.

Mercer Wealth’s clients include sovereign wealth funds, family offices, and insurance companies among others.

From its office in Dubai, Mercer also looks after Africa, India and Turkey, where they also see opportunity for growth.

Wealth creation in Middle East and Africa (MEA) grew 8.5 per cent to $8.1 trillion last year from $7.5tn in 2015, higher than last year’s global average of 6 per cent and the second-highest growth in a region after Asia-Pacific which grew 9.9 per cent, according to consultancy Boston Consulting Group (BCG). In the region, where wealth grew just 1.9 per cent in 2015 compared with 2014, a pickup in oil prices has helped in wealth generation.

BCG is forecasting MEA wealth will rise to $12tn by 2021, growing at an annual average of 8 per cent.

Drivers of wealth generation in the region will be split evenly between new wealth creation and growth of performance of existing assets, according to BCG.

Another general trend in the region is clients’ looking for a comprehensive approach to investing, according to Mr AbuShaban.

“Institutional investors or some of the families are seeing a slowdown in the available capital they have to invest and in that sense they are looking at optimizing the way they manage their portfolios and making sure they are not investing haphazardly and different parts of their investment are working together,” said Mr AbuShaban.

Some clients also have a higher appetite for risk, given the low interest-rate environment that does not provide enough yield for some institutional investors. These clients are keen to invest in illiquid assets, such as private equity and infrastructure.

“What we have seen is a desire for higher returns in what has been a low-return environment specifically in various fixed income or bonds,” he said.

“In this environment, we have seen a de facto increase in the risk that clients are taking in things like illiquid investments, private equity investments, infrastructure and private debt, those kind of investments were higher illiquidity results in incrementally higher returns.”

The Abu Dhabi Investment Authority, one of the largest sovereign wealth funds, said in its 2016 report that has gradually increased its exposure in direct private equity and private credit transactions, mainly in Asian markets and especially in China and India. The authority’s private equity department focused on structured equities owing to “their defensive characteristics.”

COMPANY PROFILE
Name: HyperSpace
 
Started: 2020
 
Founders: Alexander Heller, Rama Allen and Desi Gonzalez
 
Based: Dubai, UAE
 
Sector: Entertainment 
 
Number of staff: 210 
 
Investment raised: $75 million from investors including Galaxy Interactive, Riyadh Season, Sega Ventures and Apis Venture Partners
Related
Cryopreservation: A timeline
  1. Keyhole surgery under general anaesthetic
  2. Ovarian tissue surgically removed
  3. Tissue processed in a high-tech facility
  4. Tissue re-implanted at a time of the patient’s choosing
  5. Full hormone production regained within 4-6 months
Brief scores

Toss India, chose to bat

India 281-7 in 50 ov (Pandya 83, Dhoni 79; Coulter-Nile 3-44)

Australia 137-9 in 21 ov (Maxwell 39, Warner 25; Chahal 3-30)

India won by 26 runs on Duckworth-Lewis Method

The biog

Siblings: five brothers and one sister

Education: Bachelors in Political Science at the University of Minnesota

Interests: Swimming, tennis and the gym

Favourite place: UAE

Favourite packet food on the trip: pasta primavera

What he did to pass the time during the trip: listen to audio books

The specs

Engine: Direct injection 4-cylinder 1.4-litre
Power: 150hp
Torque: 250Nm
Price: From Dh139,000
On sale: Now

The Intruder

Director: Deon Taylor

Starring: Dennis Quaid, Michael Ealy, Meagan Good

One star

Our legal columnist

Name: Yousef Al Bahar

Advocate at Al Bahar & Associate Advocates and Legal Consultants, established in 1994

Education: Mr Al Bahar was born in 1979 and graduated in 2008 from the Judicial Institute. He took after his father, who was one of the first Emirati lawyers

The%20specs%20
%3Cp%3E%3Cstrong%3EEngine%3A%20%3C%2Fstrong%3E2.0-litre%204cyl%20turbo%0D%3Cbr%3E%3Cstrong%3EPower%3A%20%3C%2Fstrong%3E261hp%20at%205%2C500rpm%0D%3Cbr%3E%3Cstrong%3ETorque%3A%20%3C%2Fstrong%3E400Nm%20at%201%2C750-4%2C000rpm%0D%3Cbr%3E%3Cstrong%3ETransmission%3A%20%3C%2Fstrong%3E7-speed%20dual-clutch%20auto%0D%3Cbr%3E%3Cstrong%3EFuel%20consumption%3A%20%3C%2Fstrong%3E10.5L%2F100km%0D%3Cbr%3E%3Cstrong%3EOn%20sale%3A%20%3C%2Fstrong%3ENow%0D%3Cbr%3E%3Cstrong%3EPrice%3A%20%3C%2Fstrong%3EFrom%20Dh129%2C999%20(VX%20Luxury)%3B%20from%20Dh149%2C999%20(VX%20Black%20Gold)%3C%2Fp%3E%0A

Red Sparrow

Dir: Francis Lawrence

Starring: Jennifer Lawrence, Joel Egerton, Charlotte Rampling, Jeremy Irons

Three stars