• The Listener
  • North & South
  • Noted
  • RNZ
Photo/Getty Images

The rise of fake medical studies

Fake news is no longer, well, news, but now there are revelations of made-up medical studies.

High-profile cases of medical fraud, such as the now thoroughly discredited paper linking the MMR vaccine to autism published in The Lancet in 1998, may seem like occasional aberrations. Surely scientists, those systematic gatherers of empirical evidence, don’t set out to deliberately mislead people?

And surely the few bad eggs who do – such as the paper’s lead author, Andrew Wakefield – are suitably punished by being publicly shamed and, as Wakefield was, struck off the medical register?

Sadly, according to Dr Andrew Klein, a UK anaesthetist and editor-in-chief of the journal Anaesthesia, fake medical data is on the rise. Using a data-pattern analysis technique developed by his colleague Dr John Carlisle, staff at the journal are now uncovering possible scientific fraud in about one in 40 papers submitted for publication.

“There’s the odd mistake, but nine times out of 10 people have clearly made their data up or changed their data. A lot of articles that we might have published before are now not going into the journal.”

Klein, who is in New Zealand to speak at the New Zealand Anaesthesia annual scientific meeting in Queenstown, says what’s known as the Carlisle method detects suspicious data in papers based on randomised control trials – often seen as the gold standard for medical research. His journal has used the method to examine both previously published papers and those being submitted for publication. The results suggest a growing use of fake data.

“When going backwards, we found mostly genuine errors, but what we are finding now is more and more people fiddling with the data, rather than making genuine mistakes.”

And he says it’s unlikely to be just anaesthetists who are guilty of fabricating data. It’s almost certainly happening in all medical disciplines, increasing the likelihood that patients will be harmed as a result.

“We base our medical practice on evidence, but if the evidence is fabricated, then you could be putting people at risk of harm or not allowing them to get the best possible outcome.”

Klein thinks the growing use of fake data is largely a result of the pressure on academics to publish papers to advance their careers. “When money is on the line, people change their behaviour.”

And he says that, as with fake news, the use of fake data ultimately undermines the credibility of medical research. “It can be very difficult to distinguish fake data from the truth – it’s the fake-news scandal of medical publishing. If you think everything is fake, then you believe nothing, and science never advances.”

The prestigious New England Journal of Medicine (NEJM) has now joined Anaesthesia in using the Carlisle method to check all randomised control trials submitted for publication. The NEJM was one of several leading journals caught out when Carlisle used the method to detect suspicious data in 90 of more than 5000 such trials published over 16 years. As a result, the NEMJ last year retracted a widely publicised 2013 Spanish paper promoting the benefits of the Mediterranean diet because more than 1000 of the study’s 7447 participants had not been properly randomised.

Although there was no suggestion of fraud, the paper’s authors accepted they had made a mistake and removed the erroneous data. The paper was then republished with similar conclusions.

Klein says the Carlisle method has uncovered blatant fraud examples, such as cases where data from a few participants was repeated many times to inflate study sample sizes.

This year, the World Health Organisation revised a 2016 recommendation on the use of oxygen during surgery that was based on a series of papers published by an Italian anaesthetist, after the Carlisle method found they may have included fabricated data.

Klein says the sheer volume of medical publishing can make the idea of routinely checking for fake data seem impossible. However, he believes it’s vital that all medical journals start to follow the lead of Anaesthesia and the NEJM.

“If everyone worked together on this problem, we could eradicate it, one discipline at a time.”

This article was first published in the August 24, 2019 issue of the New Zealand Listener.