Continuing the series started Friday – fourteen foundation truths to set the stage for the upcoming book Superpatients: Patients who extend science when medicine’s out of answers.
Principle #1 said that good science is measurable, verifiable, and reproducible. Here’s #2, which is a shocker to a lot of people. As the articles linked below say, it’s a big problem for doctors, not just for patients:
That’s right: good science is great, but –
- Some published studies are weak (read on for the grisly details)
- Some studies are corrupt (sorry, but it’s true)
- And believe it or not, most studies have never been verified by another lab.
The point here is not to indict doctors or researchers. It’s that superpatients are entirely justified in striking out on their own, because it’s demonstrably wrong to say “Just look in the literature.” For one thing, what you need might not be in the literature. For another, even if you find something, it might be bogus. Or simply undependable. And who needs undependable science? That’s why “verifiable and reproducible” was in #1.
In short: when medical science is out of answers, it’s perfectly legitimate to look further.
[You can stop reading here if you want. If you want details, the excerpt below is adapted from Superpatients.]
1. Some published studies are scientifically weak
A big part of the culture of medicine is to trust recognized authority. Well, look at what these editors of three top medical journals have said:
- In 2009, Richard Smith (25-year editor of the BMJ) said, “Most of what appears in peer-reviewed journals is scientifically weak.” (Read the rest of what he said at that link, too.) What??? Read on.
- That same year, Marcia Angell (long-time editor-in-chief of the New England Journal of Medicine) said, “It is simply no longer possible to believe much of the clinical research that is published …. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor …”
- In 2015, Lancet editor Richard Horton said, “The case against science is straightforward: Much of the scientific literature, perhaps half, may simply be untrue.”
That’s Lancet, BMJ, and the New England Journal. Holy crap.
2. Some studies are corrupt.
This is hard to swallow, but it’s true. A real problem is that it doesn’t need to be intentional – it can just be human nature.
Studies about a drug that are funded by the drug’s maker are more likely to come out favorable. And after Marcia Angell left her role at the New England Journal, in 2012 the Times wrote that her successor “and his colleagues reported that for some subjects, so few experts without financial ties could be found that the journal’s scope was becoming artificially curtailed.”
Yes, Angell’s successor said if we exclude tainted people, the NEJM won’t have enough authors. He said that. And a decade later the Washington Post reviewed a year of NEJM studies of new drugs (73 articles) and found that 60 had been funded by the drug company, and 53 were co-written by the companies’ employees!
Oh, and then there’s Ben Goldacre’s AllTrials.net project, which has discovered that many clinical trials with results unfavorable to the funder are covered up – they’re never published. If that’s not scientific corruption, I don’t know what is. (See Ben’s entertaining TEDMED talk, and maybe my noisy iPad interview with him that night.)
Note: this isn’t just a problem for patients – it’s a hell of a problem for doctors! They’re told to obey what’s in the literature, so what are they supposed to do??
3. Most clinical trials have never been replicated by another lab.
“But wait!” you cry – “That flies in the face of rule 1, ‘good science is reproducible!'” You’re absolutely right. But it turns out scientific studies are really expensive – and scientists generally don’t want to risk the chance that another lab would disagree with their results. (When a California lab started a service to replicate studies at 1/10 the cost, it got little uptake.)
There’s this New Yorker piece on “the decline effect” – things that were supposedly proven true start turning out to be less true. OMG.
The superpatient takeaway
The point isn’t that you can’t trust anyone about anything; the reality is that people are living longer than ever, and that’s no accident, which is proof positive that medicine works better than ever. The point is that there’s still a major fly in the ointment if you assume you can trust whatever’s in the literature, and if it’s not in the literature you can’t.
Again: when medical science is out of answers, there’s plenty of reason for superpatients to look further.
Further principles will build on this, later on.
Next in the series: #3: Don’t expect docs to know all and others to know nothing.