Can we trust HIV drug trials?
posted: 05/10/2009
How far can we trust HIV drug companies? This month, while the media fretted about whether swine flu vaccine may have killed a teenager (it didn’t), the real death action is hidden in a dry report from the Journal of the American Medical Association which has published a damning scientific paper.
Dodgy Drug Trial Tricks
We have known for decades that you can’t always trust what you read in journals publishing research into new drugs and treatments. One problem is 'Missing Trials' - drug trials often go missing: a drug company might do eight trials of a drug, say, but only two have a positive result. The positive trial results are published, while the six with negative results never appear. This censoring harms people, because the results of all 8 trials when combined might show that the treatment doesn’t work, so people are given treatment that is not effective and face unnecessary side effects.
The other drug trials fiddle is researchers who change their stated goal, or "primary outcome", after their trial has finished. You might do a trial on a blood pressure pill, for example, stating that you will look to see if it can reduce heart attacks, but find that it doesn't. Then you might retrospectively change the purpose of your study, ignore the heart attacks, pretend it was only ever about blood pressure, and glowingly report a reduction in blood pressure as if this was what you were always interested in. Or you might measure so many different things that some of them will show up as positive simply by chance.
The solution isn’t working
Both of these dodgy drug research problems were supposed to have been fixed by clinical trials registers. Before you start your drug trial you should publish in the clinical trials register your protocol, saying what your primary outcome is, how many people are in your trial, when it will finish, and so on. People can check this and the published academic paper (if it appears) to see if your trial went ‘missing’, or if you misled people by changing your primary outcome.
Clinical Trials Registers only block bias in drug treatment publications if the Registers are strictly enforced. A check has shown the system doesn’t really work at all. We should not assume HIV drug treatment results published in journals are full and complete.
In 2005 the International Committee of Medical Journal Editors announced they would only publish trials that had been registered. Many journals do check initial protocols against finished academic papers.
Sylvain Matthieu and colleagues checked up how well this system is working and have published their damning findings in the Journal of the American Medical Association. They gathered all of the randomised, controlled trials from cardiology, rheumatology, and gastroenterology in the 10 biggest general medical and speciality journals from 2008.
Both dodgy tactics are widespread
Of these 323 trials less than half were adequately registered before the end of the trial, with the primary outcome clearly specified. Trial registration was entirely lacking for 89 trials. Drug companies know they can get away without registering trials, and so the deaths caused by missing data will continue.
Then they looked more closely at the trials that were properly registered, and found repeated discrepancies between the outcomes stated at registration and the outcomes published in the final paper, in a third of all papers. In almost all the papers where it was possible to assess the switch, a duff outcome was dropped in favour of one that showed a positive finding.
System Failure
You might find it boring, but our failure to ensure full, undistorted publication of all treatment trial data is the single most important issue in medicine today, because this is the only way we can know whether a treatment does good, or harm. The story may be less emotive than one dead teenager, but it costs many more lives – and you should struggle to be angry about it, because the boring regulators we trust to monitor boring problems have repeatedly failed us on this one.
Instead, we rely on good will and vague promises, monitored only by an occasional analysis from an academic on a whim. This is a broken system.
What about HIV?
While this study didn’t look into bias in publications of HIV drug trials, there is no reason to believe that HIV drug trials are better than those, criticised here, for cardiology, rheumatology, and gastroenterology. Read all with caution.
Source
image source
Permalink