Quackademic medicine.
I didn’t invent the term. (Dr. R. W. Donnell did—nearly nine years ago.) However, I sure use it a lot, because it perfectly describes a phenomenon that has proliferated and metastasized throughout the body of academic medicine like the cancer it is. I like to think that, in my own way, I’ve popularized the word to describe this particular phenomenon. But what it this phenomenon? It is nothing less than the degradation of the scientific basis of medicine through the infiltration of pseudoscience and quackery into medical academia, with academic physicians who otherwise should know better embracing pseudoscience like reiki, distance healing homeopathy, acupuncture and most of the rest of traditional Chinese medicine, and studying them scientifically—and even teaching them to medical students as though they had validity. Never mind that, for many of these modalities (I’m talking to you, homeopathy and reiki), basic science considerations are sufficient to reject them as the quackery that they are. Worse, quackademic medicine taints what should be perfectly acceptable science-based modalities, such as nutrition, exercise, and lifestyle with the stench of quackery by association when it “rebrands” them as somehow being “alternative,” “complementary,” or “integrative.” Indeed, quackademic medicine was arguably the original “sin” that ultimately led to “complementary and alternative medicine” (CAM) and now “integrative medicine” (the integration of quackery with medicine).
The history of how the NIH has funded well over $2 billion worth of dubious research through the National Center for Complementary and Alternative Medicine (NCCAM), which was recently renamed the National Center for Complementary and Integrative Health (NCCIH) has been recounted here and elsewhere many times. It’s not just NCCIH, either. The National Cancer Institute has an office, the Office of Cancer Complementary and Alternative Medicine, with the most inappropriate acronym ever, OCCAM, funding and promoting CAM in cancer medicine with a budget as large as that of NCCIH’s.
Many have been the abominations against science funded by NCCIH over its 25 year history, but from my perspective the very worst, the most egregious, the most unethical study ever funded by the NCCIH was the Trial To Assess Chelation Therapy (TACT). Basically, it was a large, multi-institution clinical trial to examine whether chelation therapy was a potentially useful treatment for coronary artery disease. Indeed, it’s one of the oldest topics covered by SBM. Indeed, SBM co-founder and former regular Kimball Atwood wrote extensively about TACT in our very first year of existence (e.g., how TACT was unethical, unscientific, and a waste of money). Before that, Kimball Atwood, Elizabeth Woeckner, Robert Baratz, and Wally Sampson wrote the definitive explanation why TACT was unjustified by science and totally unethical. Eventually I started paying more attention as well. Meanwhile, Dr. Donnell, the man himself who coined the term “quackademic medicine,” led his readers on a magical mystery tour of the quack clinics signing patients up for TACT.
TACT has clearly been the greatest triumph of quackademic medicine in the history of quackademic medicine thus far. It’s not just because quacks managed to persuade the then-NCCAM and then later the National Heart Lung and Blood Institute (NHLBI) to spend over $30 million on this clinical trial to produce at best equivocal results that were completely underwhelming. Rather, it was because quacks managed to parlay those pathetic results into another, even more expensive, trial that will spend $37 million of taxpayer money to do a followup clinical trial examining the supposed “effect” of chelation therapy in diabetics with heart disease, the only subgroup for which the investigators of TACT could seemingly find an effect, or, as it has been called, Son of TACT.
All of this serves as background as to why I was so irritated and disappointed to see an article by journalist I’ve otherwise liked, Karen Weintraub, entitled For daring to study a discredited therapy, this doctor earned scorn — and a $37 million grant. Can you say “false balance”? Sure, I knew you could.
I was actually a bit reluctant to write about this article in my inimitable manner and apply much needed not-so-Respectful Insolence to the article. the reason was the source: STAT News. I’ve already apparently completely alienated one reporter from STAT based on her reporting on Stanislaw Burzynski, and I hate to risk completely alienating another. However, sometimes, a blogger’s gotta do what a blogger’s gotta do, and, given how irritating and full of false balance this article is, I’ll take that risk.
Anyone who does any amount of writing knows that framing is everything, and that every article has to have a frame, a “spin” or point of view, if you will. In other words, there is no such thing as true “objectivity.” All a writer can hope for is to minimize his or her own personal biases. On the other hand, the obsessive fetishization of “balance” can lead to travesties like this article, which portrays Dr. Gervasio Lamas, the cardiologist who has been the prinsicpal investigator of TACT, as a “brave maverick doctor” whose only offense was following the evidence where it went, which is the sole reason why he’s being “persecuted” (i.e. ostracized) for his having become a key figure in the world of quackademic medicine based on his having spearheaded TACT. The problems start right from the beginning, with this framing:
The new heart patient asked Dr. Gervasio Lamas if he thought chelation therapy was worth a try. “Of course not!” the cardiologist replied emphatically. His Harvard training had taught him that alternative therapies were a waste of time and money, and potentially risky to boot. “I told him it was quackery.”
But Lamas went home that night unsure if he had given his patient the best medical advice. He looked up research on chelation therapy, which removes heavy metals from the body, and found very little data either supporting or contradicting the procedure.
Lamas was troubled by the idea that he had offered this man a medical opinion that wasn’t supported by science. And he decided to conduct a study himself. He had no idea what he was getting into.
Seventeen years later, his research into a therapy many of his colleagues consider bunk has earned him scorn and sideways glances at medical meetings. Some accuse him of wasting taxpayer money. But Lamas persevered — even briefly taking out a second mortgage on his house to help pay for a clinical trial.
Three years ago, he announced results of that $30 million study: Chelation was safe and potentially helpful. The conclusion shocked the mainstream medical world, including Lamas. After all, the procedure had been so discredited that doctors previously could lose their medical licenses for using it.
Actually Dr. Lamas was right the first time.Basically, TACT was basically a negative trial, when viewed critically.
I notice how Weintraub describes Dr. Lamas as having found “very little data either supporting or contradicting the procedure.” I’ll give you the translation of that: He didn’t find much in the way of clinical trials “supporting or contradicting” the procedure. However, if he had just looked a bit closer he would have seen that, even then, the existing evidence was very much like the evidence for acupuncture for various conditions. There were the occasional “positive” trials, but the preponderance of evidence was most consistent with there being no specific effect due to chelation therapy on cardiovascular disease above and beyond that of placebo effects. Moreover, the various concepts used to “explain” how chelation therapy “works” were unconvincing. then and remain unconvincing now.
Indeed, the ever-shifting concepts of how chelation therapy supposedly works remind me a lot of the same sorts of ever-morphing explanations for how acupuncture supposedly works. For example, first there was the “roto-rooter” hypothesis, in which chelation proponents claimed that calcium was an integral part of the atherosclerotic plaque and that removing calcium would cause the plaques to “melt away.” Of course, many, if not most, plaques don’t contain calcium deposits, and in those that do calcium deposition occurs quite late in the pathological process. Then there was an even more nonsensical “explanation” that when ionic calcium was removed from serum by chelation, it was replaced by calcium from bone, which would then stimulate the parathyroid glands to secrete parathormone, which then promoted remineralization of bone. The really nonsensical claim was that the calcium for this bone remineralization came from the “gradual transfer” of calcium from hardened arterial tissue and plaque, which would then “soften” the arteries and cause plaques to disintegrate. Then, of course, there was the “free radical” concept because, of course there was, and, even more implausible, there evolved the concept that chelation therapy somehow prevented “mutations” that lead to atheromas and later atherosclerotic plaques. Never mind that atheroma formation is a chronic inflammatory process, not a process involving mutation of the vascular endothelial and smooth muscle cells. Moreover, in pretty much all cases, stoichiometry just doesn’t work out, and the bottom line is that chelation therapy is incredibly implausible. Not homeopathy-grade implausible, perhaps, but plenty implausible.
In fairness, one can’t fault Weintraub for not going into detail about how implausible chelation therapy is. However, I can fault her for portraying Dr. Lamas as a brave iconoclast suffering for pure science, unafraid that his results seemingly buck the establishment and then basically dismiss critics. For example, Steve Salzberg is quoted thusly:
Now, he’s launching a second trial, with $37 million from the National Institutes of Health, to study chelation’s effect on diabetics with heart disease. That’s not sitting well with some scientists.
Steven L. Salzberg, director of the Center for Computational Biology at Johns Hopkins University, said it’s “just crazy” to do another trial. “There are better ways to use this money to study much more promising treatments of all sorts,” he said.
Salzberg, who writes the Fighting Pseudoscience blog on Forbes.com, said the history of science is littered with studies that began with good intentions or were conducted by nice people, but were still bad ideas. “There are many legitimate MDs who’ve come up with theories that were just wrong, but clung to them, despite all the evidence to the contrary,” Salzberg said. “I wouldn’t put them in jail, but I’m not going to give them $37 million.”
And dismissed by Weintraub thusly, with a blithe wave of Dr. Lamas’ “I’m just following the evidence” hand:
For his part, Lamas said he has learned to ignore the critics and just follows the evidence. If the study had shown chelation to be completely useless, he would have believed the results. So why shouldn’t he believe the data now?
That’s exactly the problem, though. Dr. Lamas doesn’t believe his own data. He spins his data as positive when, viewed critically, TACT was a negative trial and didn’t show what Lamas thought it did, which Weintraub described thusly:
The unveiling of results at a meeting in 2013 shocked everyone. A picture taken afterward shows a roomful of stunned faces. Chelation was safe, the study revealed. And it also seemed to be effective.
Overall, patients getting the active therapy had “modestly” fewer heart attacks and needed fewer bypass surgeries and hospitalizations for chest pain than those getting the saltwater placebo, according to the study, published in the Journal of the American Medical Association. When the researchers drilled down into their data, they realized that the patients who had both diabetes and heart disease saw all the benefits. Among the 633 test subjects with diabetes, there was a 41 percent reduction in cardiovascular events such as heart attacks, over as long as five years.
Not quite. TACT relied on a primary endpoint that was a composite of death, MI, stroke, coronary revascularization and hospitalization for angina. Now, I realize that other clinical trials of cardiovascular disease use composite endpoints, and no doubt TACT apologists will point that out. That doesn’t make it less of a problem. I find such composite endpoints to be highly dubious, particularly when they include something like hospitalization for angina. Death is a “hard” endpoint. You’re either dead or you’re not. MI (myocardial infarction) and stroke are “hard” endpoints. You’ve either had a stroke or MI or you haven’t. However, when composite endpoints are used, different confounding factors can be amplified, as González et al concluded:
The use of composite end points in cardiovascular trials is frequently complicated by large gradients in importance to patients and in magnitude of the effect of treatment across component end points. Higher event rates and larger treatment effects associated with less important components may result in misleading impressions of the impact of treatment.
Two elements of the composite endpoint used by Lamas, coronary revascularization and hospitalization, are subject to a great deal of clinical judgment in deciding who requires them. More problematic is the variation in usage of such interventions that has nothing to do with whether the treatment works or not. For instance, in the state of Michigan, there is up to a 2.4-fold variation in rates of percutaneous coronary interventions (PCI; i.e., angioplasty) between the highest use and lowest use areas. That’s just one state. Similar studies have shown wide variability in coronary revascularization rates just on geography alone. Adding such a variable to a composite endpoint is thus a bad idea on a scientific basis. At best, it adds unnecessary variability to the composite outcome measure for no useful benefit; at worse it adds significant bias, particularly if subjects being treated in academic centers, where patients are more likely to be referred for appropriate coronary revascularization interventions were in areas with significantly different PCI usage rates than areas where subjects being treated in “complementary and alternative medicine” (CAM) centers, where one might reasonably expect that referral for revascularization might be a bit less—shall we say?—expeditious (more on those centers later).
Let’s step back a minute and show you how negative TACT really is. TACT had a 2 x 2 factorial design
- Chelation plus high oral high dose vitamin and mineral supplement
- Chelation placebo plus oral high dose vitamin and mineral supplement
- Chelation plus oral high dose vitamin and mineral supplement placebo
- Chelation placebo plus oral high dose vitamin and mineral supplement placebo
The regimen was described in detail in a 2012 publication. The vitamin supplements included doses ranging from 25% to 6,667% of the RDA for various vitamins. For example, the dose of vitamin C was 2,000% of the RDA; thiamin, 6,667%; and vitamin A, 500%. The first results of the trial were published in JAMA in 2013. The first thing we can eliminate is the high dose oral vitamin and mineral supplements. No matter how much the data were tortured, no positive result was seen from those arms of the study, either in any of the individual endpoints, the composite endpoint, or the quality of life measures.
As for the primary endpoint (i.e., the aggregated serious cardiovascular events), there was indeed show a modest difference, namely 30% of placebo subjects versus 26.5% of the EDTA chelation subjects suffering an event (hazard ratio 0.82 for chelation). Unfortunately, as Dr. Atwood and I have noted before, the result was just barely statistically significant, p = 0.035, with the 99% confidence interval for the hazard ratio ranging from 0.69 to 0.99. (The predetermined level for statistical significance for purposes of this study was 0.036; so this is statistically significant by the barest margin.) More importantly, if you look at the individual endpoints that make up that aggregate, there was no statistically significant difference in death, myocardial infarction, stroke, coronary revascularization, and hospitalization for angina. Subgroup analysis (in fairness, preplanned) purported to show a much greater benefit for diabetics, with a hazard ratio of 0.61 (p=0.002), while patients without diabetes showed no statistically significant difference in any of the outcome measures, including the aggregated total bad outcomes.
One question that came up last time had to do with other ingredients in the chelation mixture, specifically procaine and heparin, either of which could conceivably have had an effect on cardiovascular outcomes, particularly when given over the course of months intravenously. Another question that came up was how there could have been a seemingly so much better outcome in diabetics. One notes that the placebo solution contained 1.2% glucose in order to match the osmolarities of the control and experimental solutions. That could conceivably have contributed to a slightly worse outcomes in the control group even in the absence of a therapeutic effect due to chelation. Whatever the case, one notes that in nondiabetic patients there was no statistically significant detected benefit due to chelation therapy. Finally, only 65% of subjects finished all infusions, with only 76% finishing at least 30. That’s a high drop-out rate. Moreover, 17% withdrew consent, resulting in missing data. The investigators tried to correct for this in an online supplement, but these issues remain serious. They might not be so serious as to call into doubt the effect reported if there had been a much more convincing treatment effect, but when you get equivocal results such as this such issues loom much larger.
Again, I don’t necessarily expect a journalist to go into detail about these problems, given space considerations, but perhaps if she understood them better she might have understood that the problems with TACT were real and that criticisms of the trial weren’t just “conventional” doctors closing ranks but the result of very serious issues with the trial, its interpretation, and its sale to the public. Or maybe not. There’s no way of knowing now. What I do know is that this whole article frames Dr. Lamas as a skeptic who did nothing more than “follow the science,” even having started out trying to “disprove” chelation therapy. Personally, I call BS on that claim. No physician sinks a decade of his life into a project as large as a $30 million clinical trial solely for the purpose of proving that something doesn’t work, or, as Dr. Lamas is portrayed, hoping “chelation wasn’t dangerous” and wanting to “be able to tell patients with confidence that it was also a waste of their time and money.”
Weintraub does mention some of the problems with TACT, but almost in passing:
The trial faced repeated challenges, from an investigation into whether patients were adequately safeguarded against a treatment that conventional doctors considered dangerous to a public call for its abandonment from a group of fellow doctors.
These “challenges” went far beyond that. For example, one of the highest accruing TACT sites was inspected by the FDA and garnered Form 483 filed by the FDA at least as brutal as previous Form 483s filed on Stanislaw Burzynski. Here are some of the findings:
- The investigators didn’t conduct the investigation in accordance with the signed statement and investigational plan. Several examples were given of shoddy procedures, prefilled forms, and failure to train personnel.
- Failure to report promptly to the IRB all unanticipated problems involving risk to human subjects or others. Examples are given, including failure to report the deaths of patients on the study in a timely fashion (in one case the death wasn’t reported to the IRB until four months later; in another case it was never reported at all). In other cases, adverse event reports were not submitted to the IRB.
- Failure to prepare or maintain adequate case histories with respect to observations and data pertinent to the investigation.
- Investigational drug disposition records are not adequate with respect to dates, quantity, and use by subjects.
One of TACT’s critics, Dr. Steven Nissen, chairman of cardiovascular medicine at the Cleveland Clinic, is quoted criticizing both TACT and the new study, TACT 2. Remember what I said about the composite endpoint used? Dr. Nissen was even more critical at the time in an editorial in JAMA, where he noted that because he primary composite endpoint included “softer” endpoints and these softer endpoints represented 318 of the 483 events reported, ““if any unblinding occurred, investigator biases could potentially influence the decision to hospitalize or revascularize individual patients.” He then went on to point out why we should worry about unblinding:
Differential dropout in TACT suggests unmasking, but the problem of intentional unblinding is more concerning. The sponsors of the trial, the National Heart, Lung, and Blood Institute (NHLBI) and the National Center for Complementary and Alternative Medicine (NCCAM), were unblinded throughout the trial. The National Institutes of Health policy unwisely allows the sponsor access to unblinded trial data, and both organizations sent observers to the closed sessions of the data monitoring committee. This gave them access to confidential data during each of the 11 interim analyses. The unblinding of the study sponsor represents a serious deviation from acceptable standards of conduct for supervision of clinical trials. If a pharmaceutical company sponsoring a trial were allowed access to actual outcome data during the study, there would be major objections. Like any sponsor, the NHLBI and NCCAM cannot be considered unbiased observers. These agencies made major financial commitments to the trial and may intentionally or inadvertently influence study conduct if inappropriately unblinded during the study.
That’s in addition to something in the TACT protocol first noted by, yes, Dr. R. W. Donnell over ten years ago, namely that the protocol is inadequate and that the blinding was probably not secure at many TACT sites, and there were many recruitment irregularities, including advertising that everyone gets chelation at some clinics.
So let’s backtrack. What we have here is a study of a treatment that was correctly considered to be quackery that cost $30 million and produced at best a marginally significant result in one subgroup at worst (and far more probably) a spurious result that could well have been due to bias. As a result of the incredibly flawed data from that trial, the federal government has greenlighted a new trial that not only uses the same flawed composite endpoint but is even worse in that it is a pragmatic trial, uses many of the same quack clinics to accrue patients, and will cost even more, $37 million. Now that would actually be a great story, but instead what we get is a human interest story about the principal investigator of both of these boondoggles that checks all the boxes. Not a believer at first? Check. Caring doctor? Check. Dedicated researcher? Check. Is just “following the science, no matter where it leads”? Check. Persecuted by his conventional medical colleagues? Check. I can see the pitch for a movie treatment of the story of TACT in the back of my mind already, and, not surprisingly, Weintraub is by no means the first to have spun Dr. Lamas’ story this way.
One one level, I can see the temptation to frame the story in this manner. After all, if someone were to pitch to me an idea for a story in which an expensive clinical trial of something widely derided as quackery had produced an unexpected “positive” result, I’d say, “Hmmm. That’s interesting.” The failure here is that that is not the story of TACT. The story of TACT is ideology triumphing over science to the tune of first $30 million and then $37 million to carry out unethical studies of an ineffective treatment. My spin on this story would be how, in this time of highly constrained research funding, spending $67 million to study quackery that, even if you take the results of TACT at face value, is completely ineffective in anyone other than diabetics with prior MIs and, if you are critical, is almost certainly ineffective in them too, is inexcusable, an unconscionable waste of funding that could have been used to study actual, important medical questions.
I’ll give Weintraub credit, though, for recognizing one thing correctly. Unfortunately, she also gets one thing wrong, too::
Despite the time, effort, and money, the initial trial hasn’t had much of an impact. Few mainstream doctors have changed their minds about chelation, and alternative medicine therapists apparently haven’t stopped using it in patients without diabetes — though the trial found essentially no benefit in the broader heart disease population.
At one level, that’s the way it should be. No single scientific study can be considered the truth. Findings must be repeated to be confirmed.
Well, yes and no. Yes, no single scientific study can be considered “the truth,” but, on the other hand, a large, negative clinical trial like TACT, even if viewed uncritically and taken at face value, should shut the door forever on the use of chelation therapy in non-diabetics, other than in research settings. It’s an easy prediction to make that, even if TACT2 is even more resoundingly negative than TACT was, few, if any, alternative practitioners will abandon the practice, because their practice is based on belief, not science.
I once predicted the results of TACT, long before the trial was finished, I predicted that it would be a largely negative trial but that there would be one seemingly “positive” result and that chelationists would latch on to that result to justify continuing to use chelation therapy and doing another trial. Given that TACT 2 is, like most sequels, worse than the original, I make the not-so-difficult prediction that the same sort of thing will happen here. The results will show a marginally statistically significant result in some subgroup or other (e.g., diabetics with previous MIs over the age of 70 or under the age of 50 or something like that) and will be completely negative for everyone else. A critical reading of the study results will be that it’s a negative trial, but chelation advocates will latch on to the even more narrow seemingly positive result than the one in TACT in order to justify another clinical trial, and the NCCIH and NHBLI will go along. Meanwhile, critics will be attacked as “shrill and brutish” ideologues unwilling to go where the science leads, just like what happened after TACT.
Rinse, lather, repeat.
from Science-Based Medicine http://ift.tt/2htxqqY
No comments:
Post a Comment