Yale’s Study Challenges Carragee’s Mis-Measure of BMP-2

The Spine Journal’s editor, Eugene Carragee M.D., published a flawed and openly biased study of early Infuse studies in June 2011 and set in motion a series of responses one of which is the now completed Yale University Open Data Access review of Medtronic’s BMP-2 patient study data. Carragee’s June 2011 “study” also included a uncharacteristically aggressive attack on the authors of those early studies and in the process threw the spine surgeon community into polarized camps of conflicting and confused opinions regarding the use of recombinant BMP-2 (aka: Infuse) in spine fusion cases. What are the risks of retrograde ejaculation, cancer, ectopic bone formation and which patients are more or less at risk and is Infuse superior, same or worse than iliac bone crest graft (ICBG)?

Medtronic, sponsored most of the Infuse clinical studies, tried to address surgeon concerns by turning all patient level data in its possession over to Yale University’s Open Data Access (YODA) program and providing $2.5 million of no-strings attached funding to support two independent, system reviews of the literature.

The YODA group, which was led by Harlan Krumholz, M.D., Joseph Ross, M.D. and Cary Gross, M.D., published the results of their work in the June Issue of the Annals of Internal Medicine. The issue was released Monday, June 17th. We received advance copies of the work last week.

Interestingly YODA has set up a web site for other researchers and the public to also access this same data-set.

Here is our summary and analysis.

Conclusions:

• We’re back where we were before The Spine Journal’s assault on Infuse. Infuse works in certain patients and certain indications and not in others. And it is comparable to ICBG (the gold standard) in its ability to stimulate bone growth
• Retrograde ejaculation is a non-issue
• Cancer risk is exceedingly small
• The clearest case of where not to use Infuse is anterior cervical fusion
• Early Infuse research was biased.

In their own words, here are the physician editorial recommendations for surgeons who are considering Infuse for their spine fusion patients.

“These findings are important for guiding clinical decision making. On the basis of them, using either autograft or rhBMP-2 to enhance fusion rates in patients having anterior lumbar interbody fusion or posterolateral fusion seems clinically reasonable. Patients should be counseled on the relative benefits and harms of each option and should be allowed to actively participate in decision making. In some procedures, such as anterior lumbar interbody fusion, graft harvest is a separate procedure and avoiding a second incision and associated graft site pain may be well worth the exceedingly small increased risk for cancer. In posterolateral fusion procedures, locally harvested graft and ICBG are often available through the same incision. In these cases, it may not make sense to assume any increased risk, no matter how small. Given the higher complication rates noted in anterior cervical surgery, rhBMP-2 should not be used in this setting without a compelling reason—for example, during a pseudarthrosis repair or other salvage procedure. For posterior interbody procedures, such as posterior lumbar interbody fusion or transforaminal lumbar interbody fusion, the use of rhBMP-2 is associated with ectopic bone formation, and strategies to minimize the clinical effect of excessive bone growth should be used.”

“One caution in interpreting this information is that patients in the comparator group received autologous ICBG, the gold standard. One cannot assume that other graft materials (for example, allograft, demineralized bone matrix, or ceramic) would have results similar to those of autograft or rhBMP-2. Clinical decision making for cases where autograft is not available would have to take into account the probable lower fusion rates with these alternatives compared with autograft or rhBMP-2”.

Methodology:

The methodology used in this review is clearly a step in the direction of more open and ultimately robust science. Specifically:

• Medtronic’s data dump was systematically reviewed by TWO separate and independent sets of academic research teams. Their conclusions are available for side-by-side comparisons in the Annals of Internal Medicine issue. The fact that the two fully independent research teams came to largely the same conclusions lends strong internal validity to the meta-analysis.
• A steering committee which includes representatives from academia, government and industry as well as an additional group of experts guided the research methodologies and best practices approach. The committee did all it could to wring bias out of this review. There was no single Uber-Editor driving methodology and conclusions.
• The authors held a public comment period for review of their proposed methodology.
• All data and methodology is now available for review by other researchers and the public. It is, in other words, transparent.

Consider for a moment the ramifications of this historic approach.

If other research sponsors follow the YODA/Medtronic approach, then the days of a single research entity, like a company or an academic institution, monopolizing clinical data and its conclusions or interpretations could end.

Yes, there is comfort in working with the system we know. But as we saw with both the early Infuse studies and Carragee’s own work, existing checks and balances for publication (peer review) or product approval (regulatory body review) and even the good intentions of researchers themselves can fall well short and even outright fail.

Here is the web site where anybody can apply for access to ALL of Medtronic’s patient level Infuse data.

Benefit and Harm

The two independent research groups came to largely the same conclusions after reviewing the full data sets (the Simmonds, et al. group referenced 111 papers in its footnotes and the Fu, et al. group referenced 83 papers in its footnotes). Here is what the Simmonds group said:

“Our principal analyses were based on data from 1408 individual participants in 11 eligible RCTs, including all trials sponsored by Medtronic (published and unpublished) and 1 additional trial. We found the randomization procedures to be adequate in all trials, but participants were not blinded to treatment. Although assessment of some outcomes, such as radiologic assessment of fusion, was blinded, patient-reported outcomes related to pain were not. Follow-up was reasonably complete up to our final analysis time point of 24 months. Although there is some potential for bias associated with patient-reported outcomes, in general, we consider the body of evidence for comparative effectiveness to be strong.

We found clear evidence that rhBMP-2 improves rates of fusion compared with ICBG; however, the Medtronic definitions of fusion that we used may have been stringent given that only 69% of ICBG recipients achieved fusion within 24 months, which is lower than would be expected generally. Inconsistency across trials was high, with large I2 values at all time points.

We also found that rhBMP-2 improves back pain and quality of life compared with ICBG at between 6 and 24 months after surgery. However, these improvements in pain fall below previously described, clinically meaningful thresholds (estimated as between 4 and 17 percentage points for ODI [Oswestry Disability Index]score and 5.4 points for SF-36 PCS [physical component summary]).

The IPD also indicate that rhBMP-2 may be associated with an increased risk for cancer, with nearly double the number of new cancer cases compared with ICBG recipients. The overall absolute risk for cancer is low in both groups, however, so whether this increased risk is genuine is uncertain, but it is consistent with the literature suggesting a possible link between BMP [bone morphogenetic proteins] and cancer.

Adverse event data in the literature raise concerns that rhBMP-2 may increase the risk for heterotopic bone formation, osteolysis, radiculitis, and retrograde ejaculation. However, these findings should be interpreted cautiously because they are based on only published nonrandomized studies, most of which provided little information about the comparability of groups.

Our review differs from the existing review in that we had access to more extensive and detailed data than did Carragee and colleagues, who used aggregate data extracted from publications of industry-sponsored trials and publicly available FDA summaries and public meeting documents.

The FDA materials seem to provide incomplete outcome data from a subset of trials evaluating rhBMP-2.”

Here is what the Fu group said:

“In spinal fusion, rhBMP-2 and ICBG seem to be similarly effective when used in ALIF and PLF, although the current evidence does not allow definitive conclusions about effectiveness in other surgical approaches. The Short Form-36 Physical Component Summary scores were slightly better with rhBMP-2 than with ICBG in ALIF patients through 24 months, but the difference was only 2 to 3 points on a 100-point scale and thus did not meet typical criteria for a clinically meaningful difference.

The use of rhBMP-2 in anterior cervical spine fusion was associated with statistically significant increases in overall adverse events, wound complications, and dysphagia or dysphonia. For lumbar fusion—both on-label and off-label—adverse events were common with rhBMP-2 and ICBG. Although our review raises concerns about a possible increased risk for retrograde ejaculation, urine retention, subsidence, and ectopic bone formation with rhBMP-2, the data on these harms were sparse and the quality of ascertainment was often poor. Our analysis underscores that more definitive evidence about harms was needed before rhBMP-2 became widely used.

We found that rhBMP-2 was associated with an increased risk for cancer through 24 months regardless of whether non–SEER-reportable cases were considered. This finding should be interpreted with caution because cancer cases were heterogeneous and, according to Medtronic, underreported. Seven Medtronic-sponsored trials (n =429 total) with no cancer cases in either group were not included in the meta-analysis but were not expected to affect the results. Animal studies do not suggest that rhBMP-2 is carcinogenic, but bone morphogenetic proteins are expressed by and promote the growth of some types of cancer. The development of cancer within 2 to 4 years also argues for a pro-oncogenic mechanism.”

YODA Takes on Carragee

Finally, the authors could not let Carragee’s scientifically unsound study from June 2011 go un-corrected. On page 877 of the York (Simmonds, et al.) paper, the authors reference Carragee’s comment that “the risk of adverse events is 10 to 50 times higher than reported in trial publications.”

Ten times to 50x higher. So the authors checked that claim. Here is their conclusion. “Our review differs from the existing review (the authors then cited the Carragee study) in that we had access to more extensive and detailed data than did Carragee and colleagues” and that after reviewing all of the data, the difference between the adverse events reported in the trial publications and actual adverse events was minimal.

Quoting from page 883, the results of their analysis for adverse events was inconclusive and “The only clear evidence of a difference was for pain at or shortly after surgery, which was more common in the rhBMP-2 recipients (odds ratio, 1.78).”

Not exactly 10x-50x. Not even 2x.

That’s a mis-measurement of a scale that approaches scientific malpractice.