The National Comprehensive Cancer Network often publishes clinical practice guidelines that help shape the course of treatment for cancer patients and insurance coverage.
However, a new study has called those guidelines into question.
The strength of evidence referenced by the US-based group, referred to as the NCCN, when formulating guidelines and making recommendations appears to be weak, according to the study published in the journal BMJ on Wednesday.
The study found that recommendations may include using a drug for a type of cancer for which it wasn’t necessarily approved by the US Food and Drug Administration, also known as “off-label” use, which is widely practiced in oncology care.
The study also referenced previous research that has shown 84% of NCCN members involved in developing the guidelines have received personal payments from the pharmaceutical industry.
For more than 20 years, the NCCN has developed and published guidelines, but if those guidelines are not based on strong evidence then cancer patients could be paying more than necessary for treatments, said Dr. Vinay Prasad, assistant professor of medicine at the Oregon Health and Sciences University, who led the new study.
“The average doctor in America — in my experience having worked at several hospitals — we look at this guideline all of the time to make decisions about what treatments to give. It is a very user-friendly guideline,” said Prasad.
“So, we use this for treatment, but the question is … the NCCN may recommend that, but why do they recommend it? What is the evidence supporting those recommendations? In this study we find unfortunately it is weak or lacking in many cases and I think that is of some concern.”
Public and private insurers, such as the US Centers for Medicare and Medicaid Services and UnitedHealthcare, recognize the NCCN drugs and biologics compendium, based directly on the NCCN clinical practice guidelines, as a reference for oncology coverage policies.
“These drugs are not cheap, and NCCN recommendations often mandate CMS payment and commercial insurer payment. That raises all of our premiums, and patients often have to pay the copay,” Prasad said.
The NCCN is not the only group making recommendations that influence policies. The Thomson Micromedex DrugDex Compendium and others are listed under compendia on the CMS website.
The researchers focused on the NCCN because it is widely used in clinics around the country, Prasad said, adding that his own fellows reference the NCCN often.
The NCCN Clinical Practice Guidelines for Oncology are continuously updated based on the strongest scientific evidence available, said Dr. Robert W. Carlson, Chief Executive Officer of NCCN, in a written statement on Wednesday.
“NCCN’s 1,355 panel members come from 27 leading academic cancer centers in the United States. Their expertise allows them to evaluate complex circumstances based on all available data, in order to come to a consensus about what constitutes optimal care,” Carlson said in the statement.
“The NCCN process is designed to make sure the most effective, life-saving therapies are accessible and available to the patients who need them,” he said. “CMS and other major providers have designated NCCN as the leading source for arbitrating oncology drug and biologic coverage because of the NCCN Guidelines’ proven track record for helping physicians to prolong their patients’ lives and reduce their suffering.”
The cost of guidelines
Prasad and his colleagues created a dataset of 47 drugs approved for marketing by the FDA for adult hematologic or solid cancers between 2011 and 2015.
For each drug in the dataset, the researchers compared how many drugs received FDA approval with how many drugs were NCCN recommended. They looked at where approval and recommendations overlapped. The researchers also took a close look at where the NCCN made additional recommendations beyond the FDA.
The researchers found that the drugs in the study, as of March 2016, were FDA-approved for a total of 69 treatment approaches, but the NCCN recommended those drugs for a total of 113 approaches, which included the approaches approved by the FDA but also many others.
Among those recommendations that the NCCN made beyond the FDA approvals, only 23% were cited as being based on evidence from randomized controlled trials, the researchers found, and just 16% were based on findings from phase three trials. Both types of trials are considered the gold standard in medical research.
A phase three clinical trial aims to compare the safety and effectiveness of a new treatment against what’s already available as the current standard treatment.
“This is something that is increasing costs with what appears to be low levels of evidence,” Prasad said about the study.
Another study, published in the Journal of Clinical Oncology in 2013, suggested that among the 10 most commonly prescribed chemotherapies, off-label uses supported by guidelines from the NCCN accounted for around an extra $2 billion in spending in 2010.
“If I could wave a magic wand, I would say CMS needs to strongly reevaluate whether or not newer branded $100,000 medications should be using compendium to expand their market share,” Prasad said. “Or if instead they should have to generate evidence and seek formal FDA approvals.”
In response, FDA spokeswoman Sandy Walsh wrote in an email “in general, the FDA does not comment on specific studies, but evaluates them as part of the body of evidence to further our understanding about a particular issue and assist in our mission to protect public health.”
The question patients should ask
The new study had some limitations, including that the researchers did not search evidence behind the drugs beyond what the NCCN provided in its recommendations. Also, the findings only represent the four year time period evaluated in the study, Prasad said.
Rena Conti, an associate professor at the University of Chicago and an expert in health policy and economics, said that the study appears to overstate the findings and its relevance to policy.
She added that the authors failed to assess whether off-label recommendations made by the NCCN are for diseases where there are limited treatment options or where death rates are high.
“This may induce NCCN reviewers to accept lower quality of evidence to support recommended off-label indications,” said Conti, who was first author of a separate 2013 study in the Journal of Clinical Oncology.
The new study also implies that NCCN guidelines may be the only basis for coverage and reimbursement decisions by payers and use by medical providers, but “it is not clear this is the case in practice,” said Conti.
“Payers and medical providers use both NCCN guidelines and other criteria in making these decisions. This is less well stated in the paper than it should have been,” she said.
“Although NCCN guidelines do not rate the evidence to support off-label indications, other compendia do assess the quantity and quality of evidence in support of the off-label use of cancer drugs,” she said. “In the past several years there has been a significant shift among physicians and trade groups like ASCO (American Society of Clinical Oncology) towards better understanding the benefits and the costs of using these drugs for payers and patients.”
All in all, the new study turns a spotlight on two issues, said Dr. Otis Brawley, chief medical and scientific officer and executive vice president of the American Cancer Society, who was not involved in the study.
First, “the people who are making these decisions about treatment are often times people who are consultants and paid by the drug companies that benefit from these recommendations,” Brawley said, adding that the American Cancer Society doesn’t make recommendations regarding treatments due to that risk of potential conflicts.
He noted a separate study published in the journal JAMA Oncology in 2016 found that among 125 authors behind NCCN guidelines, 108 of them, or 86%, had at least one financial conflict of interest, such as receiving some type of payment from the pharmaceutical industry. More specifically, 84% of them received personal payments while 47% of them received research payments.
Second, “the bigger problem in my mind is that many of these ‘best guesses’ that these people are making are actually sometimes very reasonable — they’re reasonable based on science — but there’s no clinical research being done to validate them,” he said. “The real shame to me is that we’re not doing the science to validate these ‘best guesses’ or these things that are professional opinion.”
When a drug is recommended for use beyond what it has been FDA approved for, one of two consequences could follow, Brawley said.
“It could be a brilliant move or it could be a disaster,” he said. “My greatest concern is these are clinical trials that need to be done. These are smart people who have come up with really good ideas who think that these drugs work in these diseases. We need to test them.”
In the meantime, “The only thing patients can do is ask their doctor, ‘Why am I being prescribed this drug?'” Brawley said. “What is the scientific basis on which I am being given this drug?”
