Flaws Found in Cancer Research Trials

In the February 1, 2007, issue of the journal Clinical Cancer Research, researchers from Memorial Sloan-Kettering Cancer Center (MSKCC; New York, NY, USA) discovered that only nine of the 70 phase II studies they assessed clearly defined measures by which an experimental drug could be judged to offer benefit to patients.

"We are facing a new and growing problem in clinical trial testing, and that is while the drugs have changed, researchers are still using the same old methods to gauge how effective they are,” said the study's lead author, Andrew Vickers, Ph.D., a research methodologist.

The problem, according to Dr. Vickers, is that for such a long time, therapies (typically chemotherapy) were assessed by seeing if tumors would shrink in patients with advanced cancer. Measuring that reduction was an accepted way to assess benefit, he said. But current new treatments, which can include targeted therapies that slow tumor progression, are frequently evaluated in less advanced cancer and in combinations "and it can be hard to answer the question of whether patients are doing better than expected,” he said.

In their study, Dr. Vickers, Howard Scher, M.D., chief of the MSKCC genitourinary oncology service, and medical student Vennus Ballen, examined phase II clinical trials reported from June 2003-June 2005 in the Journal of Clinical Oncology and in the journal Cancer, two major journals in cancer research. These studies, which usually enroll 30 to 50 patients, aim to provide a "go/no go” decision on whether the therapies studied should be evaluated in a large phase III clinical trial, the ultimate test of whether a drug should be given to cancer patients.

They specifically looked at 70 studies whose design required "historical data” to determine whether a drug was promising enough to justify a phase III trial. "When a novel agent is added to an existing standard in the hope of increasing response rates over and above those expected from the standard treatment alone, historical data on the response rates to the standard treatment are required,” Dr. Vickers said. "Similarly, some agents are thought to slow disease progression, rather than lead to rapid tumor regression, necessitating an endpoint such as progression-free survival or overall survival at one year. That survival target clearly needs to be developed by reference to historical data.”

For example, if two chemotherapy agents used in combination lead to a 30% survival rate at one year, and researchers are interested in knowing whether an addition of a third drug is of benefit, the three-drug combination has to meet that 30% hurdle and jump over it, according to Dr. Vickers. "So we have to be pretty certain that the 30% target is correct,” he said.

Of the 70 studies they examined, however, nearly half (46%) did not give any justification for the historical target. And of the studies that did refer clearly to prior data, only several (nine, or 13%), did so correctly. Furthermore, trials that failed to report a reason for the historical bar were much more likely to decide that the new therapy was "active” and therefore worthy of further study or a phase III clinical trial, according to Dr. Vickers.

The researchers could not find a single study that used advanced statistical techniques to adjust for differences between patients studied in older clinical trials that were used as the historical bar and patients treated in the new trial, who may be at an early stage of cancer.

"These studies could have been done better,” stated Vickers. "Phase II studies are all about seeing whether patients on a new treatment are doing better than expected; if so, we should investigate the new treatment in a really big trial. However, to know whether we are ‘doing better than expected,' we need some kind of benchmark of what we should expect from standard treatment. That benchmark assessment is what we find is missing from these studies.”




Related Links:
Memorial Sloan-Kettering Cancer Center