Overcoming the Efficacy-Effectiveness Gap in Cancer Therapy Research

There is an abundance of efficacy data and comparative efficacy data from clinical trials, but this is all based on treatment performance under very controlled conditions. In addition, the treatment population in a randomized controlled trial (RCT) is often much different from the patients encountered in routine clinical practice.

Treatment guidelines are based on patients in clinical trials, but these patients are only 5% of the population and they are younger and fitter than the 95% of patients treated outside of these trials, explained Miriam Koopman, MD, University Medical Center, Utrecht, the Netherlands.

Miriam Koopman, MD

Image credit: ESMO

In a session at ESMO Congress 2023, Koopman and her fellow speakers outlined the efficacy-effectiveness (E-E) gap for cancer therapies and some ways to overcome it. ESMO was held October 20-24, 2023, virtually and in Madrid, Spain.

In the real world, there is a scarcity of effectiveness data, and what there is shows that treatment effectiveness is much lower in reality. Hendrik van Halteren, MD, Admiraal de Ruijter Hospital in the Netherlands, provided an overview of the factors that can contribute to the E-E gap.

He provided the example of a patient: a 70-year-old male with advanced gastric cancer that had spread to the lymph nodes. While Halteren might prescribe a combination of chemotherapy and nivolumab, it would require referring the patient to another a hospital almost 70 miles away. Given the travel burden, the patient wanted to know what benefit he would get from nivolumab.

Hendrik van Halteren, MD

Image credit: ESMO

According to the RCTs out there, the patient would not have been eligible for the trials, which means it might not be clear what benefit he would get. While these RCTs are the gold standard or the cornerstone of cancer intervention research, the inclusion and exclusion criteria will often create a patient population that does not match the general population with the disease in the real world. For instance, criteria may mean patients of a certain age, performance status, or complex comorbidity profile cannot participate. In addition, there may be disparities regarding ethnicity and socioeconomic status if participating in a trial is burdensome between time spent and costs.

Research published in 2020 sought to determine the E-E gap by comparing the survival of patients in clinical trials with that of patients in the real world.¹ The authors used real-world evidence from the Cancer Care Ontario New Drug Funding Program database and discovered a lower probability of survival if treated with the same drug in the real world vs in a clinical trial. The median overall survival (OS) was 5.2 months lower in the real world compared with the clinical trial, and the median difference between hospitalization in the real world and serious adverse event rates in the clinical trial was 14%.

“There's a tendency towards a poorer efficacy if you're comparing effectiveness with efficacy and towards more toxicity,” Halteren said. “So, therefore, I think that comparative effectiveness data could differ from comparative efficacy data.”

Tools to measure comparative effectiveness include pragmatic trials, observational studies, and retrospective studies, but due to the uncontrolled nature of these settings, the data analysis needs to be adjusted for potential bias and confounders.

An example of a comparative effectiveness trial² evaluated 17,801 patients with resected stage III colon cancer using a Taiwan cancer registry to assess 3-year disease-free survival and 5-year OS rates before and after oxaliplatin was reimbursed. The researchers performed a boosted regression analysis, including a propensity score method. The study found, irrespective of age, no significant improvement in survival in the real world when oxaliplatin was added to chemotherapy for these patients.

Importantly, Halteren noted, the results of this study compared negatively to the MOSAIC study,³ which have led to the current guidelines of care.

“So, given the fact that in real life, oxaliplatin tends to perform poorer [and] given the fact that oxaliplatin is a pretty toxic substance, it may be worthwhile to reconsider its benefits in the real-life, stage III colon cancer setting,” he said.

The challenge facing clinicians, patients, and manufacturers is that everyone wants to get drugs to market faster, and comparative effectiveness trials take longer to conduct. Gabe Sonke, MD, PhD, MSc, Netherlands Cancer Institute in Amsterdam, the Netherlands, tried to elucidate how often the E-E gap affects treatment outcomes in daily practice.

Unfortunately, it’s hard to know how often the E-E gap is occurring, because while we know there are differences between the patients in the trial and not in the trial, there are few data on the patients not in the trial. The only way to get more data would be to randomize them, which obviously didn’t happen if they weren’t in the trial.

A recent study out of the Netherlands may be close to answering this, though.⁴ The study used the Netherlands Cancer Registry, which covers almost all of the patients in the country with cancer, to evaluate improvement in OS in primary stage IV cancers from the beginning of the registry in 1989 to 2018. The study included real-life data of patients who may have been in the studies and those who weren’t.

First the study showed that some tumor types saw big improvements in 5-year net survival as additional drugs were approved, while others didn’t. For instance, in non–small cell lung cancer, there had been 19 new drugs, but there was little change in survival. In contrast, gastrointestinal stromal tumor only had 3 new therapies, but saw the largest change in survival.

“So, there is an efficacy-effectiveness gap,” Sonke said. “These drugs do not all give what we would have expected them to give.”

Additional data through 2021 that he showed, also from the Netherlands Cancer Registry, highlighted the difference in OS of advanced cancer by age. In 2017-2021, the median OS was 6.8 months, up from 5.5 months in 2007-2011. However, that is for all age groups. The group aged 18 to 59 years, which is often the clinical trial population, had an OS of 12.7 months, up from 8.6 months, compared with an OS of 7.6 months for the group aged 60 to 74 years and just 3.9 months for the population 75 years and older, up from 6.2 months and 3.2 months, respectively. The data show that the typical patient who might be seen in the hospital has had very modest improvements during the time.

The decisions on how to design a clinical trial are made to optimize the likelihood it will be successful, and all these factors have led to not just an E-E gap, but a cancer clinical research gap, Sonke said. The clinical development side is drug centered and emphasizes speed and getting the drug to market fast, but the clinical practice side is patient centered and prioritizes completeness over speed.

“And in between there's a gap, and this gap leads to uncertainty,” he said. “And we accept all the uncertainty in favor of the speed. We want the speed, and we accept the uncertainty, but this uncertainty leads to drugs coming to the market not providing the benefits—in all aspects, at least—that we want.”

In looking for research into the E-E gap, Sonke found plenty of work being done to see just how well drugs work in clinical practice, but he thought it was noteworthy that these studies are not appearing in The Lancet or New England Journal of Medicine or Journal of Clinical Oncology.

“Apparently, the impact or the value we put to these types of studies is, at this moment, still limited compared to the randomized trials,” he said.

From Koopman’s perspective, there are 2 options for bridging the E-E gap. Option 1 is RCTs with less stringent criteria and preplanned subgroup analyses. This will mean larger trials and a longer time until results, which ultimately is undesirable. Option 2 is to maintain the stringent criteria in RCTs with registration only for patients who meet it but allow conditionally approved registration for ineligible patients who are followed in a nonrandomized setting. This can be done with real-world data (RWD).

But, in order for RWD to be useful to overcome this E-E gap, the data need to be higher quality than they are now, Koopman said. Data need to be collected and, if not already structured, need to be converted to structured data. The data need to be cleaned and then linked and harmonized, before finally being analyzed to provide answers.

However, before this becomes a reality, there are improvements that need to be made. First, every patient receiving care needs to be asked to consent to their data being used for research purposes. Clinicians also have a responsibility to ask those questions and to enter the data.

A lot of the steps of data entry and collection are still very manual, and there needs to be a way to automate this work. In addition, linking data sources simply isn’t easy.

“I cannot emphasize enough there's a big role for us as clinicians: what's not in cannot be pulled out,” she said. “So, all of you may have a role or an interest in optimizing real-world data in order to answer questions. I invite you to think about where in this process are you and where can you improve the quality of the real-world data.”

References

1. Phillips CM, Parmar A, Guo H, et al. Assessing the efficacy-effectiveness gap for cancer therapies: a comparison of overall survival and toxicity between clinical trial and population-based, real-world data for contemporary parenteral cancer therapeutics. Cancer. 2020;126(8):1717-1726. doi:10.1002/cncr.32697

2. Huang WK, Hsu HC, Chang SH, et al. Real-world effectiveness of adjuvant oxaliplatin chemotherapy in stage III colon cancer: a controlled interrupted time series analysis. Front Pharmacol. 2021:12:693009. doi:10.3389/fphar.2021.693009

3. André T, de Gramont A, Vernerey D, et al. Adjuvant fluorouracil, leucovorin, and oxaliplatin in stage II to III colon cancer: updated 10-year survival and outcomes according to BRAF mutation and mismatch repair status of the MOSAIC study. J Clin Oncol. 2015;33(35):4176-87. doi:10.1200/JCO.2015.63.4238

4. Luyendijk M, Visser O, Blommestein HM, et al. Changes in survival in de novo metastatic cancer in an era of new medicines. J Natl Cancer Inst. 2023;115(6):628-635. doi:10.1093/jnci/djad020