COTA Collaboration: Helping FDA Figure Out What's Possible and What's Not in Embrace of Real-World Evidence

When it comes to generating evidence that leads to a drug approval, the randomized controlled trial (RCT) is the gold standard. The first published trial appeared in the literature more than 70 years ago,¹ and over the past 30 years, the scientific community has developed the principles of evidence-based medicine,² in which RCT results inform clinical practice guidelines.

But the problem with the gold standard, as Andrew Norden, MD, MPH, MBA, sees it, is that too many people get left out. Norden, a neurologist and neuro-oncologist who is the chief medical officer at COTA Healthcare, described a well-documented problem with current RCTs: The typical participant in a clinical trial for a cancer drug “is more often white, more often male, more often wealthy, and certainly more often healthy” than the average person with cancer.³ That means physicians need some other way to gauge how new drugs might work on the other people who come to their clinics.

Enter real-world evidence (RWE), which is the domain of COTA, a company founded 8 years ago by cancer doctors and data scientists with the idea of harnessing the vast amounts of electronic health records (EHRs) that were accumulating, albeit in a disorganized way. COTA’s mission is to make sense of the noise so that cancer doctors can use what the data tell them about other cancer patients just like the one in front of them. The company is known for the development of the COTA Nodal Address (CNA), which condenses patient attributes into a digital code that allows providers or payers to evaluate cancer patients with similar characteristics in patient groupings.³

In an interview with Evidence-Based Oncology™ (EBO), Norden emphasized that COTA is not looking to eliminate RCTs. But by unlocking the secrets of the data sets in EHRs, real-world data can “extend” the findings of the RCT, as Norden puts it, and offer researchers, clinicians, and the FDA insight into how drugs work in populations that don’t find their way into clinical trials.

This could mean insight about patients who are poor, are racial minorities, or have chronic conditions. “As a physician, I strongly advocate for the inclusion of patients from all those groups,” Norden said. “I think it’s a real disservice that they tend not to be [included]. Thankfully, a lot of folks are working on that problem—I want them to continue that work. But in the meantime, there are lessons we can learn about the applicability of clinical trial findings to patients who may fall into underrepresented groups in the clinical trial world.”

Congress recognized this problem in 2016 when it passed the 21st Century Cures Act, which directed the FDA to develop a framework for using RWE in the course of drug regulation.⁴ The agency has responded: It published a framework in December 2018⁵ and in May 2019 published guidance documents on EHR data in clinical investigations and use of RWE in decision making for medical devices, as well as a draft document on submitting RWE for drugs and biologics.⁶

At that time, the FDA also announced it would join with COTA in a 2-year research and collaboration agreement: Starting with breast cancer, the federal regulator and the healthcare data and analytics company would create a study protocol to guide approaches to handling treatment variation within subpopulations and how RWE might be used to guide regulators.⁷

COTA comes to the partnership with several collaborations in hand. It has a pilot project with New Jersey’s largest insurer, Horizon Blue Cross Blue Shield,⁸ and joined several other technology and data sources in a project with Friends of Cancer Research.⁹ In July 2018, the Friends project published a white paper demonstrating several approaches to evaluating real-world end points using a scenario that evaluated patients with advanced non—small cell lung cancer treated with immune checkpoint inhibitors. The paper explored several end points—real-world progression-free survival, real-world time to progression, time to next treatment, time to treatment discontinuation, and overall survival (OS).

In the interview, Norden said that the project’s participants took different approaches, but the good news for RWE is: “We actually came to the same answers.”

According to the white paper, “The pilot project demonstrated that several extractable end points from EHR and claims data correlate with OS. Further validation is required to determine whether these end points are reliable surrogates for OS outside of a traditional clinical trial and whether they can support regulatory and payer decision making.”⁹

What follows is EBO’s discussion with Norden on the advancement of the CNA, plans for the FDA collaboration, and the potential for RWE (edited for clarity).

EBO: How has the CNA used data from EHRs to advance patient care to date, and how have clinicians responded to CNA?

Norden: The COTA Nodal Address is a unique way of grouping patients; in fact, we believe there isn’t another cohorting mechanism that takes into account cancer-specific information the way the CNA does. We think about it like a turbocharged ICD-10 [International Classification of Diseases, 10thRevision] code. It’s based on today’s conception of precision medicine, which says that you need to know a fair amount of clinical detail about any individual cancer patient to know what the right treatment is, estimate that patient’s prognosis, and predict cost of care. The CNA brings together all the attributes that influence outcomes, treatment decisions, and costs into a single digital code.

Ultimately, when the CNAs are assigned and you have 2 patients—perhaps in different geographies or with different physicians—if they have the [same] CNA, there’s no clinically proven reason those patients should be treated differently. You can use the CNA to identify unwarranted variation in treatment decisions or in costs. The information you glean from doing this analysis lets you develop an improvement plan around those things—it lets you target unwarranted variation based on specific, clinically defined cohorts that physicians understand.

EBO: How large is COTA’s network currently, and how does COTA intend to expand its network between now and 2020?

Norden: The COTA network is growing quickly. Patients represented are primarily from the East Coast, with a particular concentration from the Mid-Atlantic [region] and Florida. We are looking to expand on to the West Coast in 2019. One of the valuable aspects of COTA’s data set is that we have significant representation of patients who are treated in academic centers and community centers. We think that makes our data set more representative of the population at large. We know that in the United States, a lot of patients—maybe 80% or more—are treated in community centers. We think it’s really important that a real-world data set represent the patients regardless of the site of service. Often, we find interesting trends that relate to the way patients are cared for in one setting or the other.

EBO: Can you describe the broad outlines of COTA’s collaboration with FDA and how it fits into the agency’s commitment to incorporate RWE into decision making?

Norden: The FDA has clearly signaled in the last year or more that [the agency is] interested in the potential to use real-world data—and the evidence generated from that data—for regulatory decision making. The way I view this personally is that we have an enormous amount of information being entered into electronic medical records [EMRs].

Now, it’s a fact that EMRs were not designed from the get-go to efficiently enable analysis of data across a population of patients. That’s true, but we’re developing techniques—we at COTA and others in industry and academia—that allow one to efficiently extract and analyze it. In my view, it would be foolish not to take advantage of that data when we want to understand how cancer care is delivered in the United States. The FDA seems to be very much on board with that line of thinking.

That’s not to say that the FDA or COTA is advocating for the end of clinical trials.… That is not at all the viewpoint that COTA espouses. In fact, we think that real-world evidence, derived from the EMR and other sources, is a great complement to clinical trial data.

There’s no argument about the reality that clinical trial patients are highly selected, they are wealthier, they tend to be healthier, and they tend to be more often white and more often male in many cases than in the community of cancer patients at large. So, the value of real-world evidence is that you can look to extend the findings of clinical trials to a more broadly representative patient group. It’s also the case that there may be a small set of scenarios where real-world evidence could take the place of a clinical trial. I don’t think that’s true broadly, but I think there are some scenarios where it’s true. It may be true in rare diseases in which a clinical trial is unlikely to happen anyway because there just aren’t enough patients. It may be true in a setting where multiple drugs have already been approved and there’s no partner interested in funding a head-to-head comparison study. It may be true in a scenario where a drug has entered usage and has a very large effect size, and we can see clearly in real-world data that it’s superior to an existing agent.

There are these scenarios, but for the most part, the value in my mind of real-world evidence is that it can extend the findings of clinical trials; [it can] help us confirm that what we learned in a clinical trial is, in fact, true in patients who are somewhat different from the population represented in the clinical trial.

I think that’s what the FDA is looking to do in the collaboration with us. The truth is that today, the methods and the capacity of real-world evidence to answer important questions are somewhat immature.

We haven’t proved this yet. So, the FDA, like all of us in this industry, is looking to learn how this might work, and COTA is an important part of driving that forward. [The FDA is] looking to learn what kinds of elements our data can capture, how we capture those data elements, how confident we can be that they are accurate, and then look to see what’s possible—what clinical trial findings can we extend, in what scenarios might we be able to replace clinical trials, because I do think there are selected circumstances where that’s possible—and then let the science take us where it will.

This is a scientific relationship between COTA and the FDA. It’s not a commercial relationship—we’re not looking to approve a new drug. We’re looking to show what is possible and what is not possible. It’s in the interest of all the healthcare stakeholders to do this work, because there’s great potential for real-world data to accelerate the findings of clinical trials and to get drugs to market faster when there are strong data in favor of their use.

EBO: What specific milestones and deliverables should we look for as the collaboration proceeds?

Norden: At the moment, COTA and the FDA are busy talking about precisely what our work together will entail. We have agreed to start with breast cancer, because it’s a common disease with substantial areas of controversy and some areas of important unmet need—and a lot of active drugs

in development. But we have not yet settled on the specifics of what we will provide. think what you can expect is that as this work ramps up, we will submit abstracts to meetings and present posters and oral presentations. We anticipate presenting manuscripts so that the scientific community at large can learn about our work. The specific scientific deliverables and timelines are still under discussion.

EBO: During this year’s annual meeting of the American Society of Clinical Oncology, there was a presentation on the closure of the treatment gap in cancer care that was attributed to improved access under the Affordable Care Act.¹⁰ What kind of treatment improvements do you foresee by closing the information gaps for groups that are not represented in clinical trials?

Norden: I do think the work that COTA and others involved in aggregating and curating real-world data and generating evidence from that data can be valuable from the standpoint of reducing disparities in cancer care. We know that for a long time, clinical trial populations have been dominated by patients who tend to be more often white, more often male, more often wealthy, and certainly more often healthy than the average patient with cancer.

And yet, a lot of information about patients who are perhaps more representative of the population at large is captured in EMRs today. So, in COTA’s view, one of the key advantages to using real-world data is that we learn [when we] extend findings from clinical trials [how they] actually apply to patients who may come from different socioeconomic backgrounds, who may come from different racial or ethnic groups, and who certainly may be less healthy than other patients.

As a physician, I strongly advocate for the inclusion of patients from all those groups in clinical trials. I think it’s a real disservice that they tend not to be included, and a lot of folks, thankfully, are working on that problem—I want them to continue that work. But in the meantime, there are lessons we can learn about the applicability of clinical trial findings to patients who may fall into underrepresented groups in the clinical trial world.

EBO: Do you have an idea yet how new end points or regulatory benchmarks may be incorporated as RWE becomes part of the decision process?

Norden: There’s a lot of discussion about this issue.… What should the end points in a real-world evidence-based analysis be? The truth is, the field is new enough that I don’t think there’s a definitive answer to this question. COTA and a number of other entities that work on real-world data and evidence generation from that data have been engaged collaboratively in a project put together by Friends of Cancer Research.

We published a white paper online last year⁹ in which we begin to address this question: Which end points that are derived from real-world data are predictive of end points that are derived from clinical trials? I think that are used for end point determination are not well standardized, and there is some controversy about the extent to which these end points are predictive of overall survival or other clinical trial-based end points.

One thing that was heartening to see with the Friends of Cancer Research, is that 6 or 7 partners generated real-world evidence end points in different ways, but we actually came to the same answers. And we found there was a very high correlation between progression-free survival assessed using real-world data and overall survival, which is, of course, a critically important end point in cancer research. So, there’s encouraging preliminary evidence that suggests that real world—based end points like progression-free survival, time to treatment progression, time to treatment discontinuation… are correlated with clinical trials–based end points and with overall survival.

But I think we need more time to become confident about the situations where real-world end points may serve as adequate surrogates and where we must rely on clinical trials to get bias-free results.

EBO: What are some potential pitfalls or risks for health systems as they embark on the use of real-world evidence?

Norden: I do think there are some risks for potential users. To give you a sense of what some of those are, I think one of the best described is the risk of bias when interpreting real world data.

Let’s say we want to do a study based on real-world data, where we’re comparing patients who have been treated with drug A or drug B. The problem is that the doctors who prescribed drug A versus drug B probably have reasons for making those choices. And those reasons, critics fear, may not be well captured in the data sets we are using to assess these questions. That’s a real issue and one that must be addressed.

I think one of the potential ways to address it—and the approach COTA takes—is to generate a very clinically granular data set. That means we extract

from the EMRs all the key prognostic variables that we know might drive differential outcomes and account for those in our analyses, using propensity scores or other statistical methods.

Now, the naysayer might argue that there are probably unmeasured variables that we can’t extract from EMR data. I think that’s true. No one should deny that reality—that is a limitation of real-world data. And that’s why we have to be careful about how we apply this. The effect size we look for in

real-world data maybe needs to be bigger than the effect size that would be persuasive if it was drawn from a randomized clinical trial.

The other issue one has to watch out for is that the quality of the data varies by provider and by source. Some sources are simply not well designed to answer certain questions. EMR data provides a certain level of clinical granularity, but on the other hand, if the patient leaves the provider on the EMR where you’re tracking, then you may lose important pieces of information. That can be addressed with claims data, but claims data lack clinical granularity and only talk to you about healthcare transactions.

The advice I would give a potential user is to be really cautious and judicious. Whoever is producing the data [must do it] in an auditable, reproducible, high-quality way, where you have a trail of data from start to finish. [Make sure] you’re not asking the data to answer questions that can’t be satisfactorily addressed. We don’t need to start by trying to replace clinical trials. We can start by saying, “How do learnings from real-world, evidence-based sources extend our understanding of clinical trials?’” References

1. Streptomycin treatment of pulmonary tuberculosis. Br Med J. 1948;2(4582):769-782.

2. Strauss SE, Sackett DL. Using research findings in clinical practice. BMJ. 1998;317(7154):339-342. doi: 10.1136/bmj.317.7154.339.

3. Chen MS Jr, Lara PN, Dang JHT, Paterniti DA, Kelly K. Twenty years post-NIH Revitalization Act: enhancing minority participation in cancer clinical trials: (EMPaCT): laying the groundwork for improving minority trial accrual. Cancer. 2014;120(suppl 7):1091-1096. doi: 10.1002/cncr.28575.

4. Zegarelli BM. 21st Century Cures Act requires FDA to expand the role of real world evidence. Mintz website. mintz.com/insights-center/viewpoints/2146/2016-12-21st-century-cures-act-requires-fda-expand-rolereal-world. Published December 19, 2016. Accessed July 24, 2019.

5. Framework for FDA’s real-world evidence program. www.fda.gov/media/120060/download. FDA website. Published December 2018. Accessed July 24, 2019.

6. Real-world evidence. FDA website. www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence. Updated May 9, 2019. Accessed July 24, 2019.

7. COTA Healthcare website. cotahealthcare.com. Accessed July 24, 2019.

8. Caffrey M. Moving toward the “tipping point” for transformation in cancer care. Am J Manag Care. 2019;25(2):SP45.

9. Friends of Cancer Research. Establishing a framework to evaluate real-world end points. focr.org/publications/establishing-framework-evaluate-real-world-endpoints. Published July 10, 2018. Accessed July 20, 2019.

10. Adamson BJS, Cohen AB, Estevez M, et al. Affordable Care Act (ACA) Medicaid expansion impact on racial disparities in time to cancer treatment. J Clin Oncol. 2019;37(suppl; abstr LBA1). abstracts.asco.org/239/

AbstView_239_257961.html.

Download PDF: COTA Collaboration: Helping FDA Figure Out What's Possible and What's Not in Embrace of Real-World Evidence