Fine-tuning AI Tools to Make Meaningful Predictions in Cancer

It’s hard to find a cancer conference—or any health care meeting—that doesn’t offer something on the wonders of artificial intelligence (AI), and the International Myeloma Society 21st Annual Meeting & Exposition, taking place in Rio de Janiero, Brazil September 25-28, 2024, was no exception.

Anant Madabhushi, PhD | Image credit: Emory University

But the speaker, Anant Madabhushi, PhD, is careful with his optimism. He has many roles at Emory University: he’s executive director for the Empathetic AI for Health Institute, while also holding primary appointments in the Department of Biomedical Engineering and secondary appointments in Radiology and Imaging Sciences. Perhaps more critically, he’s clear-eyed about both the possibilities of AI and how far the technology has to go, which he discussed in his keynote address, “AI in Multiple Myeloma: Recent Findings and Opportunities.”

Right up front, Madabhushi shared that most of his talk would not be about multiple myeloma. His examples were largely from breast and lung cancer, both solid tumors. But the concepts he presented—from overcoming the “black box” challenges of AI to addressing the “reproducibility crisis”—are universal no matter what disease is involved. He also had exciting examples of how AI is already being used to predict which patients with lung cancer will respond to expensive therapies—something of huge interest to health systems, and, of course, to patients themselves.

Madabhushi offered prostate cancer as an example of a disease that may be overtreated; patients may suffer harm if they receive unnecessary radiation when their mortality risk is quite low. “Beyond that, certainly in the United States, we need to acknowledge that overtreatment and overdiagnosis, also beyond the patient-centric toxicity also causes financial toxicity. A lot of us are familiar with statistics where something like 42% of newly diagnosed cancer patients will lose their life savings, many within 1 to 2 years of that initial cancer diagnosis.”¹

What researchers are starting to appreciate, Madabhushi said, is how much data is collected along the way: there’s the MRI, now done in the clinic; there are other tests that feed into the electronic medical record. If surgical resection occurs, there’s the pathology specimen, which can be digitized “with computer vision AI and machine learning tools, to start to interrogate the appearance of individual cells,” Madabhushi said.

Then there’s the tumor itself, with proteomic and genomic features. All this information is being gathered from multiple places, and Madabhushi said the question becomes, “How do we align the information from radiology and pathology down to genomics?”

And once this information is gathered for an individual patient, “How do we extract features and information to better quantitatively characterize the disease, and then ultimately, how do we fuse that information so that we can create better predictors for improved diagnosis, prognosis, and predicting therapeutic response?”

All this data collection has gone on for years. What’s changed recently, he said, is that about 10 years ago, neural network algorithms became much more sophisticated, as the “computational horsepower” took a leap forward.

“Our group published one of the first papers using deep learning;² again, a particular flavor of AI for identifying cancer cells on breast pathology images. Initially, when we did this work, we were really taken aback by how powerful these algorithms could be.” It seemed that a pathologist only had to annotate a few cancer cells on breast images, “and we were able to stand up an AI algorithm to go find cancer cells on new, unseen images.”

However, the team was cautious about what it had found. There was awareness they didn’t truly understand what the AI was learning. And they’ve spent time trying to figure things out, trying to grasp just what the AI is learning from the data. Since then, an ongoing “reproducibility crisis,” of institutions not being able to regenerate results obtained with AI—typically the gold standard in science—is gaining attention in the literature, Madabhushi said.

He used an example that anyone who’s worked with ChatGPT could understand.

“There’s this phenomenon where you provide a question to ChatGPT, and it just basically makes up an answer. And this is a challenge, because the answers sound very compelling. Unfortunately, they're completely erroneous.”

If the person working with black box algorithms doesn’t understand how the AI has been trained, “You can get these very convincing responses that sound like they're correct, but they are completely erroneous.”

The worse outcome from an error-riddled term paper might be a failing grade. In oncology and hematology, the stakes are far greater. That’s why Madabhushi’s team is putting so much effort into the process of AI validations.

“We have to think about the underlying basis of the patterns and the features that we're pulling out. And when we start to build more intentional, more explainable AI algorithms, we have the opportunity to start to look at the use of these algorithms in the context of not just disease diagnosis, but really thinking about prognosis, thinking about prediction of treatment response, and treatment benefit. And it's with these more explainable AI algorithms that we can start to develop more prognostic and predictive tools that can be used for tailoring therapy for a given patient, based off their specific risk profile,” he said.

This approach will require working with multiple institutions and across many sites, and this validation process will also address the reproducibility crisis, Madabhushi said. Once the challenge of validating tools with completed trials occurs, there will be the quest to use AI in prospective trials.

“Does that mean that we throw out the black box algorithms? Not quite,” he said.

He envisions using “black box” AI in new ways, for things like identifying individual cells or tissue types, and understanding how cells work together in “the spatial geography where these cells reside, whether in the in the tumor stroma or in the epithelium, and by looking at explainable features relating to the arrangement in the architecture of these different cell types within different tissue compartments.”

This work can lead to development of new digital biomarkers to better predict treatment benefits; he showed multiple 3D images of how his team has “mapped” not just the tumor but also the associated vasculature to predict outcomes based on the “twistedness” of the blood vessels in the tumor.

As promised, there was one project involving multiple myeloma. The work with blood vessel mapping is being applied to samples from the UK BioBank. He shared fundus images, ocular photos normally used to monitor conditions such as retinopathy or glaucoma. Members of Madabhushi’s team are using AI tools to examine images of patients’ eyes over time; they are evaluating patterns of the twistedness of the vasculature in the eye vessels.

Only a few hundred patients have been studied, Madabhushi warned, but patterns are emerging. “What we're finding is that the features relating to the tortuosity of the vessels appear to predict the onset of multiple myeloma within a 10-year time frame.

“Again, we're just getting started in the multiple myeloma space,” he said. “So, early days yet.”

“Hopefully what I've conveyed is that computational analytics with routine imaging can help address questions in precision medicine,” Madabhushi said. “I want to emphasize the importance of clinical trials—completed clinical trials—where we can go back with the AI and analyze data from these completed clinical trials, and as I showed, in the context of other malignancies, there's a huge opportunity to validate these tools and demonstrate their prognostic and predictive benefit.”

References

1. Gillespie AM, Alberts DS, Roe DJ, Skrepnek GH. Death or debt? National estimates of financial toxicity in persons with newly diagnosed cancer. Am J Med. 2018;131(10):1187-1199. doi: 10.1016/j.amjmed.2018.05.020
2. Cruz-Roa A, Gilmore H, Basavanhally A, et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A deep learning approach for quantifying tumor extent. Sci Rep. 2017;18:7:46450. doi: 10.1038/srep46450.DOI: 10.1038/srep46450