Will Shapiro Discusses Ethical Challenges Across Various Artificial Intelligence Models in Oncology

Ethical challenges in artificial intelligence (AI) models include regulatory oversight, authority, and bias, according to Will Shapiro, vice president of data science, Flatiron Health.

Transcript

What are some challenges around the proper use and ethics of AI in oncology?

I mean, there's a ton of challenges around using generative AI right now in practice. And absolutely, ethics and safety are hugely important considerations. I think it's an interesting time right now. In contrast to something like workflow automation or business intelligence—these are areas that have been around for a long time and that people kind of understand how to evaluate the quality and how to think about the outputs.

Generative AI is very early. When you see companies like OpenAI and Sam Altman [CEO of OpenAI] in particular, demanding and asking for regulators to come up with frameworks to think about the appropriate use of generative AI and guardrails, there's a lot to pay attention to.

Hallucinations are very real, and generative AI algorithms come from a place of technological authority. So, even if what they're saying is complete nonsense, it still has the sheen of authority. I think it's very pernicious.

And then, of course, with any machine learning or AI algorithm, bias is incredibly important to think about. GPT [generative pretrained transformer architecture] has already been demonstrated to be biased in several different ways. If we’re using these tools, we want to make sure that they make things better for everyone, not just for the select few, and don't perpetuate inequalities across health care.

For me, a lot of that comes down to having high-quality ground truth data that you trust and that you can use to validate the output of models against.

At Flatiron, we've been abstracting data from medical charts for a decade and are very proud of the labels that we have. What that gives us is an amazing way to validate the quality of ML [machine learning] models. So that's something I think a lot about and I’m very lucky to be able to work with that kind of label data.