Dandelion Health Launches Public Service Program to Evaluate Performance and Bias of Healthcare Algorithms

— Pilot program confidentially validates electrocardiogram algorithms used in healthcare settings —

Dandelion Health, a startup offering a healthcare data platform that specializes in artificial intelligence and precision analytics, announced the launch of a new public service to rigorously evaluate the performance, fairness and equity of health algorithms.

AiThority Interview Insights: How to Get Started with Prompt Engineering in Generative AI Projects

“There is intense interest in algorithms that input electrocardiogram waveforms — which test the function of the heart by measuring its electrical activity — and output predictions about arrhythmias, lab values or the results of other, more expensive tests like echocardiograms”

The pilot program will focus on algorithms that make predictions for cardiology, using electrocardiogram waveforms, and measure their performance and bias across key racial, ethnic and geographic subgroups. Dandelion has developed this program under a patient privacy-first, open access model, to make it accessible to as many algorithm developers as possible at no cost.

This effort is generously funded by the Gordon and Betty Moore Foundation, which supports path-breaking scientific discoveries to create positive outcomes for future generations. Using Dandelion’s comprehensive set of de-identified clinical data available from 2016 to the present, the pilot program aims to catalyze the development of responsible AI to significantly improve health outcomes while maintaining patient privacy.

“When an algorithm is trained on a dataset from Cambridge or Palo Alto, and then applied on other populations — a different geography, a different care setting, or a different racial and ethnic mix of patients — the performance of the algorithm can degrade dramatically,” said Elliott Green, CEO and Co-Founder of Dandelion Health. “It’s critical to know how algorithms do, both overall and for the most vulnerable patient populations, who are not typically represented in algorithm training datasets. Our pilot program answers the question, does your algorithm do what it’s supposed to do? And does it do it fairly, for everyone?”

Securing high-quality clinical datasets that encompass diverse populations and put patient privacy first is extremely expensive and difficult, which has been a fundamental challenge to validating the performance of algorithms against intended populations.

Read More about AiThority Interview: AiThority Interview with Raj Suri, Founder at Presto Automation

Dandelion Health’s pilot program is the first to eliminate that barrier. Through its consortium of three major healthcare systems — Sharp HealthCare (San Diego, California), Sanford Health (Sioux Falls, South Dakota) and Texas Health Resources (Arlington, Texas) — Dandelion is aggregating and de-identifying clinical data for roughly 10 million patients throughout the United States across a range of geographies, ethnic and racial groups, care settings and more. The breadth and depth of the resulting raw de-identified clinical data and Dandelion’s deep expertise in creating high-quality clinical datasets uniquely positions the company to validate algorithms using data that has not previously been de-identified or available at this scale.

“There is intense interest in algorithms that input electrocardiogram waveforms — which test the function of the heart by measuring its electrical activity — and output predictions about arrhythmias, lab values or the results of other, more expensive tests like echocardiograms,” said Ziad Obermeyer, M.D., Chief Scientific Officer and Co-Founder for Dandelion Health. “Questions about data quality and patient privacy have become a global conversation among scientific and healthcare communities, governments and the public alike. The goal of our pilot program is to make it easy for developers to gain insight into how well their algorithms perform overall — and for specific groups of interest — using a scientifically rigorous validation process that protects patient privacy and is aligned with the public good.”

With the rapid growth, development and integration of AI tools in healthcare, leading industry experts such as the U.S. Food and Drug Administration and the Coalition for Health AI have called for methods to rigorously measure and validate the safety and effectiveness of AI algorithms. Dandelion’s pilot program offers the first broadly accessible solution.

The names of participating researchers and companies will never be made public. Results will only be provided to the algorithm developer unless they grant permission otherwise.

Dandelion will begin accepting electrocardiogram waveform algorithms designed to be representative of the U.S. population for validation from developers worldwide beginning July 15, 2023 for an initial period of three months. Following the pilot program’s initial launch, Dandelion may expand the validation program to include additional clinical data modalities such as clinical notes and radiology imaging.

 Latest AiThority Interview Insights : AiThority Interview with Manuvir Das, VP, Enterprise Computing at NVIDIA

 [To share your insights with us, please write to sghosh@martechseries.com]