CloudMedx Clinical AI outperforms human doctors on a US medical exam

CloudMedx Inc. announced that the AI developed by the company took a modified version of the United States Medical Licensing Examination (USMLE) Step 1 a few weeks ago and outperformed human doctors in a simulated study. This is a historic milestone that demonstrates that AI can not only understand medical concepts and narrative but also provide insights that may assist physicians. The study also demonstrated that when human intelligence is paired with machine intelligence, the combined augmented approach may generate improved results for tasks that would otherwise be very time-consuming and computationally-difficult for humans.

The simulated exam was very similar to the standard USMLE, an exam that physicians take in order to get licensed in the US. The modified exam had questions like case studies where a scenario of a patient is described, and the exam taker, both humans and the AI, were expected to apply medical knowledge and reasoning to answer multiple choice questions. The exam did not ask simple facts that may be answered through a keyword search, but rather it described complicated scenarios. For an AI to use this kind of data analysis and generate insights is phenomenal.

Read More: Interview with Nathaniel Gates, CEO & Co-Founder at Alegion

It should also be noted that the aim of CloudMedx and this study is not to provide any diagnosis, rather provide tools to read through large quantities of structured and unstructured data from health system records to generate predictive analytics that may be useful for clinicians and their care teams in their delivery of care.

CloudMedx AI takes the Medical Exam

CloudMedx has built a Clinical AI assistant that augments clinical operations and revenue cycle management for health systems. The system uses natural language understanding (NLU) and deep learning and can be integrated into Electronic Health Records to serve clinical insights within workflows to augment hospital staff to improve operations and documentation – both at population level and per patient level. The company is currently working with some of the top hospitals in the country.

Read More: Jumpstart 2019: Interview with Anand Janefalkar, Founder & CEO at UJET

To benchmark its AI, CloudMedx conducted a small-scale study where it invited a group of doctors to take a modified version of a US medical exam against their medical resident AI. The group included doctors from one of the top medical universities in the United States.  The exam had 100 questions, each presented as a case study with multiple choice answers. For example, a typical question may have a scenario where a patient describes his symptoms along with prior medical history, medications, and lab results. The physicians and the AI were required to use their medical knowledge, domain expertise, and experience to get to the right answers.

Read More: 2018 Was a Year of ‘Disruptive HR Transitions’: Tjinsite’s Yearend Conclusion

The study divided doctors into 3 groups. The task was to take the exam and see which group does better in this study. The preliminary results were fascinating. Their scores are below:

• Group 1. Human doctors: score 68 to 81 (average was 75)
• Group 2. CloudMedx AI: score 85
• Group 3. Human doctors along with CloudMedx AI: 91

CloudMedx AI outperformed all the doctors by achieving a score of 85% correct answers. The high human score was 81% and low score was 68% (with an average score of 75%). But the Human+AI group out-performed the other two groups by achieving the highest score of 91. In the last group, the AI made a recommendation for the best answer choice for each question and the physician had the option to either accept that recommendation or reject it. This group used a combination of their memory and expertise along with the recommendations from the AI to achieve an overall higher score compared to the other two groups.

Read More: Interview with Megan Kvamme, CEO at FactGem

The average time it took for the human doctors to take the exam was 73 minutes, whereas the AI completed it in under 5 minutes. This preliminary study shows that AI may augment human doctors so that their combined medical intelligence brings efficiencies and improves the overall process.

In talking to the last group (Human+AI), one thing that became evident was that for questions where doctors were on the fence, the AI was able to assist them in a data driven manner.

What’s next – Garry Kasparov Human and Machine, not Human vs Machine

CloudMedx believes this to be the future of healthcare – a machine augmented medicine that is not disruptive but assistive in nature. The significance is that Human and Machine together can achieve higher efficiency and care delivery.

According to Dr William Morris, Associate Chief Information Officer at the Cleveland Clinic Foundation, a partner CloudMedx is currently working with in the area of improving clinical workflows and documentation, 
“This is a very exciting technology that supports the vision of how Augmented Intelligence plays a pivotal role in the care of patients. While AI can master vast clinical knowledge, it is best when playing supportive role to the clinician in their workflows and daily lives.”

This clearly shows that human and machine augmented intelligence can provide a lot of reprieve from the explosion of data and assist with clinical workflows, operational efficiencies and clinical documentation that take a lot of care teams’ times and efforts.

Read More: WORBLI Calls 2018 the Year of Substantive Growth in Blockchain and Predicts Global Expansion in 2019

For the past few years, there is a lot of fear amongst people that AI will take over their jobs. While this fear is reinforced by movies and science fiction, the reality is that AI should be viewed as an augmentation tool that has the ability to analyze large amounts of data and make sense of it while letting humans focus on empathy, creativity, and important decision making. This combined approach of Human+Machine augmentation can generate amazing outcomes for the healthcare industry.

According to Alan Pitt, MD at the Barrow Neurological Institute, a premier hospital that is part of Dignity Health, 
“AI should be viewed not as a threat by physicians, but as a supporting tool that helps them build efficiency in their workflows. AI is often portrayed as technology marginalizing people. I see it differently. This is the first step showing another future for healthcare. It’s not man vs machine but man and machine improving care. The best performer was a doctor with AI. Moving forward providers will have a new tool to bring information to the bedside, I have hope AI will free providers to spend more times with patients by freeing them from their current role as scribes for the EMR.”

CloudMedx is currently working with The Barrow as part of a joint effort to identify early signals of ALS, or amyotrophic lateral sclerosis, a progressive neurodegenerative disease that affects nerve cells in the brain and the spinal cord, in order to create treatment protocols for the management of this disease using AI. This also shows how physicians and AI can work together to look at deep data to build predictive algorithms to treat rare diseases that are so hard to detect, treat, and manage.

Read More: OneLogin Appoints Courtney Harrison as Chief Human Resources Officer

Given the right focus and technology with respect to patient privacy, security, and correct data sharing, healthcare AI can be a good tool to have both for physicians and patients. Consumers are already asking to get access to their own data and be champions of their own health. At the same time, physicians are demanding better tools in the workplace so that they are not overburdened but optimize on care delivery.

Stefano Bini, MD and Professor at the University of California, San Francisco stated that 
“The premise that technology can be used to make humans smarter is not new. However, the fact that artificial intelligence can help doctors make better informed decisions based on data than the smartest amongst them can make alone is pretty astounding. This technology can make any quality disparity we see across geographic areas in our country essentially disappear. The cost savings alone would easily pay for the widespread deployment of the technology.”

Read More: Interview with Jayant Lakshmikanthan, CEO and Founder at CLARA analytics

CloudMedx is also working with different departments at the University of California, San Francisco, to automate workflows and improve clinical documentation in areas of liver cancer, orthopedic surgery, and others.