Meeting Summary for HBHI Workgroup on AI and Healthcare

 

The Hopkins Business of Health Initiative (HBHI) Workgroup on AI and Healthcare held a seminar featuring Dr. Mihaela van der Schaar, John Humphrey Plummer Professor of Machine Learning, Artificial Intelligence, and Medicine at the University of Cambridge, and Dr. David Newman-Toker, Director of the Armstrong Institute Center for Diagnostic Excellence at Johns Hopkins University. The session explored the role of artificial intelligence in clinical predictive modeling, AI adoption challenges, and the integration of large language models (LLMs) into healthcare research and decision-making.

Drs. Tinglong Dai and Risa Wolf, co-chairs of the HBHI Workgroup, welcomed participants and introduced Dr. van der Schaar as one of the world’s foremost experts in AI-powered clinical decision-making. They emphasized the growing role of AI in healthcare and the necessity of developing AI tools that clinicians can trust and use effectively.

Dr. Mihaela van der Schaar’s Talk: The CliMB Platform and AI as a Co-Pilot

Dr. van der Schaar introduced CliMB, an AI-powered co-pilot designed to help clinician-scientists and epidemiologists develop predictive models using natural language interactions, eliminating the need for programming expertise. The goal of CliMB is to provide clinicians with an interactive, intuitive AI partner that assists in complex medical decision-making rather than attempting to replace human expertise.

She used an analogy to illustrate this vision: "I would not like to board a plane without a pilot, and I would not like to board a plane without the equipment that this pilot has at their disposal." In the same way, AI should function as an assistive tool for clinicians, providing them with real-time insights, decision support, and analytical capabilities while ensuring that human expertise remains the ultimate authority.

While AI adoption in medicine has accelerated, most machine learning models remain opaque, difficult to use, and require extensive coding knowledge. Moreover, clinicians and data scientists often operate in separate silos, slowing down the development of clinically relevant AI applications.

CliMB addresses these issues by allowing clinicians to converse with AI using natural language, instead of writing code. It automates key aspects of AI modeling, including data ingestion, preprocessing, model selection, performance evaluation, and explainability.

Unlike traditional AutoML tools, which often operate as black boxes, CliMB incorporates structured reasoning, enabling users to iteratively refine their models and understand the decision-making process.

She shared findings from a blinded study involving 45 clinicians, in which over 80% of participants preferred CliMB over traditional AI tools due to its clarity, ease of use, and reliability.

Benchmark comparisons further showed that CliMB outperformed GPT-4-based predictive modeling systems, particularly in planning, error prevention, and execution.

Challenges in AI Adoption and User-Centered Design

Following the presentation, Dr. Newman-Toker reflected on why AI adoption in healthcare remains difficult despite its potential. He highlighted three primary barriers:

  • Clinician trust and usability issues. Many AI systems are designed for technical users, making them impractical for frontline clinicians who lack programming expertise. AI models that are not transparent or adaptable create skepticism and resistance among healthcare professionals.
  • Workflow integration. AI solutions must fit into clinical workflows seamlessly; otherwise, they risk being underutilized or ignored.
  • Data quality and availability. AI-driven healthcare is often limited by incomplete, biased, or unstructured data, making reliable decision-making difficult.

He emphasized that in medicine, the challenge is rarely model sophistication—it’s the data. AI tools must not only generate predictions but also help identify missing data that would improve clinical decision-making.

Dr. van der Schaar agreed, noting that AI should be designed to recognize and address data gaps rather than merely working with what happens to be available.

Q&A: Key Questions from Attendees

Clinical Decision-Making, Diagnostic Risk, and Handoff Challenges

Dr. Stuart Ray, an infectious disease and internal medicine specialist at Johns Hopkins, raised a question about how AI models account for the risk of missing critical diagnoses. Drawing from his clinical experience, he noted that in clinical practice, physicians often prioritize diagnoses based on risk severity, meaning that a potentially fatal condition (e.g., meningitis) takes precedence over a less urgent diagnosis. He questioned whether current AI-driven diagnostic models consider the asymmetry in decision-making, where missing a life-threatening diagnosis is far more consequential than issuing a false alarm.

Dr. van der Schaar acknowledged that most existing machine learning models rely on symmetric loss functions, treating false positives and false negatives as equally important. However, in reality, the cost of a missed diagnosis can vary dramatically depending on the condition. She highlighted ongoing research efforts to integrate asymmetric cost functions into AI models, ensuring that decision-making systems properly weigh the risks associated with different types of errors.

Another challenge raised by Dr. Ray was the problem of handoffs between clinicians. In hospital settings, patient care is frequently transferred between different providers, creating opportunities for information loss and miscommunication. He questioned whether AI could help preserve diagnostic continuity, ensuring that critical insights are retained and properly communicated across different shifts and medical teams.

Dr. van der Schaar suggested that trajectory modeling and dynamic patient records could help mitigate handoff challenges. By developing AI-driven disease trajectory models, clinicians could track patient progression over time, reducing the risk of important diagnoses being overlooked. She also proposed that AI systems could be integrated into electronic health records (EHRs) to provide real-time alerts when a change in patient care teams occurs, ensuring that important diagnostic pathways are not lost during transitions. 

AI Model Development: Workflow and Benchmarking Concerns

Dr. Rajanikanth Vadigepalli, a systems engineer specializing in heart failure, liver failure, and organ regeneration, raised two critical concerns:

  • Workflow structuring in AI model development. He asked whether the CliMB platform provides a structured workflow to help users formulate the right questions and select appropriate machine learning models. Even with a no-code interface, he argued, machine learning expertise remains necessary to ensure models are developed correctly.
  • Benchmarking AI models across domains. He suggested that a structured reasoning system could allow clinicians to compare and contrast AI models, identifying weaknesses and ranking models for reliability.

Dr. van der Schaar confirmed that CliMB offers both guided pathways and full customization options, allowing users to either follow recommended best practices or take control of individual modeling decisions. She acknowledged that many clinical researchers are unfamiliar with optimal AI workflows, which is why integrating structured guidance within the platform is a priority.

Large Language Models and Nutrition Data

Dr. Vijaya Parameswaran, a dietitian and expert in technology adoption for nutrition research, expressed enthusiasm for using AI to analyze nutrition and lifestyle data. However, she pointed out that data quality and standardization remain major barriers, making it difficult to translate AI-driven insights into public health interventions.

Dr. van der Schaar agreed, noting that nutrition data often lacks consistency across studies, making it challenging to develop reliable predictive models. She emphasized the importance of interdisciplinary collaboration to refine AI methodologies for lifestyle and dietary analysis.

Asymmetric Cost Functions in Machine Learning

Dr. Harold Lehman raised an important point about asymmetric cost functions in machine learning, noting that many AI models in healthcare continue to use symmetric loss functions, despite the fact that certain errors—such as missing a life-threatening diagnosis—carry far greater consequences than others. He emphasized that the medical field needs to move beyond traditional machine learning approaches and adopt frameworks that reflect the real-world consequences of decision-making errors.

Dr. van der Schaar responded by acknowledging that while some efforts have been made in this direction, machine learning research has often overlooked real-world clinical challenges due to a lack of incentives for engaging with them. She suggested that a concerted effort to catalog and standardize key challenges in AI for healthcare could encourage the machine learning community to prioritize these issues in future research.

AI for Comorbidities in Non-Communicative Populations

Dr. Serge Soudoplatoff, who has worked on AI-based virtual environments for people with dementia, asked whether AI could be used to detect comorbidities in patients who cannot verbally communicate symptoms.

Dr. van der Schaar responded that dynamic comorbidity networks and trajectory modeling could help identify patterns in disease progression, even when patients are unable to describe their conditions. AI models could infer comorbidities based on prior patient data and known disease pathways, allowing clinicians to make more informed predictions.

Thank you for your continued support! We'll be announcing our March seminar soon! Stay tuned!