The Impact of Explainable AI on the Life Sciences Industry

Have you ever wondered why certain medications aren’t equally effective for all patients suffering from the same condition? You’ve probably concluded that since every patient is unique, their distinct traits influence drug efficacy. Within the complex network of biological processes, even minor details – such as how an individual metabolizes a compound apparently unrelated to the disease – can significantly impact therapy effectiveness.

Additionally, factors related directly to the disease – such as variant, progression, or severity – are never identical among patients. There are exceptions, of course. For example, combining of sofosbuvir and ledipasvir (e.g., Harvoni) for Hepatitis C Virus (HCV) genotype 1, achieves a sustained virologic response (SVR) of ~95–99%. However, most medications have significantly lower efficacy. For instance, atopic dermatitis (AD) treatments like Tralokinumab and Nemolizumab typically achieve around 50% effectiveness (EASI-75), though Tralokinumab reaches 81.7% effectiveness for head and neck lesions, and Nemolizumab is notably more effective in patients with high IgE levels.

What is agentic and explainable AI?

Medical data and biological processes are rarely binary or yes/no scenarios. To begin to understand Agentic Explainable AI (AAI), this introduction captures the essence. When asking an AI model, “Does Tralokinumab treat AD?” you might get a simple “yes” but this answer is only accurate in about half the cases. A deeper understanding, currently provided by experienced clinicians and scientists, is necessary. AAI, designed to independently explain its decisions and actions in context, is tailored to user needs and/or patient scenarios. Just as a physician doesn’t base a diagnosis solely on a single number but integrates patient history, imaging, symptoms and context, AAI doesn’t limit itself to simple yes/no answers. Instead, it provides context-based reasoning and explanations. Thus, asking an AAI model whether Tralokinumab treats AD prompts it to contextualize the information it has, recognize missing data and acknowledge patient uniqueness.

How does explainable AI support companies?

Many people argue it’s impossible to fully explain decisions made by AI models, highlighting a lack of trust, transparency and understanding. These elements are critical for regulatory compliance, particularly given the ethical implications of AI-driven decisions. The first step to achieving AAI is Explainable AI itself. Explainable AI refers to methods and techniques that make the outputs of machine learning models understandable to humans. Explainable artificial intelligence (XAI) aims to provide clear, interpretable explanations for how and why specific decisions were reached.

For example, imagine a deep-learning model trained to detect brain tumors from MRI images can achieve high accuracy (e.g., 96%). However, if asked why a particular patient has cancer, it can’t pinpoint the exact tumor area, identify decisive image features, or demonstrate adherence to medical knowledge. The model has merely “learned” associations between pixel patterns and labels.

The key is understanding the details behind an answer. But should a model understand every detail? Not necessarily. XAI (not to be confused with Agentic Explainable AI) aims to provide the level of explainability that users require. A physician might want specific insights into drug effectiveness tailored to a patient’s unique traits, but excessively detailed explanations about drug mechanisms may not always be practical or necessary.

Instead of exclusively relying on complex, opaque models like deep learning, consider using simpler and more transparent models – if they offer comparable performance. However, simpler models might lack the nuance needed for complex biomedical contexts, like advanced imaging analyses.

The challenge is to balance accuracy with understandability. Another approach involves training smaller, cost-effective “student” large language models (LLMs) to replicate the performance of larger, highly accurate “teacher” models. These student models, trained on teacher-provided cases, maintain similar effectiveness with simpler architectures, fewer parameters and enhanced explainability.

Explainable AI in practice – case studies?

Digital pathology

Paige Prostate was the first AI system for digital pathology approved by the FDA (2021). It uses deep learning and computer vision to analyze large, microscopic images of prostate tissue and identify areas suspicious for cancer for further specialist review. Although the model showed high sensitivity and specificity, pathologists were initially skeptical when it marked areas as cancerous without clear explanations, uncertain whether the AI was mistakenly classifying artifacts or abnormalities. The model’s highlighted areas didn’t always align with traditional diagnostic criteria.

The solution was to implement explainability strategies, such as local interpretations – heatmaps allowing pathologists to verify AI recommendations against their diagnostic knowledge – as well as rigorous retrospective testing, data standardization and robust validation involving pathologists. Thus, Paige Prostate, although not strictly XAI, applied a strategy of clinical explainability – limited, visual and practical – which facilitated FDA approval.

Coronary disease

Another example is the XGBoost model (gradient-boosted trees), which predicts coronary disease risk from patient data (age, gender, blood pressure, cholesterol, ECG, glucose). While accurate, doctors asked: “Why does the system indicate an 84% risk for this patient?” Explainable AI was applied using SHAP (SHapley Additive exPlanations), a method that explains predictions by distributing the influence of each feature on outcomes, which revealed that cholesterol and age had significant impacts:

Diagnostics

A particularly intriguing approach is Ada Health, a multilingual app that enables users to input symptoms and receive personalized suggestions about potential causes and next steps. Ada employs a practical approach to explainability and ensures system outputs are understandable and actionable for both patients and clinicians. Instead of formal XAI techniques like SHAP or LIME, Ada’s entire system is designed around logically justified, transparent diagnostic suggestions. Ada uses symbolic reasoning based on decision models, medical ontologies and adaptive logic, thereby aligning with regulatory definitions like Software as a Medical Device (SaMD). Its dynamic, adaptive question-answering approach transparently demonstrates the application’s reasoning. Each diagnostic suggestion (e.g., dehydration, viral infection) clearly outlines supporting symptoms and reasoning, so in this way it mirrors traditional clinical consultations.

In clinical partnership integrations, Ada provides simplified logical documentation in order to aide clinician comprehension of AI-generated suggestions. Crucially, Ada doesn’t diagnose – it  informs users about possible scenarios and encourages timely medical consultation. Despite its advantages, Ada’s reliance on explicit IF-THEN logic can miss nuanced or atypical patient presentations – underscoring the importance of clinician oversight. Ada Health, CE-marked as a Class IIa medical device, is also eligible for FDA approval in the U.S.

Data protection

When discussing transparency and explainability, data protection and privacy naturally arise. Explainable AI significantly aids in understanding how AI models process sensitive or identifiable data. “Black-box” models obscure potential unintended data use or retention of identifiable information. Explainability helps reveal these risks by highlighting data features influencing decisions or identifying reliance on sensitive personal information.

This isn’t just a technical issue but also a regulatory compliance matter, as legal frameworks increasingly mandate transparency in algorithmic decisions affecting individuals directly. Explainable AI thus becomes vital for building trust and proactively addressing privacy concerns.

Emerging AI technologies are rapidly reshaping the life sciences industry from research and development to clinical trials, drug development, diagnostics and patient care. The benefits AI can bring are significant – so are the risks. That’s why companies around the world team up with Software Mind – a partner that provides expert engineering services and domain consultancy that understands the unique challenges and opportunities companies in the life sciences industry face. If you’d like to learn how our team can support the research, development, deployment, maintenance and evolution of your life sciences solutions, get in touch using the contact form.

FAQ

What is explainable AI in healthcare and why is it important for the life sciences industry?

Explainable AI (XAI) refers to the strategies that people can follow to better understand how and why AI models work. In practice, this means moving away from ‘black box’ models to interpretable models that provide clear explanations of why a certain model acts a particular way. In life sciences, XAI can support decision-making and regulatory compliance, help eliminate bias and ensure better understanding of diagnoses and treatment – both for clinicians and patients.

How does explainable AI improve drug development, diagnostics, and patient care?

Along with delivering increased transparency that builds trust, explainable AI (XAI) enables prediction capabilities for determining drug interactions and toxicity, which can lead to more precise diagnoses. Additionally, XAI supports efforts to personalize treatments and engage patients, which translates into higher value care.

What are real-world examples of explainable AI applications in medicine?

Explainable AI has proven to be effective in supporting digital pathology, prediction of coronary diseases, diagnostics and data protection. It can also provide assistance by automatically transcribing clinical notes, monitoring patient treatments and facilitating patient care.

How does explainable AI help life sciences companies meet regulatory and data protection requirements?

Explainable AI provides transparent, clear and dependable insights that are easy to validate and audit. Along with ensuring accountability and providing an understanding of algorithms, XAI provides tools to support decision-making – an important part of compliance with regulations like the EU AI Act, the GDPR and HIPAA.

What is the difference between explainable AI (XAI) and agentic explainable AI (AAI) in medical research?

Explainable AI (XAI) aims to make it easier for humans to understand why an AI solution makes certain decisions. Agentic explainable AI (AAI) takes this a step further by empowering an AI solution with the independence to act and perform tasks on its own, while still delivering explanations and reasoning behind its actions.