AI Cancer Tools: Relying on Visual Shortcuts Over True Biology

Recent research highlights significant concerns regarding the reliability of deep learning systems employed in cancer pathology. These sophisticated tools, designed to analyze cancer biology from microscopic images, may rely on misleading shortcuts rather than actual biological data.

Artificial intelligence is rapidly advancing in the field of cancer diagnosis, offering the potential for quicker and cost-effective assessments. However, a study from the University of Warwick, published in Nature Biomedical Engineering, raises alarms that many of these AI systems might be operating on visual heuristics instead of fundamental biological signals, posing a risk to their efficacy in real patient scenarios.

“It’s similar to evaluating a restaurant’s quality by its line of customers; while it offers insight, it’s not a direct indication of the culinary experience. Numerous AI models in pathology are following this pattern, relying on correlations between biomarkers or readily identifiable tissue features, instead of isolating specific biomarker signals. When conditions shift, these shortcuts tend to fail.”

Dr. Fayyaz Minhas, Associate Professor and Lead Researcher of the Predictive Systems in Biomedicine (PRISM) Lab, University of Warwick

The researchers conducted an extensive analysis of over 8,000 patient samples spanning four major cancer types: breast, colorectal, lung, and endometrial. They compared the performance of various leading machine learning models. While the results often appeared impressively accurate, the underlying mechanisms frequently hinged on statistical shortcuts.

For instance, rather than detecting mutations in the cancer-linked BRAF gene, a model might learn that BRAF mutations frequently occur alongside clinical features like microsatellite instability (MSI). Consequently, the AI predicts BRAF status based on this combination rather than the actual BRAF signal, meaning its predictive accuracy is contingent upon the simultaneous presence of these biomarkers, which could lead to unreliable outcomes if they are not present.

Kim Branson, Senior Vice President and Global Head of Artificial Intelligence and Machine Learning at GSK, comments: “Predicting a BRAF mutation based on correlated features like MSI is akin to forecasting rain by merely observing umbrellas—it can work, but it doesn’t equate to a proper understanding of meteorology. Importantly, if a model cannot provide insights beyond a basic pathologist-assigned grade, we aren’t advancing the field; we’re merely automating shortcuts. The path for the next wave of pathology AI should focus on more rigorous evaluation standards that compel models to move past these easy fixes and truly understand biological processes.”

When the AI models’ performance was evaluated among stratified patient groups, such as only high-grade breast cancers or exclusively MSI-positive tumors, the accuracy significantly declined. This revealed a heavy reliance on shortcut signals that vanished once outside variables were accounted for.

In specific prediction tasks, the performance differential between deep learning systems and human-generated clinical data was marginal. AI achieved accuracy scores just above 80% in predicting biomarkers, while tumor grade alone—an aspect already assessed by pathologists—yielded approximately 75% accuracy.

Professor Nasir Rajpoot, Director of the Tissue Image Analytics (TIA) Centre at the University of Warwick and CEO of the spin-out company Histofy, states: “This study underlines a crucial point concerning the integration of AI in medicine: to produce meaningful and lasting benefits, we must evaluate the clinical relevance of AI-driven predictions through comprehensive, bias-aware assessments rather than relying solely on superficial accuracy metrics.”

While machine learning techniques can be useful for research, drug development, candidate screening, and supporting clinical decisions, the researchers stress the necessity for future AI tools to transcend correlation-focused learning and instead adopt strategies that actively model biological relationships and causal links. They also advocate for stringent evaluation standards, incorporating subgroup analysis and comparative measures against straightforward clinical benchmarks, prior to deploying these technologies for regular patient care.

Dr. Minhas concludes: “This research isn’t meant to denounce AI in pathology; rather, it serves as a wake-up call. Current models might perform well in controlled environments, but they often depend on statistical shortcuts instead of authentic biological comprehension. Without robust evaluation standards, these tools should not substitute for molecular testing; it’s vital for clinicians and researchers to recognize their limitations and apply them cautiously.”

Co-author Professor Sabine Tejpar, Head of Digestive Oncology at KU Leuven, adds: “The clinical relevance of innovative tools necessitates careful adaptation to what is precise, accurate, and attainable for each individual patient. Too often, oncology becomes engulfed in ‘innovation’ that has minimal or no impact on patient care, driven more by what can be marketed rather than by thorough evaluations of what is genuinely relevant for unique patients and their specific characteristics.”

“While progress typically entails initial missteps, we must learn from past experiences and avoid oversimplifications or overextensions through inappropriate methodologies. Complexity and variability are fundamental challenges, yet they are also the very aspects these new technologies need to master.”