A Clinically Translatable Multimodal Deep Learning Model for HRD Detection from Histopathology Images
Abstract
1. Introduction
2. Materials and Methods
2.1. Study Cohorts
2.2. Data Modality Acquisition and Preprocessing: The TRINITY AI Multimodal Architecture
- Morphological features used a Vision Transformer embedding network.
- Molecular/pathological features were processed via a multilayer perceptron (MLP) encoder.
- AI-inferred transcriptomic features were encoded using an MLP optimized for high-dimensional biological data.
2.3. Molecular Data
2.4. Digital Pathology (WSI) Processing and Morphological Feature Extraction
2.5. AI-Inferred Transcriptomic Feature Generation
- Differential expression analysis identified genes associated with HRD/HRP classification [30].
- Genes were then filtered based on their correlation between predicted and actual transcriptomic values.
- A genetic algorithm searched for the optimal subset of genes exhibiting maximal predictive synergy [31].
2.6. Contrastive Alignment in a Shared Latent Space
2.7. The Final TRINITY Embedding
2.8. ML Model Training and Testing Methodologies
2.9. Model Configuration and Hyperparameter Optimization
2.10. Data Leakage Prevention During Training
- Patient-level separation: All data (imaging, transcriptomic, and molecular) from the same patient were exclusively assigned to a single split (training, validation, or external test). No patient data appeared across multiple sets.
- Modality-specific preprocessing isolation:
- WSI preprocessing (including HistoQC filtering and patch extraction) was performed independently within each data split, without sharing statistics or parameters across splits.
- Gene selection using differential expression analysis and a genetic algorithm was conducted only on the training set, and the resulting gene panel was subsequently applied unchanged to the validation and test sets.
- Multi-stage pipeline safeguards:
- Stage 1 (multimodal embedding via contrastive learning):The multimodal embedding framework was trained using a contrastive learning paradigm, which does not rely on HRD ground-truth labels. Instead, the model learns a shared embedding space by pulling semantically similar (positive) cross-modal pairs closer together while pushing dissimilar (negative) pairs farther apart. Since no outcome labels are used at this stage, and training is restricted to the training split, this design inherently reduces the risk of label leakage while preserving strict data separation.
- Stage 2 (Gradient Boosting Classifier):Used pre-generated embeddings from Stage 1, ensuring no information flow back to embedding generation.
- Hyperparameter tuning:Hyperparameter optimization for the Gradient Boosting Classifier was performed using stratified cross-validation within the training set only. A completely held-out validation set was used solely for final model selection, without participation in training or tuning.
- No data reuse: The 260-gene panel selected during training was not refined on validation/test data. Clinical variables used for molecular encoder were not selected based on validation set correlation. This is completely based on the training set.
- Independent test set: The external cohort (n = 74) was never viewed during model development, training, or hyperparameter selection and was used for blind evaluation only.
3. Results and Discussion
- Data scope: DeepHRD and Kather Lab used broader multi-institutional cohorts; our TCGA-centric approach limits direct comparison but provides internal harmonization benefits.
- Modality differences: DeepHRD and Kather Lab rely on morphology alone; TRINITY uniquely integrates imaging, AI-inferred transcriptomics, and clinical data. This multimodal approach explains performance advantages but reflects different clinical utility (more data-intensive).
- Task definition: All three methods predict HRD status, but TRINITY additionally incorporates clinico-molecular data (BRCA status, aneuploidy, and mutational signatures), making it more clinically actionable for patient stratification.
- Evaluation metrics: We now report all methods using consistent metrics (sensitivity, specificity, AUC, PPV, and NPV) rather than selective reporting.
- Generalization: We acknowledge that our external validation (n = 74 ovarian) shows AUC = 0.88, directly comparable to Kather Lab’s 0.61 on external OV cohorts, suggesting improvement. However, we note their 10-cancer-type validation is broader in scope.
3.1. Cost-Effectiveness and Turnaround Time Advantages
3.2. Limitations and Future Directions
3.3. Broader Impact and Future Research Directions
- Gene Ontology enrichment analysis: Top genes from the 260-gene panel were analyzed for functional categories. Enriched terms include the following:
- DNA damage response (p < 0.001): ATM, CHEK2, TP53BP1, and RAD51 pathway members
- Homologous recombination repair (p < 0.001): BRCA1, BRCA2, RAD51D, PALB2, and BARD1.
- Apoptosis regulation (p < 0.01): BAX, BAD, and TP53 targets
- Highlighted clinically actionable genes: a.APOBEC3B: High expression in HRD-High samples; implicated in PARP inhibitor sensitivity and genomic instability b.CXCL13: Immune infiltration marker; elevated in HRD-High, suggesting immunogenic phenotype c.ESR1: Interaction with HR repair pathway; explains differential response in ER+ breast cancers
- AI prediction validation: Median Pearson correlation coefficient (PCC = 0.6061) between AI-predicted and true expression is discussed in context of recent literature.
- Limitations acknowledged: We note that while genes are enriched for HRD biology, some selected genes may reflect tumor microenvironment or immune context rather than intrinsic HR deficiency, suggesting future work should integrate single-cell transcriptomics.
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
| AI | Artificial intelligence |
| HRD | Homologous recombination deficiency |
| TAT | Turn-around time |
| H&E | Hematoxylin and eosin |
| TCGA | The Cancer Genome Atlas |
| HRR | Homologous recombination repair |
| AUC | Area under curve |
| PPV | Positive predictive value |
| NPV | Negative predictive value |
| WSI | Whole-slide image |
| PARP | Poly (ADP-ribose) polymerase |
| NGS | Next-generation sequencing |
| NMD | No mutations detection |
| HRP | Homologous recombination proficient |
| QC | Quality control |
| DL | Deep learning |
References
- Janowczyk, A.; Zuo, R.; Gilmore, H.; Feldman, M.; Madabhushi, A. HistoQC: An open-source quality control tool for digital pathology slides. JCO Clin. Cancer Inform. 2019, 3, 1–7. [Google Scholar] [CrossRef] [PubMed]
- Bilal, M.; Nimir, M.; Snead, D.; Taylor, G.S.; Rajpoot, N. Role of AI and digital pathology for colorectal immuno-oncology. Br. J. Cancer 2023, 128, 3–11. [Google Scholar] [CrossRef] [PubMed]
- Hu, J.; Lv, H.; Zhao, S.; Lin, C.J.; Su, G.H.; Shao, Z.M. Prediction of clinicopathological features, multi-omics events and prognosis based on digital pathology and deep learning in HR+/HER2− breast cancer. J. Thorac. Dis. 2023, 15, 2528. [Google Scholar] [CrossRef]
- Roy, R.; Chun, J.; Powell, S.N. BRCA1 and BRCA2: Different roles in a common pathway of genome protection. Nat. Rev. Cancer 2012, 12, 68–78. [Google Scholar] [CrossRef]
- Konstantinopoulos, P.A.; Ceccaldi, R.; Shapiro, G.I.; D’Andrea, A.D. Homologous recombination deficiency: Exploiting the fundamental vulnerability of ovarian cancer. Cancer Discov. 2015, 5, 1137–1154. [Google Scholar] [CrossRef]
- Färkkilä, A.; Rodríguez, A.; Oikkonen, J.; Gulhan, D.C.; Nguyen, H.; Domínguez, J.; Ramos, S.; Mills, C.E.; Pérez-Villatoro, F.; Lazaro, J.B.; et al. Heterogeneity and clonal evolution of acquired PARP inhibitor resistance in TP53- and BRCA1-deficient cells. Cancer Res. 2021, 81, 2774–2787. [Google Scholar] [CrossRef] [PubMed]
- Murciano-Goroff, Y.R.; Schram, A.M.; Rosen, E.Y.; Won, H.; Gong, Y.; Noronha, A.M.; Janjigian, Y.Y.; Stadler, Z.K.; Chang, J.C.; Yang, S.R.; et al. Reversion mutations in germline BRCA1/2-mutant tumors reveal a BRCA-mediated phenotype in noncanonical histologies. Nat. Commun. 2022, 13, 7182. [Google Scholar] [CrossRef]
- Morimoto, A.; Matsumoto, T.; Teramoto, N.; Murakami, S.; Usami, T.; Murai, J.; Sugiyama, T. Dynamic changes in homologous recombination status before and after chemotherapy in advanced ovarian, fallopian tube, and primary peritoneal cancers. medRxiv 2025. [Google Scholar] [CrossRef]
- van Wijk, L.M.; Vermeulen, S.; ter Haar, N.T.; Kramer, C.J.H.; Terlouw, D.; Vrieling, H.; Cohen, D.; Vreeswijk, M.P.G. Performance of a RAD51-based functional HRD test on paraffin-embedded breast cancer tissue. Breast Cancer Res. Treat. 2023, 202, 607–616. [Google Scholar] [CrossRef]
- Kim, D.; Nam, H.J. PARP inhibitors: Clinical limitations and recent attempts to overcome them. Int. J. Mol. Sci. 2022, 23, 8412. [Google Scholar] [CrossRef]
- Kwon, J.S.; Tinker, A.V.; Santos, J.; Compton, K.; Sun, S.; Schrader, K.A.; Karsan, A. Germline testing and somatic tumor testing for BRCA1/2 pathogenic variants in ovarian cancer: What is the optimal sequence of testing? JCO Precis. Oncol. 2022, 6, e2200033. [Google Scholar] [CrossRef]
- Bayley, R.; Sweatman, E.; Higgs, M.R. New perspectives on epigenetic modifications and PARP inhibitor resistance in HR-deficient cancers. Cancer Drug Resist. 2023, 6, 35. [Google Scholar] [CrossRef]
- Xiang, J.; Wang, X.; Wang, X.; Zhang, J.; Yang, S.; Yang, W.; Han, X.; Liu, Y. Automatic diagnosis and grading of prostate cancer with weakly supervised learning on whole slide images. Comput. Biol. Med. 2023, 152, 106340. [Google Scholar] [CrossRef]
- Bergstrom, E.N.; Abbasi, A.; Díaz-Gay, M.; Galland, L.; Ladoire, S.; Lippman, S.M.; Alexandrov, L.B. Deep learning artificial intelligence predicts homologous recombination deficiency and platinum response from histologic slides. J. Clin. Oncol. 2024, 42, 3550–3560. [Google Scholar] [CrossRef] [PubMed]
- Hoang, D.T.; Dinstag, G.; Hermida, L.C.; Ben-Zvi, D.S.; Elis, E.; Caley, K.; Sammut, S.J.; Sinha, S.; Sinha, N.; Dampier, C.H.; et al. Prediction of cancer treatment response from histopathology images through imputed transcriptomics. Res. Sq. 2023, rs-3193270, Update in Nat. Cancer 2024, 5, 1305–1317. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.; Zeng, H.; Xiang, Y.; Huang, Y.; Luo, Y.; Ma, X. Histopathological images and multi-omics integration predict molecular characteristics and survival in lung adenocarcinoma. Front. Cell Dev. Biol. 2021, 9, 720110. [Google Scholar] [CrossRef] [PubMed]
- Marconato, N.; De Tommasi, O.; Paladin, D.; Boscarino, D.; Spagnol, G.; Saccardi, C.; Maggino, T.; Tozzi, R.; Noventa, M.; Marchetti, M. Unraveling homologous recombination deficiency in ovarian cancer: A review of currently available testing platforms. Cancers 2025, 17, 1771. [Google Scholar] [CrossRef]
- Wang, J.; Shen, J.; Zhang, Z.; Ye, L.; Lu, J.; Xiao, Z.; Zhang, L.; Liu, X.; Zhu, C.; Wang, X.; et al. Homologous recombination deficiency score for predicting efficacy of platinum-based chemotherapy and PARPi maintenance therapy in ovarian cancer. J. Clin. Oncol. 2024, 42, e17543. [Google Scholar] [CrossRef]
- McGenity, C.; Clarke, E.L.; Jennings, C.; Matthews, G.; Cartlidge, C.; Freduah-Agyemang, H.; Stocken, D.D.; Treanor, D. Artificial intelligence in digital pathology: A systematic review and meta-analysis of diagnostic test accuracy. npj Digit. Med. 2024, 7, 114. [Google Scholar] [CrossRef]
- Kim, J.H.; Kim, S.I.; Kim, E.T.; Ha, H.I.; Lee, D.E.; Lee, Y.J.; Lee, J.Y.; Kim, S.; Kim, S.W.; Kim, Y.T.; et al. The association between location of BRCA mutation and efficacy of PARP inhibitor as a frontline maintenance therapy in advanced epithelial ovarian cancer. Cancers 2025, 17, 756. [Google Scholar] [CrossRef]
- Norquist, B.; Wurz, K.A.; Pennil, C.C.; Garcia, R.; Gross, J.; Sakai, W.; Karlan, B.Y.; Taniguchi, T.; Swisher, E.M. Secondary somatic mutations restoring BRCA1/2 predict chemotherapy resistance in hereditary ovarian carcinomas. J. Clin. Oncol. 2011, 29, 3008–3015. [Google Scholar] [CrossRef]
- Jackson, L.M.; Moldovan, G.L. Mechanisms of PARP1 inhibitor resistance and their implications for cancer treatment. NAR Cancer 2022, 4, zcac042. [Google Scholar] [CrossRef]
- Radford, A.; Kim, J.W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. Learning Transferable Visual Models from Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML), Online, 18–24 July 2021; PMLR: Cambridge, MA, USA; Volume 139, pp. 8748–8763. [Google Scholar] [CrossRef]
- Xu, Y.; Wang, Y.; Zhou, F.; Ma, J.; Jin, C.; Yang, S.; Li, J.; Zhang, Z.; Zhao, C.; Zhou, H.; et al. A multimodal knowledge-enhanced whole-slide pathology model for molecular prediction. arXiv 2024, arXiv:2407.15362. [Google Scholar] [CrossRef]
- Bergstra, J.; Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 2012, 13, 281–305. [Google Scholar]
- Loeffler, C.M.L.; El Nahhas, O.S.M.; Muti, H.S.; Carrero, Z.I.; Seibel, T.; van Treeck, M.; Cifci, D.; Gustav, M.; Bretz, K.; Gaisa, N.T.; et al. Prediction of homologous recombination deficiency from routine histology with attention-based multiple instance learning in nine different tumor types. BMC Biol. 2024, 22, 225. [Google Scholar] [CrossRef] [PubMed]
- Jiao, Z.; Chen, S.; Shi, H.; Xu, J. Multimodal feature selection with feature correlation and feature structure fusion for MCI and AD classification. Brain Sci. 2022, 12, 80. [Google Scholar] [CrossRef]
- Xie, M.; Lee, K.; Lockhart, J.H.; Cukras, S.D.; Carvajal, R.; Beg, A.A.; Flores, E.R.; Teng, M.; Chung, C.H.; Tan, A.C. TIMEx: Tumor-immune microenvironment deconvolution web-portal for bulk transcriptomics using pan-cancer scRNA-seq signatures. Bioinformatics 2021, 37, 3681–3683. [Google Scholar] [CrossRef]
- Ding, T.; Wagner, S.J.; Song, A.H.; Chen, R.J.; Lu, M.Y.; Zhang, A.; Vaidya, A.J.; Jaume, G.; Shaban, M.; Kim, A.; et al. A multimodal whole-slide foundation model for pathology. Nat. Med. 2025, 31, 3749–3761. [Google Scholar] [CrossRef]
- Pan, J.W.; Tan, Z.C.; Ng, P.S.; Zabidi, M.M.A.; Nur Fatin, P.; Teo, J.Y.; Hasan, S.N.; Islam, T.; Teoh, L.Y.; Jamaris, S.; et al. Gene expression signature for predicting homologous recombination deficiency in triple-negative breast cancer. npj Breast Cancer 2024, 10, 60. [Google Scholar] [CrossRef]
- Mowlaei, M.E.; Shi, X. FSFGA: A Feature Selection Framework for Phenotype Prediction Using Genetic Algorithms. Genes 2023, 14, 1059. [Google Scholar] [CrossRef]
- Oord, A.V.D.; Li, Y.; Vinyals, O. Representation learning with contrastive predictive coding. arXiv 2018, arXiv:1807.03748. [Google Scholar] [CrossRef]
- Friedman, J.H. Greedy function approximation: A gradient boosting machine. Ann. Stat. 2001, 29, 1189–1232. [Google Scholar] [CrossRef]
- Leibowitz, B.D.; Dougherty, B.V.; Bell, J.S.; Kapilivsky, J.; Michuda, J.; Sedgewick, A.J.; Munson, W.A.; Chandra, T.A.; Dry, J.R.; Beaubier, N.; et al. Validation of genomic and transcriptomic models of homologous recombination deficiency in a real-world pan-cancer cohort. BMC Cancer 2022, 22, 587. [Google Scholar] [CrossRef]
- Rizvi, A.A.; Nagpal, K.; Hu, I.; Joshi, R.; Schau, G.; Baits, R.; Muller, Y.; Stumpe, M.C.; Beaubier, N. Homologous recombination deficiency is detectable from H&E wholeslide images in realworld prostate needle core biopsies. Lab. Investig. 2023, 103, S792–S793. [Google Scholar]
- Loeffler, C.M.L.; El Nahhas, O.S.; Muti, H.S.; Seibel, T.; Cifci, D.; van Treeck, M.; Gustav, M.; Carrero, Z.I.; Gaisa, N.T.; Lehmann, K.V.; et al. Direct prediction of homologous recombination deficiency from routine histology in ten different tumor types with attention-based multiple instance learning: A development and validation study. medRxiv 2023. [Google Scholar] [CrossRef] [PubMed]
- Guffanti, F.; Mengoli, I.; Damia, G. Current HRD assays in ovarian cancer: Differences, pitfalls, limitations, and novel approaches. Front. Oncol. 2024, 14, 1405361. [Google Scholar] [CrossRef] [PubMed]
- Jahn, S.W.; Plass, M.; Moinfar, F. Digital pathology: Advantages, limitations and emerging perspectives. J. Clin. Med. 2020, 9, 3697. [Google Scholar] [CrossRef] [PubMed]



| Cancer Type | Stages | Race | Ethnicity |
|---|---|---|---|
| BRCA (n = 1117) | Stage I (174) Stage II (640) Stage III (261) Stage IV (20) Unknown (22) | White (783) Black or African American (175) Asian (61) American Indian or Alaska Native (1) Not Available (97) | Not Hispanic or Latino (908) Hispanic or Latino (40) Not Available (169) |
| OV (n = 106) | Stage I (2) Stage II (4) Stage III (74) Stage IV (23) Unknown (3) | Not Available (106) | Not Hispanic or Latino (93) Hispanic or Latino (2) Not Available (11) |
| Cohort | Cancer Type | Training (HRD-High) | Training (HRD-Low) | Validation (HRD-High) | Validation (HRD-Low) | External (HRD-High) | External (HRD-Low) | Total |
|---|---|---|---|---|---|---|---|---|
| TCGA | BRCA | 145 | 593 | 62 | 254 | – | – | 1054 |
| 738 | 316 | |||||||
| OV | 36 | 33 | 12 | 15 | – | – | 96 | |
| 69 | 27 | – | – | |||||
| BRCA + OV | 738 + 69 = 807 (Total Training) | 316 + 27 = 343 (Total Testing) | – | – | 1150 | |||
| External | OV | – | – | – | – | 32 | 42 | 74 |
| Model | AUROC | Accuracy | Sensitivity (Recall) | Specificity | PPV (Precision) | NPV |
|---|---|---|---|---|---|---|
| Logistic Regression | 0.5917 | 0.5319 | 0.8750 | 0.3548 | 0.4118 | 0.8462 |
| Random Forest | 0.6129 | 0.5213 | 0.8125 | 0.3710 | 0.4000 | 0.7931 |
| Support Vector Classifier | 0.6436 | 0.6383 | 0.6562 | 0.6290 | 0.4773 | 0.7800 |
| Gradient Boosting | 0.5766 | 0.5213 | 0.7500 | 0.4032 | 0.3934 | 0.7576 |
| K-Nearest Neighbors | 0.5393 | 0.4149 | 0.8750 | 0.1774 | 0.3544 | 0.7333 |
| Decision Tree | 0.6174 | 0.5851 | 0.7188 | 0.5161 | 0.4340 | 0.7805 |
| Parameter | Value | Description |
|---|---|---|
| learning_rate | 0.01 | Controls how much the model updates its weights after each iteration; smaller values make training slower but stable. |
| max_depth | 7 | Sets the maximum depth of each decision tree, allowing the model to capture complex relationships while controlling overfitting. |
| max_features | log2 | Determines the number of features to consider when splitting a node, helping to reduce overfitting by introducing feature randomness. |
| min_samples_leaf | 1 | Ensures that each terminal leaf node contains at least one training sample, allowing fine-grained decision boundaries. |
| min_samples_split | 5 | Specifies the minimum number of samples required to split an internal node, preventing the creation of overly specific branches. |
| n_estimators | 100 | Sets the total number of boosting stages (trees) to be built; more estimators can improve performance but increase computation. |
| subsample | 0.8 | Uses 80% of the training data for each tree to introduce randomness and improve the model’s generalization. |
| Model Predict on | Total Samples | Sensitivity | NPV | PPV | Specificity | AUC ROC |
|---|---|---|---|---|---|---|
| TCGA- breast and ovarian | 343 | 0.7973 | 0.9395 | 0.6211 | 0.8662 | 0.9001 |
| TCGA-breast | 316 | 0.7742 | 0.9417 | 0.6316 | 0.8898 | 0.9121 |
| TCGA-ovarian | 27 | 0.9167 | 0.875 | 0.5789 | 0.4667 | 0.7111 |
| External validation set | 74 | 0.812 | 0.854 | 0.765 | 0.81 | 0.888 |
| Source | Modal | Data Source | BRCA AUC | OV AUC | Key Differences | |
|---|---|---|---|---|---|---|
| DeepHRD [14] | Image | TCGA | 0.81 | - | ResNet18 CNN; no multimodal integration | |
| Kather Lab HRD [36] | Image | Multi-cohort | 0.78 | 0.61 | Attention- based MIL; single modality; 10 cancer types tested | |
| Trinity-HRD | TCGA | Image + Gene + Clinical | TCGA | 0.9121 | 0.7111 | Integrated multimodal; contrastive learning; |
| 0.9001 (BRCA-OV) | ||||||
| External cohort | - | 0.88 | ||||
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Uttarwar, M.; Khandare, J.; Shivamurthy, P.M.; Satpute, A.; Panwar, M.; Kothavade, H.; Ramesh, A.; Iyer, S.; Shafi, G. A Clinically Translatable Multimodal Deep Learning Model for HRD Detection from Histopathology Images. Diagnostics 2026, 16, 356. https://doi.org/10.3390/diagnostics16020356
Uttarwar M, Khandare J, Shivamurthy PM, Satpute A, Panwar M, Kothavade H, Ramesh A, Iyer S, Shafi G. A Clinically Translatable Multimodal Deep Learning Model for HRD Detection from Histopathology Images. Diagnostics. 2026; 16(2):356. https://doi.org/10.3390/diagnostics16020356
Chicago/Turabian StyleUttarwar, Mohan, Jayant Khandare, P. M. Shivamurthy, Aditya Satpute, Mohit Panwar, Hrishita Kothavade, Aarthi Ramesh, Sandhya Iyer, and Gowhar Shafi. 2026. "A Clinically Translatable Multimodal Deep Learning Model for HRD Detection from Histopathology Images" Diagnostics 16, no. 2: 356. https://doi.org/10.3390/diagnostics16020356
APA StyleUttarwar, M., Khandare, J., Shivamurthy, P. M., Satpute, A., Panwar, M., Kothavade, H., Ramesh, A., Iyer, S., & Shafi, G. (2026). A Clinically Translatable Multimodal Deep Learning Model for HRD Detection from Histopathology Images. Diagnostics, 16(2), 356. https://doi.org/10.3390/diagnostics16020356
