Image Feature Fusion of Hyperspectral Imaging and MRI for Automated Subtype Classification and Grading of Adult Diffuse Gliomas According to the 2021 WHO Criteria
Abstract
1. Introduction
2. Materials and Methods
2.1. Dataset Structure
2.2. Hyperspectral Data Acquisition and Preprocessing
2.3. MRI Acquisition and Preprocessing
2.4. MRI Analysis
2.5. Mutual Information-Based Wavelength Selection
2.6. Hyperspectral Attention Fusion Network
2.6.1. Overview
2.6.2. Convolutional Backbone with Residual Connections
2.6.3. Channel Attention Mechanism
2.6.4. Multimodal Fusion Branch
2.6.5. Training and Optimization
3. Results
3.1. Area Under the Receiver Operating Characteristic Curve (ROC–AUC) Comparison Between HSI-Only and Multimodal Configurations
3.2. Classification Accuracy Comparison Between HSI-Only and Multimodal Configurations
3.3. Confusion Matrix Analysis: Performance Comparison Between HSI-Only and Multimodal Configurations
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- DeAngelis, L.M. Global consequences of malignant CNS tumours: A call to action. Lancet Neurol. 2019, 18, 324–325. [Google Scholar] [CrossRef] [PubMed]
- Louis, D.N.; Perry, A.; Wesseling, P.; Brat, D.J.; Cree, I.A.; Figarella-Branger, D.; Hawkins, C.; Ng, H.K.; Pfister, S.M.; Reifenberger, G.; et al. The 2021 WHO classification of tumors of the central nervous system: A summary. Neuro Oncol. 2021, 23, 1231–1251. [Google Scholar] [CrossRef]
- Wen, P.Y.; Packer, R.J. The 2021 WHO classification of tumors of the central nervous system: Clinical implications. Neuro Oncol. 2021, 23, 1215–1217. [Google Scholar] [CrossRef]
- van den Bent, M.J. Interobserver variation of the histopathological diagnosis in clinical trials on glioma: A clinician’s perspective. Acta Neuropathol. 2010, 120, 297–304. [Google Scholar] [CrossRef]
- Komori, T. Grading of adult diffuse gliomas according to the 2021 WHO classification of tumors of the central nervous system. Lab. Investig. 2022, 102, 126–133. [Google Scholar] [CrossRef]
- Wang, W.; Zhao, Y.; Teng, L.; Yan, J.; Guo, Y.; Qiu, Y.; Ji, Y.; Yu, B.; Pei, D.; Duan, W.; et al. Neuropathologist-level integrated classification of adult-type diffuse gliomas using deep learning from whole-slide pathological images. Nat. Commun. 2023, 14, 6359. [Google Scholar] [CrossRef]
- Ma, L.; Rathgeb, A.; Mubarak, H.; Tran, M.; Fei, B. Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. J. Biomed. Opt. 2022, 27, 056502. [Google Scholar] [CrossRef]
- Ma, L.; Little, J.V.; Chen, A.Y.; Myers, L.; Sumer, B.D.; Fei, B. Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging. J. Biomed. Opt. 2022, 27, 046501. [Google Scholar]
- Wang, J.; Zhang, B.; Wang, Y.; Zhou, C.; Zou, D.; Vonsky, M.S.; Mitrofanova, L.B.; Li, Q. Dual-modality image feature fusion network for gastric precancerous lesions classification. Biomed. Signal Process. Control 2024, 87, 105516. [Google Scholar] [CrossRef]
- Du, J.; Tao, C.; Xue, S.; Zhang, Z. Joint diagnostic method of tumor tissue based on hyperspectral spectral-spatial transfer features. Diagnostics 2023, 13, 2002. [Google Scholar] [CrossRef] [PubMed]
- Zhou, X.; Ma, L.; Mubarak, H.K.; Palsgrove, D.; Sumer, B.D.; Chen, A.Y.; Fei, B. Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma. J. Biomed. Opt. 2024, 29, 016005. [Google Scholar] [CrossRef] [PubMed]
- Ortega, S.; Halicek, M.; Fabelo, H.; Camacho, R.; Plaza, M.d.l.L.; Godtliebsen, F.; Callicó, G.M.; Fei, B. Hyperspectral imaging for the detection of glioblastoma tumor cells in H&E slides using convolutional neural networks. Sensors 2020, 20, 1911. [Google Scholar] [CrossRef] [PubMed]
- Chen, J.; Yang, J.; Wang, J.; Zhao, Z.; Wang, M.; Sun, C.; Song, N.; Feng, S. Study on an automatic classification method for determining the malignancy grade of glioma pathological sections based on hyperspectral multi-scale spatial-spectral fusion features. Sensors 2024, 24, 3803. [Google Scholar] [CrossRef]
- Wang, X.; Wang, T.; Zheng, Y.; Yin, X. Recognition of liver tumors by predicted hyperspectral features based on patient’s computed tomography radiomics features. Photodiagnosis Photodyn. Ther. 2023, 42, 103638. [Google Scholar] [CrossRef]
- Tan, R.; Sui, C.; Wang, C.; Zhu, T. MRI-based intratumoral and peritumoral radiomics for preoperative prediction of glioma grade: A multicenter study. Front. Oncol. 2024, 14, 1401977. [Google Scholar] [CrossRef]
- Zerweck, L.; Klose, U.; Würtemberger, U.; Richter, V.; Nägele, T.; Gohla, G.; Grundmann-Hauser, K.; Estler, A.; Ruff, C.; Erb, G.; et al. Preoperative adult-type diffuse glioma subtype prediction with dynamic contrast-enhanced MR imaging and diffusion weighted imaging in tumor cores and peritumoral tissue-a standardized multicenter study. Diagnostics 2025, 15, 532. [Google Scholar] [CrossRef]
- Bhattacharya, I.; Seetharaman, A.; Kunder, C.; Shao, W.; Chen, L.C.; Soerensen, S.J.; Wang, J.B.; Teslovich, N.C.; Fan, R.E.; Ghanouni, P.; et al. Selective identification and localization of indolent and aggressive prostate cancers via CorrSigNIA: An MRI-pathology correlation and deep learning framework. Med. Image Anal. 2022, 75, 102288. [Google Scholar] [CrossRef]
- Kaneko, M.; Fukuda, N.; Nagano, H.; Yamada, K.; Yamada, K.; Konishi, E.; Sato, Y.; Ukimura, O. Artificial intelligence trained with integration of multiparametric MR-US imaging data and fusion biopsy trajectory-proven pathology data for 3D prediction of prostate cancer: A proof-of-concept study. Prostate 2022, 82, 793–803. [Google Scholar] [CrossRef] [PubMed]
- Caglic, I.; Sushentsev, N.; Shah, N.; Warren, A.Y.; Lamb, B.W.; Barrett, T. Integration of prostate biopsy results with pre-biopsy multiparametric magnetic resonance imaging findings improves local staging of prostate cancer. Can. Assoc. Radiol. J. 2022, 73, 515–523. [Google Scholar] [CrossRef]
- Gomaa, A.; Huang, Y.; Hagag, A.; Schmitter, C.; Höfler, D.; Weissmann, T.; Breininger, K.; Schmidt, M.; Stritzelberger, J.; Delev, D.; et al. Comprehensive multimodal deep learning survival prediction enabled by a transformer architecture: A multicenter study in glioblastoma. Neurooncol. Adv. 2024, 6, vdae122. [Google Scholar] [CrossRef]
- Mahootiha, M.; Tak, D.; Ye, Z.; Zapaishchykova, A.; Likitlersuang, J.; Pardo, J.C.C.; Boyd, A.; Vajapeyam, S.; Chopra, R.; Prabhu, S.P.; et al. Multimodal deep learning improves recurrence risk prediction in pediatric low-grade gliomas. Neuro Oncol. 2025, 27, 277–290. [Google Scholar] [CrossRef] [PubMed]
- Rathore, S.; Chaddad, A.; Iftikhar, M.A.; Bilello, M.; Abdulkadir, A. Combining MRI and histologic imaging features for predicting overall survival in patients with glioma. Radiol. Imaging Cancer 2021, 3, e200108. [Google Scholar] [CrossRef]
- Ortega, S.; Fabelo, H.; Camacho, R.; de la Luz Plaza, M.; Callicó, G.M.; Sarmiento, R. Detecting brain tumor in pathological slides using hyperspectral imaging. Biomed. Opt. Express 2018, 9, 818–831. [Google Scholar] [CrossRef]
- Xin, L.; Zheng, W.; Lin, K.; Lin, S.; Huang, Z. Deciphering metabolic alterations associated with glioma grading using hyperspectral stimulated Raman scattering imaging. Anal. Chem. 2025, 97, 7897–7905. [Google Scholar] [CrossRef]
- Zhu, R.; He, H.; Chen, Y.; Yi, M.; Ran, S.; Wang, C.; Wang, Y. Deep learning for rapid virtual H&E staining of label-free glioma tissue from hyperspectral images. Comput. Biol. Med. 2024, 180, 108958. [Google Scholar]
- Zhou, H.; Xu, R.; Mei, H.; Zhang, L.; Yu, Q.; Liu, R.; Fan, B. Application of enhanced T1WI of MRI radiomics in glioma grading. Int. J. Clin. Pract. 2022, 2022, 3252574. [Google Scholar] [CrossRef]
- Fan, H.; Luo, Y.; Gu, F.; Tian, B.; Xiong, Y.; Wu, G.; Nie, X.; Yu, J.; Tong, J.; Liao, X. Artificial intelligence-based MRI radiomics and radiogenomics in glioma. Cancer Imaging 2024, 24, 36. [Google Scholar] [CrossRef] [PubMed]
- Lin, D.; Liu, J.; Ke, C.; Chen, H.; Li, J.; Xie, Y.; Ma, J.; Lv, X.; Feng, Y. Radiomics analysis of quantitative maps from synthetic MRI for predicting grades and molecular subtypes of diffuse gliomas. Clin. Neuroradiol. 2024, 34, 817–826. [Google Scholar] [CrossRef]
- Li, G.; Li, L.; Li, Y.; Qian, Z.; Wu, F.; He, Y.; Jiang, H.; Li, R.; Wang, D.; Zhai, Y.; et al. An MRI radiomics approach to predict survival and tumour-infiltrating macrophages in gliomas. Brain 2022, 145, 1151–1161. [Google Scholar] [CrossRef] [PubMed]
- Graham, S.; Vu, Q.D.; Raza, S.E.A.; Azam, A.; Tsang, Y.W.; Kwak, J.T.; Rajpoot, N. Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 2019, 58, 101563. [Google Scholar] [CrossRef]
- Abel, J.; Jain, S.; Rajan, D.; Padigela, H.; Leidal, K.; Prakash, A.; Conway, J.; Nercessian, M.; Kirkup, C.; Javed, S.A.; et al. AI powered quantification of nuclear morphology in cancers enables prediction of genome instability and prognosis. NPJ Precis. Oncol. 2024, 8, 134. [Google Scholar] [CrossRef] [PubMed]
- Noh, M.-G.; Oh, S.-J.; Ahn, E.-J.; Kim, Y.-J.; Jung, T.-Y.; Jung, S.; Kim, K.-K.; Lee, J.-H.; Lee, K.-H.; Moon, K.-S. Prognostic significance of E-cadherin and N-cadherin expression in gliomas. BMC Cancer 2017, 17, 583. [Google Scholar] [CrossRef]
- Shinoura, N.; E Paradies, N.; E Warnick, R.; Chen, H.; Larson, J.J.; Tew, J.J.; Simon, M.; A Lynch, R.; Kanai, Y.; Hirohashi, S. Expression of N-cadherin and alpha-catenin in astrocytomas and glioblastomas. Br. J. Cancer 1995, 72, 627–633. [Google Scholar] [CrossRef]
- Utsuki, S.; Sato, Y.; Oka, H.; Tsuchiya, B.; Suzuki, S.; Fujii, K. Relationship between the expression of E-, N-cadherins and beta-catenin and tumor grade in astrocytomas. J. Neurooncol. 2002, 57, 187–192. [Google Scholar] [CrossRef] [PubMed]
- Chen, Q.; Cai, J.; Jiang, C. CDH2 expression is of prognostic significance in glioma and predicts the efficacy of temozolomide therapy in patients with glioblastoma. Oncol. Lett. 2018, 15, 7415–7422. [Google Scholar] [PubMed]






Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Su, Y.; Sun, J.; Fu, R.; Li, X.; Bai, J.; Li, F.; Yang, H.; Cheng, Y.; Lu, J. Image Feature Fusion of Hyperspectral Imaging and MRI for Automated Subtype Classification and Grading of Adult Diffuse Gliomas According to the 2021 WHO Criteria. Diagnostics 2026, 16, 458. https://doi.org/10.3390/diagnostics16030458
Su Y, Sun J, Fu R, Li X, Bai J, Li F, Yang H, Cheng Y, Lu J. Image Feature Fusion of Hyperspectral Imaging and MRI for Automated Subtype Classification and Grading of Adult Diffuse Gliomas According to the 2021 WHO Criteria. Diagnostics. 2026; 16(3):458. https://doi.org/10.3390/diagnostics16030458
Chicago/Turabian StyleSu, Ya, Jiazheng Sun, Rongxin Fu, Xiaoran Li, Jie Bai, Fengqi Li, Hongwei Yang, Ye Cheng, and Jie Lu. 2026. "Image Feature Fusion of Hyperspectral Imaging and MRI for Automated Subtype Classification and Grading of Adult Diffuse Gliomas According to the 2021 WHO Criteria" Diagnostics 16, no. 3: 458. https://doi.org/10.3390/diagnostics16030458
APA StyleSu, Y., Sun, J., Fu, R., Li, X., Bai, J., Li, F., Yang, H., Cheng, Y., & Lu, J. (2026). Image Feature Fusion of Hyperspectral Imaging and MRI for Automated Subtype Classification and Grading of Adult Diffuse Gliomas According to the 2021 WHO Criteria. Diagnostics, 16(3), 458. https://doi.org/10.3390/diagnostics16030458

