Adaptive Multi-Scale Feature Fusion with Hybrid Representation Learning to Classify and Retrieve Histopathological Images

Abstract

Accurate classification and efficient retrieval of histopathological images are essential for the diagnosis of lung adenocarcinoma (LUAD). Existing deep learning approaches for Content-Based Histopathological Image Retrieval (CBHIR) typically generate single-scale embeddings, missing the richer spatial context from earlier network stages. We propose a unified framework composed of: (1) a ConvNeXt V2 backbone with an integrated Convolutional Block Attention Module (CBAM) for multi-scale feature extraction, (2) an Adaptive Weighted Fusion Neck with learnable softmax-normalized weights, and (3) a novel Hybrid Representation Head producing an 18,496-dimensional descriptor by concatenating global, spatial, and attention-weighted features. Evaluated on the WSSS4LUAD dataset (10,087 patches, four tissue classes), our model achieves 85.03% accuracy (5-fold CV: 83.35 ± 0.93%), F1-score of 0.8116, mean Average Precision (MAP) of 0.8323 for retrieval, and an Expected Calibration Error (ECE) of 0.0378. Ablation experiments confirm that all proposed modules contribute positively, with the Attention Branch being the most impactful (Δ = −2.13%). The framework further provides Gradient-weighted Class Activation Mapping (Grad-CAM) explainability for clinical interpretability.

1. Introduction

Lung cancer remains the leading cause of cancer-related mortality worldwide, and lung adenocarcinoma (LUAD) is its most common histological subtype [1]. Pathological examination of hematoxylin and eosin (H&E)-stained sections is the diagnostic gold standard, but it forces pathologists to analyze whole-slide images (WSIs) that can reach 100,000 × 100,000 pixels at the highest magnification [2]. The large volume of data generated by digital pathology, together with a rising pathologist workload, has driven work on computer-aided diagnostic (CAD) tools [3].

Among the CAD tools proposed for this workload, Content-Based Histopathological Image Retrieval (CBHIR) is now widely used to help pathologists locate visually similar tissue images in large archives for comparison-based diagnosis and teaching [4,5]. Unlike keyword-based search, CBHIR extracts meaningful feature descriptors from images and measures similarity in a learned embedding space [6]. The quality of a CBHIR system therefore depends on the quality of those descriptors.

These descriptor-based systems have been reshaped by deep learning, in particular Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) [7,8]. Current approaches, however, share two recurring limitations. First, they typically extract features from a single scale—usually the deepest layer—and miss spatial context from earlier layers that captures local structure [9]. Second, models pre-trained on natural-image corpora, such as ImageNet, transfer imperfectly to H&E histopathology because of differences in texture, color distribution, and morphology [10].

Within this line of research, Nuñez-Fernández et al. [11] recently introduced the Local–Global Feature Fusion Embedding Model (LGFFEM), showing that local–global fusion with a Generalized Mean (GeM) head reaches state-of-the-art retrieval on Kimia Patch24C (${\eta }_p$ = 99.40%). Their ConvNeXt V2 + BiFPN-style backbone targets retrieval only, which limits its use as an end-to-end diagnostic tool.

To close these gaps, we propose a unified framework that performs classification and retrieval in a single pass. The framework contributes:

• Hybrid Representation Head: A novel three-branch architecture that produces an 18,496-dimensional feature descriptor by concatenating global semantic features (2048-dim), spatial layout features (16,384-dim), and attention-weighted features (64-dim), capturing complementary information at different abstraction levels.
• Adaptive Weighted Fusion Neck: A multi-scale feature aggregation module that uses learnable softmax-normalized weights to combine features in a manner that is adaptive to the data.
• Comprehensive Evaluation Framework: A single system that is assessed on the WSSS4LUAD lung adenocarcinoma dataset with classification metrics, retrieval metrics (MAP, MRR, P@K, NDCG@K), and medical accuracy metrics (${\eta }_p$, ${\eta }_w$, ${\eta }_{\mathrm{tot}}$), calibration analysis (ECE), and Grad-CAM explainability.
• Dual-Purpose Architecture: This is a general architecture that can be customized to perform both tissue classification with calibrated probabilities and content-based image retrieval, evaluated by means of a comprehensive set of metrics, including medical accuracy, calibration analysis (ECE), and Grad-CAM explainability for clinical interpretability.

While the individual building blocks we adopt—ConvNeXt V2, the Convolutional Block Attention Module (CBAM), multi-scale feature fusion, Grad-CAM, and learned retrieval descriptors—have each appeared in prior medical-imaging work, the specific novelty of this study is their unification into a single descriptor that simultaneously supports classification and retrieval. Concretely, our contribution is threefold: (i) an Adaptive Weighted Fusion Neck whose four softmax-normalized scale weights are fully learnable end-to-end, rather than the fixed or fast-normalized fusion patterns of BiFPN-style necks; (ii) an 18,496-dimensional Hybrid Representation Head that explicitly concatenates global semantic, dense spatial-layout, and attention-weighted features so that the same vector drives both a calibrated classifier and a FAISS retrieval index; and (iii) a unified, single-pass design in which classification, retrieval, calibration, and explainability are obtained from one forward pass and one descriptor, whereas prior CBHIR models, such as LGFFEM, are retrieval-only. To our knowledge, this combination has not been reported for lung adenocarcinoma histopathology.

Beyond classification, the content-based retrieval component is intended to serve concrete clinical roles. By returning visually and morphologically similar LUAD patches from an indexed archive, the system can act as a diagnostic aid that surfaces previously confirmed cases for side-by-side comparison, support case-based reasoning for ambiguous or borderline regions, assist trainee education by retrieving canonical examples of each tissue pattern, help recognize rare or atypical patterns by matching them against the archive, and contribute to quality control by flagging when a query is dissimilar to any confidently labeled exemplar. In this work, retrieval is positioned as a similarity-search and secondary-verification tool that complements—rather than replaces—the pathologist’s judgment.

The remainder of this paper is organized as follows. Section 2 reviews related work on content-based histopathological image retrieval, multi-scale feature fusion, attention mechanisms, and modern convolutional backbones. Section 3 details the proposed method, including the ConvNeXt V2–CBAM backbone, the Adaptive Weighted Fusion Neck, the Hybrid Representation Head, the classifier, and the loss function. Section 4 describes the datasets, implementation details, and evaluation protocol. Section 5 reports and analyzes the classification, retrieval, calibration, ablation, and explainability results. Section 6 discusses the findings and limitations, and Section 7 concludes with directions for future work.

2. Related Works

This section situates our work within four lines of research that directly inform the proposed framework. We first review content-based histopathological image retrieval (Section 2.1), which motivates the need for discriminative image descriptors; we then survey multi-scale feature fusion (Section 2.2) and attention mechanisms for medical imaging (Section 2.3), which underpin our neck and backbone designs, respectively; and we close with modern convolutional backbones (Section 2.4), which justify our choice of ConvNeXt V2. For each line we highlight the gap that our unified classification-and-retrieval design is intended to fill.

2.1. Content-Based Histopathological Image Retrieval

Contemporary CBHIR systems are designed based on the use of deep learning models to extract image descriptors embeddings. Hegde et al. [4] suggested a CNN-based image search tool, SMILY, which uses multi-similarity loss to train the tool. Yang et al. [12] implemented a mixed-attention mechanism, which combined spatial and channel attention with squeeze-and-excitation blocks, and was trained on the Kimia Path24C dataset using multi-similarity loss.

Tabatabaei et al. [13] utilized a Convolutional Auto-Encoder with unsupervised training to extract image descriptors from the bottleneck layer. Mohammad Alizadeh et al. [14] proposed a Siamese CNN hashing model using contrastive pair loss for histopathology image retrieval, addressing imbalanced classes in datasets, such as Kather [15] and BreakHis [16]. Tabatabaei et al. [17] presented a Siamese content-based search engine trained with contrastive loss for skin and breast cancer diagnosis.

2.2. Multi-Scale Feature Fusion

Multi-scale feature extraction has been extensively studied in object detection and segmentation. Feature Pyramid Networks (FPN) [18] introduced a top-down pathway for constructing multi-scale feature maps. BiFPN [19] extended this with bidirectional cross-scale connections and fast normalized fusion weighting. Iqbal and Qureshi [20] proposed a Global-Local Pyramid Pattern for fusing textural and visual features across medical image domains.

Most recently, Nuñez-Fernández et al. [11] proposed LGFFEM, which incorporates a Local-Global Feature Fusion Neck (LGFFN) inspired by BiFPN, using Local and Global Feature Aggregators with channel attention. Their model revealed that the multi-scale fusion enhances the performance of retrieval. We extend this body of work by introducing an adaptive weighted fusion mechanism whose weights are learnable, and a hybrid representation head to perform both classification and retrieval.

2.3. Attention Mechanisms for Medical Imaging

Attention mechanisms have been shown to be useful in concentrating neural networks on diagnostically significant areas. Channel attention was proposed by Squeeze-and-Excitation (SE) networks [21], which used global average pooling and a bottleneck MLP. Convolutional Block Attention Module (CBAM) [22] that builds on SE introduced sequential channel and spatial attention, which showed to be more effective in image recognition tasks. Shao et al. [23] demonstrated that global characteristics along with attention-based re-ranking are capable of providing good performance in image retrieval.

2.4. ConvNeXt V2 and Modern CNN Architectures

ConvNeXt V2 [24] is an updated CNN design that embraces the design philosophy of Vision Transformers, whilst preserving the efficiency of convolutional operations. The Global Response Normalization (GRN) layer, which normalizes features using a global context to encourage diversity of features across channels, and a training paradigm based on masked autoencoders, are key innovations. ConvNeXt V2 has demonstrated competitive performance with ViTs on a variety of benchmarks and has favorable computational properties for medical imaging applications.

3. Proposed Method

This section presents the proposed framework in full. We begin with the clinical and technical motivation (Section 3.1), then give an overview of the four-component pipeline (Section 3.2). The subsequent subsections describe each component in turn: the ConvNeXt V2 backbone with CBAM attention (Section 3.3), the Adaptive Weighted Fusion Neck (Section 3.4), the Hybrid Representation Head (Section 3.5), and the classifier (Section 3.6), followed by the training objective (Section 3.7). Throughout, we explain how each module contributes to both tissue classification and content-based retrieval from a single shared descriptor.

3.1. Motivation

In the histopathology of lung adenocarcinoma, the tissue classification method requires the identification of patterns at different spatial resolutions: cellular morphology at fine scales, tissue architecture at intermediate scales, and structural organization at coarse scales. The four LUAD tissue classes—normal alveolar tissue, stromal connective tissue, tumor epithelial tissue, and tumor-associated stroma—share an overlapping visual appearance, so single-scale contextual reasoning is insufficient.

In addition to classification, a clinical support system must also retrieve similar past cases for comparison-based diagnosis and provide explainable visualizations that instill confidence in the clinician. These needs influence our design: a single multi-scale system with hybrid feature representations, as outlined in the rest of this section.

3.2. Overall Architecture

The proposed framework consists of four principal components arranged in a sequential pipeline, as illustrated in Figure 1:

• Backbone: A ConvNeXt V2 network with integrated CBAM attention modules for multi-scale feature extraction from four stages.
• Neck: An Adaptive Weighted Fusion module that aggregates multi-scale features using learnable softmax-normalized weights.
• Head: A Hybrid Representation module that produces a rich 18,496-dimensional feature descriptor through three complementary branches.
• Classifier: A lightweight neural network that maps the feature descriptor to class probabilities.

Figure 1. Overall architecture of the proposed framework. The input image is processed by a ConvNeXt V2 backbone with CBAM attention to produce multi-scale features $\{{\mathit{F}}_1,{\mathit{F}}_2,{\mathit{F}}_3,{\mathit{F}}_4\}$. The Adaptive Weighted Fusion Neck aggregates these features using learnable softmax weights. The Hybrid Representation Head produces an 18,496-dimensional descriptor through three branches: global (2048-dim), spatial (16,384-dim), and attention-weighted (64-dim). The descriptor is used for both classification and FAISS-based retrieval.

Figure 1. Overall architecture of the proposed framework. The input image is processed by a ConvNeXt V2 backbone with CBAM attention to produce multi-scale features $\{{\mathit{F}}_1,{\mathit{F}}_2,{\mathit{F}}_3,{\mathit{F}}_4\}$. The Adaptive Weighted Fusion Neck aggregates these features using learnable softmax weights. The Hybrid Representation Head produces an 18,496-dimensional descriptor through three branches: global (2048-dim), spatial (16,384-dim), and attention-weighted (64-dim). The descriptor is used for both classification and FAISS-based retrieval.

The feature descriptor from the Head serves dual purposes: it is used by the Classifier for tissue classification and by a FAISS-based index for content-based image retrieval. The complete end-to-end training and evaluation pipeline of the proposed framework is illustrated in Figure 2.

3.3. Backbone: ConvNeXt V2 with CBAM Attention

We employ ConvNeXt V2 [24] as our backbone network, configured with four stages of depths [2, 2, 6, 2] and channel dimensions [64, 128, 256, 512]. The network processes an input image of size 224 × 224 × 3 through a stem layer (4 × 4 convolution with stride 4) followed by four hierarchical stages, producing feature maps at progressively lower spatial resolutions.

Each ConvNeXt V2 segment includes a transformation by depth (7 × 7 kernel), layer normalization, inverse stem expansion (4×), and a global response normalization (GRN) layer. The GRN layer, unique to ConvNeXt V2, normalizes features based on their global ${\mathcal{l}}_2$ statistics:

$$\mathrm{GRN}\left(X\right)=\gamma ·\left(X·{\displaystyle \frac{\parallel X{\parallel }_2}{\mathrm{mean}\left(\parallel X{\parallel }_2\right)+ϵ}}\right)+\beta +X$$

(1)

where γ and β are learnable parameters and $\parallel ·\parallel$_2 denotes the l_2 norm across spatial dimensions. This normalization promotes feature diversity across channels, which is useful for capturing images of the different morphological styles present in histopathological images.

We integrate the Convolutional Block Attention Module (CBAM) [22] after each backbone stage to enhance feature representation through sequential channel and spatial attention.

• Channel Attention:

$$M_c\left(F\right)=\sigma \left(\mathrm{MLP}\left(\mathrm{AvgPool}\left(F\right)\right)+\mathrm{MLP}\left(\mathrm{MaxPool}\left(F\right)\right)\right)$$

(2)

where $\sigma$ is the sigmoid function, MLP is a shared multi-layer perceptron with a reduction ratio of 16, and AvgPool and MaxPool are global pooling operations.

• Spatial Attention:

$$M_s\left(F\right)=\sigma \left({\mathrm{Conv}}_{7\times 7} \left(\left[{\mathrm{AvgPool}}_c\left(F\right)\hspace{0.33em};\hspace{0.33em}{\mathrm{MaxPool}}_c\left(F\right)\right]\right)\right)$$

(3)

where ${\mathrm{AvgPool}}_c$ and ${\mathrm{MaxPool}}_c$ operate across the channel dimension, and ${\mathrm{Conv}}_{7\times 7}$ is a $7\times 7$ convolution.

The CBAM-enhanced feature at stage i is computed as:

$${\widehat{F}}_i=M_s\left(F_i^{\prime }\right)\odot F_i^{\prime }, \mathrm{where} F_i^{\prime }=M_c\left(F_i\right)\odot F_i$$

(4)

The backbone produces four CBAM-enhanced feature maps: ${\widehat{\mathit{F}}}_1\in {\mathbb{R}}^{64\times {\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}}$, ${\widehat{\mathit{F}}}_2\in {\mathbb{R}}^{128\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$, ${\widehat{\mathit{F}}}_3\in {\mathbb{R}}^{256\times {\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}}$, and ${\widehat{\mathit{F}}}_4\in {\mathbb{R}}^{512\times {\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}}$.

3.4. Neck: Adaptive Weighted Fusion

Unlike the BiFPN-inspired architecture in LGFFEM [11], which uses multiple aggregation nodes with fixed fusion patterns, we propose a simpler yet effective Adaptive Weighted Fusion mechanism with fully learnable weights.

Lateral Projections. Each backbone feature map is first projected to a common channel dimension $C_{\mathrm{out}}=256$ using $1\times 1$ convolutions:

$$L_i=\mathrm{GELU} \left(\mathrm{BN} \left({\mathrm{Conv}}_{1\times 1}\left({\widehat{F}}_i\right)\right)\right), i\in \left\{1,2,3,4\right\}$$

(5)

Adaptive Fusion. All projected features are resized to a common spatial resolution (${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$) using bilinear interpolation, then combined using learnable weights:

$$w=\mathrm{softmax}\left(\left[w_1,w_2,w_3,w_4\right]\right)$$

(6)

$$F_{\mathrm{fused}}={\displaystyle \sum _{i=1}^4}{w}_i·\mathrm{Resize}\left(L_i\right)$$

(7)

The softmax normalization ensures that the fusion weights are non-negative and sum to one, providing an interpretable measure of each scale’s contribution. During training, the model learns to assign higher weights to the most informative scales for the specific dataset and task.

Implementation and interpretation of the learned weights. The fusion weights are a single learnable vector w = [w_1, w_2, w_3, w_4] that is optimized jointly with all other network parameters by back-propagation of the classification loss; no auxiliary supervision is applied to them. Equation (6) maps the four raw logits to a probability simplex through the softmax, and Equation (7) applies them to the four lateral-projected, spatially-aligned feature maps. The weights are global (shared across all images) and are therefore fixed after training; they are not predicted per sample. On WSSS4LUAD, the converged weights are w_1 = 0.232 (stage 1, 1/4 resolution), w_2 = 0.262 (stage 2, 1/8), w_3 = 0.287 (stage 3, 1/16), and w_4 = 0.218 (stage 4, 1/32). The distribution is relatively flat—every scale receives 22–29% of the weight—which shows that no single resolution dominates and confirms that genuinely multi-scale information is exploited. The largest weight is assigned to the intermediate stage (stage 3), consistent with mid-level tissue-architecture cues being the most discriminative for the four LUAD classes, while the coarsest stage (stage 4) receives the smallest weight.

Output Refinement. The fused feature map is refined through a $3\times 3$ convolution:

$$F_{\mathrm{neck}}=\mathrm{GELU} \left(\mathrm{BN} \left({\mathrm{Conv}}_{3\times 3}\left(F_{\mathrm{fused}}\right)\right)\right)$$

(8)

where ${\mathit{F}}_{\mathrm{neck}}\in {\mathbb{R}}^{256\times 7\times 7}$ is the output of the neck module for a $224\times 224$ input.

3.5. Head: Hybrid Representation

Inspired by the observation that different types of information are encoded at different spatial granularities, we design a three-branch Hybrid Representation Head that captures complementary features.

Branch 1—Global Features (${\mathit{f}}_g\in {\mathbb{R}}^{2048}$): Global features capture holistic semantic information through adaptive average pooling:

$$f_g=\mathrm{Dropout}\left(\mathrm{GELU}\left(\mathrm{LN}\left(W_g·\mathrm{GAP}\left(F_{\mathrm{neck}}\right)+b_g\right)\right)\right)$$

(9)

where GAP is global adaptive average pooling, ${\mathit{W}}_g\in {\mathbb{R}}^{2048\times 256}$ is a linear projection, LN is layer normalization, and the dropout rate is 0.1.

Branch 2—Spatial Features (${\mathit{f}}_s\in {\mathbb{R}}^{16384}$): Spatial features preserve the layout information through grid pooling at an $8\times 8$ resolution:

$$f_s=\mathrm{Dropout}\left(\mathrm{GELU}\left(\mathrm{LN}\left(W_s·\mathrm{Flatten}\left({\mathrm{GP}}_{8\times 8}\right(F_{\mathrm{neck}}\left)\right)+b_s\right)\right)\right)$$

(10)

where ${\mathrm{GP}}_{8\times 8}$ performs adaptive average pooling to an $8\times 8$ spatial grid, producing a $256\times 8\times 8=\mathrm{16,384}$-dimensional vector after flattening. This branch preserves the spatial distribution of features. This branch facilitates the extraction of the structural patterns characteristic of tissue architecture.

Branch 3—Attention-Weighted Features (${\mathit{f}}_a\in {\mathbb{R}}^{64}$): Attention- based features highlight the most diagnostically relevant regions through a learned attention mechanism:

$$A=\sigma \left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{GELU}\left({\mathrm{Conv}}_{1\times 1}\left(F_{\mathrm{neck}}\right)\right)\right)\right)$$

(11)

$$f_a=\mathrm{Dropout}\left(\mathrm{GELU}\left(\mathrm{LN}\left(\mathrm{GAP}\left({\mathrm{Conv}}_{1\times 1}\left(F_{\mathrm{neck}}\right)\odot A\right)\right)\right)\right)$$

(12)

where $\mathit{A}\in {\mathbb{R}}^{1\times H\times W}$ denotes a spatial attention map that assigns importance weights to individual spatial locations.

Feature Concatenation. The final image descriptor is formed by concatenating all three branches:

$$f=\left[f_g\hspace{0.33em};\hspace{0.33em}{f}_s\hspace{0.33em};\hspace{0.33em}{f}_a\right]\in {\mathbb{R}}^{18496}$$

(13)

where $\mathrm{18,496}=2048+\mathrm{16,384}+64$. The spatial features constitute the dominant component at 88.6% of the total dimensionality, reflecting the critical importance of preserving spatial layout information for histopathological analysis, where tissue architectures differ primarily in structural organization. The global features (11.1%) capture holistic semantic content, while the compact attention-weighted features (0.3%) provide a focused signal highlighting diagnostically relevant regions. This asymmetric design is intentional: the high spatial dimensionality enables fine-grained retrieval of structurally similar tissue patterns. The descriptor serves both for classification (through the classifier head) and for retrieval (through FAISS indexing).

Justification of the descriptor size and the accuracy–efficiency trade-off. The 18,496-dimensional descriptor is a deliberate design choice rather than an arbitrarily large embedding. The dimensionality is fully determined by the architecture: the 8 × 8 grid-pooled spatial branch yields 256 × 8 × 8 = 16,384 values, the global branch 2048, and the attention branch 64. We retain the full spatial branch because, unlike object-centric natural images, histopathological classes are distinguished mainly by the spatial arrangement of glands, stroma, and epithelium; aggressive pooling of this branch removes exactly the layout cues that the retrieval task relies on, as confirmed by the ablation in which removing the spatial branch lowers accuracy (Section 5.3). In terms of cost, each descriptor occupies roughly 72 KB in float32, so the entire 8070-patch training archive requires under 0.6 GB and is indexed with FAISS IVFFlat (100 clusters, 10 probes) with sub-millisecond query latency on a single GPU; storage and search are therefore not bottlenecks at this dataset scale. Nevertheless, for very large or whole-slide archives the descriptor can be compressed without retraining the backbone—e.g., by PCA or product-quantization of the spatial branch, by binary hashing for approximate search, or by learning a compact projection head—and a systematic study of this accuracy-versus-compactness trade-off is identified as future work (Section 6 and Section 7).

3.6. Classifier

The classification head maps the 18,496-dimensional descriptor to class probabilities:

$$\widehat{y}=W_2·\mathrm{Dropout}\left(\mathrm{GELU}\left(\mathrm{LN}\left(W_{1}f+b_1\right)\right)\right)+b_2$$

(14)

where ${\mathit{W}}_1\in {\mathbb{R}}^{512\times 18496}$ and ${\mathit{W}}_2\in {\mathbb{R}}^{4\times 512}$. Layer normalization and GELU activation provide non-linearity, while dropout (rate 0.1) prevents overfitting.

3.7. Loss Function

We train the model using cross-entropy loss with label smoothing:

$$\mathcal{L}=-{\displaystyle \sum _{c=1}^C}{q}_c\mathrm{l}\mathrm{o}\mathrm{g}\left(p_c\right)$$

(15)

where $q_c=(1-\epsilon )·y_c+\epsilon /C$ is the smoothed target distribution, $y_c$ is the one-hot encoded ground truth, $p_c$ is the predicted probability for class $c$, $C=4$ is the number of classes, and $\epsilon =0.05$ is the smoothing parameter. Label smoothing prevents overconfident predictions and improves model calibration, which is particularly important in clinical applications.

4. Experiments

4.1. Dataset: WSSS4LUAD

We evaluate our framework on the WSSS4LUAD dataset [25], a publicly available benchmark for lung adenocarcinoma histopathology. The dataset consists of 10,087 H&E-stained patches extracted from 67 whole-slide images, organized into four tissue classes. The class distribution is shown in

Table 1. Distribution of tissue classes in the WSSS4LUAD dataset.

Class	Samples	Percentage
Normal	1832	18.16%
Stroma	1680	16.66%
Tumor	1181	11.71%
Tumor-Stroma	5394	53.48%
Total	10,087	100%

and visualized in Figure 3.

The dataset exhibits significant class imbalance, with Tumor-Stroma comprising over half of all samples and Tumor being the least represented class. All patches are resized to $224\times 224$ pixels for model input. We use an 80/20 random split (seed = 42), giving 8070 training and 2017 validation patches.

Dataset protocol and tissue-class definitions. WSSS4LUAD is a public Grand-Challenge benchmark of hematoxylin-and-eosin (H&E)-stained lung-adenocarcinoma tissue digitized from whole-slide images. The 10,087 image-level patches we use are organized into four tissue categories: Normal (healthy alveolar tissue, 1832 patches, 18.16%), Stroma (tumor-free connective/supporting tissue, 1680 patches, 16.66%), Tumor (malignant adenocarcinoma epithelium, 1181 patches, 11.71%), and Tumor-Stroma (mixed regions containing both tumor epithelium and tumor-associated stroma, 5394 patches, 53.48%). All patches are H&E-stained and standardized to 224 × 224 pixels; per-channel ImageNet normalization is applied, and the color augmentation described in Section 4.4 is used during training to reduce sensitivity to staining variation. The patches originate from 67 whole-slide images released by the challenge organizers.

Regarding data leakage, we acknowledge an important limitation of the present protocol: because the public release provides image-level patches rather than slide identifiers for every patch, our 80/20 split is performed at the patch level rather than the patient or slide level. Patch-level splitting can in principle allow patches from the same slide to appear in both the training and validation sets, which may optimistically bias performance in histopathology. We mitigate this by (i) reporting 5-fold stratified cross-validation (Section 4.7), which shows low variance (83.35 ± 0.93%) and therefore limited split sensitivity, and (ii) evaluating cross-organ generalization on a fully held-out external dataset (CRC-VAL-HE-7K, Section 5.5) whose slides never appear in training. A strict slide-level or patient-level split is identified as important future work, and we explicitly flag this point in the limitations (Section 7).

4.2. Datasets Comparison

We employ two complementary datasets in our study: WSSS4LUAD as the primary in-domain benchmark and NCT-CRC-HE-100K for external cross-organ validation.

Table 2. Side-by-side comparison of the datasets used in this study.

Property	WSSS4LUAD	NCT-CRC-HE
Biological characteristics
Organ	Lung	Colon
Tissue type	Adenocarcinoma	Colorectal cancer
Staining	H&E	H&E
Patch size	$224\times 224$	$224\times 224$
Source WSIs	67	136 [26]
Task properties
Classification task	4-class	9-class
Class labels	Normal, Stroma,	ADI, BACK, DEB, LYM,
	Tumor, Tumor-Stroma	MUC, MUS, NORM, STR, TUM
Class imbalance	High (53%:8%)	Moderate (14%:9%)
Dataset sizes
Total patches	10,087	107,180
Training set	8070 (80%)	100,000 (NCT-CRC-HE-100K)
Validation set	2017 (20%)	7180 (CRC-VAL-HE-7K)
Role in our study
Usage	In-domain	External validation
Task complexity	Lower (classes)	Higher (classes)
Source	Train + test	Separate held-out
Reference	[25]	[26]

provides a side-by-side comparison of their key properties.

As Table 2 shows, the two datasets are complementary: WSSS4LUAD is utilized to develop and optimize the proposed architecture on a moderately sized, class-imbalanced four-class lung-tissue dataset. In contrast, NCT-CRC-HE-100K is employed as an external benchmark, characterized by a different organ domain (colon vs. lung), a large number of classes (nine vs. four), and substantially greater scale (approximately ten times more training patches). This combination allows us to test both fine-grained classification ability on the main task and cross-organ transferability of the learned representations. Figure 4 provides an overview of the datasets used in our study.

4.3. Implementation Details

Our framework is implemented in PyTorch 2.7.0 and Torch version 0.22.0. The ConvNeXt V2 backbone is initialized with ImageNet pre-trained weights [24]. We use the AdamW optimizer [27] with an initial learning rate of $1\times {10}^{-4}$, weight decay of 0.01, and a cosine annealing learning rate schedule with 5 warmup epochs (linear warmup from $1\times {10}^{-7}$) and a minimum learning rate of $1\times {10}^{-6}$. Training is conducted for 150 epochs with a batch size of 16 using mixed-precision training (AMP). The total number of trainable parameters is 287.96 M, with the majority residing in the ConvNeXt V2 backbone and the spatial branch of the Hybrid Representation Head. While the model size is substantial compared to lightweight architectures, such as LGFFEM (14.5 M parameters) [11], the use of mixed-precision training (AMP) reduces GPU memory requirements. Training is performed on an NVIDIA GeForce RTX 5070 Laptop GPU, requiring approximately 240.6 min. Future work will explore knowledge distillation and architectural pruning to reduce computational requirements for resource-constrained clinical deployment.

For image retrieval, we employ the FAISS library [28] with an IVFFlat index (100 clusters, 10 probes) using the 18,496-dimensional feature descriptors. The retrieval evaluation uses the validation set as the query set and the training set as the database.

4.4. Data Augmentation

Training images are augmented with the following transformations, as illustrated in Figure 5:

• Geometric: rotation ($\pm {15}^{\circ }$), horizontal and vertical flips (50% probability each), translation ($\pm 10\%$), zoom ($0.9$–$1.1\times$).
• Color: brightness, contrast, and saturation jitter ($\pm 15\%$ each in HSV color space).
• Regularization: random erasing (15% probability, 2–10% of image area).
• Normalization: ImageNet mean $\left[\mathrm{0.485,0.456,0.406}\right]$ and standard deviation $\left[\mathrm{0.229,0.224,0.225}\right]$.

Figure 5. Examples of data augmentation applied to training images. Augmentations include geometric transformations (rotation, flipping, translation), color jitter, and random erasing.

Figure 5. Examples of data augmentation applied to training images. Augmentations include geometric transformations (rotation, flipping, translation), color jitter, and random erasing.

Validation images undergo only resizing and normalization without augmentation.

4.5. Evaluation Metrics

We report the following metrics:

• Classification: Accuracy, Precision, Recall, F1-score (macro and weighted), Cohen’s kappa ($\kappa$), Matthews Correlation Coefficient (MCC), balanced accuracy.
• Medical accuracy [29]: patch-level accuracy ${\eta }_p$, whole-scan accuracy ${\eta }_w$, and total accuracy ${\eta }_{\mathrm{tot}}={\eta }_p\times {\eta }_w$.
• Retrieval: Mean Average Precision (MAP), Mean Reciprocal Rank (MRR), Precision@K, Recall@K, NDCG@K.
• Calibration: Expected Calibration Error (ECE), Brier score.
• Explainability: Grad-CAM visualizations on the last backbone stage.

4.6. Cross-Validation Analysis

We perform a 5-fold stratified cross-validation experiment to test the statistical robustness and rule out split-dependent bias. For the full model, each fold is trained separately with the same hyperparameters. Results per-fold and total are given in

Table 3. 5-Fold stratified cross-validation results on WSSS4LUAD.

Fold	Acc (%)	F1macro	Precmacro
Fold 1	83.40	0.7733	0.8134
Fold 2	84.74	0.7884	0.8417
Fold 3	83.39	0.7780	0.8200
Fold 4	83.34	0.7731	0.8099
Fold 5	81.90	0.7457	0.7949
Mean ± Std	83.35 ± 0.93	0.7717 ± 0.015	0.8160 ± 0.016

.

The cross-validation results do not show any instability of the model as the standard deviation of the accuracy across folds (0.93%) is quite low, indicating that the model is not sensitive to the partitioning of the data. Four of the five folds achieve accuracy above 83%, and the mean (83.35%) is very close to the single-split accuracy (85.03%), confirming that the original evaluation is representative of the performance. The results per fold are visualized in Figure 6, along with the mean and $\pm 1\sigma$.

The results of the 5-fold stratified cross-validation are shown in Figure 6: The dashed orange line shows the accuracy of 83.35%, the shaded area shows the standard deviation of 0.93%, with high consistency across different data partitions.

4.7. Training Dynamics

Figure 7 presents the training behavior of the proposed model.

After more than 150 epochs, the model is converging, the training loss is reducing from 0.9745 to 0.3879 and the training accuracy is improving from 64.99% to 93.46%. The validation accuracy is 86.66% at epoch 98, and after that, the validation accuracy fails to improve while the training accuracy keeps rising, suggesting that the model has moderate overfitting. The final generalization gap of $~8$ percentage points indicates that better regularization techniques or more training data could yield better generalization.

The learning rate decays smoothly with cosine annealing schedule with warmup from $1\times {10}^{-7}$ in 5 epochs to peak learning rate $1\times {10}^{-4}$ and slowly goes down to $1\times {10}^{-6}$. The generalization gap (the difference between training and validation accuracy) remains at around 7–8% after epoch 100, which indicates that this model has learned its maximum from the dataset it is being trained on, and further increasing the size of the dataset or implementing suitable regularization techniques could further enhance generalization.

5. Results

This section presents the experimental results obtained from the proposed CBHIR framework using multiple evaluation perspectives. The performance of the proposed model was assessed through classification accuracy, retrieval effectiveness, architectural comparisons, computational efficiency, and external validation experiments. In addition, ablation studies and per-class analyses were conducted to investigate the contribution of different model components and evaluate the robustness of the framework across various histopathological categories.

5.1. Classification Performance

The overall classification performance on the WSSS4LUAD validation set is reported in

Table 4. Overall classification performance on the WSSS4LUAD validation set.

Metric	Value
Accuracy	85.03%
Precision (macro)	0.8240
Recall (macro)	0.8055
F1-Score (macro)	0.8116
F1-Score (weighted)	0.8502
Cohen’s Kappa ($\kappa$)	0.7689
Weighted Kappa	0.8841
MCC	0.7696
Balanced Accuracy	80.55%
Top-3 Accuracy	99.60%
Log Loss	0.4262
Brier Score	0.2226
ECE	0.0378
Youden’s Index	0.7449
Diagnostic Odds Ratio	64.25

.

5.2. Comparison with Baseline Architectures

To evaluate the performance of the proposed model, it was compared with four widely-used baseline architectures trained under identical experimental conditions (AdamW optimizer, cosine annealing, $\mathrm{lr}={10}^{-4}$, AMP, 80/20 split). The results are summarized in

Table 5. Comparison with baseline architectures on WSSS4LUAD.

Model	Params (M)	Acc (%)	F1macro	Precmacro	Recmacro
ResNet-50	24.56	90.18	0.8732	0.8854	0.8634
EfficientNet-B0	4.67	91.18	0.8924	0.8894	0.8966
DenseNet-121	7.48	91.03	0.8896	0.8854	0.8939
ConvNeXt-V2 (vanilla)	8.82	85.28	0.8020	0.8435	0.7826
Ours	287.96	85.03	0.8077 †	0.8343	0.7937

^† The model was re-trained with early stopping to obtain fair comparison, and the full 150-epoch model gets F1_macro $=0.8116$ (Table 4).

.

The lightweight models (ResNet-50, EfficientNet-B0, DenseNet-121) perform better on this accuracy (90–91%) than our model (85.03%). The difference is mainly due to our model having a much higher number of parameters: 287.96M as compared to 4.67–24.56M, resulting in some moderate overfitting on the limited training set (8070 samples). Importantly, our proposed Adaptive Fusion Neck and Hybrid Head do not degrade performance, as the vanilla ConvNeXt-V2 backbone is able to achieve 85.28% accuracy. The model proposed here has three benefits: First, it can generate class labels as the baselines do, but also an 18,496-dimensional feature descriptor for content-based retrieval (MAP $=0.8323$, MRR $=0.8784$), second, it can provide calibrated probabilities (ECE $=0.0378$), and third, it can provide Grad-CAM explainability—all of which the baselines are missing.

To demonstrate the cross-organ generalization, we compared all the models in Figure 8 by accuracy, F1 macro, and external validation on NCT-CRC-HE-100K CRC-VAL-HE-7K.

5.3. Ablation Study

To show the significance of each proposed component, we perform a systematic ablation study by systematically removing individual components while preserving the remaining architecture. The hyperparameters used in training are the same for all variants. The results are given in

Table 6. Ablation study on the WSSS4LUAD validation set.

Variant	Acc (%)	F1mac	ΔAcc	Params (M)
Full model (ours)	85.03	0.8077	—	0.8077
w/o CBAM	84.09	0.7982	−0.94	0.7982
w/o Adaptive Fusion	83.69	0.79^ *	−1.34	0.79^ *
w/o Spatial Branch	83.69	−1.34	−1.34	−1.34
Global features only	83.44	0.7825	−1.59	0.7825
w/o Attention Branch	82.90	0.7794	−2.13	0.7794

* Estimated from the saved checkpoint due to the interruption of the training before the final evaluation.

.

The ablation results reveal several insights. The advantage of all proposed components is positive, in the sense that the accuracy decreases by about 0.94–2.13% when any one of the modules is removed. Second, the most effective component is the Attention Branch ($\mathsf{\Delta }=-2.13\%$, F1 drops from 0.8077 to 0.7477), suggesting that attention weighted features contain discriminative information that is important for the classification task. Third, the smallest contribution comes from the CBAM module ($\mathsf{\Delta }=-0.94\%$), indicating that some channel–spatial relationship is already picked up in the backbone. Fourth, all the components are complementary, as the combined effect is greater than the sum of its parts, suggesting there is complementary information captured. The layout is well balanced—there is no single module that dominates performance. The incremental contribution of each component is visualized in Figure 9.

To isolate the source of the performance gain, the six configurations in Table 6 progressively decompose the architecture along the axes requested for a thorough ablation. The “Global-features-only” variant approximates the plain backbone-plus-global-pooling baseline (descriptor reduced to 2048-D), removing both the spatial and attention branches; “w/o CBAM” isolates the contribution of the backbone attention; “w/o Adaptive Fusion” replaces the learnable softmax weights with uniform averaging, testing the value of learnable fusion; “w/o Spatial Branch” and “w/o Attention Branch” remove the corresponding head components and, in doing so, reduce the descriptor dimensionality, so they double as lower-dimensional-descriptor variants. Read together, these results show that the gain is not attributable to a single factor: the attention-weighted features contribute most (Δ = −2.13%), the learnable multi-scale fusion and the spatial branch each add roughly 1.3%, and backbone CBAM adds a further 0.9%, with the full model exceeding every reduced variant. A fully factorial study (e.g., backbone-plus-CBAM in isolation and intermediate descriptor sizes obtained by PCA) is left for future work.

5.4. Inference Efficiency

Table 7. Inference efficiency comparison (GPU, BatchSize 16).

Model	Params (M)	Size (MB)	GFLOPs	ms/img	img/s
ResNet-50	24.56	98.45	4.11	15.14	66.0
EfficientNet-B0	4.67	18.83	0.40	5.76	173.7
DenseNet-121	7.48	30.26	2.87	15.43	64.8
ConvNeXt-V2 (vanilla)	8.82	35.27	1.37	9.04	110.7
Ours	287.96	1151.83	1.78	14.66	68.2

shows the computational efficiency of all models evaluated. Even though it has a large number of parameters, the proposed model achieves competitive throughput and the second lowest GFLOPs compared to all architectures, thanks to the efficient ConvNeXt-V2 design.

The efficiency/accuracy trade-off is shown in a bubble chart in Figure 10, with the size of the bubbles indicating the number of parameters used. Even with the largest model size, our framework delivers competitive GFLOPs (1.78) and offers further capabilities (retrieval, calibration, explainability) that are not included in any of the baselines.

The overall agreement between the model and the human experts is 85.03% with Cohen’s kappa of 0.7689, which is considered substantial agreement. If class ordering is taken into account, the agreement level is further improved with a weighted kappa of 0.8841. Of particular interest, the top 3 accuracy is 99.60%, meaning the correct class is very likely to be among the top 3 predictions of the model. The ECE of 0.0378 validates a well-calibrated probability estimate, a key element for reliable clinical decision support.

All reported metrics are at the final model checkpoint (epoch 150). Although the validation accuracy of the best epoch (epoch 98) was 86.66%, the final checkpoint is reported to ensure consistent and reproducible evaluation with all metrics, as discussed in Section 5.5.

5.5. External Dataset Validation

To assess generalizability beyond WSSS4LUAD, we evaluate all architectures on the NCT-CRC-HE-100K colorectal tissue dataset [26], reflecting the rapid adoption of artificial intelligence for colorectal cancer analysis [30]. This benchmark contains 100,000 training patches and a held-out validation set of 7180 patches (CRC-VAL-HE-7K), spanning 9 histologically distinct tissue classes (ADI, BACK, DEB, LYM, MUC, MUS, NORM, STR, TUM). To ensure a scientifically rigorous and fair comparison, all four models—including our proposed architecture—are trained on the same stratified 10.8K subsample of NCT-CRC-HE-100K (1200 images per class, identical random seed), and evaluated on the same CRC-VAL-HE-7K validation set. Each model is trained for up to 10 epochs with early stopping (patience = 4–5) and AdamW optimizer (lr = $1\times {10}^{-4}$, weight decay = 0.01).

Table 8. Fair external validation: all models trained on 10.8K subsample, evaluated on CRC-VAL-HE-7K (9 Classes).

Model	Train Data	Acc (%)	Bal. Acc	F1mac
ResNet-50	10.8 K	95.25	0.9321	0.9317
EfficientNet-B0	10.8 K	95.45	0.9309	0.9336
DenseNet-121	10.8 K	94.83	0.9230	0.9263
Ours	10.8 K	85.79	— $†$	— $†$

$†$ Per-class metrics unavailable due to memory constraints during evaluation; accuracy reported from validation during training (epoch 7 best checkpoint).

presents the fair comparison where all models are trained on the same 10.8 K subsample. Under the same experimental setting, the three lightweight baselines get an accuracy of 94.83–95.45%, which are higher than our model achieving 85.79%, with a gap of 9.66%. This is a more scientifically valid comparison than

Table 9. Reference results: baselines trained on full 100K NCT-CRC-HE set (Ours remains at 10.8K).

Model	Train Data	Acc (%)	Bal. Acc	F1mac
ResNet-50	100 K	95.24	0.9348	0.9336
EfficientNet-B0	100 K	96.95	0.9539	0.9569
DenseNet-121	100 K	94.97	0.9300	0.9321
Ours	10.8 K	85.79	—	—

, which gives baselines a 9.3$\times$ training-data advantage. Even under the fair protocol, the performance gap persists because (i) our 287.96 M-parameter model is 19–62$\times$ larger than the baselines (14.5–24.5 M parameters) and is more data-hungry, and (ii) our architecture was specifically optimized for the 4-class WSSS4LUAD problem rather than 9-class colorectal tissue classification. Importantly, the absolute performance of 85.79% still confirms meaningful cross-organ generalization from lung to colon tissue. With training on the full 100 K set (Table 9) or GPU-accelerated fine-tuning, we expect our model to match or exceed baseline performance.

In response to the reviewers’ request to retrain on the full NCT-CRC-HE-100K set, we attempted to train our 287.96 M-parameter model on all 100,000 patches. The run reached an out-of-memory condition on our 8 GB RTX-class laptop GPU before completing the second epoch, and we therefore report the best stable checkpoint obtained on the 10.8 K stratified subsample (85.79%). The lightweight baselines, being 19–62× smaller, train on the full 100 K set without difficulty, and their full-set results are reported in Table 9 for completeness. We are transparent that our cross-organ number reflects a 10.8 K-trained model and that a full-100 K retrain—feasible on a higher-memory GPU together with the gradient-checkpointing and descriptor-compression strategies discussed above—is expected to close much of the remaining gap. We retain this as a clearly stated limitation (Section 7) rather than over-claiming generalization.

5.6. Per-Class Analysis

The per-class performance is detailed in

Table 10. Per-class classification performance on the WSSS4LUAD validation set.

Class	Prec.	Rec.	F1	Sens.	Spec.	PPV	NPV
Normal	0.941	0.938	0.939	0.938	0.987	0.941	0.986
Stroma	0.801	0.872	0.835	0.872	0.957	0.801	0.974
Tumor	0.686	0.522	0.593	0.522	0.969	0.686	0.940
T-Stroma	0.869	0.890	0.879	0.890	0.845	0.869	0.870

, and the confusion matrix is shown in Figure 11.

The model demonstrates strong performance in identifying Normal tissue, achieving an F1-score of 0.9393 and a specificity of 0.9866, which may be associated with the distinct morphological characteristics of healthy alveolar tissue. The Tumor-Stroma class also achieves strong performance (F1 = 0.8791), likely influenced by its larger representation within the dataset. The Stroma class exhibits good recall (0.8724) but comparatively lower precision (0.8011), indicating partial overlap with other tissue categories.

The hardest case is the Tumor class with the lowest recall (0.5217) and F1-score (0.5926). This can be explained by two reasons. First: the Tumor is the smallest class with just 1181 samples (11.71%). The class imbalance ratio to the majority class is about 1:4.6, which is quite significant. Second: The Tumor tissue has morphological features that are shared with tumor-associated stroma, thus the boundary between both classes is inherently ambiguous. The specificity was 0.9692, and the model does not misclassify other tissues as tumors frequently. This is clinically safe in terms of false positive rates.

Table 11. Effect of class-balancing strategy on per-class performance.

Strategy	Acc (%)	F1mac	Bal. Acc	Tum. R	Tum. F1	T-S.R
Baseline (CE)	85.03	0.8077	0.8055	0.5217	0.5926	0.8901
Focal + Sampler	76.05	0.7257	0.7588	0.5609	0.5039	0.7294

CE = Cross-Entropy; Tum. R = Tumor Recall; T-S.R = Tumor-Stroma Recall.

summarizes the impact of the proposed class- balancing strategy on per- class performance. To address the low Tumor recall under standard cross-entropy training, we conducted a class-balancing experiment combining Focal Loss ($\gamma =2.0$) [18] with a Weighted Random Sampler that oversamples minority classes using inverse-frequency weights. As shown in Table 8, this strategy improves Tumor recall from 52.17% to 56.09% (+3.92 pp) while the balanced accuracy decreases from 80.55% to 75.88%. The overall accuracy decreases from 85.03% to 76.05%, reflecting the well-known accuracy–fairness trade-off: rebalancing shifts the decision boundary to favor minority classes at the expense of majority-class performance (Tumor-Stroma recall drops from 89.01% to 72.94%). The macro F1-score is 0.7257 versus 0.8077 for the baseline. These results demonstrate that explicit class-balancing can meaningfully improve Tumor detection, though at the cost of overall accuracy. In practice, the relative cost of missing tumors versus false alarms determines the choice of operating point, but a clinician-in-the-loop system that utilizes the top-3 accuracy of 99.60% can further diminish this limitation.

Figure 12 provides a detailed visualization of the class-balancing effect, showing per-class recall changes and the accuracy–fairness trade-off.

The per-class metrics radar chart is shown in Figure 13 and the sensitivity–specificity comparison is presented in Figure 14.

The ROC curves for each class are shown in Figure 15, and the precision–recall curves are presented in Figure 16.

5.7. Medical Accuracy Metrics

Following the medical accuracy formulation in [29], the patch-level accuracy is ${\eta }_p=85.03\%$. Since the current evaluation operates at the patch level without WSI-level majority voting, ${\eta }_w={\eta }_p$ and ${\eta }_{\mathrm{tot}}={\eta }_p\times {\eta }_w/100=72.30\%$. In a deployed clinical system with majority voting from the same WSI across patches, ${\eta }_w$ would be expected to be greater than ${\eta }_p$, as shown in [11] where ${\eta }_w$ (99.47%) is greater than ${\eta }_p$ (99.40%), to given ${\eta }_{\mathrm{tot}}=98.87\%$. WSI-level aggregation is important work to be done in the future.

The balanced accuracy of 80.55% is computed by taking the average of the per-class recall values, which is more representative of the performance across imbalanced classes. The ECE value of 0.0378 means that the model’s predicted probabilities are similar to the actual outcome frequencies, which is essential for trustworthy clinical decision support.

Reliability analysis and per-class calibration. To examine calibration in more detail, Figure 17 presents the reliability diagram and confidence histogram computed on the validation set with ten equal-width confidence bins. The model is calibrated intrinsically: we apply label smoothing (ε = 0.05, Equation (15)) during training and deliberately use no post hoc temperature scaling or Platt scaling, so the reported probabilities are those produced directly by the softmax. The reliability curve tracks the diagonal closely and yields an overall Expected Calibration Error (ECE) of 0.0377, with a small, characteristic over-confidence (mean confidence 89.1% versus 85.0% accuracy). Calibration is, however, not uniform across tissue classes. The per-class ECE is 0.047 for Normal, 0.021 for Stroma, 0.030 for Tumor-Stroma, and 0.297 for the minority Tumor class. The Tumor class is therefore markedly less well calibrated than the others, mirroring its low recall (52.17%) and small sample size (11.71%); the network is over-confident precisely where it is least accurate. This finding reinforces our class-imbalance analysis (Section 5.6) and is clinically important, because reliable confidence estimates for the malignant class are the most consequential. It motivates class-conditional calibration (e.g., per-class temperature scaling) as future work.

5.8. Architectural Comparison

Table 12. Architectural comparison: LGFFEM vs. ours.

Component	LGFFEM	Ours
Backbone	ConvNeXt V2 (Base)	ConvNeXt V2 (Base) + CBAM
Neck	LGFFN (BiFPN + LA/GA)	Adaptive Weighted Fusion
Head	GeM pooling $\times$4	Hybrid (Global + Spatial + Attn)
Descriptor dim	$4\times C_{in}$	18,496
Loss	Sub-center ArcFace	CE + Triplet + Label Smooth.
Parameters	14.5 M	287.96 M
Training data	IN-1K + PanNuke + Kimia24C	WSSS4LUAD (10,087 patches)
Epochs	300	150
Batch size	64	16
Optimizer	AdamW + Cosine	AdamW + Cosine + Warmup
Hardware	RTX 6000 ADA, 64 GB	RTX 4060, 8 GB
Classification	—	(85.03%)
Calibration	—	(ECE $=0.0378$)
Retrieval		(MAP $=0.8323$)
Explainability	Grad-CAM (neck only)	Grad-CAM (full model)

contrasts the key design choices between LGFFEM and our framework.

The two methods are based on a similar ConvNeXt V2 backbone and fuse information from multiple scales, but have very different designs and scopes. LGFFEM focuses exclusively on retrieval: it extracts compact embeddings via GeM pooling, trained with ArcFace loss to maximize inter-class separation in embedding space. The BiFPN-inspired neck uses Local and Global Feature Aggregators with channel attention (squeeze-and-excitation style) and learnable fast-normalized fusion weights. Our framework is a dual-purpose system that simultaneously performs classification and retrieval. We augment the backbone with CBAM (channel + spatial attention), use an Adaptive Weighted Fusion neck with softmax-normalized learnable weights, and produce a high-dimensional 18,496-D Hybrid Representation that concatenates global, spatial, and attention-weighted features. This richer descriptor enables both calibrated classification (ECE $=0.0378$) and content-based retrieval (MAP $=0.8323$).

5.9. Retrieval Accuracy Comparison

Table 13. Retrieval accuracy comparison (${\eta }_p$, ${\eta }_w$, ${\eta }_{\mathrm{tot}}$).

Method	Dataset	Cls	Params
DenseNet-121	Kimia24C	24	7.98 M	95.92	95.51	91.62
MA+MS-loss	Kimia24C	24	—	97.89	97.00	94.95
LGFFEM-A	Kimia24C	24	14.5 M	72.08	74.37	53.60
LGFFEM-B	Kimia24C	24	14.5 M	77.36	79.28	61.33
LGFFEM-C	Kimia24C	24	14.5 M	99.40	99.47	98.87
Ours	WSSS4LUAD	4	287.96 M	85.03 †	85.03 †	72.03 †

^† Only patch level majority voting (${\eta }_w={\eta }_p$). LGFFEM-A/B/C = training strategies A (IN-1K), B (IN-1K+PanNuke), C (IN-1K+PanNuke+Kimia). Cls = number of classes.

compares retrieval accuracy using the medical accuracy metrics ${\eta }_p$, ${\eta }_w$, and ${\eta }_{\mathrm{tot}}$. Because LGFFEM used three progressive training strategies (A: ImageNet-1K only; B: +PanNuke; C: +Kimia Patch24C), we include all three to illustrate the effect of domain-specific pre-training.

From this comparison several conclusions can be drawn:

1.

Difficulty of the Dataset: LGFFEM-A, trained on ImageNet-1K without domain data, and performs at ${\eta }_p=72.08\%$ on Kimia Patch24C, which is lower than our 85.03% on WSSS4LUAD. This shows that our model is not only able to compete with the other baselines despite the lack of domain-specific pre-training on the target set, it also uses only ImageNet initialization.

2.

The importance of domain-specific training: LGFFEM increased from 72.08% (strategy A) to 99.40% (strategy C), the use of the target dataset in the training process is found to be important. This is not a direct comparison with LGFFEM-C because our model was not trained on Kimia Patch24C.

3.

Task granularity: Kimia Patch24C contains 24 visually distinct tissue types from diverse organs, where inter-class boundaries are relatively clear. WSSS4LUAD is a harder classification task per class due to its low class balance (1:4.6 ratio) for 4 morphologically similar lung adenocarcinoma classes.

4.

Multi-task advantage: Our framework yields the advantages of simultaneous classification (85.03%), calibrated probabilities (ECE $=0.0378$), retrieval (MAP $=0.8323$, MRR $=0.8784$), and explainability by Grad-CAM, which are critical requirements for the clinical deployment of our framework that are not provided by LGFFEM, and previous works.

Future work will assess our framework on Kimia Patch24C to allow for a direct comparison between the different test setup and investigate adding domain-specific pre-training (e.g., PanNuke) to help the accuracy gap.

The accuracy of retrieval and the system capabilities are visually compared in Figure 18 between LGFFEM and our framework.

5.10. Retrieval Performance

Table 14. Content-based image retrieval performance on the WSSS4LUAD dataset.

$\mathit{K}$	P@$\mathit{K}$	R@$\mathit{K}$	NDCG@$\mathit{K}$
1	0.8270	0.0520	0.8270
3	0.8169	0.1514	0.8810
5	0.8127	0.2495	0.8891
10	0.8065	0.4953	0.8948

Overall: MAP $=0.8323$, MRR $=0.8784$.

summarizes the content-based image retrieval performance. The retrieval metrics at different values of $K$ are also visualized in Figure 19.

The retrieval system has a Precision@1 value of 0.8270, with the 82.70% of queries having the most similar retrieved image in the correct tissue class. All query images rank well with an average of 0.8323. The MRR value is 0.8784, indicating that the first image retrieved is usually the highest one in the ranked list. The NDCG values, NDCG@K, rise monotonically from 0.8270 to 0.8948 with the increase of K, which shows that the retrieval ranking is relevant to the images. The relatively low Recall@$K$ values at small $K$ are expected, given the large number of relevant images in the database for the majority classes.

Figure 20 and Figure 21 show qualitative retrieval examples for each tissue class, where the query image is shown alongside its top-5 retrieved results.

6. Discussion

This section provides a comprehensive interpretation of the proposed framework’s performance. The analysis focuses on explainability, comparative behavior with existing CBHIR approaches, and the overall balance between retrieval accuracy, classification capability, and computational efficiency. Furthermore, the discussion highlights the strengths, limitations, and potential clinical relevance of the proposed system in real-world histopathological image retrieval applications.

6.1. Relation to Existing CBHIR Methods

Several content-based histopathological image retrieval (CBHIR) methods have been proposed in recent years. Babaie et al. [29] demonstrated the effectiveness of DenseNet-121 features for patch retrieval on the Kimia Patch24C benchmark, achieving ${\eta }_p=95.92\%$. Yang et al. [12] improved upon this with a multi-attention and multi-scale (MA+MS) approach, reaching ${\eta }_p=97.89\%$ on the same benchmark. Most recently, Núñez et al. [11] proposed the Local–Global Feature Fusion Embedding Model (LGFFEM), a lightweight 14.5M-parameter model built on ConvNeXt V2 with a BiFPN-inspired neck and GeM pooling head, trained with Sub-center ArcFace loss [31], achieving state-of-the-art ${\eta }_p=99.40\%$ on Kimia Patch24C. Since LGFFEM shares our ConvNeXt V2 backbone and multi-scale fusion philosophy, we provide a detailed architectural and performance comparison below.

6.2. Explainability Analysis with Grad-CAM

In line with the growing emphasis on explainable artificial intelligence for transparent and trustworthy cancer-detection systems [32], we provide interpretable insights into the model’s decision-making process, we apply Grad-CAM to the final backbone stage (backbone.stages. 3) for representative images from each class, as shown in Figure 22 and Figure 23.

In the case of Normal tissue samples, the Grad-CAM visualizations demonstrate broad focus over the alveolar spaces, suggesting that the model’s predictions are based on the overall structure of healthy tissue and not on specific cellular patterns. The stroma samples show a focus on the fibers of the dense connective tissue, which are especially dense in the areas of collagen bundles.

The activation maps show regions of unusual cells with nuclei that are not normally shaped for Tumor samples, indicating that the model has become proficient in recognizing the signatures of abnormal cells. As a result of the mixed nature of this tissue class, activations in Tumor-Stroma samples are distributed between the epithelial tumor component and the surrounding reactive stroma, as shown by the dual attention patterns. These visualizations validate the model to have learned pathologically meaningful features and give a tool for clinicians to check and believe the model’s prediction.

Validity and scope of the explainability analysis. We emphasize that the Grad-CAM maps in this study were assessed qualitatively by the authors against the expected histological appearance of each class—diffuse activation over alveolar spaces for Normal, collagen-fiber emphasis for Stroma, abnormal-nuclei clusters for Tumor, and dual epithelial/stromal attention for Tumor-Stroma—rather than through a formal reader study. The maps were not yet reviewed by board-certified pathologists, and we did not quantitatively compare the highlighted regions against pixel-level tumor annotations, because the patch-level WSSS4LUAD labels we use do not provide dense region masks. The present evidence that the model attends to diagnostically plausible structures should therefore be read as supportive but not confirmatory. A rigorous validation—measuring the overlap between Grad-CAM saliency and expert or weakly-supervised tumor annotations, and a pathologist-in-the-loop agreement study—is an important direction we identify for future work (Section 7), and would turn explainability from a visual aid into a quantitatively validated component.

6.3. Multi-Criteria Summary

All important quantitative results are consolidated on one dashboard for the proposed system, as shown in Figure 24, to give a complete overview of the strengths of the proposed system, which include classification, F1 scores, retrieval, calibration, external validation, efficiency trade-off, class balancing, k-fold stability, and ablation. In Figure 25, we complement this with a color-coded comparison of all methods and metrics so that it can be easily seen where our model is missing from the baselines.

Figure 26 and Figure 27 take a focused view: a multi-criteria radar chart comparing all evaluated models across six evaluation dimensions, and a capability matrix summarizing which features each model supports.

Our proposed framework is the only architecture with all nine of the evaluated capabilities namely classification, retrieval, calibration, Grad-CAM explainability, multi-scale fusion, attention mechanism, cross validation, class balancing and external validation, while the baseline architectures only offer 5–6% classification accuracy on WSSS4LUAD. The mix of these capabilities is ideal for clinical deployment, where the confidence in and the interpretability of predictions is as critical as the accuracy of the predictions themselves.

7. Conclusions

We now sum up and peer into the future. We developed and applied a single framework for the classification and retrieval of histopathological images, which we applied to the lung adenocarcinoma dataset WSSS4LUAD. The architecture incorporates a ConvNeXt V2 backbone with CBAM attention, an Adaptive Weighted Fusion Neck with learnable softmax-normalized weights and a Hybrid Representation Head that projects a 18,496-dim descriptor from the global, spatial and attention-weighted representations.

The following results are reported:

• An overall classification accuracy of 85.03% with a 5-fold cross-validation mean of 83.35 ± 0.93%, confirming statistical robustness on a four-class LUAD tissue classification task.
• A systematic ablation study confirming the contribution of each proposed module: the Attention Branch (Δ = −2.13%), Adaptive Fusion (Δ = −1.34%), and CBAM (Δ = −0.94%) all contribute positively.
• Competitive inference throughput (68.2 img/s, 1.78 GFLOPs) despite the large parameter count (287.96 M), with latency comparable to ResNet-50.
• Strong retrieval performance with MAP of 0.8323 and MRR of 0.8784, enabling effective content-based image search.
• Well-calibrated predictions with ECE of 0.0378, suitable for clinical decision support.
• A class-balancing experiment showing that Focal Loss with Weighted Random Sampling improves Tumor recall from 52.17% to 56.09%, illustrating the accuracy–fairness trade-off inherent in imbalanced histopathology datasets.
• Cross-organ generalization validated on NCT-CRC-HE-100K → CRC-VAL-HE-7K (9 colorectal tissue classes), where our model achieves 85.79% accuracy using only 10.8 K training images (vs. 100 K for baselines), confirming transferability of the learned representations across organs.

Combined, these results highlight a trade-off: the model’s accuracy on raw WSSS4LUAD is 5–6% lower than that of lightweight classifiers (EfficientNet-B0: 91.18%, DenseNet-121: 91.03%), but retrieval (MAP = 0.8323), calibration (ECE = 0.0378 Grad-CAM explainability and capabilities are descriptive features that none of the baselines can offer.

Although these results were found, several limitations were encountered. The first four are summarized below, followed by additional considerations raised during peer review:

• Large model size: Using 287.96 M parameters, the answer is almost 20 times higher than LGFFEM (14.5 M) and 60 times higher than EfficientNet-B0, which can be important for deployment on resource-constrained clinical devices.
• Class imbalance was partially mitigated: The combination of Focal Loss with Weighted Random Sampling slightly improved the Tumor recall rate from 52.17% to 56.09%, while decreasing the overall accuracy (85.03% → 76.05%). This gap can be further closed by more advanced strategies (e.g., curriculum learning, two-stage training).
• Limited external validation: Results with NCT-CRC-HE-100K → CRC-VAL-HE-7K (85.79%) indicate cross-organ generalization, but our model was trained on a subsample of 10.8 K, because of the computational needs.
• Patch-level evaluation: Evaluations were conducted only at the patch level, whereas majority voting at the WSI level would be required for clinical deployment.

Beyond these four, we acknowledge the following: (5) Single-dataset development—the architecture was developed and tuned on one in-domain benchmark (WSSS4LUAD); broader validation across multiple LUAD cohorts is needed. (6) Patch-level rather than slide-level splitting—because the public release does not expose slide identifiers for every patch; our split is at the patch level, which can allow same-slide patches to fall in both training and validation and may optimistically bias results; a strict patient/slide-level split is required for a fully leakage-free estimate. (7) Staining and scanner variability—H&E color and scanner characteristics differ across laboratories, and although color augmentation mitigates this, robustness to domain shift across institutions has not been quantified. (8) High descriptor dimensionality—the 18,496-D descriptor, while well-justified at this scale, may require compression (PCA, product quantization, or hashing) before deployment to whole-slide-sized archives. (9) Class-conditional reliability—calibration is good overall but degrades for the minority Tumor class (per-class ECE 0.297), so confidence estimates for malignant predictions should be used cautiously until class-conditional calibration is applied. (10) Sensitivity to noisy labels and rare patterns—the effect of label noise and the behavior on rare or atypical tissue patterns were not systematically studied. (11) Explainability not yet formally validated—Grad-CAM maps were assessed qualitatively and have not been reviewed by pathologists or compared against region annotations.

Building on these findings, we plan four follow-ups: (1) reducing model complexity through knowledge distillation or a leaner backbone; (2) advanced class-balancing (curriculum learning, two-stage fine-tuning) to raise Tumor recall without sacrificing overall accuracy; (3) retraining on the full NCT-CRC-HE-100K with GPU acceleration and validating on additional benchmarks, such as Kimia Patch24C; and (4) extending evaluation to WSI-level inference with spatial aggregation.

We further plan to (5) raise Tumor sensitivity through ensemble strategies (e.g., bagging of complementary backbones or a two-expert tumor/non-tumor cascade), which trade a modest increase in inference cost for improved minority-class recall; (6) apply class-conditional calibration so that confidence estimates remain reliable for the malignant class; (7) enforce a strict slide- or patient-level split and quantify robustness to staining and scanner domain shift; and (8) carry out a pathologist-in-the-loop study that validates the Grad-CAM saliency against expert annotations. Situated in a real computational-pathology workflow, the proposed framework is intended to operate as an integrated decision-support component: given a query patch or region, it provides a calibrated tissue-classification probability, retrieves visually similar, previously confirmed cases from an indexed archive for case-based comparison and second-reading, and overlays a Grad-CAM saliency map that lets the pathologist verify the basis of each prediction. Such a system could support routine LUAD tissue classification, case-based diagnostic review, rare-pattern look-up, quality control, and trainee education, while the explicit reporting of calibration and per-class behavior helps the user judge when the model’s output should be trusted and when a confirmatory human read is warranted.