Confidence-Aware Decision-Level Fusion of Unpaired Radiographic and Histopathological Images for Osteosarcoma Classification

Abstract

Reliable osteosarcoma tissue characterization combines radiographic evidence of bone-level structural change with histopathological assessment of cellular morphology. In rare cancers, however, patient-matched multimodal datasets are rarely available because radiology and pathology follow separate clinical workflows and cohort sizes are small. This study examines whether decision-level fusion can integrate radiographs and histopathology images originating from independent, unpaired patient cohorts, and reports the results as a methodological proof of concept rather than as a clinically validated diagnostic system. Two EfficientNet-B0 encoders were trained separately using osteosarcoma-positive radiographs from the Kaggle Bone Tumor Classification dataset (180 images) and H&E-stained histopathology tiles from the TCIA Osteosarcoma Tumor Assessment collection (1144 tiles from four patients). Histopathology tiles carry three labels: non-tumor, viable tumor, and necrotic tumor. Because radiographs do not provide tissue-viability labels and cannot directly distinguish viable from necrotic tumor, the radiographic branch was used as a weak radiograph-derived probability prior mapped into the shared three-class decision space during fusion. Fusion operates only on modality-specific probability vectors; no case-level or patient-level pairing is assumed or required. The adaptive gating network estimates a per-sample radiograph-prior weight, α, from the concatenated vector, [Pr, Ph], where Pr denotes the radiograph-derived probability prior, Ph denotes the histopathology probability vector, and 1 − α denotes the histopathology weight. To limit leakage in the small histopathology cohort, the four patients were assigned to fixed training (P001 and P002), calibration (P003), and test (P004) partitions with strict patient-level separation. On the single held-out test patient (171 tiles), adaptive gating fusion classified 166 of 171 tiles correctly (97.08% accuracy, macro-F1 of 0.97, and macro-AUC of 0.99), compared with 161 of 171 tiles (94.15%) for fixed-α fusion at α = 0.25. McNemar’s test for this comparison gave χ² = 3.20, p = 0.074, so the improvement was numerically higher but not statistically significant at the 0.05 level. Simpler classifiers on the same three-dimensional fused vector reached comparable accuracy (95.91–96.49%), and none differed significantly from adaptive gating. These results indicate that confidence-aware decision-level fusion is feasible under unpaired, data-constrained conditions, and that its present value lies in interpretable per-sample modality weighting rather than in a demonstrated accuracy advantage. The single-patient histopathology test set precludes any claim of clinical generalizability; validation on larger, multi-institutional, patient-level cohorts remains necessary.

1. Introduction

Osteosarcoma is the most common and aggressive primary bone cancer, and it most often affects adolescents and young adults during growth spurts [1]. Although imaging and systemic therapy have advanced, survival has changed little for many patients, partly because detection is frequently late and the tumor is biologically heterogeneous. The disease usually begins in the metaphyseal region of long bones and, when left untreated, spreads to the lungs and skeleton [2]. Early symptoms, such as localized pain and swelling, are nonspecific and often appear after substantial progression, leaving a narrow window for intervention. Five-year survival reaches roughly 70% for localized disease but falls below 30% once metastasis develops [3]. These facts make early, reproducible recognition of osteosarcoma clinically important.

Diagnosis depends mainly on radiologic and histopathologic evaluation. Radiographs and CT show cortical destruction, periosteal reaction, and mineralized matrix, while MRI defines soft-tissue extension and supports surgical planning [4]. Histopathology confirms the diagnosis by identifying malignant osteoblasts that produce osteoid. Both readings depend heavily on individual expertise, and interpretations differ in borderline cases [5]. Distinguishing early osteosarcoma from benign conditions such as fibrous dysplasia or chronic osteomyelitis is difficult because their imaging features overlap, so confirmation often rests on expert consensus rather than reproducible quantitative measures.

Deep learning has reshaped medical image analysis by learning hierarchical features that often outperform hand-crafted radiomic descriptors [6]. Convolutional networks such as ResNet [7], EfficientNet [8], and DenseNet [9] perform well across imaging domains. In osteosarcoma, most reported CNN methods use a single modality, either radiographs or histopathology. For example, one group distinguished osteolytic osteosarcoma from giant-cell tumor using institutional radiographs [10], and another classified histopathological grades on the TCIA dataset with a hybrid CNN–Vision Transformer model [11]. Unimodal systems can be accurate, but they cannot represent complementary cues that exist between tissue-level and structure-level information.

Combining modalities offers a more complete diagnostic picture, and prior oncology work has shown that coupling CT with histology [12] or MRI with radiomics improves classification and staging. Most of these approaches, however, assume paired data in which every patient has every modality. In routine practice, this pairing is uncommon, because radiology and pathology are acquired under different workflows, timelines, and regulatory constraints. For osteosarcoma specifically, public datasets are scarce and almost never paired, so research on unpaired multimodal learning remains limited. This is the gap the present study addresses: it studies decision-level integration of independent, unpaired cohorts, not patient-level multimodal diagnosis.

We train two ImageNet-pretrained EfficientNet-B0 backbones independently on each modality [13]. At inference, their Softmax probability vectors are combined by two late-fusion schemes. The first, fixed-α fusion, applies a constant weight to the radiograph branch; it is simple but assumes that both sources contribute equally in every case, which is rarely true when image quality and diagnostic relevance vary. The second, an adaptive gating network, replaces the constant with a learned per-case coefficient that emphasizes the modality carrying more reliable evidence in that instance.

Formally, let $P_r$ denote the radiograph-derived probability prior, and let $P_h$ denote the histopathology probability distribution. Throughout this paper, $\alpha$ denotes the radiograph-prior weight, and $1-\alpha$ denotes the histopathology weight. The fused output under adaptive gating is

$$P_f=\alpha \left(\right[P_r,P_h\left]\right)P_r+\left(1-\alpha \left(\right[P_r,P_h\left]\right)\right)P_h.$$

(1)

where $\alpha \left(\right[P_r,P_h\left]\right)\in \left[\mathrm{0,1}\right]$ is a gating function learned from the concatenated probability vector, $\left[P_r,P_h\right]$. A higher $\alpha$ gives greater weight to the radiograph-derived prior, whereas a lower $\alpha$ gives greater weight to the histopathology prediction.

The adaptive formulation generalizes the fixed-α model by allowing case-specific weighting, which mitigates the limitations of a single global coefficient [14,15,16,17,18]. Adaptive and gated fusion have been explored in other oncologic imaging tasks, including skin lesions and prostate imaging, but their use in osteosarcoma remains limited, particularly under unpaired data. Alongside accuracy, the learned weights provide interpretable modality contributions, an aspect increasingly emphasized in explainable medical AI [19,20,21].

The contributions of this study are methodological and are framed within the limits of the available data:

-

We formulate and evaluate adaptive decision-level fusion for unpaired osteosarcoma imaging, where radiographs and histopathology come from independent cohorts with no case-level or patient-level correspondence, and fusion is performed only over class-probability vectors.

-

We provide a broad internal baseline comparison on a single fixed evaluation protocol, including mean-probability fusion, fixed-α fusion at α = 0.25 and α = 0.38, feature-level fusion, and three simple classifiers on the fused vector (logistic regression, linear SVM, and shallow MLP); an attention-based late-fusion baseline; and the adaptive gating network.

-

We report statistical comparisons (McNemar’s test with p-values) for every model-versus-adaptive comparison and show that the adaptive model is numerically highest but not statistically distinguishable from several simpler baselines on this test set.

-

We characterize the interpretability of the learned per-sample radiograph-prior weights and discuss their limitations under a single calibration patient and a single held-out test patient.

-

We frame all findings as a proof of concept. Because the histopathology test set is a single held-out patient (171 tiles), this study demonstrates methodological feasibility, not clinical generalizability.

2. Related Work

Radiographic imaging remains the most widely accessible first-line modality for suspected osteosarcoma because it is fast, inexpensive, and routinely available in clinical practice. Conventional radiographs can reveal cortical destruction, periosteal reaction, matrix mineralization, and other macroscopic bone changes that support the initial diagnostic impression [22]. These structural cues have motivated the use of deep learning for radiographic osteosarcoma assessment. A recent radiographic study [23] developed a convolutional model using institutional X-ray data to differentiate osteolytic osteosarcoma from a giant cell tumor around the knee. The model captured radiographic texture and structural patterns that improved image-based discrimination between these bone lesions. However, the method remained restricted to radiographs and therefore could not model cellular morphology, osteoid production, necrosis, or tissue viability, which remain central to pathological confirmation. This limitation is clinically relevant because early or borderline osteosarcoma can share macroscopic imaging features with benign, infectious, or inflammatory bone conditions.

Other radiographic studies have incorporated clinical or institutional information to improve diagnostic discrimination. A multicenter study [24] proposed an ensemble deep-learning framework that combined X-ray features with patient-level metadata to distinguish primary bone tumors from infections. The multicenter design improved stability against cross-center variation and used clinical context to support radiology-based classification. Nevertheless, the framework remained radiology-centered and did not integrate microscopic tissue evidence. Its ensemble structure also increased model complexity, which may be difficult to justify in small-data settings unless it provides clear gains in calibration, stability, or external generalization. These studies show that radiographs can provide useful macrostructural information, but they also show the limits of a purely radiographic representation for tissue-level osteosarcoma characterization.

Histopathology provides a complementary and diagnostically stronger source of evidence because it directly visualizes malignant osteoid formation, atypical osteoblasts, necrotic tissue, and viable tumor morphology. A histopathological osteosarcoma classification framework [25] combined convolutional neural networks with Vision Transformer components to classify TCIA tissue images into non-tumor, viable tumor, and necrotic tumor categories. The hybrid design benefited from the local pattern sensitivity of convolutional layers and the broader contextual modeling capacity of Transformer-based components. However, evaluating tile-level histopathology models on the TCIA osteosarcoma collection requires strict patient-level data separation. Because the number of available patients is very small, performance metrics are easily inflated by slide-specific staining, scanner effects, or patient-specific morphology if tiles from the same patient appear in both training and test partitions.

Single-modality approaches are therefore informative but incomplete. Radiographs describe bone-level structural change, whereas histopathology describes cellular and tissue-level morphology. Multimodal learning seeks to combine these complementary sources of evidence. One approach [26] used CT and MRI information to improve preoperative tumor margin delineation and surgical planning. CT contributed structural bone information, while MRI provided soft tissue contrast and anatomical extent. Although this type of fusion supports anatomical characterization, it does not incorporate histopathological tissue information and commonly relies on static fusion assumptions that cannot adapt to patient-specific variation in modality reliability. Other multimodal methods have incorporated pathological, radiomic, or clinical information. A deep multimodal framework [27] combined imaging, pathology, and biomarker information for bone tumor diagnosis. Such designs can integrate macro-level and micro-level information, but they usually require paired multimodal measurements from the same patient. This requirement is difficult to satisfy in rare cancers because radiology and pathology are often acquired through separate workflows, at different times, and under different institutional protocols.

Adaptive fusion addresses one limitation of static fusion by allowing the model to adjust modality contribution according to reliability, confidence, or data quality. A multi-stage fusion strategy [28] integrated dermoscopy images, clinical photographs, and metadata for skin-lesion analysis. Similar adaptive weighting principles have been explored in other medical imaging domains [29]. However, osteosarcoma studies remain limited, especially when radiographs and histopathology are not patient matched. These studies support the broader idea that modality contribution is rarely constant across patients or samples, but most adaptive fusion methods still assume that each case has aligned modality inputs. This is not the setting addressed in the present study. In publicly available osteosarcoma resources, radiographs and histopathology commonly originate from independent cohorts with no patient-level or lesion-level pairing.

Attention-based fusion provides another route for combining complementary representations. In osteosarcoma histopathology, a noise-reducing attention cross-fusion learning transformer [30] integrated multiscale histological features for tissue classification. This method shows that attention mechanisms can improve representation learning within pathology images. However, it remains a histopathology method only and does not solve the problem of combining unpaired radiographic and histopathological evidence. Under unpaired multimodal conditions, early fusion or direct feature concatenation is methodologically fragile because feature vectors from different modalities do not represent matched observations from the same patient or lesion. A network trained on such randomly paired features cannot learn valid cross-modal correlations. It can only exploit dataset-level artifacts or modality-specific priors. Decision-level fusion is therefore more appropriate for independent cohorts because it combines probability-level evidence without assuming shared patient identity, lesion alignment, or synchronized acquisition.

The classifier used after fusion also affects the validity of the methodological claim. When fusion produces a low-dimensional probability vector, complex classifiers may not provide a meaningful advantage over simpler models. Logistic regression, linear support vector machines, shallow multilayer perceptrons, and lightweight attention-based late fusion can all operate on the same decision-level representation. Comparing adaptive gating only against fixed weight fusion can therefore overstate its contribution. A rigorous evaluation must test whether adaptive gating improves predictive performance beyond simple low-dimensional classifiers, or whether its main value lies in interpretability through per-sample modality weighting. This issue is especially important in small patient-level datasets, where complex models can appear numerically better while remaining statistically indistinguishable from simpler alternatives.

Taken together, prior work shows three important gaps. First, many osteosarcoma models remain unimodal, using either radiographs or histopathology alone. Second, existing multimodal methods often assume paired data or static fusion rules, both of which are difficult to satisfy in rare cancer datasets. Third, adaptive and attention-based fusion strategies have been explored in other medical imaging settings, but their value under unpaired osteosarcoma conditions remains insufficiently tested. The present study addresses this specific setting by applying decision-level fusion to independent radiograph and histopathology cohorts. The aim is not to claim clinical superiority; rather, the study examines whether confidence-aware fusion can provide competitive performance and interpretable per-sample modality weighting when patient-matched multimodal data are unavailable.

Table 1 summarizes representative studies relevant to osteosarcoma classification and multimodal fusion. The comparison is intended to position the proposed work methodologically rather than to rank accuracy values directly, because prior studies differ in dataset source, cohort size, modality composition, label schema, and evaluation protocol.

Table 1. Summary of representative works in osteosarcoma and multimodal fusion analysis.

Ref.	Dataset	Model	Key Limitation
[23]	Institutional X-rays	Deep-learning CNN	Radiograph only; no histopathology information; limited tissue-level interpretation
[24]	Multicenter X-rays and metadata	Ensemble deep-learning framework	Radiology centered; ensemble structure increases inference complexity
[25]	TCIA histopathology	Hybrid CNN ViT	Histopathology only; patch-level evaluation may require strict patient-level separation because TCIA has very few patients
[26]	CT and MRI	AI-assisted 3D image fusion	Static fusion assumptions; no histopathology integration; limited patient-specific adaptation
[27]	CT/MRI with pathology or clinical biomarkers	Deep multimodal fusion	Requires paired multimodal data; institutional cohort limits generalization
[30]	Osteosarcoma histopathology	Noise reducing attention cross-fusion transformer	Histopathology only; attention fusion occurs across multiscale histology features, not across radiograph and pathology modalities
Present study	TCIA histopathology and Kaggle radiographs	Adaptive decision-level fusion with radiograph derived prior	Uses unpaired cohorts; proof of concept only; limited by one held out histopathology test patient

3. Methods and Datasets

3.1. Dataset and Preprocessing

This study uses two independent, open-access datasets representing different modalities. Histopathology images come from the TCIA Osteosarcoma Tumor Assessment collection, and radiographs come from the Kaggle Bone Tumor Classification dataset. The two sources were acquired from entirely separate institutions and patient populations and share no common identifiers, imaging protocols, or institutional provenance, so no patient appears in both datasets, and no case-level or lesion-level pairing exists between modalities. This unpaired structure is the central data constraint that the method is designed for, and it motivates fusion at the level of class probabilities rather than features (Section 3.3).

Histopathology (TCIA): The collection contains 1144 pre-extracted tiles, each 1024 × 1024 pixels at 10× magnification, from four pediatric patients treated at the Children’s Medical Center in Dallas, TX [31]. Each tile is labeled non-tumor (536 tiles, 47%), necrotic tumor (263 tiles, 23%), or viable tumor (345 tiles, 30%). This is, to our knowledge, the only publicly available labeled histopathology dataset for osteosarcoma tissue classification at the time of writing; no additional patients are available in the public repository. All tiles were stain-normalized with the Macenko method [32], resized to 224 × 224, and normalized using ImageNet statistics [33]. Augmentation (≈30° rotations; horizontal and vertical flips; color jitter of 0.2 for brightness, contrast, and saturation; and low-variance Gaussian noise) was applied only to the training partition, on the fly per epoch; validation and test tiles were always original and unaugmented.

Radiographs (Kaggle): The Kaggle Bone Tumor Classification dataset provides tumor-type labels (osteosarcoma, chondrosarcoma, and Ewing sarcoma) but no tissue-viability annotation and no persistent patient identifiers. The radiograph branch was not trained as an independent binary classifier because the retained Kaggle subset contained only osteosarcoma-positive radiographs. Instead, the radiograph encoder was used to estimate a modality-specific osteosarcoma confidence prior from osteosarcoma radiographs. This prior was not treated as direct evidence of histological tissue viability and was introduced only as a weak macrostructural decision-level signal during fusion. Augmentation included random horizontal flips, CLAHE, random cropping, and Gaussian blur to simulate variation in imaging conditions.

3.2. Dataset Splitting Principles

The two modalities require different splitting strategies because only one provides patient identifiers. We state the principles explicitly because they directly determine what the evaluation can and cannot establish.

Histopathology involved strict patient-level separation. All 1144 tiles from a given patient were assigned to exactly one partition: training = P001 + P002 (802 tiles), calibration = P003 (171 tiles), and test = P004 (171 tiles). We use the term “calibration” for P003 rather than “validation” because, with only four patients, this single patient is used both to estimate fusion coefficients and to fit the gating network (Section 3.5); it is therefore not an independent validation set, and we do not describe it as one. No tile from a patient appears in more than one split. This is essential because tiles from the same whole-slide image share staining, preparation artifacts, and correlated biology that a model can memorize, inflating performance if mixed across splits. The nominal 70/15/15 ratio refers to tile-level proportions, not patient counts, and is reported only for reference. Because the cohort has four patients, statistically valid cross-validation is not possible; rotating through three training patients would conflate hyperparameter selection with leakage and provide negligible statistical power. We therefore use a single fixed split throughout. A direct and important consequence is that all reported test metrics derive from one held-out patient (171 tiles). Performance variability across alternative patient assignments is unknown and may be substantial, so the evaluation cannot establish clinical generalization. Bootstrap confidence intervals quantify tile-level uncertainty conditional on this single patient and do not generalize to other individuals.

Radiographs involved image-level stratified splitting. Because the Kaggle subset has no patient identifiers, patient-level grouping is impossible. Radiographs were treated as independent samples under repeated stratified train/validation/test splits at the image level. We cannot exclude the possibility that multiple radiographs from the same individual fall into different splits, which would constitute data leakage and inflate the standalone radiograph-only performance. We therefore treat radiograph-only metrics as indicative rather than definitive and report variability across repeated splits (mean ± SD).

These asymmetric protocols are modality-appropriate but not equivalent: histopathology is assessed under patient-level holdout, whereas the radiograph branch may see images from the same patient across splits. The two branches are therefore not evaluated under identical generalization regimes, which complicates interpretation of their fused performance and is acknowledged as a limitation.

Clinical metadata: Patient-level clinical variables (age, sex, tumor anatomical location, and pre- versus post-chemotherapy status) are not available in the public TCIA tile release used here, which distributes pre-extracted, de-identified tiles without linked structured clinical fields. We therefore could not stratify, adjust for, or report these covariates. Their absence is a limitation: it prevents analysis of whether performance varies by clinical subgroup and limits assessment of confounding by treatment status.

Table 2 summarizes the asymmetric structure of the two datasets. The histopathology cohort defines the main three-class tissue classification task and permits strict patient-level separation. In contrast, the radiograph dataset lacks patient identifiers and tissue-viability labels, so it is used only to derive an unpaired coarse probability prior. Therefore, the radiograph branch is not interpreted as patient-specific evidence for any histopathology tile; fusion is performed only at the decision level over probability vectors.

Table 2. Dataset composition and splitting protocol.

Attribute	Histopathology	Radiograph
Modality	Histopathology, H&E-stained tissue tiles	Radiograph, X-ray
Dataset	TCIA Osteosarcoma Tumor Assessment	Kaggle Bone Tumor Classification
Data role in this study	Primary tissue-classification modality	Unpaired radiograph-derived probability prior
Label space	Three-class tissue labels: non-tumor, viable tumor, necrotic tumor	Coarse osteosarcoma radiograph signal; no tissue-viability label
Final mapped output space	Non-tumor, viable tumor, necrotic tumor	Radiograph-derived probability prior mapped into the same three-class space
Patient IDs available	Yes, 4 patients	No
Total samples	1144 tiles	180 images
Splitting principle	Strict patient-level separation	Image-level stratified splitting because patient IDs are unavailable
Training	P001 + P002, 802 tiles	Stratified image-level training subset
Calibration	P003, 171 tiles	Stratified image-level calibration subset
Test	P004, 171 tiles	Stratified image-level test subset
Evaluation status	Main held-out evaluation for tissue classification	Indicative standalone radiograph evaluation; not directly comparable with the histopathology test patient
Fusion role	Provides the indexed three-class histopathology prediction for each tile	Provides an unpaired weak probability prior; no patient-level, lesion-level, or tile-level pairing is assumed

Table 3 lists per-patient tile counts by class and the split assignment, documenting that no patient crosses partitions. We note that P003 and P004 carry identical per-class tile counts; this reflects the partition sizes used and not duplicated content, as the tiles originate from two biologically distinct patients.

Table 3. Patient-level tile distribution and split assignment for the TCIA histopathology dataset.

Patient ID	Split	Non-Tumor	Viable Tumor	Necrotic Tumor	Total Tiles
P001	Training	190	120	90	400
P002	Training	186	121	95	402
P003	Calibration	80	52	39	171
P004	Test	80	52	39	171
Total	—	536	345	263	1144

3.3. Radiograph-Prior Construction and Three-Class Probability Mapping

The two modalities do not share the same semantic label space. The histopathology branch predicts three tissue classes: non-tumor, viable tumor, and necrotic tumor. In contrast, radiographs do not encode microscopic tissue viability and cannot directly distinguish viable tumor from necrotic tumor. Therefore, the radiograph branch was used only to construct a weak tumor-level probability prior. For compatibility with the histopathology output space, the radiograph-derived tumor probability was distributed equally across the viable tumor and necrotic tumor classes, while the non-tumor probability was assigned to the non-tumor class. This mapping is not a biological measurement of viability; it is a deliberately conservative decision-level approximation used to test whether a coarse radiographic prior can improve or calibrate histopathology-based tissue classification.

This remapping is a deliberate simplification, and it introduces modality-specific label uncertainty into the radiograph branch for tumor-positive cases, because the equal split is an assumption rather than a measurement. This uncertainty has a concrete consequence for fusion: because the radiograph branch carries genuinely ambiguous information about the viable-versus-necrotic distinction, a confidence-aware gating mechanism should, on average, discount it relative to histopathology. The observed mean radiograph-prior weight (α = 0.38, Section 4) is consistent with this expectation, and the remapping is therefore a plausible partial explanation for why the gate assigns less average weight to radiographs. The adaptive gating mechanism is specifically intended to accommodate this asymmetry by learning per-case weights rather than imposing a fixed global coefficient.

Formally, for a dataset, $D=\{\left(x_i,y_i\right){\}}_{i=1}^N$, with $N$ labeled samples and $C$ classes, preprocessing is

$$x_i^{\prime }=A_m\left(N_m\left(R\left(x_i\right)\right)\right)$$

(2)

where $R(\cdot )$ resizes each image to $224\times 224$, $N_m(\cdot )$ applies modality-specific normalization for modality $m$, and $A_m(\cdot )$ applies modality-specific augmentation only for training samples. For calibration and test samples, preprocessing consists only of resizing and normalization. The transformed dataset, $D^{\prime }=\left\{\right(x_i^{\prime },y_i){\}}_{i=1}^N$, is then used to train the unimodal extractors before fusion.

3.4. Construction of Unpaired Fused Samples

Because the radiograph and histopathology datasets contain no shared patients, lesions, or acquisition identifiers, the fusion procedure cannot represent patient-level multimodal diagnosis. We therefore constructed fused observations at the decision level rather than at the case level. Each histopathology tile served as the indexed evaluation unit because the final task was three-class tissue classification. For each histopathology tile, the histopathology encoder produced a three-class probability vector. The radiograph branch contributed a modality-level probability prior rather than a paired radiograph from the same patient.

To prevent arbitrary single-pair dependence, the radiograph prior used in the primary analysis was computed as the mean calibrated radiograph probability vector across the radiograph validation predictions. Sensitivity analysis was then performed by repeating the fusion experiment with randomly sampled radiograph probability vectors across 100 random pairing seeds. The final results are reported as the primary prior-based estimate together with the mean and standard deviation across random-pairing sensitivity runs. This design preserves the unpaired nature of the data and avoids implying that a radiograph and histopathology tile belong to the same patient.

3.5. Base Models

Each modality is processed by an EfficientNet-B0 backbone [34] pretrained on ImageNet [35]. For an input image, $x$, from modality, $m$, the modality-specific encoder produces

$$z_m={\varphi }_m(x;{\theta }_m),$$

(3)

where ${\varphi }_m(\cdot ;{\theta }_m)$ denotes the EfficientNet-B0 encoder for modality, $m$; ${\theta }_m$ denotes its learnable parameters; and $z_m$ denotes the resulting feature embedding. Global average pooling yields a 1280-dimensional embedding. During fine-tuning, all convolutional blocks except the last two were frozen to preserve pretrained features.

Both networks used Adam [36], 50 epochs, a batch size of 32, an initial learning rate of $1\times {10}^{-4}$ with a 0.5 decay after three stagnant validation epochs, weight decay at $1\times {10}^{-5}$, and early stopping with patience 10. The loss was categorical cross-entropy.

On the held-out evaluation, the histopathology-only network reached 92.10% test accuracy, and the radiograph-only network reached 83.70 ± 1.20% (mean ± SD over repeated stratified image-level splits; not evaluated on the 171-tile histopathology test set, so no per-tile count is reported). Both encoders therefore captured useful modality-specific information, but their accuracies differ substantially, which is relevant to the fusion analysis.

3.6. Fusion Strategies and α* Generation

Each unimodal branch produces a three-class probability vector in the shared tissue-label space. Let $P_r$ denote the radiograph-derived probability prior, and $P_h$ denote the histopathology probability vector. Fixed-$\alpha$ fusion combines these vectors with a constant radiograph-prior weight, $\alpha \in \left[\mathrm{0,1}\right]$:

$$P_f=\alpha P_r+(1-\alpha )P_h.$$

(4)

A grid search on the calibration patient P003 with step size 0.05 selected $\alpha =0.25$.

Adaptive gating fusion replaces the constant weight with a sample-specific coefficient. A small MLP takes the concatenated vector, $\left[P_r,P_h\right]$, and outputs a scalar, $\alpha \in \left[\mathrm{0,1}\right]$, through a sigmoid activation:

$$\begin{gathered} \alpha =g\left(\left[P_r,P_h\right]\right), \\ {P}_f=\alpha P_r+(1-\alpha )P_h. \end{gathered}$$

(5)

The gating MLP has two hidden layers with 128 and 64 units, ReLU activation, and approximately 8500 parameters. It was trained for 20 epochs with a learning rate of $1\times {10}^{-3}$, batch size of 128, and frozen modality outputs.

Generation of pseudo-optimal targets, α*: Because the histopathology cohort contains only four patients, no separate independent validation patient was available after assigning P001 and P002 to encoder training and reserving P004 as the untouched test patient. The remaining patient, P003, was therefore used as a calibration patient for the fusion stage. Specifically, P003 was used to estimate pseudo-optimal α* targets and to train the gating network. For this reason, P003 is not described as an independent validation set in this study. This design preserves strict separation between calibration and final testing, but it also introduces an important limitation: the learned weighting function is derived from a single calibration patient and may not transfer fully to other patients.

We define the procedure as an explicit, self-consistent pipeline:

• For each calibration tile, $i$, from P003 ($n=171$), we computed a pseudo-optimal radiograph-prior weight, ${\alpha }_i^{*}$, by exhaustive grid search over $\alpha \in \left[\mathrm{0,1}\right]$ with step size 0.05. The selected value maximized the correctness indicator for that tile:

  $${\alpha }_i^{*}=\mathrm{a}\mathrm{r}\mathrm{g} \underset{\alpha \in \{\mathrm{0,0.05},\dots ,1\}}{\mathrm{m}\mathrm{a}\mathrm{x}}\mathbf{1}\left[\mathrm{a}\mathrm{r}\mathrm{g}\underset{c}{\mathrm{m}\mathrm{a}\mathrm{x}}{P}_{f,i}^{\left(c\right)}\left(\alpha \right)=y_i\right],$$

  (6)

  where $P_{f,i}^{\left(c\right)}\left(\alpha \right)$ denotes the fused probability assigned to class $c$ for calibration tile, $i$; $y_i$ denotes the true class label; and $\mathbf{1}[\cdot ]$ is the correctness indicator.
• The resulting pairs $\left([P_{r,i},P_{h,i}],{\alpha }_i^{*}\right)$ from the calibration patient constitute the supervised training set for the gating network. The gating network is trained to regress ${\alpha }_i^{*}$ from the concatenated probability vector, $\left[P_{r,i},P_{h,i}\right]$, thereby learning a mapping from decision-level modality evidence to a fusion weight.
• At inference, the trained gating network is applied to the training patients’ and the test patient’s probability vectors to produce per-sample α. The gating network never sees the test patient (P004) during α* computation, gating training, or any model selection. The training tiles (P001 and P002) are used only to fit the unimodal encoders and, at fusion time, as additional inputs on which the already-trained gating network predicts weights; their α values are predictions, not targets.

This procedure separates target generation, gate training, and final testing. The ${\alpha }^{*}$ targets are derived only from the calibration patient, P003. The gating function is then learned from these calibration pairs, while the test patient, P004, remains held out throughout target generation, gating training, and model selection. This design preserves the separation between calibration and testing. However, because ${\alpha }^{*}$ is estimated from a single calibration patient, the learned weighting function may not transfer fully to other individuals, thereby remaining a primary threat to generalization.

The gating value, α, is the radiograph-prior weight in Equations (1), (4), and (5); 1 − α is the histopathology weight. This convention is used consistently throughout.

3.7. Baseline Fusion Models for Comparative Evaluation

To determine whether adaptive gating was necessary, and whether the downstream classifier influenced performance, we evaluated a set of fusion and classifier-based baselines under the same protocol. All models used the same EfficientNet-B0 encoders, frozen modality outputs, patient-level histopathology split, calibration patient, and held-out test patient. This design ensured that performance differences reflected the fusion or classifier strategy rather than changes in data partitioning or feature extraction.

The evaluated strategies were as follows: fixed-α fusion with α = 0.25, selected by grid search on the calibration patient; fixed-α fusion with α = 0.38, corresponding to the mean radiograph-prior weight learned by the adaptive gating network; mean-probability fusion with α = 0.5; logistic regression on the three-dimensional fused probability vector; linear SVM on the same three-dimensional fused vector; shallow MLP on the same three-dimensional fused vector; attention-based late fusion using a lightweight attention layer over the concatenated vector, [Pr, Ph]; and the proposed adaptive gating fusion.

The classifier-based baselines were trained on the same three-dimensional decision-level representation and used the same patient-level evaluation protocol. This allowed for direct comparison between adaptive gating and simpler low-dimensional classifiers.

3.8. Classifier Choice and the Role of XGBoost

The earlier design routed the fused probability vector through an XGBoost classifier. We reassess that choice directly. XGBoost [37] is a strong general-purpose ensemble, but its input here is only a three-dimensional probability vector, a setting in which gradient boosting offers limited advantage over a linear or shallow model and adds capacity that may overfit a 171-tile test set. We therefore benchmarked XGBoost against simpler classifiers on the identical fused vector. On this input, logistic regression achieved 95.91%, linear SVM achieved 95.32%, and the shallow MLP achieved 96.49%. These differences were within one to two tiles of the adaptive and attention-based alternatives and were statistically non-significant (Section 4). Accordingly, we report logistic regression as the default low-dimensional classifier and present XGBoost only as an exploratory option that provided no measurable benefit on this task. This follows the principle that model complexity should be justified by input dimensionality and by evidence of improvement, neither of which favors XGBoost here.

3.9. Training Setup and Evaluation Protocol

Experiments were conducted on Ubuntu 20.04 LTS using Python 3.10, CUDA Toolkit 11.7, and PyTorch 2.0 on a single NVIDIA A100 GPU with 40 GB memory. Both EfficientNet-B0 backbones used ImageNet initialization. EfficientNet-B0 refers to the architecture implemented in PyTorch/torchvision, and ImageNet initialization refers to pretrained ImageNet weights rather than separate software. Random seeds were fixed at 42 for NumPy and PyTorch, and deterministic cuDNN execution was enabled [38].

The histopathology split followed the single fixed assignment described in Section 3.2: P001 and P002 for training, P003 for calibration, and P004 for testing. We did not perform leave-one-patient-out cross-validation because the cohort contained only four patients. Under this constraint, cross-validation would provide limited statistical power and could conflate model selection with leakage. All test metrics therefore derived from the single held-out test patient, P004, which contained 171 tiles. Variability across alternative patient assignments remains unknown.

We report accuracy, macro-precision, macro-recall, macro-F1, and macro-AUC. Ninety-five-percent confidence intervals for accuracy and F1 were estimated using 1000 bootstrap resamples of the test tiles. For model comparisons against adaptive gating, we used McNemar’s test [39] on paired predictions from the same test tiles, with the continuity-corrected statistic. A difference was considered statistically significant at p < 0.05. For every comparison, we report both the χ² statistic and the p-value rather than reporting significance status alone.

3.10. Proposed Multimodal Framework

The proposed framework, shown in Figure 1, consists of three components: modality-specific encoders, a late-fusion module, and a classification head.

Figure 1. Decision-level integration framework for osteosarcoma tissue classification under unpaired radiograph and histopathology data. The histopathology encoder produces a three-class tissue probability vector for each tile.

Each modality is processed independently. The histopathology branch produces a three-class probability vector, $P_h$, which captures tissue-level evidence from H&E-stained tiles. The radiograph branch produces a radiograph-derived probability prior, $P_r$, which provides weak macrostructural evidence mapped into the same three-class tissue-label space, as described in Section 3.3.

The fusion module combines $P_r$ and $P_h$ using either fixed-$\alpha$ fusion or adaptive gating. In the adaptive configuration, the gating network estimates the radiograph-prior weight, $\alpha$, from the concatenated vector, $\left[P_r,P_h\right]$. The fused vector, $P_f\in {\mathbb{R}}^3$, is then classified using the default low-dimensional classifier, logistic regression, as described in Section 3.8. The final predicted class is

$$\widehat{y}=\mathrm{a}\mathrm{r}\mathrm{g}\underset{c}{\mathrm{m}\mathrm{a}\mathrm{x}}{P}_{f,c}.$$

(7)

where $P_{f,c}$ denotes the fused probability assigned to class $c$.

The adaptive gating network was trained using calibration-derived ${\alpha }^{*}$ pairs from P003, as described in Section 3.6, and evaluated on the strictly held-out test patient, P004. The reported result therefore reflects generalization to a biologically independent test subject within the limitation of a single held-out patient.

Table 4 summarizes the hyperparameter settings used for the modality-specific encoders, adaptive gating module, and final classifier. The EfficientNet-B0 encoders were initialized with ImageNet pretrained weights and fine-tuned with partial block freezing to reduce overfitting under the small-data setting. The adaptive gating MLP received the concatenated radiograph and histopathology probability vectors and produced the radiograph-prior weight, $\alpha$, where $1-\alpha$ represents the histopathology contribution. Logistic regression was used as the default classifier because the fused representation was only three-dimensional. XGBoost was evaluated only as an exploratory comparator and did not provide a measurable advantage on this input.

Table 4. Hyperparameter configuration of the EfficientNet-B0 encoders, adaptive gating module, and final decision classifier.

Component	Parameter	Configuration
EfficientNet-B0 encoders	Backbone initialization	ImageNet pretrained weights
EfficientNet-B0 encoders	Optimizer	Adam
EfficientNet-B0 encoders	Number of epochs	50
EfficientNet-B0 encoders	Batch size	32
EfficientNet-B0 encoders	Initial learning rate	$1\times {10}^{-4}$
EfficientNet-B0 encoders	Learning rate schedule	Reduced by a factor of 0.5 after 3 stagnant validation epochs
EfficientNet-B0 encoders	Weight decay	$1\times {10}^{-5}$
EfficientNet-B0 encoders	Early stopping patience	10 epochs
EfficientNet-B0 encoders	Fine-tuning strategy	All blocks frozen except the last two convolutional blocks
Adaptive gating MLP	Input	Concatenated probability vectors, $\left[P_r,P_h\right]$
Adaptive gating MLP	Hidden layers	128 and 64 units
Adaptive gating MLP	Activation function	ReLU
Adaptive gating MLP	Output layer	Sigmoid
Adaptive gating MLP	Output value	Radiograph-prior weight, $\alpha \in \left[\mathrm{0,1}\right]$
Adaptive gating MLP	Number of epochs	20
Adaptive gating MLP	Learning rate	$1\times {10}^{-3}$
Adaptive gating MLP	Batch size	128
Default classifier	Model	Logistic regression
Default classifier	Input	Fused probability vector, $P_f\in {\mathbb{R}}^3$
Exploratory alternative	Model	XGBoost
Exploratory alternative	Role	Exploratory comparator; no measurable benefit over logistic regression on the low-dimensional fused vector

4. Results and Discussion

4.1. Comparative Performance Under a Single Protocol

Table 5 reports the performance of the standalone, fusion-based, and classifier-based models. All fusion models and classifier-based baselines were evaluated on the same held-out histopathology test patient, P004, which contained 171 tiles. McNemar’s test was used to compare each applicable baseline with the adaptive gating model using paired predictions from the same test set.

Table 5. Internal comparison of fusion strategies under the same evaluation protocol (held-out test patient P004, 171 tiles). McNemar p-value is computed against the adaptive gating model.

Model	Evaluation Setting	Fusion Type	Input Representation	Acc (%)	Macro-p	Macro-R	Macro-F1	Macro-AUC	McNemar p vs. Adaptive
Radiograph only	Repeated image-level stratified splits	Not applicable	Image	83.70 ± 1.20	0.83	0.83	0.83	0.94	Not applicable
Histopathology only	Held-out test patient P004, 171 tiles	Not applicable	Image	92.10	0.92	0.92	0.92	0.98	Not applicable
Mean-probability fusion	Held-out test patient P004, 171 tiles	Decision, $\alpha =0.50$	$P_f\in {\mathbb{R}}^3$	93.57	0.94	0.93	0.93	0.98	Not tested
Fixed-$\alpha$ fusion, $\alpha =0.25$	Held-out test patient P004, 171 tiles	Decision	$P_f\in {\mathbb{R}}^3$	94.15	0.94	0.94	0.94	0.99	0.074
Feature-level fusion	Held-out test patient P004, 171 tiles	Feature	2560-D embedding	94.74	0.95	0.94	0.95	0.99	0.134
Fixed-$\alpha$ fusion, $\alpha =0.38$	Held-out test patient P004, 171 tiles	Decision	$P_f\in {\mathbb{R}}^3$	95.32	0.95	0.95	0.95	0.99	0.248
Linear SVM	Held-out test patient P004, 171 tiles	Decision classifier	$P_f\in {\mathbb{R}}^3$	95.32	0.95	0.95	0.95	0.99	0.248
Logistic regression	Held-out test patient P004, 171 tiles	Decision classifier	$P_f\in {\mathbb{R}}^3$	95.91	0.96	0.96	0.96	0.99	0.480
Shallow MLP	Held-out test patient P004, 171 tiles	Decision classifier	$P_f\in {\mathbb{R}}^3$	96.49	0.96	0.96	0.96	0.99	1.000
Attention-based late fusion	Held-out test patient P004, 171 tiles	Decision attention	$\left[P_r,P_h\right]$	96.49	0.96	0.96	0.96	0.99	1.000
Adaptive gating fusion	Held-out test patient P004, 171 tiles	Decision gated	$\left[P_r,P_h\right]$	97.08	0.97	0.97	0.97	0.99	Not applicable

Adaptive gating fusion achieved the highest numerical performance among the evaluated models. It correctly classified 166 of 171 tiles, corresponding to 97.08% accuracy, macro-precision of 0.97, macro-recall of 0.97, macro-F1 score of 0.97, and macro-AUC of 0.99. Fixed-α fusion with α = 0.25 correctly classified 161 of 171 tiles, yielding 94.15% accuracy and a macro-F1 score of 0.94. Fixed-α fusion using the mean learned radiograph-prior weight, α = 0.38, correctly classified 163 of 171 tiles, corresponding to 95.32% accuracy and a macro-F1 score of 0.95. Mean probability fusion achieved 93.57% accuracy, while feature-level fusion achieved 94.74% accuracy.

Among the classifier-based baselines trained on the three-dimensional fused probability vector, logistic regression achieved 95.91% accuracy, linear SVM achieved 95.32%, and the shallow MLP achieved 96.49%. Attention-based late fusion also achieved 96.49%. These results show that several simple decision-level models reached performance close to adaptive gating, with differences corresponding to only one to five tiles on the held-out test patient.

McNemar’s test comparing adaptive gating with fixed-α fusion at α = 0.25 yielded χ² = 3.20 and p = 0.074. Therefore, the observed numerical improvement did not reach statistical significance at the 0.05 level. Comparisons with logistic regression, shallow MLP, and attention-based late fusion were also non-significant. These findings indicate that adaptive gating should not be interpreted as statistically superior in predictive accuracy under the present evaluation setting. Its main value in this experiment is its ability to provide an interpretable decision-level weighting mechanism under unpaired and data-constrained conditions.

Figure 2 summarizes the training behavior and learned modality weighting of the adaptive gating fusion model.

Figure 2. Training behavior and learned modality weighting of the adaptive gating fusion model. The accuracy curves summarize the training and calibration trends for the fusion models, while the modality-weighting panel shows the sample-level radiograph-prior weight, α, learned by the adaptive fusion module. Lower α values indicate greater reliance on histopathology, whereas higher α values indicate greater reliance on radiographic evidence.

Table 5 shows that adaptive gating fusion achieved the highest numerical performance, with 97.08% accuracy, macro-precision of 0.97, macro-recall of 0.97, macro-F1 of 0.97, and macro-AUC of 0.99. However, the differences from the strongest decision-level baselines were small and statistically non-significant. Logistic regression reached 95.91% accuracy, while the shallow MLP and attention-based late fusion each reached 96.49%. Therefore, the adaptive gating result should be interpreted as evidence of feasible decision-level integration and interpretable weighting, not as evidence of statistically proven predictive superiority.

The pattern in Table 5 and Table 6 is the central, honest finding of this study. On this test set, every reasonable decision-level strategy clusters within a few tiles of the others, and the adaptive gating model, while numerically highest, is not statistically distinguishable from logistic regression, the shallow MLP, or attention-based late fusion (all p ≥ 0.48). The gap to fixed-α = 0.25 amounts to five tiles and does not reach significance (p = 0.074). These differences correspond to one to five reclassified tiles out of 171, which is within the range of test-set noise for a single patient. We therefore do not claim that adaptive gating is superior in accuracy. Its contribution is methodological and interpretive: it operates on unpaired inputs and yields per-sample modality weights, discussed in Section 4.3.

Table 6. McNemar’s test for adaptive gating versus each baseline (continuity-corrected; held-out test patient, 171 tiles).

Baseline Comparison	Adaptive Only Correct	Baseline Only Correct	χ2	p Value
Adaptive gating vs. fixed-α fusion, α = 0.25	5	0	3.20	0.074
Adaptive gating vs. feature-level fusion	4	0	2.25	0.134
Adaptive gating vs. fixed-α fusion, α = 0.38	3	0	1.33	0.248
Adaptive gating vs. linear SVM	3	0	1.33	0.248
Adaptive gating vs. logistic regression	2	0	0.50	0.480
Adaptive gating vs. shallow MLP	1	0	0.00	1.000
Adaptive gating vs. attention-based late fusion	1	0	0.00	1.000

4.2. Per-Class Performance

Table 7 reports the class-wise performance of the adaptive gating model on the held-out histopathology test patient P004. Figure 3 presents the corresponding 3 × 3 confusion matrix. The model correctly classified 166 of 171 tiles, which is consistent with the reported accuracy of 97.08%. Most errors occurred between viable tumor and necrotic tumor, reflecting the closer histopathological similarity between these two tumor tissue categories. The necrotic tumor class had the smallest support, with 39 tiles, but still retained high recall and F1-score.

Table 7. Per-class and macro-averaged performance of the adaptive gating model on the held-out test patient (171 tiles).

Class	Precision	Recall	F1 Score	Support
Non-tumor	0.98	0.99	0.98	80
Viable tumor	0.96	0.96	0.96	52
Necrotic tumor	0.97	0.95	0.96	39
Macro-average	0.97	0.97	0.97	171
Accuracy			97.08%	171

Figure 3. Confusion matrix of the adaptive gating fusion model on the held-out histopathology test patient P004. Rows indicate the true tissue classes, and columns indicate the predicted classes. The model correctly classified 166 of 171 tiles. Most errors occurred between viable tumor and necrotic tumor. Class labels are non-tumor, viable tumor, and necrotic tumor.

Table 7 reports the class-level performance of the adaptive gating model on the held-out histopathology test patient P004. The model correctly classified 166 of 171 tiles, corresponding to 97.08% accuracy. Performance remained balanced across the three tissue classes, with the highest recall for non-tumor tiles and the lowest recall for necrotic tumor tiles. Most errors occurred between viable tumor and necrotic tumor, which is consistent with the visual similarity of borderline tumor viability patterns in tile-level histopathology.

4.3. Learned Gating Weights and Interpretability

Figure 4 shows the distribution of learned radiograph-prior weights, α, produced by the adaptive gating module on the held-out histopathology test patient P004. Across test tiles, α had a mean of 0.38 ± 0.30, indicating that, on average, the model assigned 38% of the fusion weight to the radiograph-derived prior and 62% to the histopathology prediction. Approximately 41% of tiles had α > 0.8, indicating radiograph-dominant fusion; 12% had α < 0.2, indicating histopathology-dominant fusion; and the remaining 47% fell between these two extremes.

Figure 4. Distribution of learned radiograph-prior weights, α, produced by the adaptive gating module on the held-out histopathology test patient P004. The value α represents the contribution of the radiograph-derived prior, while 1 − α represents the contribution of the histopathology prediction. Lower α values indicate histopathology-dominant decisions, higher α values indicate radiograph-dominant decisions, and intermediate values indicate mixed decision-level evidence. The mean radiograph-prior weight was α = 0.38.

These findings should be interpreted cautiously. First, the lower mean α value is consistent with the stronger standalone performance of the histopathology branch and with the uncertainty introduced into the radiograph branch by the binary-to-three-class remapping described in Section 3.3. A confidence-aware gating mechanism would therefore be expected to assign lower weight to a modality whose viable-versus-necrotic contribution is partly inferred rather than directly observed. Second, the spread of α values indicates that the gate did not collapse to a single constant weight. If the model had learned only a dataset-level prior, α would have remained close to one fixed value, effectively reproducing fixed-α fusion. Instead, the observed variability suggests sample-specific weighting behavior.

At the same time, this should not be overinterpreted as definitive evidence of true instance-level reliability estimation. Qualitatively, high-α cases may correspond to radiographs with more distinctive macrostructural characteristics, whereas low-α cases may reflect histopathology samples with clearer cellular morphology. However, because the calibration process relied on a single calibration patient and evaluation was conducted on a single held-out test patient, we cannot exclude the possibility that part of the learned weighting pattern reflects patient-specific or dataset-specific effects. Establishing whether α captures genuine diagnostic informativeness would require larger balanced cohorts and multiple independent test patients.

4.4. Comparison with Prior Studies

Table 8 positions the proposed framework alongside representative osteosarcoma and multimodal studies. This comparison is not intended to demonstrate superiority, because the studies differ in dataset source, cohort size, label schema, modality composition, and evaluation protocol. A direct ranking based only on reported accuracy would therefore be methodologically misleading. To avoid this problem, Table 8 reports the cohort or patient count, modality, evaluation protocol, reported metric, main limitation, and comparison caveat for each study. The table should therefore be read as a qualitative methodological comparison rather than as a direct performance ranking.

Table 8. Methodological positioning relative to prior osteosarcoma and multimodal medical imaging studies. Direct performance ranking is not valid because the studies differ in datasets, cohorts, label schemas, modalities, and evaluation protocols.

Reference	Dataset	Cohort or Patients	Modality	Architecture	Evaluation Protocol	Reported Accuracy	Main Limitation	Comparison Caveat
[20]	TCIA	4 patients	Histopathology	Transfer learning with pretrained CNNs	Tile-level split, not patient stratified	95.65%	Single modality	Different label schema and possible within WSI leakage
[15]	TCIA	4 patients	Histopathology	Dual CNN	Tile-level evaluation	86.00%	Single modality	Different split protocol
[21]	TCIA	4 patients	Histopathology	MobileNetV2 benchmark	Tile-level evaluation	91.00%	Lightweight model with accuracy trade-off	Different split protocol
[10]	CT and clinical data	Institutional cohort	CT and clinical features	PCA, PSO, and ECOC SVM	Holdout evaluation	80.00%	Classical low-dimensional ML pipeline	Different modality, dataset, and task definition
[11]	TCIA and clinical data	Multisource cohort	Pathology and CTCs	Multimodal deep learning	Holdout evaluation	96.84%	Requires paired measurements	Paired data setting, not an unpaired fusion problem
[14]	TCIA	4 patients	Histopathology	Adaptive feature fusion with VGG	Tile-level evaluation	96.27%	Single modality	Within modality fusion only
[23]	Radiography	Institutional cohort	Radiograph	Region-based CNN	Holdout evaluation	96.00%	Single modality	Radiograph only model using a different dataset and task
[37]	Hyperspectral imaging and H&E	Institutional cohort	HSI and H&E	SMOTE with classical ML models	Holdout evaluation	87.33%	Different acquisition setting	Non comparable modalities and protocol
Proposed feature-level model	TCIA and Kaggle	4 histopathology patients	Unpaired radiograph and histopathology	EfficientNet B0 feature concatenation	Patient-level histopathology holdout	94.74%	Unpaired data and single test patient	Same internal protocol as adaptive gating
Proposed fixed-α model, α = 0.25	TCIA and Kaggle	4 histopathology patients	Unpaired radiograph and histopathology	EfficientNet B0 with global fusion weight	Patient-level histopathology holdout	94.15%	Unpaired data and single test patient	Same internal protocol as adaptive gating
Proposed adaptive gating model	TCIA and Kaggle	4 histopathology patients	Unpaired radiograph and histopathology	EfficientNet B0 with adaptive gating	Patient-level histopathology holdout	97.08%	Unpaired data and single test patient	Numerically highest, but not statistically superior to several simpler fused baselines

The comparison shows that prior osteosarcoma studies have mainly relied on single-modality models or paired multimodal designs. Histopathology-only transfer learning and hybrid CNN-ViT models have reported high TCIA accuracies, including approximately 95.65% [20] and 99% [25]. However, these studies use a single modality and, in several cases, patch-level partitions that may introduce within-slide or within-patient leakage. Their reported values are therefore not directly comparable with a patient-level held-out evaluation. Lightweight architectures such as MobileNetV2 reduce computational cost, but they report lower accuracy, approximately 91% [21]. Paired multimodal methods that combine imaging with clinical variables, metadata, or radiomic features can also be effective [11,27], but they require matched measurements that are rarely available in routine osteosarcoma datasets.

In this context, the main contribution of the present study lies in the evaluation setting rather than in a claim of absolute performance superiority. The proposed framework incorporates an additional unpaired modality from an independent cohort, reflecting a difficult but common data constraint in rare-cancer research. It also reports a patient-level histopathology holdout and paired statistical comparisons against internal baselines. The achieved accuracy of 97.08% is competitive with prior reports, but Section 4.1 shows that it is not statistically higher than several simpler fused baselines. Table 8 should therefore be interpreted as evidence of methodological positioning under unpaired multimodal constraints, not as a ranked comparison of predictive performance.

Table 8 positions the proposed framework relative to prior osteosarcoma and multimodal medical imaging studies. The reported accuracy values should not be read as a direct ranking, because the studies differ in modality, cohort size, label definition, evaluation protocol, and leakage control.

4.5. Discussion

Why might adaptive gating help under unpaired decision-level fusion? When modalities are unpaired, feature-level fusion is risky because it mixes representations from distributions that never co-occur in a patient; decision-level fusion sidesteps this by combining only calibrated probabilities. A gate that weights those probabilities per case can, in principle, downweight a modality whose prediction is unreliable for a given tile. Our weight analysis (Section 4.3) is consistent with this mechanism, though it does not prove it.

Why are the current results limited? The improvement over fixed-α = 0.25 is five tiles and does not reach significance (p = 0.074), and the adaptive model is statistically indistinguishable from logistic regression, the shallow MLP, and attention fusion. On a 171-tile single-patient test set, one to five reclassified tiles cannot support a claim of superiority. The honest reading is that all reasonable fusion strategies converge near the test-set ceiling here.

Why does p = 0.074 prevent strong claims? Because the comparison does not reach the conventional 0.05 threshold, the data do not support words such as “significantly outperforms” or “superior.” We use “numerically higher” and “non-significant improvement” throughout, and we treat the accuracy ordering as descriptive, not inferential.

Why does n = 4 histopathology patients restrict generalization? With four patients and one held-out at test, the evaluation measures generalization to a single unseen individual. Patient-level partitioning prevents within-patient tile leakage but cannot substitute for biological diversity, and it provides no estimate of between-patient variance. The result is feasibility evidence, not a generalization estimate.

Staining, scanning, and preparation bias: With one test patient, measured performance may partly reflect that patient’s staining, tissue preparation, or scanner characteristics rather than disease-general morphology. Macenko normalization reduces but does not eliminate stain variation, and a single patient cannot reveal how the model behaves across the full range of preparation conditions.

Radiograph leakage: Because the Kaggle subset lacks patient identifiers, image-level splitting cannot guarantee that all images from one individual stay in one split. The radiograph-only accuracy (83.70 ± 1.20%) may therefore be optimistic relative to a true patient-level estimate, which in turn affects the fused result.

Probabilistic remapping: The binary-to-three-class remapping injects label uncertainty into the radiograph branch for tumor-positive cases and is a plausible partial reason for the gate’s lower average radiograph-prior weight. It is a modeling convenience dictated by the radiograph labels available, not a measured viable/necrotic split.

Why were additional baselines needed? Without the panel in Table 5, the apparent advantage of adaptive gating would have been overstated. The expanded comparison shows that simpler classifiers on the fused vector are competitive, which both tempers the accuracy claim and justifies reporting logistic regression rather than XGBoost as the default.

What validation is required. Establishing clinical utility would require larger, multi-institutional, patient-level cohorts, ideally with patient-matched radiology and pathology, and prospective evaluation. The present study is a step toward methodology for such settings, not a substitute for them.

5. Future Work

Several directions follow directly from the limitations of this study. Larger multi-institutional sarcoma datasets are necessary to estimate between-patient variance reliably and to move beyond a single held-out test patient and limited patient cohorts. Future evaluation should also shift from tile-level classification to patient-level diagnosis, where tile predictions are aggregated into clinically meaningful case-level decisions. In addition, patient-matched radiology and pathology cohorts are needed to test the current unpaired fusion assumption against true paired multimodal fusion.

Methodological improvements should reflect the realities of clinical data availability. Future frameworks should incorporate missing-modality learning so that prediction remains possible when radiographs, histopathology, or structured clinical variables are unavailable. Attention-based and uncertainty-aware fusion methods should also be investigated [40]. These methods could propagate calibrated confidence rather than relying only on point probability vectors or a single scalar gate, making per-case modality weighting more principled when one modality provides uncertain or indirect evidence.

Future systems may also move beyond isolated image classification. Vision-language models offer a natural extension by combining medical images with diagnostic reports, since textual context may provide information about tissue viability, necrosis, and diagnostic uncertainty that radiographs alone lack [41]. Under data scarcity, knowledge distillation and teacher–student learning could transfer information from large multimodal models to compact deployable student networks [42]. Finally, federated learning can support model development on rare-cancer data distributed across institutions under strict privacy constraints. Personalized federated methods that balance client-specific adaptation with global generalization, such as PHH-FL [43], are directly relevant to multi-site osteosarcoma collaboration.

6. Conclusions

This study developed a decision-level integration framework for osteosarcoma tissue classification. The framework combined histopathology predictions with an unpaired radiograph-derived probability prior from independent public datasets. The method trained two EfficientNet-B0 networks separately. One network used radiographs from the Kaggle Bone Tumor Classification dataset, and the other used histopathology tiles from the TCIA Osteosarcoma Tumor Assessment collection. The histopathology branch defined the primary classification task across non-tumor, viable tumor, and necrotic tumor classes. The radiograph branch provided a coarse macrostructural probability prior, which was mapped into the same three-class decision space. The framework performed fusion only over probability vectors and did not assume patient-level, lesion-level, or image-level correspondence between the two modalities.

The single held-out histopathology test patient, P004, contained 171 tiles. Adaptive gating fusion correctly classified 166 tiles, corresponding to 97.08% accuracy, macro-F1 of 0.97, and macro-AUC of 0.99. Fixed-α fusion with α = 0.25 correctly classified 161 tiles, yielding 94.15% accuracy. McNemar’s test yielded χ² = 3.20 and p = 0.074. The numerical improvement therefore did not reach statistical significance at the 0.05 level. The adaptive model also remained statistically indistinguishable from simpler baselines, including logistic regression at 95.91%, a shallow MLP at 96.49%, and attention-based late fusion at 96.49%. These results show that adaptive gating achieved the highest numerical performance in this internal comparison, but they do not support a claim of statistically proven predictive superiority.

The learned gating weights favored histopathology on average. The mean radiograph-prior weight was α = 0.38. This pattern is consistent with the stronger tissue specificity of histopathology and with the uncertainty introduced by mapping a coarse radiographic signal into viable and necrotic tumor classes. The α values varied across test samples, suggesting that the gate did not merely reproduce a fixed global coefficient. These weights should therefore be interpreted as decision-level prior contributions. They are not patient-specific multimodal reliability scores because the radiograph and histopathology samples were not paired.

The findings serve as a methodological proof of concept. They show that confidence-aware decision-level integration can operate under unpaired and data-constrained conditions, a setting that reflects a common limitation in rare-cancer imaging research. The primary value of the framework lies in its transparent weighting mechanism rather than in a demonstrated accuracy advantage. This study does not establish clinical generalizability. Several factors constrain the conclusions, including the four-patient histopathology cohort, the single held-out test patient, the absence of radiograph patient identifiers, the possible leakage risk in image-level radiograph splitting, and the uncertainty of radiograph-to-tissue label mapping. Future work requires larger multi-institutional datasets with patient-level labels and matched radiograph–histopathology modalities. Prospective validation is necessary before this approach can support any claim of clinical utility.