Explainable Multi-Label Machine Learning Framework for Patient-Specific Antibiotic Recommendation

Abstract

Rapid selection of appropriate antimicrobial therapy is important for improving patient outcomes and supporting antimicrobial stewardship, particularly in the context of increasing antimicrobial resistance (AMR). Conventional antimicrobial susceptibility testing (AST), although essential for clinical decision-making, may not provide actionable susceptibility information immediately. This study develops an explainable multi-label machine learning framework for patient-specific antibiotic recommendation based on susceptibility prediction and ranked decision support using routinely collected clinical microbiology data. The framework integrates leakage-controlled preprocessing, microbiological and demographic feature representation, multi-label susceptibility encoding, One-vs-Rest ensemble learning, probability-based antibiotic ranking, statistical evaluation, temporal validation, and SHapley Additive exPlanations (SHAP). The dataset comprised 234 clinical records, of which 196 bacterial/other records were retained for the primary analysis after excluding 38 fungal records. A chronological partition produced 156 development records and 40 temporally held-out test records. Thirty-three antibiotic susceptibility labels were retained based on development-set availability. XGBoost, Random Forest, and LightGBM were evaluated using five-fold out-of-fold (OOF) validation. Random Forest was selected for the final recommendation and explainability analyses based on its overall performance, achieving a Micro-F1 of 0.4204, Macro-F1 of 0.3003, AUROC of 0.7069, AUPRC of 0.3979, and Precision@5 of 0.3962. A leakage-safe local antibiogram was additionally evaluated as a population-level ranking baseline, achieving a Precision@5 of 0.3077. Feature ablation showed that the combination of organism and age produced the highest OOF Micro-F1 (0.4635) and AUROC (0.7186), while the addition of gender and specimen type improved selected ranking measures but did not consistently improve aggregate classification performance. On the temporally held-out test cohort, Random Forest achieved a Micro-F1 of 0.4267, AUROC of 0.7447, AUPRC of 0.4892, and Precision@5 of 0.4350. SHAP analysis was completed for all 33 antibiotic-specific classifiers, providing global and antibiotic-level explanations of model behavior. The framework provides an interpretable approach for ranking potentially susceptible antibiotics at the patient level and is intended as clinical decision support rather than an autonomous prescribing system. Further external and prospective multicentre validation, including dedicated evaluation of challenging cases such as pan-drug resistance, is required before clinical deployment.

1. Introduction

Antimicrobial resistance (AMR) has emerged as one of the most pressing global public health challenges, threatening the effective prevention and treatment of bacterial infections. The rapid rise in AMR, driven by the inappropriate, excessive, and unnecessary use of antibiotics, has accelerated the emergence and spread of multidrug-resistant (MDR) bacteria, leading to prolonged hospitalization, increased healthcare costs, treatment failure, higher mortality rates, and diminished long-term treatment effectiveness [1,2,3]. Because conventional antimicrobial susceptibility testing often requires considerable processing time, clinicians frequently rely on empirical antibiotic prescribing before laboratory results become available, increasing the risk of suboptimal treatment and further contributing to the development of antimicrobial resistance [4].

Rapid identification of effective antimicrobial therapy is therefore essential for improving patient outcomes, minimizing unnecessary antibiotic exposure, and slowing the spread of resistant microorganisms [5,6]. In recent years, advances in artificial intelligence (AI) and machine learning (ML) have demonstrated considerable potential for predicting antimicrobial susceptibility and supporting evidence-based antibiotic selection [7,8]. Nevertheless, most existing machine learning approaches formulate antimicrobial susceptibility prediction as a binary or single-label classification problem, predicting the effectiveness of only one antibiotic at a time. Such approaches provide limited support for real-world clinical decision-making, where clinicians must evaluate multiple therapeutic options simultaneously. Furthermore, many existing models lack interpretability, making it difficult for healthcare professionals to understand the factors influencing model predictions and reducing trust in AI-assisted clinical decision support [8,9,10].

To address these limitations, this study proposes an end-to-end explainable multi-label machine learning framework for patient-specific antibiotic recommendation support using routine clinical and microbiological data. The proposed framework integrates automated data preprocessing, feature engineering, multi-label prediction, explainable artificial intelligence (XAI) using SHAP, statistical evaluation, patient-specific antibiotic ranking, and automated reporting within a unified clinical decision-support pipeline. By combining these components, the framework transforms heterogeneous clinical and microbiological information into transparent, patient-specific antibiotic susceptibility rankings that can support evidence-based antimicrobial decision-making and stewardship [11,12,13].

Unlike conventional single-label prediction methods, the proposed framework introduces a patient-specific multi-label recommendation strategy that simultaneously predicts susceptibility probabilities for multiple antibiotics and ranks them for each patient. This formulation better reflects situations in which an individual isolate may be susceptible to more than one antimicrobial option and therefore provides a ranked set of potentially suitable candidates rather than a single binary prediction [13,14]. The framework reports predicted susceptibility probabilities rather than interpreting these values as clinical certainty or prescribing instructions.

The framework is comprehensively evaluated using three ensemble learning algorithms—XGBoost, Random Forest, and LightGBM—to identify the most effective model for patient-specific antimicrobial susceptibility prediction. Model development and comparison are performed using five-fold out-of-fold (OOF) validation within a development cohort, while a temporally held-out patient cohort is reserved for final evaluation. In addition to conventional multi-label classification measures, including Micro-F1 and Macro-F1, the study evaluates AUROC and AUPRC together with ranking-based measures, including Precision@1, Precision@3, Precision@5, Recall@1, Recall@3, and Recall@5. A leakage-safe local antibiogram is included as a population-level baseline to determine whether patient-specific modeling provides additional value beyond empirical population-level susceptibility patterns.

To improve transparency and clinical interpretability, SHAP-based explainable artificial intelligence is employed to quantify the contribution of input features to antibiotic-specific model predictions. The final framework provides SHAPs for all 33 retained antibiotic-specific classifiers. In addition, a Random Forest feature-ablation analysis examines the contribution of organism, age, gender, and specimen-related information under the same out-of-fold evaluation protocol. Statistical uncertainty is assessed using bootstrap confidence intervals, and paired model comparisons are evaluated with multiple-comparison correction. The framework also generates ranked Top-K antibiotic susceptibility outputs and population-level summaries of predicted susceptibility probabilities, providing complementary information that may support antimicrobial stewardship and evidence-based decision-making within the study setting in Erbil City, Kurdistan Region of Iraq.

Importantly, the proposed framework is intended as an interpretable decision-support approach rather than an autonomous prescribing system. Patient-specific rankings should be interpreted together with laboratory AST results and clinical considerations that are not represented in the current dataset, such as infection site, allergy history, renal and hepatic function, drug interactions, contraindications, dosing requirements, antimicrobial availability, and local treatment guidelines.

2. Materials and Methods

2.1. Study Design

This study developed an explainable multi-label machine learning framework for patient-specific antibiotic recommendation support using routinely collected clinical microbiology data. The aim was to estimate the probability that an individual patient’s isolate would be reported as susceptible to multiple antibiotics and to use these probabilities to produce a ranked list of potential antibiotic options.

Because the dataset originated from a single clinical center and contained one record per patient, hospital-level grouping was not applicable; no outbreak identifier was available for additional cluster-based partitioning. The analytical workflow included clinical data collection, microbiological identification and antimicrobial susceptibility testing (AST), data quality assessment, cohort construction, chronological data partitioning, leakage-controlled preprocessing, feature construction, multi-label target generation, machine learning model development, model comparison, statistical analysis, feature ablation, temporal validation, and explainability analysis using SHAP. Because a single patient isolate may be susceptible to multiple antibiotics simultaneously, the prediction task was formulated as a multi-label classification problem. XGBoost, Random Forest, and LightGBM were evaluated using a One-vs-Rest (OvR) strategy. Model development was performed using five-fold out-of-fold (OOF) validation within the development cohort, while the most recent observations were reserved as an independent temporal-test cohort.

The framework therefore consisted of seven major stages: (i) clinical data acquisition and initial cohort construction, (ii) microbiological identification and antimicrobial susceptibility testing, (iii) data preprocessing, feature engineering, and multi-label target construction, (iv) multi-label machine learning model development using a One-vs-Rest strategy, (v) model evaluation and statistical analysis within the development cohort, (vi) final evaluation on an untouched temporal-test cohort, and (vii) explainable artificial intelligence and patient-specific antibiotic recommendation support. The overall workflow is presented in Figure 1.

Figure 1. Overall workflow of the proposed explainable multi-label framework for patient-specific antibiotic recommendation support. The study begins with 234 unique patients from a single-center dataset; 38 fungal records are excluded, leaving 196 bacterial/other records. Chronological partitioning produces 156 development and 40 untouched temporal-test patients, with 33 antibiotic labels selected using development data only. Leakage-controlled preprocessing is followed by five-fold OOF evaluation of XGBoost, Random Forest, and LightGBM using One-vs-Rest. Model comparison includes a local antibiogram baseline, bootstrap confidence intervals, paired statistical testing, and Random Forest feature ablation. Random Forest is then refitted on the 156 development patients and evaluated on the 40 temporal-test patients. SHAP analysis provides antibiotic-specific explanations, while the framework generates susceptibility probabilities and patient-specific Top-5 antibiotic rankings for decision support.

2.2. Sample Collection and Clinical Dataset Acquisition

A retrospective clinical microbiology dataset was collected from a single clinical center in Erbil, Kurdistan Region of Iraq. The original dataset comprised 234 records from 234 unique patients and contained demographic, sampling, microbiological, and antimicrobial susceptibility information, including Gender, Age, Sample Date, Strain, and Resistant, Intermediate, and sensitive results. The study population comprised 155 male patients (66.2%) and 79 female patients (33.8%). Patients were distributed across five age groups: <20, 20–39, 40–59, 60–79, and ≥80 years (Table 1).

Table 1. Demographic characteristics of the clinical dataset.

Age Group (Years)	Male (n)	Female (n)	Total (n)	Percentage (%)
<20	10	7	17	7.3
20–39	23	12	35	15
40–59	31	17	49	20.6
60–79	53	33	86	36.6
≥80	38	10	48	20.5
Total	155	79	234	100.0

Clinical specimens included urine, sputum, blood, wound swabs, Foley catheter urine, throat and other swab specimens, central venous line tips, tissue, bone, pus, abdominal pus, cerebrospinal fluid, chest-tube drainage, aspiration fluids, and other sterile body fluids. Specimen Type was retained as a predictive feature in the revised machine-learning framework to account for the heterogeneity of the clinical samples.

Each record corresponded to one unique patient and contained demographic information, specimen information, microorganism identification, and antimicrobial susceptibility results. Microorganism identification and AST were performed using the VITEK^® 2 Compact automated identification and antimicrobial susceptibility testing system (bioMérieux S.A., Marcy l’Étoile, France). Ethical approval was obtained from the Scientific Research Ethics Committee of Erbil Polytechnic University (Approval No. 214/00218 HRE, 9 December 2025), and all study procedures were conducted in accordance with the Declaration of Helsinki.

The unequal distribution across age groups reflects the observed composition of the retrospective clinical cohort and was not imposed through sampling or artificial balancing; therefore, the age-group frequencies should be interpreted as descriptive characteristics of the study population rather than as predefined sampling proportions.

2.3. Culture of Clinical Specimens

All clinical specimens were examined and processed according to standard microbiological procedures. Depending on the specimen type, samples were inoculated onto appropriate culture media, including Blood Agar, MacConkey Agar, Chocolate Agar, Mannitol Salt Agar (MSA), Cysteine Lactose Electrolyte Deficient (CLED) Agar, Sabouraud Dextrose Agar (SDA), and other selective or differential media as required. Blood culture specimens were processed using an automated BACT/ALERT® blood culture system (bioMérieux S.A., Marcy l’Étoile, France) according to the manufacturer’s instructions. Following incubation, typically overnight at 37 °C, clinical isolates were initially identified based on colony characteristics, Gram-stain morphology, hemolysis on blood agar, and biochemical tests.

2.4. Identification and Antimicrobial Susceptibility Testing

The VITEK^® 2 Compact system was used for the automated identification of bacterial and fungal isolates and for antimicrobial susceptibility testing. Several colonies from pure 18–24 h cultures were transferred using sterile loops into clear plastic test tubes containing 3 mL of 0.45% sterile saline solution (pH 4.5–7.0). Bacterial suspensions were adjusted to 0.5–0.63 McFarland using a DensiCHEK™ turbidity meter (bioMérieux S.A., Marcy l’Étoile, France), whereas yeast suspensions were adjusted to 1.96–2.25 McFarland.

Gram-negative bacteria were identified using VITEK® 2 GN cards, Gram-positive bacteria using VITEK® 2 GP cards, and yeast isolates using VITEK® 2 YST cards (bioMérieux S.A., Marcy l’Étoile, France), and yeast isolates using YST cards. For antimicrobial susceptibility testing, VITEK® 2 AST-GN417 and AST-GN419 cards were used for Gram-negative bacteria, AST-GP580 and AST-GP592 cards for Gram-positive bacteria, and AST-YS08 cards for yeast (bioMérieux S.A., Marcy l’Étoile, France). After inoculation, the cards and corresponding suspension tubes were loaded into the VITEK^® 2 instrument, which automatically processed the isolates and generated the identification and antimicrobial susceptibility results [15,16].

The antimicrobial susceptibility results were interpreted according to the Clinical and Laboratory Standards Institute (CLSI) Performance Standards for Antimicrobial Susceptibility Testing (M100), 35th edition (2025). The resulting classifications were recorded as Resistant (R), Intermediate (I), or Sensitive/Susceptible (S) according to the applicable CLSI breakpoints. For the machine-learning analysis, only the recorded Sensitive/Susceptible results were used to define positive multi-label targets. Resistant and Intermediate results were not modeled as separate target classes; therefore, the prediction task focused on identifying antibiotics for which a Sensitive/Susceptible result was recorded.

2.5. Data Preprocessing and Leakage Control

Before model development, the clinical dataset underwent data-quality assessment and preprocessing to ensure consistency and compatibility with the machine-learning models. The audit confirmed 234 records from 234 unique patients, with no duplicate records identified in the analytical dataset. The records were classified according to organism type, comprising 196 bacterial/other records and 38 fungal records. Because the primary objective was antibacterial susceptibility prediction, the 38 fungal records were excluded from the primary analysis, leaving 196 patients for subsequent analysis.

The 196-patient cohort was ordered chronologically according to Sample Date and divided into 156 development patients and 40 temporal-test patients. The temporal-test cohort remained completely untouched during model development and was not used for preprocessing, fitting, antibiotic-label selection, model training, model comparison, feature ablation, or model selection.

To prevent information leakage, preprocessing was performed within each training fold of the five-fold out-of-fold (OOF) procedure. Numerical variables were imputed, where necessary, using medians calculated from the training subset only, while categorical variables were imputed using the training-fold mode and encoded using one-hot encoding fitted on the training subset. The resulting preprocessing transformations were then applied unchanged to the corresponding validation fold without refitting. For the final temporal-test evaluation, all preprocessing parameters were estimated using the complete 156-patient development cohort and then applied unchanged to the 40 temporal-test patients. No standardization or z-score scaling was applied.

This fold-specific preprocessing strategy ensured that information from validation and temporal-test observations could not influence the transformations used for model training or evaluation [17].

2.6. Feature Engineering

The original demographic and microbiological variables were converted into a structured representation suitable for machine learning. Age was represented numerically in years, while Gender was retained as a categorical variable. The microorganism information was represented as Organism, and the source of the clinical specimen was represented separately as Specimen Type. Sample Date was converted into a standardized date representation, from which Month and Year were extracted.

The final predictive feature set consisted of Age, Gender, Organism, Specimen Type, Month, and Year, as summarized in Table 2. These variables represent demographic, microbiological, specimen-related, and temporal characteristics of each patient record.

Table 2. Predictive features used in the machine-learning framework.

Feature	Type	Description
Age	Numerical	Patient age in years
Gender	Categorical	Gender recorded in the clinical dataset
Organism	Categorical	Identified microorganism
Specimen Type	Categorical	Clinical specimen source/type
Month	Numerical	Month of specimen collection
Year	Numerical	Year of specimen collection

For patient $i$, the resulting feature vector was defined as

$$x_i=\left[A,g,e_i,\hspace{0.25em},G,ende{r}_i\hspace{0.25em}Organis{m}_i\hspace{0.25em}SpecimenTyp{e}_i\hspace{0.25em}Mont{h}_i\hspace{0.25em}Yea{r}_i\right].$$

No antimicrobial susceptibility result was included among the predictor variables.

2.7. Multi-Label Representation of Antibiotic Susceptibility

The susceptibility prediction task was formulated as a multi-label classification problem because a single patient isolate may be susceptible to several antibiotics simultaneously. This representation allows multiple antibiotic labels to be assigned to the same patient rather than forcing each observation into a single class [18,19].

For patient $i$, the susceptibility target vector was defined as

$$y_i=\left(y_{i1},y_{i2},\dots ,y_{iL}\right),$$

where $L$ denotes the total number of antibiotic labels included in the final analysis, and

$$y_{ij}=\left\{\begin{array}{cc}1,& \mathrm{if} \mathrm{antibiotic} j \mathrm{was} \mathrm{recorded} \mathrm{as} \mathrm{Sensitive}/\mathrm{Susceptible},\\ 0,& \mathrm{otherwise}.\end{array}\right.$$

The susceptible antibiotic lists were converted into binary multi-label vectors using the MultiLabelBinarizer procedure. Initially, 45 candidate antibiotic labels were identified from the dataset. To prevent information from the temporal-test cohort from influencing the definition of the prediction task, antibiotic-label selection was performed using the 156-patient development cohort only. This procedure resulted in 33 antibiotic labels being retained for the final analysis.

The resulting target matrices were therefore:

$Y_{dev}\in {\left\{0,1\right\}}^{156\times 33}$ for the development cohort and

$Y_{test}\in {\left\{0,1\right\}}^{40\times 33}$ for the temporal-test cohort.

The Resistant (R) and Intermediate (I) fields were not encoded as separate target labels. Consequently, a value of zero indicates that a Sensitive/Susceptible label was not recorded for the corresponding antibiotic and should not automatically be interpreted as laboratory-confirmed resistance. This binary multi-label representation provides the basis for simultaneously estimating susceptibility probabilities for multiple antibiotics and generating patient-specific ranked antibiotic recommendations.

2.8. Proposed Explainable Multi-Label Machine Learning Framework

Three ensemble-based machine learning algorithms were evaluated within the proposed multi-label framework: XGBoost, Random Forest, and LightGBM [20,21,22]. Because each patient may have susceptibility labels for multiple antibiotics, all three models were implemented using a One-vs-Rest (OvR) strategy. Under this approach, an independent binary classifier was fitted for each of the 33 antibiotic labels.

For patient $i$ and antibiotic $j$, the corresponding classifier estimates the probability of a Sensitive/Susceptible result as

$$p_{ij}=P\left(y_{ij}=1\mid x_i\right),$$

where $x_i$ represents the feature vector of the patient $i$, and $p_{ij}$ denotes the predicted probability that the patient’s isolate will be recorded as Sensitive/Susceptible to the antibiotic $j$.

Accordingly, the model produces a 33-dimensional probability vector for each patient:

$$p_i=\left[p_{i1},p_{i2},\dots ,p_{i33}\right].$$

These predicted probabilities were used for quantitative model evaluation and for ranking antibiotics according to their estimated susceptibility probability. The resulting rankings form the basis of the patient-specific antibiotic recommendation-support component described in Section 2.12.

The three candidate models were compared using the same development-cohort OOF validation procedure. Following model comparison, the best-performing model was selected according to the predefined evaluation criteria and subsequently refitted using the complete development cohort before evaluation on the untouched temporal-test cohort.

2.9. Model Validation and Performance Evaluation

2.9.1. Five-Fold Out-of-Fold Validation

Model development and internal validation were performed exclusively on the 156-patient development cohort using five-fold out-of-fold (OOF) validation. In each iteration, four folds were used for training and the remaining fold for validation, ensuring that every patient was used once for validation. The five validation predictions were combined to obtain OOF predictions for all development patients. All preprocessing and model fitting were performed using the corresponding training folds only. The 40 temporal-test patients were excluded from all development-stage analyses, including model comparison, feature ablation, and model selection, and were reserved for final temporal evaluation.

2.9.2. Classification and Ranking Metrics

Model performance was evaluated using Micro-F1, Macro-F1, AUROC, AUPRC, Precision@1, Precision@3, Precision@5, Recall@1, Recall@3, and Recall@5. These metrics were selected to provide complementary information on overall multi-label classification performance, antibiotic-specific discrimination, and the quality of the ranked susceptibility outputs [18,19,23,24].

Micro-F1 summarizes performance across all antibiotic labels by aggregating true positives, false positives, and false negatives before calculating the F1-score:

$$F{1}_{\mathrm{m}\mathrm{i}\mathrm{c}\mathrm{r}\mathrm{o}}={\displaystyle \frac{2TP}{2TP+FP+FN}}.$$

Macro-F1 calculates the F1-score separately for each antibiotic and then averages the results, giving equal weight to all modeled antibiotics:

$$F{1}_{\mathrm{m}\mathrm{a}\mathrm{c}\mathrm{r}\mathrm{o}}={\displaystyle \frac{1}{L}}{\displaystyle \sum _{j=1}^L}F{1}_j,$$

where $L$ represents the total number of modeled antibiotic labels.

AUROC was used to measure the ability of each antibiotic-specific classifier to distinguish between patients with and without a recorded Sensitive/Susceptible outcome across different probability thresholds. The true-positive rate (TPR) and false-positive rate (FPR) were calculated as

$$TPR={\displaystyle \frac{TP}{TP+FN}},FPR={\displaystyle \frac{FP}{FP+TN}}.$$

AUROC represents the area under the resulting receiver operating characteristic curve:

$$AUROC={\int }_0^{1}TPR\left(FPR\right)\hspace{0.17em}d\left(FPR\right).$$

Higher AUROC values indicate better discrimination between positive and negative susceptibility outcomes.

AUPRC was used as a complementary discrimination measure by summarizing the relationship between precision and recall across probability thresholds. Precision and recall were calculated as

$$Precision={\displaystyle \frac{TP}{TP+FP}},Recall={\displaystyle \frac{TP}{TP+FN}},$$

and the area under the precision–recall curve was defined as

$$AUPRC={\int }_0^{1}Precision\left(Recall\right)\hspace{0.17em}d\left(Recall\right).$$

AUPRC is particularly useful for assessing performance when susceptibility labels have unequal frequencies across antibiotics.

For patient-specific recommendation ranking, antibiotics were ordered according to their predicted Sensitive/Susceptible probabilities. Precision@$K$ measured the proportion of the $K$ highest-ranked antibiotics that corresponded to antibiotics recorded as Sensitive/Susceptible for the patient:

$$Precision@K={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \frac{\mid Top{K}_i\cap R_i\mid }{K}},$$

where $N$ is the number of evaluated patients, $Top{K}_i$ is the patient’s $K$ highest-ranked antibiotics, and $R_i$ is the set of antibiotics recorded as Sensitive/Susceptible for the patient $i$.

Recall@$K$ measured the proportion of the patient’s recorded Sensitive/Susceptible antibiotics recovered within the Top-$K$ ranking:

$$Recall@K={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \frac{\mid Top{K}_i\cap R_i\mid }{\mid R_i\mid }},$$

Considering patients with at least one recorded Sensitive/Susceptible antibiotic.

Because the proposed framework generates a Top-5 antibiotic ranking, Precision@5 and Recall@5 were treated as the principal ranking measures. Precision@1, Precision@3, Recall@1, and Recall@3 were additionally reported to assess the performance of shorter ranked antibiotic lists.

In addition to the aggregate multi-label metrics, antibiotic-specific performance was evaluated separately for each modeled antibiotic using AUROC, AUPRC, precision, recall, and F1-score, where both outcome classes were present in the corresponding validation data.

Accuracy was not used as a primary evaluation measure because the task is a multi-label classification problem with unequal antibiotic-label frequencies and substantial class imbalance. In this setting, aggregate accuracy can be dominated by the majority negative decisions across labels and may therefore provide a less informative assessment of susceptibility prediction and ranking quality. Micro-F1, Macro-F1, AUROC, AUPRC, and Precision@K/Recall@K were therefore prioritized to capture overall classification performance, label-balanced performance, discrimination, and patient-level ranking quality.

2.10. Statistical Analysis

Uncertainty in model performance was estimated using 2000 patient-level bootstrap resamples of the development-cohort OOF predictions. For each bootstrap resample, patients were sampled with replacement, and the corresponding performance metrics were recalculated. The 2.5th and 97.5th percentiles of the bootstrap distributions were used to obtain 95% confidence intervals (CIs) [25].

Pairwise comparisons between models were performed using paired patient-level bootstrap resampling, preserving the correspondence between predictions for the same patients. The primary comparison used Precision@5, reflecting the Top-5 antibiotic ranking generated by the framework. For two models $A$ and $B$, the paired performance difference was calculated as

$${\Delta }_{A-B}=M_A-M_B,$$

where $M_A$ and $M_B$ represent the corresponding performance measures of models $A$ and $B$, respectively.

Two-sided bootstrap comparisons produced raw p-values, and the Holm correction was applied to adjust for multiple pairwise comparisons among the evaluated models. Statistical evidence for a difference was assessed using the adjusted p-values together with the corresponding 95% bootstrap CIs.

This patient-level bootstrap procedure avoids treating the five OOF folds as independent clinical samples and provides uncertainty estimates and paired comparisons at the patient level.

2.11. SHAP-Based Model Explainability

To improve model transparency and facilitate interpretation of the predicted susceptibility patterns, SHapley Additive exPlanations (SHAPs) were used to investigate the contribution of individual input features to the predictions of the final Random Forest model [26]. SHAP provides feature-attribution values that indicate how individual predictors contribute to a model output and can be used to obtain both global and patient-level explanations.

Because the final model uses a One-vs-Rest (OvR) architecture, SHAP analysis was performed at the antibiotic-specific classifier level. Thus, the contribution of the input features was examined separately for the individual antibiotic susceptibility predictions rather than treating the complete multi-label model as a single output.

For a model prediction, the SHAP decomposition was expressed as

$$f\left(x\right)={\varphi }_0+{\displaystyle \sum _{m=1}^M}{\varphi }_m,$$

where $f\left(x\right)$ represents the model output for a given patient and antibiotic classifier, ${\varphi }_0$ is the baseline model output, ${\varphi }_m$ represents the contribution of feature $m$, and $M$ denotes the total number of input features.

The SHAP analysis provided two complementary levels of interpretation. Global explanations were used to summarize the overall contribution of the available predictors across observations, whereas local explanations were used to examine how individual features contributed to the prediction for a specific patient and antibiotic classifier.

This analysis was used to improve the transparency of the patient-specific susceptibility predictions and to identify the features most strongly associated with the model’s predicted outputs [26].

2.12. Patient-Specific Antibiotic Recommendation Support

For each patient, the final model generated a predicted susceptibility probability for each of the 33 modeled antibiotics:

$$p_i=\left[p_{i1},p_{i2},\dots ,p_{i33}\right].$$

The antibiotics were ranked in descending order according to their predicted susceptibility probabilities, and the five highest-ranked antibiotics were retained as the patient’s Top-5 susceptibility ranking.

The reported values represent predicted susceptibility probabilities, not confidence scores or probabilities of clinical treatment success. Accordingly, patient-specific antibiotic recommendation support refers only to the ranked susceptibility information generated by the framework and does not imply autonomous antibiotic prescribing. The ranked outputs are intended to support interpretation of susceptibility patterns and must not replace laboratory antimicrobial susceptibility testing (AST), antimicrobial stewardship principles, or clinical judgment.

The available dataset did not contain several clinical variables relevant to antibiotic selection, including infection site and severity, allergy history, renal and hepatic function, drug interactions, contraindications, previous antimicrobial exposure, dosing requirements, and drug availability. Therefore, the ranked outputs require confirmation against laboratory AST and consideration of relevant clinical factors before any treatment decision.

No clinical probability threshold was imposed to exclude a candidate antibiotic from the ranked Top-5 output because an evidence-based threshold could not be established from the available dataset. Instead, a sensitivity audit evaluated the number of ranked candidates exceeding probability levels of 0.20, 0.30, 0.50, and 0.70 using development-cohort OOF predictions. These thresholds were used only for transparent auditing and were not used for model selection or applied to the temporal-test cohort. Thus, the predicted probabilities should be interpreted as model-derived susceptibility estimates rather than measures of clinical treatment suitability.

2.13. Local Antibiogram Baseline

A local antibiogram baseline was included to determine whether the patient-specific machine-learning models provided additional ranking information beyond a conventional population-level susceptibility profile. For each OOF training fold, the susceptibility prevalence of antibiotic $j$ was calculated using the training patients only:

$$q_j={\displaystyle \frac{{\sum }_{i\in T}{y}_{ij}}{\mid T\mid }},$$

where $T$ denotes the patients in the corresponding training fold and $y_{ij}$ indicates whether antibiotic $j$ was recorded as Sensitive/Susceptible for patient $i$.

The resulting prevalence values were used to rank antibiotics for patients in the corresponding validation fold. Because the antibiogram does not use patient-specific predictors, patients within the same validation fold received the same population-level ranking.

The baseline was evaluated using AUROC, AUPRC, Precision@1, Precision@3, Precision@5, Recall@1, Recall@3, and Recall@5. Threshold-dependent Micro-F1 and Macro-F1 were not applicable because the antibiogram provides population-level rankings rather than patient-specific binary predictions. The implementation explicitly treated these F1 measures as unavailable for this baseline.

2.14. Feature Ablation Methodology

Feature ablation was performed using the Random Forest model to assess the incremental contribution of the available predictors. Feature ablation was performed using the Random Forest model to assess the incremental contribution of the available predictors (Table 3).

Table 3. Five feature configurations were evaluated.

Variant	Feature Set
RF-A1	Organism
RF-A2	Organism + Age
RF-A3	Organism + Age + Gender
RF-A4	Organism + Age + Gender + Specimen Type
RF-A5	Age + Gender + Organism + Specimen Type + Month + Year

Each configuration was evaluated using the same five-fold OOF procedure, leakage-controlled preprocessing, Random Forest configuration, and evaluation metrics. The ablation analysis was used to assess the contribution of demographic, specimen-related, and temporal information beyond microorganism identity. The full six-feature representation was retained for the primary model. These five configurations are directly supported by the implementation.

2.15. Temporal-Test Evaluation

Following development-cohort model comparison, Random Forest was selected as the final recommendation model.

The selected model was refitted using all 156 development patients and evaluated once on the 40 temporal-test patients. The temporal-test cohort was not used for antibiotic-label selection, preprocessing fitting, model training, model comparison, feature ablation, or model selection.

The final temporal evaluation used Micro-F1, Macro-F1, Micro-Precision, Micro-Recall, AUROC, AUPRC, Precision@1, Precision@3, Precision@5, Recall@1, Recall@3, and Recall@5. The resulting performance therefore represents an out-of-time evaluation of the selected model rather than another internal cross-validation estimate.

2.16. Model Training and Hyperparameter Configuration

XGBoost, Random Forest, and LightGBM were trained using the One-vs-Rest strategy within the five-fold OOF development procedure. Preprocessing and model fitting were restricted to the corresponding training folds.

The final recommendation model was a Random Forest, configured with 300 trees, a maximum tree depth of 10, and balanced class-subsample weighting. Model selection was performed using the development-cohort OOF results, after which the selected Random Forest model was refitted to all 156 development patients before the final temporal-test evaluation.

No information from the temporal-test cohort was used for model selection or parameter estimation. No systematic hyperparameter search or nested cross-validation was performed. The models were evaluated using fixed, prespecified configurations, with all model fitting conducted within the corresponding training folds.

All machine-learning and statistical analyses were implemented in Python 3.12. Data preprocessing, MultiLabelBi-narizer, One-vs-Rest classification, Random Forest modeling, and performance evaluation were implemented using scikit-learn (version 1.9.1). XGBoost (version 3.4.1) and LightGBM (version 4.7.0) were used for the corresponding machine-learning models, while SHAP (version 0.52.0) was used for model explainability and feature-attribution analysis.

3. Results

3.1. Dataset Characteristics

The study dataset comprised 234 records from 234 unique patients. The data-quality assessment identified no duplicate records. The records included demographic information together with specimen collection dates, microorganism identification, and antimicrobial susceptibility results. The organism-type assessment identified 196 bacterial/other records and 38 fungal records. Because the primary objective was antibacterial susceptibility prediction, the fungal records were excluded from the subsequent modeling analysis.

The 196 retained records were ordered chronologically according to the sample date. The earliest 156 patients constituted the development cohort, whereas the most recent 40 patients were reserved as an untouched temporal-test cohort. Following development-only antibiotic-label selection, 33 antibiotic labels were retained from the initial set of 45 candidate labels. The resulting target matrices contained 33 antibiotic columns, giving dimensions of $156\times 33$ for the development cohort and $40\times 33$ for temporal testing. The principal dataset and analytical cohort characteristics are summarized in Table 4.

Table 4. Characteristics of the study dataset and analytical cohorts.

Dataset Characteristic	Number
Original records	234
Unique patients	234
Bacterial/other records	196
Fungal records excluded	38
Development cohort	156
Temporal-test cohort	40
Initial candidate antibiotic labels	45
Final modeled antibiotic labels	33
Development target matrix	156 × 33
Temporal-test target matrix	40 × 33

The chronological partition established a clear separation between model development and evaluation on later observations. The temporal-test cohort was not used during preprocessing, antibiotic-label selection, model comparison, feature ablation, or model selection.

3.2. Distribution of Microorganisms

The 234-record cohort showed substantial heterogeneity in microorganism composition (Table 5; Figure 2). Klebsiella pneumoniae was the most frequently represented organism, accounting for 29.06% (68/234) of the records, followed by Escherichia coli (14.53%, 34/234) and Acinetobacter baumannii (12.82%, 30/234). Among fungal organisms, Candida albicans was the most frequently recorded species (7.69%, 18/234), followed by Candida spp. (3.42%, 8/234). Pseudomonas aeruginosa accounted for 5.98% (14/234), while Staphylococcus aureus and MRSA each accounted for 3.85% (9/234).

Table 5. Distribution of microorganisms and no-growth records in the original 234-record cohort.

No.	Microorganism	Number of Records	Percentage (%)
1	Achromobacter denitrificans	1	0.43
2	Achromobacter xylosoxidans	1	0.43
3	Acinetobacter baumannii	30	12.82
4	Aspergillus spp.	3	1.28
5	Brucella melitensis	1	0.43
6	Burkholderia cepacia	1	0.43
7	Candida albicans	18	7.69
8	Candida glabrata	2	0.85
9	Candida kefyr	1	0.43
10	Candida parapsilosis	2	0.85
11	Candida tropicalis	4	1.71
12	Candida spp.	8	3.42
13	Enterobacter cloacae	4	1.71
14	Enterococcus faecium	2	0.85
15	Escherichia coli	34	14.53
16	Klebsiella pneumoniae	68	29.06
17	Proteus mirabilis	2	0.85
18	Proteus vulgaris	2	0.85
19	Pseudomonas aeruginosa	14	5.98
20	Raoultella ornithinolytica	1	0.43
21	Serratia marcescens	1	0.43
22	Sphingomonas paucimobilis	1	0.43
23	Staphylococcus aureus	9	3.85
24	Staphylococcus aureus (MRSA)	9	3.85
25	Staphylococcus epidermidis	5	2.14
26	Staphylococcus haemolyticus	1	0.43
27	Staphylococcus hominis	2	0.85
28	Stenotrophomonas maltophilia	2	0.85
29	Streptococcus anginosus	1	0.43
30	Streptococcus gallinarum	1	0.43
31	Yersinia kristensenii	1	0.43
32	No bacterial pathogen isolated/no growth	2	0.85
Total		234	100.00

Figure 2. Distribution of microorganisms in the original 234-record clinical cohort. The horizontal bar chart presents the frequency of each identified microorganism, with the corresponding number and percentage of records shown at the end of each bar. The cohort was dominated by Klebsiella pneumoniae (68, 29.06%), Escherichia coli (34, 14.53%), and Acinetobacter baumannii (30, 12.82%), while the remaining microorganisms occurred at lower frequencies; fungal organisms and records with no bacterial pathogen isolated/no growth are distinguished separately.

The remaining organisms occurred at substantially lower frequencies, with most represented by two or fewer records. This heterogeneous distribution is relevant to the susceptibility-prediction task because the multi-label outcomes reflect the antimicrobial susceptibility profiles associated with individual microorganisms. More frequently represented organisms consequently contribute more observations to the overall susceptibility patterns, whereas estimates for rarely observed organisms are based on fewer records.

The observed distribution also supports the inclusion of Organism as a core predictive feature. However, microorganism frequency alone does not establish a causal relationship with antimicrobial resistance. Rather, it describes the composition of the study population and provides context for interpreting the subsequent organism-dependent susceptibility predictions.

3.3. Five-Fold Out-of-Fold Model Performance

Five-fold out-of-fold (OOF) validation was performed within the 156-patient development cohort to compare XGBoost, Random Forest, and LightGBM under the same multi-label prediction framework. Each model was evaluated using the OOF predictions generated across the five validation folds, and the resulting performance measures are summarized in Table 6 and Figure 3. The implementation completed all five folds for each of the three models.

Table 6. Five-fold out-of-fold performance of the evaluated models.

Model	Micro-F1	Macro-F1	Micro-Precision	Micro-Recall	AUROC	AUPRC	Precision@1	Precision@3	Precision@5	Recall@1	Recall@3	Recall@5
XGBoost	0.4011	0.2778	0.5068	0.3319	0.6538	0.3149	0.5513	0.4231	0.3564	0.2360	0.4369	0.5429
Random Forest	0.4204	0.3003	0.6072	0.3215	0.7069	0.3979	0.5769	0.4466	0.3962	0.2433	0.4410	0.5881
LightGBM	0.2841	0.1933	0.4118	0.2168	0.6046	0.2515	0.4359	0.3739	0.3179	0.1770	0.3862	0.4869
Local Antibiogram	—	—	—	—	0.3563	0.1118	0.4679	0.3440	0.3077	0.1454	0.3192	0.4485

Figure 3. Five-fold out-of-fold (OOF) performance comparison of the evaluated models. The figure presents (A) Micro-F1, (B) Macro-F1, (C) AUROC, (D) AUPRC, and (E) Precision@5. Random Forest achieved the highest values among the three machine-learning models across all five measures, while the local antibiogram provided a population-level reference for the ranking-based measures.

Among the three machine-learning models, Random Forest achieved the strongest overall OOF performance. It obtained the highest Micro-F1 (0.4204), Macro-F1 (0.3003), AUROC (0.7069), and AUPRC (0.3979). It also produced the highest-ranking performance, with Precision@1, Precision@3, and Precision@5 of 0.5769, 0.4466, and 0.3962, respectively, and Recall@1, Recall@3, and Recall@5 of 0.2433, 0.4410, and 0.5881.

XGBoost showed the second-highest overall performance for most measures, with a Micro-F1 of 0.4011, Macro-F1 of 0.2778, AUROC of 0.6538, and AUPRC of 0.3149. Its Precision@5 and Recall@5 were 0.3564 and 0.5429, respectively. LightGBM produced lower values across the principal measures, with a Micro-F1 of 0.2841, Macro-F1 of 0.1933, AUROC of 0.6046, and AUPRC of 0.2515. Its Precision@5 and Recall@5 were 0.3179 and 0.4869, respectively.

The ranking measures further showed that Random Forest achieved the highest Precision@1, Precision@3, and Precision@5, as well as the highest Recall@1, Recall@3, and Recall@5 among the three machine-learning models. This indicates that its predicted susceptibility probabilities provided the strongest Top-K ranking performance within the development cohort.

For comparison, the local antibiogram was evaluated separately as a leakage-safe population-level ranking baseline. It achieved an AUROC of 0.3563, AUPRC of 0.1118, Precision@1 of 0.4679, Precision@3 of 0.3440, Precision@5 of 0.3077, Recall@1 of 0.1454, Recall@3 of 0.3192, and Recall@5 of 0.4485. Micro-F1 and Macro-F1 were not calculated for this baseline because they provide population-level rankings rather than threshold-dependent binary predictions.

The model-comparison plots in Figure 3 provide a visual representation of these results across Micro-F1, Macro-F1, AUROC, AUPRC, and Precision@5. The Local Antibiogram is included only in the panels for which its ranking-based evaluation is defined; it is not assigned artificial Micro-F1 or Macro-F1 values.

Bootstrap uncertainty analysis further supported the observed performance differences. Based on 2000 patient-level bootstrap resamples of the OOF predictions, Random Forest achieved a Micro-F1 of 0.4200 (95% CI: 0.3636–0.4766), a Macro-F1 of 0.2920 (95% CI: 0.2240–0.3538), and Precision@5 of 0.3964 (95% CI: 0.3474–0.4474). The corresponding estimates for XGBoost were 0.4000 (95% CI: 0.3414–0.4609), 0.2698 (95% CI: 0.2025–0.3438), and 0.3564 (95% CI: 0.3102–0.4051), respectively. LightGBM achieved 0.2836 (95% CI: 0.2340–0.3382), 0.1876 (95% CI: 0.1353–0.2445), and 0.3184 (95% CI: 0.2756–0.3641), respectively. The local antibiogram achieved a Precision@5 of 0.3082 (95% CI: 0.2679–0.3474). These bootstrap intervals provide uncertainty estimates around the principal OOF performance measures and complement the paired Precision@5 comparisons reported in Section 3.4. The corresponding bootstrap confidence intervals are summarized in Table 7.

Table 7. Bootstrap 95% confidence intervals for principal out-of-fold performance measures.

Model	Metric	Mean	95% CI
XGBoost	Micro-F1	0.4000	0.3414–0.4609
XGBoost	Macro-F1	0.2698	0.2025–0.3438
XGBoost	Precision@5	0.3564	0.3102–0.4051
Random Forest	Micro-F1	0.4200	0.3636–0.4766
Random Forest	Macro-F1	0.2920	0.2240–0.3538
Random Forest	Precision@5	0.3964	0.3474–0.4474
LightGBM	Micro-F1	0.2836	0.2340–0.3382
LightGBM	Macro-F1	0.1876	0.1353–0.2445
LightGBM	Precision@5	0.3184	0.2756–0.3641
Local Antibiogram	Precision@5	0.3082	0.2679–0.3474

Note: 95% CIs were obtained from 2000 patient-level bootstrap resamples of the development-cohort OOF predictions.

3.4. Statistical Comparison of Model Performance

To assess whether differences in recommendation performance were statistically meaningful, paired patient-level bootstrap comparisons were performed using 2000 bootstrap resamples of the OOF predictions. Because the final recommendation output consists of five ranked antibiotics, Precision@5 was used as the primary metric for pairwise comparison. The resulting p-values were adjusted using the Holm correction for multiple comparisons.

As shown in Table 8, Random Forest significantly outperformed XGBoost in Precision@5, with a mean difference of 0.0400 in favor of Random Forest (95% CI: 0.0141–0.0667; raw p = 0.003, Holm-adjusted p = 0.006). Random Forest also significantly outperformed LightGBM (mean difference = 0.0780, 95% CI: 0.0487–0.1103; adjusted p < 0.001) and the local antibiogram baseline (mean difference = 0.0882, 95% CI: 0.0564–0.1205; adjusted p < 0.001).

Table 8. Paired patient-level bootstrap comparisons of Precision@5.

Model Comparison	Mean Difference	95% CI	Raw p	Holm-Adjusted p
XGBoost vs. Random Forest	−0.0400	−0.0667 to −0.0141	0.003	0.006
XGBoost vs. LightGBM	0.0380	0.0141 to 0.0628	0.001	0.004
XGBoost vs. Local Antibiogram	0.0482	0.0141 to 0.0833	0.002	0.006
Random Forest vs. LightGBM	0.0780	0.0487 to 0.1103	<0.001	<0.001
Random Forest vs. Local Antibiogram	0.0882	0.0564 to 0.1205	<0.001	<0.001
LightGBM vs. Local Antibiogram	0.0102	−0.0244 to 0.0449	0.586	0.586

Note: Mean differences were calculated as Model A − Model B using paired patient-level bootstrap resampling of the OOF predictions (2000 resamples). Positive values favor Model A, whereas negative values favor Model B. p-values were adjusted using the Holm procedure for multiple comparisons.

XGBoost also significantly outperformed LightGBM (mean difference = 0.0380, 95% CI: 0.0141–0.0628; adjusted p = 0.004) and the local antibiogram (mean difference = 0.0482, 95% CI: 0.0141–0.0833; adjusted p = 0.006). In contrast, the difference between LightGBM and the local antibiogram was not statistically significant (mean difference = 0.0102, 95% CI: −0.0244–0.0449; adjusted p = 0.586).

These results provide additional statistical support for selecting Random Forest as the final recommendation model. Importantly, the analysis was performed at the patient level using paired bootstrap resampling rather than treating the five cross-validation folds as independent clinical observations.

3.5. Feature Ablation Analysis

As shown in Table 9 and Figure 4, feature ablation was performed using Random Forest with five progressively expanded feature configurations. The results showed that Organism + Age achieved the highest Micro-F1 (0.4635) and AUROC (0.7186), while adding Gender and Specimen Type produced the best AUPRC (0.4000), Precision@5 (0.4000), and Recall@5 (0.5966). The full six-feature representation was retained for the primary analysis because the ablation experiment was designed to assess the contribution of individual feature groups rather than to replace the prespecified primary feature representation through post hoc feature selection.

Table 9. Random Forest feature ablation results.

Feature Configuration	Micro-F1	Macro-F1	AUROC	AUPRC	Precision@5	Recall@5
Organism	0.4299	0.3945	0.7115	0.3362	0.3692	0.5420
Organism + Age	0.4635	0.3792	0.7186	0.3869	0.3872	0.5782
Organism + Age + Gender	0.4420	0.3525	0.7136	0.3820	0.3859	0.5788
Organism + Age + Gender + Specimen Type	0.4325	0.3173	0.7178	0.4000	0.4000	0.5966
Full feature set	0.4204	0.3003	0.7069	0.3979	0.3962	0.5881

Figure 4. Random Forest feature-ablation performance across five feature configurations, showing the effect of progressively adding Age, Gender, Specimen Type, Month, and Year to the Organism-based representation using Micro-F1, Macro-F1, AUROC, AUPRC, Precision@5, and Recall@5.

The ablation therefore indicates that Age contributed to overall predictive performance, whereas the addition of Gender and Specimen Type improved several ranking-oriented measures. The temporal variables Month and Year did not further improve the full configuration. Because the study period covered only approximately 13 months, Month and Year were included as available temporal predictors but were not interpreted as evidence of long-term antimicrobial-resistance trends.

3.6. Temporal-Test Performance

After completion of the five-fold OOF model comparison, Random Forest was selected as the final model and refitted using the complete 156-patient development cohort. It was then evaluated once on the independent 40-patient temporal-test cohort, which was defined chronologically at the patient-record level as the later records in the dataset and remained completely untouched throughout model development and model selection. The final temporal-test results are presented in Table 10.

Table 10. Random Forest performance on the independent temporal-test cohort.

Metric	Temporal-Test Result
Micro-F1	0.4267
Macro-F1	0.2249
Micro-Precision	0.7442
Micro-Recall	0.2991
AUROC	0.7447
AUPRC	0.4892
Precision@1	0.6250
Recall@1	0.1873
Precision@3	0.5000
Recall@3	0.3292
Precision@5	0.4350
Recall@5	0.4410

On the temporal-test cohort, Random Forest achieved a Micro-F1 of 0.4267, Macro-F1 of 0.2249, AUROC of 0.7447, and AUPRC of 0.4892. For recommendation ranking, Precision@1, Precision@3, and Precision@5 were 0.6250, 0.5000, and 0.4350, respectively, while Recall@1, Recall@3, and Recall@5 were 0.1873, 0.3292, and 0.4410.

The temporal evaluation provides an out-of-time assessment of the selected model using patients that were not involved in preprocessing, fitting, model training, model comparison, or feature selection. The results therefore provide a more stringent assessment of model generalization than the internal OOF estimates. The implementation also confirms that the development and temporal-test target matrices contained 156 × 33 and 40 × 33 observations, respectively.

3.7. Antibiotic-Specific Performance

To complement the aggregate multi-label results, the selected Random Forest model was evaluated separately for each of the 33 modeled antibiotics using development-cohort out-of-fold (OOF) predictions. Performance varied substantially across antibiotic labels. The highest F1-scores were observed for Tigecycline (0.6517), Gentamicin (0.6364), Fosfomycin (0.6000), and Amikacin (0.5783), whereas several less frequent labels produced an F1-score of zero.

The strongest discrimination was observed for Fusidic Acid (AUROC = 0.9063), followed by Clindamycin (0.8852), Tigecycline (0.8764), and Rifampicin (0.8739). AUPRC was highest for Tigecycline (0.7517), Fusidic Acid (0.7442), Fosfomycin (0.7071), and Gentamicin (0.6824).

For each antibiotic, positive susceptibility prevalence was calculated from the 156-patient development cohort as follows:

$$\mathrm{Positive} \mathrm{prevalence} (\%)={\displaystyle \frac{\mathrm{Positive} N}{156}}\times 100$$

where Positive N denotes the number of development-cohort patients with a recorded Sensitive/Susceptible result for the corresponding antibiotic. Thus, the reported prevalence reflects the proportion of development-cohort patients with a positive susceptibility label for each antibiotic. Antibiotic-specific performance was calculated from development-cohort OOF predictions.

These results demonstrate substantial heterogeneity in predictive performance across antibiotic labels. Labels with greater positive susceptibility support generally provided more stable performance, whereas several less frequent labels showed limited predictive performance, including F1-scores of zero. The complete antibiotic-specific results, including positive counts, positive prevalence, and performance measures, are presented in Table 11, with the variation in F1-score illustrated in Figure 5.

Table 11. Antibiotic-specific performance of the Random Forest model.

Antibiotic	Positive N	Positive Prevalence (%)	Precision	Recall	F1	AUROC	AUPRC
Amikacin	48	30.77	0.6857	0.5000	0.5783	0.7919	0.6368
Amoxicillin/Clavulanic Acid	14	8.97	0.2000	0.0714	0.1053	0.7462	0.2406
Cefazolin	9	5.77	0.0000	0.0000	0.0000	0.4206	0.0599
Cefepime	21	13.46	0.6667	0.1905	0.2963	0.7425	0.4471
Ceftazidime	22	14.10	0.0000	0.0000	0.0000	0.5517	0.1618
Ceftazidime/Avibactam	11	7.05	1.0000	0.2727	0.4286	0.6677	0.3782
Ceftolozane/Tazobactam	8	5.13	0.0000	0.0000	0.0000	0.5971	0.1143
Ceftriaxone	13	8.33	0.0000	0.0000	0.0000	0.5481	0.1123
Cefuroxime	10	6.41	0.0000	0.0000	0.0000	0.4918	0.0703
Cefuroxime Axetil	9	5.77	0.0000	0.0000	0.0000	0.3364	0.0458
Ciprofloxacin	24	15.38	0.0000	0.0000	0.0000	0.6473	0.2197
Clindamycin	13	8.33	1.0000	0.3077	0.4706	0.8852	0.6678
Colistin	16	10.26	0.5000	0.1875	0.2727	0.8438	0.3653
Doxycycline	9	5.77	0.5000	0.2222	0.3077	0.8133	0.3625
Erythromycin	10	6.41	1.0000	0.3000	0.4615	0.8062	0.4899
Fosfomycin	39	25.00	0.6774	0.5385	0.6000	0.7974	0.7071
Fusidic Acid	9	5.77	1.0000	0.3333	0.5000	0.9063	0.7442
Gentamicin	73	46.79	0.6049	0.6712	0.6364	0.7305	0.6824
Imipenem	36	23.08	0.6000	0.3333	0.4286	0.7248	0.4647
Levofloxacin	6	3.85	0.0000	0.0000	0.0000	0.6511	0.0894
Linezolid	17	10.90	0.8333	0.2941	0.4348	0.8599	0.5077
Meropenem	45	28.85	0.5938	0.4222	0.4935	0.7934	0.6009
Minocycline	7	4.49	0.6667	0.2857	0.4000	0.7363	0.4302
Moxifloxacin	8	5.13	0.0000	0.0000	0.0000	0.3961	0.1579
Nitrofurantoin	14	8.97	0.6000	0.2143	0.3158	0.8214	0.4875
Piperacillin/Tazobactam	27	17.31	0.2857	0.1481	0.1951	0.6069	0.2582
Rifampicin	17	10.90	1.0000	0.3529	0.5217	0.8739	0.6141
Teicoplanin	14	8.97	0.8000	0.2857	0.4211	0.8204	0.5350
Tetracycline	19	12.18	0.6000	0.1579	0.2500	0.7353	0.3940
Tigecycline	48	30.77	0.7073	0.6042	0.6517	0.8764	0.7517
Tobramycin	10	6.41	0.6667	0.4000	0.5000	0.7110	0.4845
Trimethoprim/Sulfamethoxazole	37	23.72	0.4615	0.1622	0.2400	0.5916	0.3370
Vancomycin	15	9.62	0.8000	0.2667	0.4000	0.8057	0.5108

Figure 5. Per-antibiotic F1-scores obtained by the Random Forest model across the 33 modeled antibiotic susceptibility labels. The results demonstrate substantial variation in predictive performance across antibiotics, with the highest F1-scores observed for Tigecycline (0.652), Gentamicin (0.636), Fosfomycin (0.600), and Amikacin (0.578), while several lower-frequency antibiotic labels achieved an F1-score of 0.000.

3.8. SHAP-Based Explainability

SHAP analysis was applied to the selected Random Forest model to quantify the contribution of the available input features to antibiotic susceptibility predictions. Because the prediction task was formulated using a One-vs-Rest (OvR) structure, SHAPs were generated separately for each antibiotic-specific classifier. The SHAP completion audit confirmed successful analysis for all 33 modeled antibiotics (33/33; 100%).

The global analysis showed that Age_years had the largest mean absolute SHAP value (0.074971), followed by Month (0.043720), Klebsiella pneumoniae (0.042860), Escherichia coli (0.031589), and Acinetobacter baumannii (0.029380). Other influential encoded predictors included sputum and MRSA specimen categories, Staphylococcus aureus, gender, Pseudomonas aeruginosa, and urine specimen type. These findings indicate that the model predictions were influenced by a combination of demographic, temporal, organism-related, and specimen-related information.

The antibiotic-specific SHAP results were retained in Supplementary Table S1 (SHAP_per_antibiotic.csv), allowing the feature contributions to be examined separately for each of the 33 antibiotic classifiers. This documents the application of SHAP within the OvR multi-label framework rather than treating the complete prediction task as a single output. The global feature-importance results are summarized in Table 12 and visualized in Figure 6. Importantly, SHAP values represent feature contributions to model predictions and not causal clinical effects. Therefore, the identified features should be interpreted as factors influencing model behavior rather than as independent determinants of antibiotic susceptibility.

Table 12. Global SHAP feature importance of the selected Random Forest model.

Rank	Feature	Mean Absolute SHAP
1	Age_years	0.074971
2	Month	0.043720
3	Organism_Klebsiella pneumoniae	0.042860
4	Organism_Escherichia coli	0.031589
5	Organism_Acinetobacter baumannii	0.029380
6	Specimen_Type_sputum	0.023649
7	Organism_Staphylococcus aureus	0.020113
8	Specimen_Type_MRSA	0.018903
9	Gender_male	0.018010
10	Gender_female	0.016107
11	Organism_Pseudomonas aeruginosa	0.013448
12	Specimen_Type_urine	0.013119
13	Organism_Staphylococcus epidermidis	0.009605
14	Organism_Enterobacter cloacae	0.006195
15	Specimen_Type_blood	0.005142

Note: The table reports the 15 highest-ranked encoded features from the global SHAP analysis.

Figure 6. Global SHAP feature-importance plot showing the contribution of the input features to the Random Forest antibiotic susceptibility predictions. Features are ranked by mean absolute SHAP value; Age_years and Month show the largest overall contributions, followed by organism- and specimen-related features. The plot presents the 15 most influential encoded features from the complete global SHAP analysis.

3.9. Patient-Specific Antibiotic Recommendation Based on Top-5 Susceptibility Rankings

The selected Random Forest model generated patient-specific susceptibility rankings by estimating a predicted susceptibility probability for each of the 33 final antibiotic labels and ordering the antibiotics from highest to lowest probability. The five highest-ranked antibiotics were retained as the patient-specific Top-5 output.

Representative OOF rankings are presented in Table 13. The results demonstrate substantial variation between patients. For example, Gentamicin was ranked first for DEV_P1 with a predicted susceptibility probability of 0.7771, whereas Fosfomycin was ranked first for DEV_P3 (0.5295) and DEV_P6 (0.8709). DEV_P7 received Ceftazidime/Avibactam as its highest-ranked option (0.8216), while DEV_P9 was ranked highest for Minocycline (0.6522).

Table 13. Representative patient-specific Top-5 susceptibility rankings from out-of-fold predictions.

Patient	Rank 1	p	Rank 2	p	Rank 3	p	Rank 4	p	Rank 5	p
DEV_P1	Gentamicin	0.7771	Ceftazidime/Avibactam	0.7426	Amikacin	0.7201	Cefepime	0.5894	Ciprofloxacin	0.4333
DEV_P2	Gentamicin	0.6471	Meropenem	0.6319	Imipenem	0.6051	Amikacin	0.5892	Piperacillin/Tazobactam	0.5448
DEV_P3	Fosfomycin	0.5295	Meropenem	0.5122	Amikacin	0.4181	Gentamicin	0.4179	Imipenem	0.3743
DEV_P4	Gentamicin	0.7573	Rifampicin	0.5953	Linezolid	0.5171	Teicoplanin	0.5107	Vancomycin	0.5093
DEV_P5	Gentamicin	0.3783	Amikacin	0.2660	Ciprofloxacin	0.2649	Ceftazidime/Avibactam	0.2641	Colistin	0.2403
DEV_P6	Fosfomycin	0.8709	Meropenem	0.8474	Imipenem	0.8257	Gentamicin	0.7486	Piperacillin/Tazobactam	0.6715
DEV_P7	Ceftazidime/Avibactam	0.8216	Ceftolozane/Tazobactam	0.8018	Gentamicin	0.7717	Ciprofloxacin	0.7421	Amikacin	0.6498
DEV_P8	Gentamicin	0.6332	Piperacillin/Tazobactam	0.4198	Tetracycline	0.3193	Ceftazidime/Avibactam	0.3107	Amikacin	0.2809
DEV_P9	Minocycline	0.6522	Doxycycline	0.6484	Tetracycline	0.6004	Gentamicin	0.4816	Piperacillin/Tazobactam	0.3959
DEV_P10	Gentamicin	0.6408	Cefepime	0.3118	Piperacillin/Tazobactam	0.3024	Amikacin	0.2750	Ceftriaxone	0.2749

Note: p denotes the predicted susceptibility probability.

The corresponding OOF patient–antibiotic heatmap (Figure 7) provides a broader view of these patient-level differences across the modeled antibiotics. Each row represents a patient, and each column represents an antibiotic, with the intensity indicating the predicted probability of susceptibility.

Figure 7. Patient–antibiotic heatmap showing OOF predicted susceptibility probabilities for the 33 modeled antibiotics. Each row represents a patient, and each column represents an antibiotic; color intensity indicates the predicted susceptibility probability, with higher intensities corresponding to higher predicted probabilities. The heatmap illustrates patient-level variation in susceptibility profiles used to generate the Top-5 rankings.

These results demonstrate that the framework can produce individualized antibiotic rankings rather than a single population-level recommendation. The rankings are intended as decision-support outputs and should be interpreted together with laboratory AST results, clinical characteristics, and antimicrobial stewardship considerations; they are not autonomous prescribing recommendations.

3.10. Probability Calibration

The predicted susceptibility probabilities generated by the selected Random Forest model were visually assessed for calibration using the out-of-fold predictions from the development cohort. This analysis provides a visual assessment of the agreement between predicted probabilities and observed susceptibility outcomes; it does not constitute formal clinical validation of probability calibration. The calibration analysis complements the ranking results presented in Section 3.9 by providing an additional assessment of the reliability of the predicted probabilities used for patient-specific ranking. The corresponding OOF calibration results are presented in Figure 8.

Figure 8. Out-of-fold calibration plot for the selected Random Forest model, comparing predicted susceptibility probabilities with observed susceptibility outcomes. The plot assesses the reliability of the probabilities used for patient-specific antibiotic ranking.

3.11. Population-Level Antibiotic Ranking

Beyond patient-specific recommendations, the selected Random Forest model was used to summarize antibiotic rankings at the population level. Antibiotics were ranked according to their aggregated predicted susceptibility probabilities, providing an overall view of the antibiotics with higher predicted susceptibility across the evaluated patients. This analysis complements the patient-specific Top-5 rankings presented in Section 3.9. The resulting population-level ranking is presented in Figure 9.

Figure 9. Population-level ranking of antibiotics based on mean out-of-fold (OOF) predicted susceptibility probabilities from the selected Random Forest model. Antibiotics are ordered from highest to lowest mean predicted susceptibility, with Gentamicin showing the highest population-level predicted susceptibility, followed by Amikacin, Meropenem, and Tigecycline.

3.12. Clinical Decision-Support Interpretation

The results demonstrate that the proposed framework can provide patient-specific susceptibility rankings and offer interpretable information about the factors influencing model predictions. The Random Forest outputs should therefore be regarded as clinical decision-support information rather than autonomous antibiotic prescribing recommendations.

The predicted susceptibility probabilities and Top-5 rankings can complement conventional antimicrobial susceptibility testing (AST), local resistance patterns, and patient-specific clinical assessment. In particular, the population-level ranking in Section 3.11 provides an aggregate view of predicted susceptibility, whereas the patient-specific rankings in Section 3.9 account for differences between individual patients.

Importantly, a higher predicted susceptibility probability does not by itself establish that an antibiotic is clinically appropriate for treatment. Final antibiotic selection should remain subject to microbiological evidence, clinical characteristics, antimicrobial stewardship principles, and clinician judgment. This interpretation maintains the intended role of the framework as an explainable decision-support tool rather than a replacement for clinical decision-making.

4. Discussion

The present study developed an explainable multi-label machine learning framework for patient-specific antibiotic recommendation using clinical microbiology data. The revised analysis strengthens the original framework by combining leakage-controlled evaluation, temporal testing, One-vs-Rest modeling of 33 antibiotic labels, model comparison, feature ablation, SHAP-based explainability, and probability-based patient-specific ranking. These components provide a more comprehensive assessment of the feasibility of using machine learning to support antibiotic susceptibility-based decision-making.

The microbial profile was dominated by Klebsiella pneumoniae, Escherichia coli, and Acinetobacter baumannii. This pattern is consistent with previous reports identifying these organisms as important contributors to healthcare-associated infections and antimicrobial resistance (AMR) [6,27]. The prominence of K. pneumoniae is particularly relevant because of its association with extended-spectrum β-lactamase and carbapenem-resistant strains, which can substantially restrict therapeutic options [28,29]. Likewise, E. coli and A. baumannii are important causes of urinary, bloodstream, and respiratory infections and have considerable capacity to acquire multidrug resistance [6,30].

Although fungal isolates were excluded from the primary bacterial susceptibility prediction analysis, Candida albicans was the predominant fungal species in the original cohort, followed by other Candida spp., consistent with previous reports identifying C. albicans as a leading cause of invasive candidiasis despite the increasing emergence of non-albicans Candida species with reduced antifungal susceptibility [31]. Other clinically important organisms, including Pseudomonas aeruginosa, Staphylococcus aureus, methicillin-resistant S. aureus (MRSA), and Staphylococcus epidermidis, were also present in the original microbiological profile [5,32]. These findings reinforce the importance of incorporating organism information into antimicrobial susceptibility prediction.

Random Forest was selected as the final model based on its overall performance across the evaluated measures. The use of multiple complementary metrics is important because the objective of the framework extends beyond independent binary classification to ranking potentially susceptible antibiotics for each patient. The revised analysis therefore considers both classification and ranking performance rather than relying on a single metric. Previous antimicrobial susceptibility studies have also reported competitive performance from ensemble tree-based approaches, supporting their continued investigation for AMR prediction [3,33].

The feature-ablation results further indicate that organism-related information provides substantial predictive information, while demographic, specimen-related, and temporal variables contribute additional information across different evaluation measures. This supports the use of a broader feature set rather than relying exclusively on organism identity. Nevertheless, these predictive contributions should not be interpreted as causal determinants of antimicrobial susceptibility.

SHAP analysis provided an additional perspective on model behavior. Across the 33 antibiotic-specific classifiers, Age_years showed the largest global mean absolute SHAP value, followed by Month and several organism-related features, including K. pneumoniae, E. coli, and A. baumannii. Specimen-related and demographic variables also contributed to predictions. This suggests that the model uses multiple sources of information when estimating susceptibility rather than depending on a single predictor. Explainability is particularly relevant to healthcare AI because transparent model behavior can facilitate clinical interpretation and support more trustworthy use of predictive systems [26,34]. Previous studies have likewise identified organism-related information as an important predictor of antimicrobial resistance [3,35]. However, SHAP values represent contributions to model predictions and should not be interpreted as causal clinical effects.

An important feature of the framework is its probability-based multi-label recommendation strategy. The OvR models estimate susceptibility probabilities for the 33 antibiotic labels, which are then ranked to generate patient-specific Top-5 outputs. This approach recognizes that more than one antibiotic may have relatively high predicted susceptibility for a patient and provides several candidate options rather than forcing a single binary decision. Similar recommendation-oriented approaches have been explored as a means of supporting individualized antimicrobial selection and stewardship [2,4,36,37,38]. In the present study, however, the rankings are interpreted as susceptibility-based decision support rather than autonomous treatment recommendations.

The population-level ranking provides a complementary view of the patient-specific results by summarizing predicted susceptibility across the study population. Such information may complement local antibiograms and provide additional data for antimicrobial stewardship activities [6,27]. Nevertheless, the population ranking is specific to the investigated cohort and should not be interpreted as a universal prescribing hierarchy. Its value lies in providing an aggregate description of the model’s predicted susceptibility patterns that can be considered alongside microbiological surveillance and established stewardship practices.

The revised validation strategy also strengthens the interpretation of the findings. The analysis used an untouched chronological temporal-test cohort in addition to out-of-fold development evaluation, reducing the risk that performance estimates were driven solely by randomly partitioned observations. The temporal results showed that the selected model retained measurable discrimination and ranking performance on the later held-out cohort; however, the small size of this cohort limits the precision and generalizability of these estimates, while the lower Macro-F1 observed for the temporal cohort also indicates that performance remains more challenging for less frequent antibiotic labels. Thus, the temporal evaluation provides encouraging evidence of generalization but does not substitute for independent external validation.

Taken together, the framework integrates multi-label susceptibility prediction, model comparison, feature ablation, SHAP explainability, patient-specific Top-5 ranking, and population-level analysis within a unified workflow. Previous research has demonstrated the potential of machine learning for antimicrobial susceptibility prediction [3,33]. Recent work has emphasized the importance of explainability, validation, and integration with antimicrobial stewardship for clinical adoption [34,37,39,40,41]. The present study extends this direction by combining susceptibility prediction with interpretable ranking-oriented decision support while maintaining a clear distinction between model output and clinical treatment decisions.

Despite these strengths, several limitations should be acknowledged. First, the relatively small, single-centre dataset may limit the generalizability of the findings to other hospitals and geographical settings; therefore, the temporal-test evaluation does not replace independent external validation. Second, the available predictors were mainly demographic, microbiological, specimen-related, and temporal variables, while important clinical and molecular factors such as previous antimicrobial exposure, comorbidities, infection severity, renal function, allergy history, and resistance determinants were not available. Their inclusion may improve future prediction and recommendation quality. Third, the relatively short study period limits assessment of longer-term changes in antimicrobial resistance patterns. Finally, although PDR cases were present in the original dataset, a dedicated PDR subgroup performance analysis was not performed; therefore, the reliability of the generated antibiotic rankings specifically for PDR patients cannot be established from the current results. Future studies should evaluate PDR cases separately using larger datasets with comprehensive susceptibility profiles, richer clinical and molecular information, and prospective multicenter validation before clinical deployment.

5. Conclusions

The proposed explainable multi-label machine learning framework provides a practical approach for patient-specific antibiotic recommendations based on susceptibility ranking and clinical decision support. By combining multi-label prediction, probability-based Top-5 ranking, SHAP-based explainability, and population-level analysis, the framework provides interpretable information that may complement antimicrobial susceptibility testing and antimicrobial stewardship.

The results support the potential of the selected Random Forest model for susceptibility-based ranking, while the temporal-test evaluation provides additional evidence of its predictive utility on later unseen patients. However, the model should not be interpreted as an autonomous prescribing system, and its predictions should be considered together with microbiological evidence, patient-specific clinical factors, local resistance patterns, and clinician judgment.

Further validation on larger, multicenter, and prospective datasets, including dedicated evaluation of challenging cases such as PDR, is required before the framework can be considered for routine clinical deployment.