Zendal:A Federated Framework for Critical Emergency Triage Escalation

Abstract

Emergency triage artificial intelligence (AI) is usually evaluated as a centralized prediction problem, yet many hospitals cannot pool patient-level records across governance boundaries. We studied a narrower and safety-relevant task: within-dataset federated simulation for detecting Emergency Severity Index (ESI)-2/ESI-1 encounters using information available before or during triage. Zendal is a reproducible within-dataset federated evaluation framework for critical emergency triage escalation. It defines a triage-time feature boundary, site-aligned cross-silo splits, training-split-only imputation and scaling, validation-selected operating thresholds, bootstrap uncertainty, decision-curve analysis, calibration analysis, local-only baselines, per-site and subgroup diagnostics, and leave-one-site-out transfer stress testing. In 558,029 public emergency department encounters, the endpoint was binary critical-band ESI assignment (ESI-2/ESI-1 vs. ESI-3/ESI-4/ESI-5), not an independently adjudicated clinical outcome. The feature set contained 240 pre-triage variables: demographics, arrival metadata, previous utilization counts, triage vital signs, and chief-complaint indicators; disposition, diagnoses, medications, procedures, imaging, laboratories, and target-derived imputations were excluded. A centralized logistic reference reached AUROC 0.879 (95% CI: 0.876–0.882), average precision 0.767, sensitivity 0.900 (95% CI: 0.895–0.905), specificity 0.654 (95% CI: 0.649–0.658), and positive predictive value (PPV) 0.530 at a validation-selected threshold targeting at least 90% sensitivity. Federated logistic training across three site-aligned clients nearly matched the centralized reference: AUROC 0.878 (95% CI: 0.875–0.880), average precision 0.767, sensitivity 0.900 (95% CI: 0.896–0.904), specificity 0.650 (95% CI: 0.645–0.655), and PPV 0.527. Post hoc Platt scaling reduced the federated expected calibration error from 0.160 to 0.011, supporting recalibration before direct risk interpretation. Model-guided escalation produced a positive mean decision-curve net benefit across thresholds 0.05–0.50 (0.164 for federated logistic vs. −0.003 for flag-all and 0 for flag-none). In leave-one-site-out transfer tests, federated source-site models retained AUROC 0.846–0.876 and sensitivity 0.871–0.909 on held-out sites, but the low-prevalence site had PPV 0.107 and negative net benefit. An open MIMIC-IV-ED Demo smoke test supported schema feasibility but was too small for external validation. Zendal is therefore a transparent framework for no-raw-row-exchange triage model evaluation, not a deployment-ready clinical system or a demonstration of formal privacy protection.

1. Introduction

Emergency triage is a safety-critical ranking problem under time pressure. An emergency department (ED) must decide which patients need immediate attention while operating with incomplete data, crowding, and limited clinical capacity. In the United States, EDs handle more than 155 million visits annually; 11.5% result in hospital admission and 2.0% require transfer to a critical care unit [1]. In this setting, the main error types are not symmetric. Under-triage can delay treatment for high-acuity patients, whereas over-triage consumes scarce resources and can slow care for others. A useful AI triage system therefore needs more than high average accuracy: it needs explicit operating points, locally governed probability thresholds, and transparent safety trade-offs.

Most emergency triage models are still studied as centralized predictors. Data are pooled within one site or governance boundary, a classifier is optimized on the pooled data, and privacy, calibration, communication, and operating-point selection are discussed separately. This leaves a practical knowledge gap. Hospitals that cannot move patient records still need to know whether a distributed triage workflow can preserve sensitivity for ESI-2/ESI-1 patients using only information available at triage.

To address that gap, we define the target setting as triage-time federated critical-band escalation: cross-silo training in which raw records remain local, the primary endpoint is ESI-2/ESI-1 escalation, and features are restricted to demographics, arrival metadata, previous utilization counts, triage vital signs, and chief-complaint indicators. We intentionally exclude disposition, diagnoses, medications, procedures, imaging, laboratory results, and target-conditioned imputations from the primary analysis. Federated learning is useful here because it allows hospitals to train a shared model without exchanging raw records [2,3], but federation alone does not solve the safety problem. Model updates can reveal sensitive information [4], client updates can be corrupted [5,6], and high average accuracy can still hide clinically unacceptable critical misses.

We introduce Zendal, a reproducible framework for studying this setting. The framework specifies the pre-triage feature boundary, site-aligned train/validation/test construction, train-split-only preprocessing, centralized and federated reference models, validation-time threshold selection, decision-curve analysis, per-site diagnostics, leave-one-site-out transfer stress testing, and subgroup safety checks. The present manuscript centers its claims on the completed triage-time binary federated experiment. The name refers to Isabel Zendal, the nurse associated with the 1803 Balmis smallpox-vaccine expedition, and is intended to mark the same systems problem: extending care across distance without centralizing all resources in one site.

We evaluated Zendal on a public retrospective dataset of 558,029 ED encounters labeled with ESI. This study used site-aligned train/validation/test splits inside three hospital-department partitions. Validation data were reserved for selecting the critical-band operating threshold, and held-out test data were used once for final reporting. This is a proxy for cross-silo federation, not an independent multi-hospital validation study. To make that boundary explicit, we report global held-out results, operating-point sensitivity sweeps, per-site diagnostics, leave-one-site-out transfer stress tests, subgroup diagnostics by age and gender, decision-curve net benefit, feature-boundary sensitivity analysis, and a small open MIMIC-IV-ED Demo schema smoke test limited to binary critical-band transfer [7].

This paper is organized around the following empirical questions:

• Can federated logistic training match a centralized logistic reference for triage-time ESI-2/ESI-1 escalation?
• Does validation-selected model-guided escalation provide positive decision-curve net benefit across plausible probability thresholds?
• How do PPV, alert rate, and false negatives change when the validation target is tightened from 85% to 95% sensitivity?
• How does the primary federated model behave across the three site-aligned client partitions?
• How does a model trained on two site partitions transfer to a held-out third site when that site contributes no training or threshold-selection rows?
• How do sensitivity, specificity, PPV, and alert rate vary across age and gender subgroups?
• What does a feature-boundary sensitivity analysis imply about the boundary of defensible five-class claims?
• Are the probability scores reliable enough to support direct risk interpretation, or should the present system be treated as a thresholded escalation model?

The main findings are concrete. With 240 triage-time features, centralized logistic regression reaches AUROC 0.879 and 90.0% held-out critical-band sensitivity at a validation-selected threshold. Federated logistic training across three site-aligned clients reaches AUROC 0.878 and the same 90.0% sensitivity, with specificity 65.0%, PPV 52.7%, and alert rate 51.6%. A federated operating-point sweep shows why the 90% target is a governance choice rather than a fixed clinical rule: tightening the validation target from 85% to 95% reduces held-out false negatives from 2525 to 818 but increases the alert rate from 44.8% to 62.5% and lowers PPV from 57.4% to 46.0%. Decision-curve analysis shows a positive mean net benefit for federated model-guided escalation across thresholds of 0.05–0.50 (0.164), compared with −0.003 for flag-all and 0 for flag-none. Calibration diagnostics are weaker before recalibration: the federated logistic model has a Brier score of 0.159 and a 10-bin expected calibration error (ECE) of 0.160, but Platt scaling fitted on validation scores reduces the test ECE to 0.011. Per-site AUROC ranges from 0.834 to 0.896. In leave-one-site-out transfer tests, federated source-site models retain AUROC 0.846–0.876 and sensitivity 87.1–90.9%, but PPV falls to 10.7% and mean net benefit becomes negative on the low-prevalence site. Subgroup diagnostics show sensitivity of 87.4% in female encounters and 92.4% in male encounters, and a wider age-band range from 78.0% in ages 18–34 to 96.3% in ages 65+. A broader five-class feature-boundary sensitivity analysis without target-conditioned preprocessing is much weaker (mean five-class accuracy 48.4%, macro-F1 29.8%, critical-band AUROC 74.0%), so this manuscript treats binary pre-triage escalation as the primary defensible endpoint rather than claiming deployment-ready five-class ESI assignment.

The contributions are as follows:

• We specify a triage-time feature boundary for critical-band triage modeling and make post-triage information an explicit reporting concern.
• We present Zendal, a reproducible cross-silo evaluation framework for ESI-2/ESI-1 escalation with site-aligned partitions, validation-selected thresholds, operating-point sensitivity sweeps, decision-curve analysis, site-transfer stress testing, and subgroup diagnostics.
• We provide a retrospective proof-of-concept evaluation on 558,029 public ED encounters showing that federated logistic training nearly matches centralized logistic discrimination and sensitivity without pooling raw records.
• We quantify clinical utility and safety trade-offs through bootstrap confidence intervals, probability-score reliability metrics, per-site diagnostics, gender and age-band subgroup results, and decision-curve net benefit.

The rest of this paper reviews related work (Section 2), describes the Zendal workflow (Section 3), defines the experimental protocol (Section 4), reports the empirical results (Section 5), and discusses limitations and deployment requirements (Section 6 and Section 7).

2. Related Work

2.1. Emergency Department Triage Modeling

Early machine learning approaches to emergency department triage have shown that routinely available signals, such as vitals, demographics, and chief complaints, can outperform conventional rule-based systems like the Emergency Severity Index in predicting critical outcomes and hospitalization [8,9,10]. Large-scale studies using decision-curve analysis have also reported net-benefit gains, indicating clinical utility for risk stratification. Subsequent systems have integrated structured features with chief-complaint text or clinical notes, improving both discrimination and calibration [11]. Attention-based architectures have also been explored for triage modeling [12].

Our work builds on this triage modeling literature but differs in three ways: First, we restrict the primary endpoint to ESI-2/ESI-1 escalation using only pre-triage variables, which is closer to the information available when an ED triage decision is made. Second, we evaluate the centralized-versus-federated trade-off under the same split, preprocessing, and validation-selected operating point. Third, we report the clinical quantities that determine whether such a model is usable as decision support: sensitivity, specificity, PPV, alert rate, net benefit, per-site behavior, subgroup behavior, and explicit validation boundaries.

2.2. Safety-Aware Learning and Decision-Focused Evaluation

Accuracy and AUROC alone do not capture the full picture for clinical adoption, where explicit cost trade-offs and careful operating-point selection are essential. Decision-curve analysis (DCA), for example, offers a principled way to compare the net benefit of different models across a range of decision thresholds [13]. Cost-sensitive learning formalizes the asymmetric penalties of mis-triage by using class-weighted or instance-weighted objectives [14,15]. Neyman–Pearson (NP) classification takes a related approach by prioritizing one type of error while constraining another [16]. Recent applied healthcare AI work has used the same policy-aware operating-point framing for edge-level myocardial infarction screening, where sensitivity, latency, and energy budgets must be considered together [17].

Our primary experiment operationalizes this safety-first principle through a binary critical-band endpoint and a validation-selected threshold that must reach at least 90% sensitivity before test evaluation. This keeps the clinical operating point explicit rather than allowing average accuracy to determine the decision rule.

2.3. Calibration and Reliability Under Distribution Shift

In the context of clinical decision-making, probability scores are useful only when connected to an operating decision. Temperature scaling is a strong post hoc calibration method [18], but calibration can degrade under dataset shift [19,20]. We therefore emphasize pre-specified threshold selection on held-out validation data and report the resulting sensitivity, specificity, PPV, alert rate, and decision-curve net benefit rather than relying on discrimination alone.

2.4. Federated Learning in Healthcare

Federated learning offers a way to train models across multiple sites without sharing raw patient data [2,3]. In healthcare, federated learning has been explored for outcome prediction across institutions and modalities [21,22,23]. Systematic reviews through 2024 identify hundreds of applications but note that relatively few have reached real-world deployment [24]; reporting on calibration and decision thresholds also remains uncommon.

More recently, research has started to address calibration in federated learning more directly; for example, FedCal improves local and global calibration under heterogeneity [25]. In the present triage study, we do not claim a new calibration method. We instead make the validation-time sensitivity constraint explicit and test how that threshold behaves across sites.

2.5. Formal Privacy Techniques in Federated Learning

Federated learning reduces direct data exposure by keeping training data local, but model updates can still reveal sensitive membership information [4]. Differential privacy (DP) provides a formal framework for bounding privacy risk [26]. In practice, DP-SGD adds noise to clipped gradients during training [27], and modern accounting tools often use Rényi DP or related analyses [28].

DP-SGD can be deployed in both centralized and federated settings; practical implementations in PyTorch 2.8.0 are available through Opacus 1.5.4 [29]. Recent work also explores alternative DP definitions and accounting approaches (e.g., Gaussian DP) that can yield tighter privacy–utility trade-offs in some regimes [30].

2.6. Secure Aggregation and Byzantine Robustness

Even with the protections of differential privacy, federated systems are still vulnerable to Byzantine attacks, where malicious clients contribute poisoned updates to degrade or bias the global model [5,6]. Secure aggregation can reduce server visibility into individual client updates—for example, via cryptographic protocols or related techniques [31,32].

Byzantine-robust aggregation, on the other hand, provides a defense against malicious clients through statistical methods. Common approaches include coordinate-wise median and trimmed-mean [6]. The primary experiment uses weighted FedAvg [2]; targeted poisoning attacks are outside the present evidence boundary.

2.7. Trustworthy and Explainable AI in Distributed Settings

Trustworthy clinical AI also requires interpretable behavior and subgroup evaluation. Post hoc explanation methods such as SHAP and LIME can provide feature-level interpretations of model predictions [33,34]. Fairness metrics can be used to assess disparate impact across demographic groups, with equality of opportunity often used when sensitivity for a clinically important outcome is the relevant constraint [35].

In federated settings, ensuring fairness across distributed populations can introduce additional challenges due to heterogeneity. Recent work on agnostic and distributionally robust federated learning addresses worst-case performance gaps across clients [36]. We support subgroup fairness evaluation (accuracy/F1 and rate-based checks across age and gender) alongside optional SHAP analysis.

2.8. Positioning Relative to Prior Work

Our work unifies several research directions. Dayan et al. demonstrated healthcare FL for clinical outcomes, but not for emergency triage operating-point selection [23]. Raita et al. established triage ML gains at scale using DCA, but their model was trained centrally and did not evaluate federated training [8]. Clinical deployments have shown improvements compared to traditional ESI, but they often rely on centralized data, limiting scalability across institutions [9]. The privacy and FL literature provides foundational systems guidance [3,26]; the open gap is a reproducible, safety-focused triage study that reports feature-timing boundaries, site transfer, subgroup behavior, and clinically interpretable operating-point trade-offs together.

Zendal is positioned at this intersection. It does not claim a new privacy definition, a new federated optimizer, or external clinical validation. Its contribution is the integration of established components into a reproducible triage workflow that reports the quantities that a safety-aware distributed clinical decision-support system would need: critical-band recall, PPV, alert rate, decision-curve net benefit, client-level diagnostics, site-transfer stress tests, subgroup diagnostics, and communication cost.

Table 1. Positioning of Zendal relative to selected emergency triage and federated clinical prediction studies. DCA = decision-curve analysis; FL = federated learning.

Study	Main Endpoint	Feature-Timing Boundary	External Validation	Calibration Reported	DCA Reported	FL Used	Main Distinction from Zendal
Levin et al. [9]	Clinical outcomes relative to ESI	Operational ED data	Single health system	Limited	No	No	Centralized electronic triage deployment evidence.
Raita et al. [8]	Critical care and hospitalization outcomes	Routinely available ED predictors	Retrospective multi-hospital data	Limited	Yes	No	Strong centralized triage benchmark with DCA.
Liu et al. [10]	Need for critical care	Practical ED variables	Retrospective validation	Limited	No	No	Centralized critical-care detection model.
Chang et al. [11]	ED disposition	Structured data plus language features	Retrospective single-system evaluation	Yes	No	No	Incorporates NLP for disposition prediction.
Dayan et al. [23]	COVID-19 clinical outcome	Multi-institutional clinical predictors	International hospitals	Not triage-specific	No	Yes	Real healthcare FL, but outside ED triage operating-point evaluation.
Zendal	ESI-2/ESI-1 vs. ESI-3/4/5	Strict pre-triage tabular features	Within-dataset site simulation plus MIMIC demo schema smoke test	Yes, including post hoc recalibration	Yes	Yes	Integrates no-raw-row-exchange FL, local thresholds, site-transfer, subgroup, and calibration diagnostics for critical-band ESI assignment.

summarizes this positioning relative to selected emergency triage and federated clinical prediction studies.

3. Methods

3.1. Framework Overview and Task Definition

Zendal is organized as a reproducible workflow rather than as a single classifier. The input is a set of ED encounters distributed across N clients. In the primary empirical instantiation, each encounter contains pre-triage tabular features and a binary target $y_{\mathrm{crit}}$, where $y_{\mathrm{crit}}=1$ denotes ESI-2/ESI-1 and $y_{\mathrm{crit}}=0$ denotes ESI-3/ESI-4/ESI-5. The model outputs a critical-band probability score. The primary safety event is a false negative: a true ESI-2/ESI-1 encounter not selected for escalation at the validation-selected operating threshold.

The workflow has six stages: First, each site maps local records into a pre-triage feature set and excludes post-triage variables. Second, missing values and scaling parameters are fitted on the training split only. Third, centralized and federated reference models are trained under matched hyperparameters. Fourth, the validation split selects an escalation threshold targeting at least 90% critical-band sensitivity. Fifth, the held-out test split is evaluated once with bootstrap confidence intervals, decision-curve net benefit, and per-site diagnostics. Sixth, leave-one-site-out transfer tests, subgroup diagnostics, and external-schema checks define the boundary between retrospective proof of concept and external validation.

This design keeps the methodological contract explicit: no raw patient-level records are exchanged between clients, safety is measured on held-out labels rather than inferred from aggregate accuracy, and every operating threshold is selected on validation data before final test evaluation.

3.2. Dataset, Labels, and Feature Engineering

Our analysis is based on a publicly available hospital triage dataset that contains 558,029 emergency department encounters. Each encounter is labeled with the Emergency Severity Index (ESI), a validated 5-level triage scale that ranges from ESI-5 (the lowest urgency) to ESI-1 (the highest urgency, indicating an immediate life-threatening condition). The primary endpoint is binary critical-band escalation: ESI-2/ESI-1 versus ESI-3/ESI-4/ESI-5.

We use a triage-time pre-triage feature definition. The 240 model features include demographics, arrival metadata, previous utilization counts, triage vital signs, and chief-complaint indicators. We exclude disposition, previous disposition, diagnosis groups, medications, procedures, imaging, laboratory results, interaction features derived from later information, and any preprocessing rule that conditions on the ESI target. Missing values are left unresolved during feature construction and are imputed only after splitting, using medians fitted on the training rows.

Figure 1 summarizes the original 5-class label distribution and highlights the severe class imbalance at ESI-1 (0.9% of encounters), which motivated the binary critical-band endpoint.

Features are grouped as vitals, symptoms, risk factors, and context. The vital group includes age, triage heart rate, systolic and diastolic blood pressure, respiratory rate, oxygen saturation (SpO₂), and temperature. The symptom group contains 200 binary chief-complaint indicators supplied in the public dataset. Zendal does not learn a complaint vocabulary from validation or test rows; it uses the fixed indicator columns released with the dataset. Risk features include previous ED visits and previous admissions. Contextual features include gender, arrival mode, arrival month, and arrival-hour bins encoded as one-hot indicators. A complete feature list with source field, encoding, missingness rate, timing, and post-triage exclusion status is provided in Supplementary Table S1.

We split within each site-aligned hospital-department partition using a fixed seed of 42, producing 446,422 training encounters, 55,802 validation encounters, and 55,805 held-out test encounters. The training split was partitioned across three site-aligned clients. Validation and test rows retained their site labels for threshold selection, per-site diagnostics, and subgroup analysis. Continuous features were imputed with training-split medians and standardized with training-split scaling parameters; validation and test rows were transformed with those fitted preprocessing statistics.

Table 2. Client heterogeneity across the three site-aligned hospital-department partitions. Critical rate denotes ESI-2/ESI-1 prevalence after dropping rows with missing ESI labels. Mean vital missingness is averaged across age and six triage vital-sign fields.

Site	Encounters	Critical Rate	ESI-1	ESI-2	ESI-3	ESI-4	ESI-5	Mean Age	Mean Vital Missing	Mean ED Visits
A	320,033	0.393	4577	121,296	123,800	52,565	17,795	48.4	0.248	3.55
B	166,335	0.236	675	38,505	75,950	44,595	6610	50.9	0.380	4.51
C	71,661	0.052	19	3733	36,479	27,843	3587	54.2	0.236	1.32

reports the resulting heterogeneity across the three site-aligned client partitions.

3.3. Threat Model and Privacy Assumptions

We adopted an honest-but-curious server as the threat model for the primary experiment. In this scenario, the central aggregator follows the federated protocol but may inspect the parameter updates that it receives. The primary experiment keeps raw rows at their site-aligned clients and communicates only model parameters for weighted FedAvg. It does not enable secure aggregation, differential privacy, or adversarially robust aggregation; therefore, it should not be interpreted as providing a formal privacy guarantee. It is a reproducible cross-silo learning simulation under a no-raw-row-exchange constraint.

For deployment, secure aggregation, authenticated transport, access control, audit logging, and model documentation would be required. Differential privacy should be enabled only after task-specific validation confirms that rare but clinically critical classes remain identifiable at the chosen privacy budget. Differentially private and robust-aggregation variants are outside the primary results reported here.

3.4. Primary Model and Training Objective

The primary analysis uses logistic models because the main question is whether federation preserves a transparent centralized reference under a strict triage-time feature boundary. For encounter i with feature vector ${\mathbf{x}}_i\in {\mathbb{R}}^{240}$, the model estimates

$${\widehat{p}}_i=\sigma ({\mathbf{w}}^{\top }{\mathbf{x}}_i+b),$$

(1)

where ${\widehat{p}}_i$ is the estimated probability of ESI-2/ESI-1 escalation. We train with class-weighted binary cross-entropy,

$$\mathcal{L}=-{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\left[\alpha y_{i}log\left({\widehat{p}}_i\right)+(1-y_i)log(1-{\widehat{p}}_i)\right],$$

(2)

where $\alpha$ is the noncritical-to-critical ratio in the training split. The safety-weighted federated variant multiplies $\alpha$ by 1.5. The validation-selected threshold, not a different model architecture, sets the final sensitivity–specificity trade-off. This keeps the main result inspectable and reproducible.

More expressive neural, hierarchical, and differentially private variants are reserved for future experiments. They are intentionally not used as headline evidence in this manuscript because the 5-class feature-boundary sensitivity analysis did not support deployment-ready full ESI assignment.

3.5. Federated Learning Protocol

To simulate a multi-institution clinical decision-support scenario, we partitioned the training set across $N=3$ site-aligned clients using the hospital-department identifier. Each client trained the same logistic model locally for one pass per communication round using AdamW, learning rate 0.002, weight decay ${10}^{-4}$, batch size 4096, and the class weight described above. After each round, the clients uploaded the model parameters to the server. The server computed weighted FedAvg by local sample count [2] and broadcast the updated global parameters. We ran 12 communication rounds.

The primary federated model has 240 weights plus one bias. In FP32 precision, one model copy is 964 bytes; uplink plus downlink is therefore approximately 1.9 KB per client per round, or 5.8 KB system-wide for three clients per round before protocol overhead. Bandwidth is therefore negligible for this reference model; the harder deployment work is governance, preprocessing version control, security, monitoring, and local threshold management.

3.6. Operating-Point Selection

For the primary binary logistic experiment, we selected a probability threshold on the validation split by scanning thresholds from 0.000 to 1.000 in increments of 0.001 and choosing the most specific threshold that still reached at least 90% critical-band sensitivity on validation data.

This operating-point rule is intentionally simple: it fixes the clinical safety target before test evaluation and makes the sensitivity–specificity–alert-rate trade-off reproducible. More complex conformal or subgroup-constrained thresholding was not used for the primary result reported here. Figure 2 summarizes the end-to-end pipeline used for feature construction, local training, aggregation, threshold selection, and evaluation.

4. Experiments

The experiments were designed to separate claims that are often conflated in clinical AI papers. First, we tested whether federated training preserves centralized binary critical-band discrimination without pooling raw records. Second, we selected the operating threshold on validation data and reported sensitivity, specificity, PPV, and alert rate on the held-out test split. Third, we repeated threshold selection at 85%, 90%, and 95% validation sensitivity targets to quantify the safety–capacity trade-off. Fourth, we evaluated critical-band net benefit across escalation thresholds. Fifth, we reported per-site and subgroup diagnostics so that heterogeneity was visible. Sixth, we ran leave-one-site-out transfer stress tests in which the held-out site contributed no rows to model training or source-threshold selection. Seventh, we ran a feature-boundary sensitivity analysis and an external-schema smoke test to define what the retrospective evidence does and does not support.

4.1. Evaluation Metrics

For binary critical-band escalation, we report AUROC, average precision (AP), sensitivity, specificity, PPV, and alert rate. Sensitivity is the recall for ESI-2/ESI-1 encounters. Alert rate is the fraction of all encounters assigned to the critical pathway at the selected threshold. To quantify uncertainty, we computed bootstrap 95.0% confidence intervals using 500 resamples on the held-out test split. For critical-band escalation utility, we computed decision-curve net benefit across probability thresholds $p_t=0.05$–$0.50$:

$$\mathrm{NB}\left(p_t\right)={\displaystyle \frac{\mathrm{TP}}{n}}-{\displaystyle \frac{\mathrm{FP}}{n}}{\displaystyle \frac{p_t}{1-p_t}}.$$

(3)

This compares model-guided escalation against flag-none and flag-all reference policies. We also report an operating-point sweep in which the validation sensitivity target was set to 85%, 90%, and 95%, with the resulting fixed thresholds applied once to the held-out test split. We report Brier score and 10-bin expected calibration error (ECE) to assess whether probability scores can be interpreted directly as critical-band risk estimates. To test whether recalibration can improve score reliability, we fit Platt scaling, isotonic regression, and temperature scaling on the validation scores and evaluate the recalibrated scores on the held-out test split. To support fairness-aware deployment planning, we report subgroup metrics across gender and age groups, including sensitivity, specificity, PPV, alert rate, observed critical-band prevalence, and Wilson intervals for proportion metrics. We treat these as descriptive diagnostics rather than fairness certification.

4.2. Systems Compared

We compared three primary systems: The centralized logistic reference was trained on the pooled training split and served as the upper-bound governance case in which raw rows could be centralized. The federated logistic model trained the same linear binary classifier across three site-aligned clients with weighted FedAvg and no raw-row exchange. The federated safety-weighted logistic model used the same federated protocol but increased the positive-class weight by 50% during local optimization. All three systems used the same pre-triage features, train-split-only imputation and scaling, validation threshold rule, and held-out test set.

We added four reviewer-requested comparison analyses: First, we trained local-only logistic models separately on each site and evaluated each model on its local site and on the other two sites using the threshold selected on the source site’s validation rows. Second, we trained a reduced-feature centralized logistic baseline using only age, triage vital signs, previous ED visits, and previous admissions. Third, we trained a centralized histogram gradient-boosting baseline with the same 240-feature matrix to test whether a stronger non-linear centralized model changes the interpretation. Fourth, we applied post hoc calibration to the centralized and federated logistic scores. For site-transfer stress testing, we trained source-site centralized and federated logistic models on two site partitions, selected the threshold using source-site validation rows only, and evaluated on the withheld third site. We also report a threshold-only local recalibration variant in which 20% of the held-out site was used only to choose a local threshold, with no local model refitting. For an external-schema smoke test, we trained a separate minimal shared-feature logistic model for binary critical-band detection on Kaggle and evaluated it, without refitting, on the open MIMIC-IV-ED Demo linked with the MIMIC-IV Clinical Database Demo patient table for age [7,37].

4.3. Hyperparameters

We used a fixed random seed of 42 for reproducibility. The centralized logistic model was trained for 6 epochs with AdamW, learning rate 0.002, weight decay ${10}^{-4}$, batch size 4096, and a positive-class weight equal to the noncritical-to-critical ratio in the training split. The federated logistic models used 12 communication rounds, one local pass per client per round, AdamW, learning rate 0.002, weight decay ${10}^{-4}$, batch size 4096, and weighted FedAvg by local sample count. The safety-weighted federated variant multiplies the positive-class weight by 1.5. The histogram gradient-boosting baseline used 80 boosting iterations, learning rate 0.06, maximum 31 leaf nodes, ${\ell }_2$ regularization ${10}^{-4}$, and the same positive-class weighting. The primary threshold was selected on validation data to target at least 90% sensitivity while maximizing specificity. The operating-point sweep used the same trained model and repeated validation-threshold selection for 85%, 90%, and 95% sensitivity targets; no model was refitted for the sweep.

4.4. Reproducibility Specification

Table 3. Reproducibility specification for the primary triage-time binary critical-band experiment.

Component	Specification
Dataset	Public Kaggle hospital triage and patient history dataset; 558,029 ED encounters labeled with ESI.
Primary endpoint	Binary ESI-2/ESI-1 escalation vs. ESI-3/ESI-4/ESI-5.
Data split	Site-aligned train/validation/test split inside each hospital-department partition with seed 42: 446,422 train, 55,802 validation, and 55,805 held-out test encounters.
Preprocessing	Missing values are imputed with medians estimated from the training split only; standardization parameters are fitted on the training split only.
Feature groups	A total of 240 triage-time pre-triage features: age and triage vital signs, chief-complaint indicators, previous utilization counts, gender, arrival mode, arrival month, and arrival-hour bins; labs and post-triage variables are excluded.
Federated clients	Three site-aligned client partitions derived from the hospital-department identifier; local rows remain client-side during federated training.
Training	Logistic binary classifier; AdamW, learning rate 0.002, batch size 4096, six centralized epochs or 12 federated rounds, and weighted FedAvg by client sample count.
Operating point	The validation split selects the most specific threshold that reaches at least 90% critical-band sensitivity; the selected threshold is then fixed for test reporting.
Operating-point sweep	The same trained primary models are evaluated at validation-selected thresholds targeting 85%, 90%, and 95% critical-band sensitivity to report the PPV, alert-rate, and false-negative trade-off.
Uncertainty	Bootstrap 95.0% confidence intervals use 500 resamples on the held-out test set; Wilson intervals are computed for subgroup proportion metrics.
Probability reliability	Brier score, 10-bin ECE, and 10 reliability-bin summaries are reported for primary test predictions.
Post hoc calibration	Platt scaling, isotonic regression, and temperature scaling are fitted on validation scores and evaluated once on the held-out test split.
Decision-curve analysis	Net benefit is reported across escalation thresholds of 0.05–0.50 and compared with flag-all and flag-none reference policies.
Added baselines	Local-only site logistic models, a numeric-only centralized logistic model, and a centralized histogram gradient-boosting model are evaluated with the same validation-threshold rule.
Site-transfer stress test	Each site is withheld once; source-site models are trained on the other two sites, source validation rows select the operating threshold, and the withheld site is evaluated without model refitting.
External-schema smoke test	A minimal shared-feature logistic model is trained on Kaggle for binary ESI-2/ESI-1 detection and evaluated, without refitting, on MIMIC-IV-ED Demo acuity 1/2 vs. 3/4 among 207 stays with non-missing acuity; this is schema compatibility evidence, not external validation.
Subgroup diagnostics	The primary binary models are evaluated on the held-out test split by gender, age band, and site using the same validation-selected threshold.
Feature-boundary sensitivity	A separate 5-class HistGradientBoosting sensitivity analysis reruns the broader advanced-feature path after removing target-conditioned imputation; it is used to bound claims, not as the primary endpoint.

summarizes the analysis choices required to reproduce the reported experiments. We include this table to make the methodological boundary explicit: this study evaluates a reproducible federated triage framework on a public dataset, using site-aligned client proxies and held-out testing, but it does not constitute prospective or independent multi-hospital validation. We used TRIPOD+AI as the primary reporting reference for retrospective prediction-model development and performance evaluation [38]. DECIDE-AI is not directly applicable to this study because it addresses early-stage live clinical evaluation of AI decision-support systems; it instead frames the prospective deployment boundary described in the Discussion [39].

5. Results

5.1. Primary Triage-Time Binary Federated Result

The primary experiment evaluated ESI-2/ESI-1 escalation using only the 240 triage-time pre-triage features described above. The training split contained 446,422 encounters, the validation split 55,802 encounters, and the held-out test split 55,805 encounters. Critical-band prevalence was 30.3% in the training and test splits.

Table 4. Primary triage-time binary critical-band results on the held-out test split. Thresholds are selected on validation data to target at least 90% sensitivity.

Model	AUROC (95% CI)	AP	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	Alert Rate (95% CI)
Centralized logistic	0.879 (0.876–0.882)	0.767	0.900 (0.895–0.904)	0.654 (0.649–0.658)	0.530 (0.525–0.536)	0.514 (0.510–0.518)
Federated logistic	0.878 (0.875–0.880)	0.767	0.900 (0.896–0.904)	0.650 (0.645–0.655)	0.527 (0.521–0.533)	0.516 (0.512–0.521)
Federated safety-weighted logistic	0.878 (0.875–0.880)	0.766	0.901 (0.896–0.905)	0.650 (0.646–0.655)	0.527 (0.522–0.533)	0.516 (0.512–0.521)

Note: AP = average precision; PPV = positive predictive value. Bootstrap confidence intervals use 500 held-out test resamples.

reports the held-out test performance. The centralized logistic reference reaches AUROC 0.879 (95% CI: 0.876–0.882) and AP 0.767. At the validation-selected threshold, it reaches 90.0% sensitivity, 65.4% specificity, PPV 53.0%, and alert rate 51.4%. Federated logistic training nearly matches this reference: AUROC falls by only 0.0014, sensitivity remains 90.0%, specificity is 65.0%, PPV is 52.7%, and the alert rate is 51.6%. Increasing the positive-class weight in the federated safety-weighted variant does not materially improve the operating point; it reaches AUROC 0.878 and sensitivity 90.1% with similar specificity and PPV.

This result is the central empirical claim of the manuscript: under a strict pre-triage feature boundary, simple cross-silo federated learning preserves the discrimination and sensitivity of a centralized logistic reference without moving raw patient records.

5.2. Additional Baselines and Local-Only Models

Table 5. Additional centralized baselines evaluated on the held-out test split. Thresholds were selected on validation data to target at least 90% sensitivity.

Model	AUROC	AP	Sens.	Spec.	PPV	Alert	Brier	ECE
Numeric-only logistic	0.681	0.465	0.900	0.252	0.343	0.794	0.225	0.179
Histogram gradient boosting	0.877	0.769	0.896	0.653	0.528	0.513	0.144	0.123

contextualizes the primary models against two additional centralized baselines. A numeric-only logistic model using age, triage vital signs, and prior utilization reaches the target sensitivity but has much lower discrimination (AUROC 0.681) and a high alert rate (79.4%). The centralized histogram gradient-boosting model reaches AUROC 0.877 and AP 0.769, similar to the logistic reference, but does not improve the operating point. This supports the interpretation that the main result is driven by the triage-time feature boundary and threshold rule rather than by a complex model class.

Table 6. Local-only logistic baselines. Each model was trained on one site’s training rows, selected its threshold on that site’s validation rows, and was evaluated on each site-specific held-out test subset.

Train Site	Test Site	AUROC	Sens.	Spec.	PPV	Alert Rate	Mean NB
A	A	0.894	0.900	0.683	0.648	0.546	0.279
A	B	0.846	0.868	0.650	0.433	0.472	0.094
A	C	0.830	0.856	0.616	0.110	0.409	−0.111
B	A	0.844	0.907	0.550	0.567	0.630	0.249
B	B	0.856	0.911	0.603	0.415	0.518	0.087
B	C	0.820	0.875	0.568	0.101	0.455	−0.127
C	A	0.834	0.916	0.523	0.554	0.650	0.231
C	B	0.814	0.908	0.488	0.353	0.605	0.033
C	C	0.835	0.907	0.605	0.112	0.422	−0.140

reports local-only logistic models trained separately on each site. Local-only training performs well on Site A but does not solve cross-site or low-prevalence behavior. Models trained on Sites A or B still have a negative mean net benefit on Site C, and the Site C-only model has PPV 11.2% even on its own local test split. These results show why the site-transfer problem is not only an aggregation problem: prevalence, threshold selection, and alert capacity must be governed locally.

5.3. Operating-Point Policy Sweep

The 90% sensitivity target is a safety-oriented design choice, not a universal clinical constant. A hospital with lower alert capacity could prefer a lower sensitivity target; a setting where missed critical-band cases are especially costly could prefer a higher target. We therefore repeated validation-threshold selection for the primary federated logistic model at 85%, 90%, and 95% target sensitivity and evaluated each fixed threshold on the held-out test split.

This sweep makes the safety–capacity trade-off explicit. Moving from the 90% to the 95% target prevents 871 additional false negatives in the held-out split but produces 6083 additional alerts and lowers PPV from 52.7% to 46.0%. Moving from 90% to 85% reduces the alert burden by 3800 encounters but adds 836 false negatives. The primary 90% target is therefore a transparent reference operating point rather than a claim that the same threshold should be used at every hospital.

Table 7. Operating-point sweep for the primary federated logistic model. Thresholds were selected on validation data at each target and then applied to the held-out test split.

Validation Target	Threshold	Test Sensitivity	Test Specificity	Test PPV	Alert Rate	False Negatives
85% Sensitivity	0.490	0.850	0.726	0.574	0.448	2525
90% Sensitivity	0.399	0.900	0.650	0.527	0.516	1689
95% Sensitivity	0.270	0.952	0.516	0.460	0.625	818

reports the held-out results at the 85%, 90%, and 95% validation sensitivity targets.

5.4. Probability-Score Reliability

The operating threshold was selected on validation data, so the primary result does not require raw probability scores to be perfectly calibrated. We nevertheless evaluated score reliability because deployment teams may be tempted to interpret the model output as an absolute critical-band risk.

Table 8. Held-out probability-score reliability for the primary binary models. Lower values are better. Bootstrap 95% confidence intervals use 500 held-out test resamples.

Model	Brier Score (95% CI)	10-bin ECE (95% CI)
Centralized logistic	0.142 (0.141–0.144)	0.114 (0.110–0.116)
Federated logistic	0.159 (0.157–0.161)	0.160 (0.156–0.163)
Federated safety-weighted logistic	0.185 (0.184–0.187)	0.216 (0.213–0.219)

reports the unrecalibrated Brier score and 10-bin ECE for the primary binary models.

These calibration results limit how the probability scores should be used. The centralized reference has the best Brier score and ECE, whereas the federated and safety-weighted federated models overestimate critical-band probability more strongly, especially in high-score bins. The model output is therefore appropriate in this manuscript as a validation-thresholded escalation score. It should not be presented to clinicians as an absolute risk estimate without site-local recalibration and prospective calibration monitoring.

Post hoc calibration materially improves score reliability without changing the underlying discrimination. 

Table 9. Post hoc calibration results on the held-out test split. Calibration functions were fitted on validation scores only. Thresholds were reselected on recalibrated validation scores to target at least 90% sensitivity.

Model	Calibration	Brier	ECE	AUROC	Sens.	PPV
Centralized logistic	Uncalibrated	0.142	0.114	0.879	0.900	0.530
Centralized logistic	Platt scaling	0.126	0.007	0.879	0.900	0.529
Centralized logistic	Isotonic regression	0.126	0.004	0.879	0.901	0.528
Centralized logistic	Temperature scaling	0.142	0.114	0.879	0.900	0.529
Federated logistic	Uncalibrated	0.159	0.160	0.878	0.900	0.527
Federated logistic	Platt scaling	0.126	0.011	0.878	0.900	0.527
Federated logistic	Isotonic regression	0.126	0.004	0.877	0.915	0.508
Federated logistic	Temperature scaling	0.158	0.164	0.878	0.900	0.527

shows that Platt scaling reduces the federated logistic Brier score from 0.159 to 0.126 and ECE from 0.160 to 0.011. Isotonic regression reduces ECE further to 0.004 but changes the operating point more noticeably, increasing sensitivity to 91.5% and lowering PPV to 50.8%. Temperature scaling does not improve this class-weighted logistic setting. We therefore treat Platt scaling as the most stable recalibration baseline for this retrospective analysis, while still requiring site-local recalibration before clinical use.

Table 10. Site-specific post hoc calibration for the primary federated logistic model. Calibration functions were fitted on site-specific validation rows and evaluated on the corresponding held-out site test rows.

Site	Calibration	Brier	ECE	Sens.	PPV
A	Uncalibrated	0.138	0.097	0.898	0.654
A	Platt scaling	0.126	0.008	0.898	0.654
A	Isotonic regression	0.126	0.009	0.906	0.644
B	Uncalibrated	0.179	0.202	0.910	0.409
B	Platt scaling	0.126	0.026	0.911	0.409
B	Isotonic regression	0.125	0.009	0.916	0.404
C	Uncalibrated	0.204	0.340	0.880	0.107
C	Platt scaling	0.044	0.006	0.880	0.107
C	Isotonic regression	0.044	0.007	0.904	0.101

extends the calibration analysis using functions fitted separately on each site’s validation rows.

Site-specific recalibration reduces ECE at all three client partitions, especially Site C, where the uncalibrated federated ECE is 0.340. This improvement does not resolve the clinical utility issue at Site C: PPV remains near 10% because the local critical-band prevalence is low. Calibration therefore addresses risk-score reliability, whereas deployment readiness still depends on local PPV, alert burden, and net-benefit governance. Figure 3 visualizes the corresponding held-out reliability pattern before and after validation-fitted recalibration.

5.5. Decision-Curve Net Benefit

Sensitivity alone does not determine whether a critical-band escalation policy is useful. A policy that flags nearly everyone can achieve high sensitivity while creating an unmanageable alert burden. We therefore computed the decision-curve net benefit across thresholds from 0.05 to 0.50 and compared model-guided escalation with flag-all and flag-none policies.

All three model-guided policies produce a positive mean net benefit across the threshold range, whereas flag-none has net benefit 0 and flag-all has a negative mean net benefit. The federated logistic model has a lower mean net benefit than the centralized reference, but it remains decisively above both reference policies. The safety-weighted federated variant does not improve the net benefit, which supports using the simpler federated logistic model as the primary result.

Table 11. Decision-curve summary for binary ESI-2/ESI-1 escalation.

Model	Mean Net Benefit	Mean Flag-All Net Benefit
Centralized logistic	0.1768	−0.0027
Federated logistic	0.1640	−0.0027
Federated safety-weighted logistic	0.1457	−0.0027

summarizes the mean net benefit for the three model-guided policies and the flag-all reference policy.

5.6. Site-Aligned Held-Out Diagnostics

Because this study used one public dataset, we do not claim external validation across independent hospitals. We instead report site-stratified diagnostics for the three client partitions used in the federated experiment. These diagnostics make the remaining transportability risk visible.

Table 12. Per-site held-out diagnostics for the primary federated logistic model.

Site	Test n	Critical n	AUROC	Sens.	Spec.	PPV	Alert Rate
A	32,004	12,588	0.896	0.905	0.679	0.647	0.551
B	16,634	3918	0.853	0.884	0.640	0.431	0.484
C	7167	375	0.834	0.883	0.585	0.105	0.439

reports the resulting site-stratified held-out diagnostics.

Sensitivity remains above 88% in all three site partitions, but discrimination and PPV vary substantially. Site C has only 375 critical-band cases in the held-out split and a 5.2% observed critical-band prevalence, which explains the low PPV despite acceptable sensitivity. These findings support the feasibility of a shared federated signal while showing that local threshold recalibration and site-specific monitoring would be necessary before deployment.

5.7. Leave-One-Site-Out Transfer Stress Test

Per-site diagnostics still train on all three site partitions. To stress-test transportability more directly, we withheld each site once, trained source-site models on the other two sites, selected the operating threshold using source-site validation rows only, and evaluated the fixed model on the withheld site. This is not external validation because all rows came from the same public dataset but the held-out site contributed no rows to model fitting or source-threshold selection.

Table 13. Leave-one-site-out transfer stress test for the federated source-site logistic model. The source threshold was selected without using the held-out site. The local recalibration column reports sensitivity after using a stratified 20% calibration subset from the held-out site only for threshold selection, with no model refitting.

Held-Out Site	Test n	Crit. Prev.	AUROC	AP	Sens.	Spec.	PPV	Alert Rate	Mean NB	Local-Recal. Sens.
A	320,033	0.393	0.876	0.828	0.882	0.670	0.634	0.547	0.261	0.900
B	166,335	0.236	0.846	0.619	0.871	0.642	0.428	0.479	0.086	0.901
C	71,661	0.052	0.846	0.263	0.909	0.580	0.107	0.445	−0.128	0.895

reports the federated source-site transfer results and the threshold-only local recalibration sensitivity analysis.

The source-site federated model retains useful discrimination on all withheld sites (AUROC 0.846–0.876), but the source-selected threshold does not preserve the intended 90% sensitivity on Sites A and B. A threshold-only local recalibration step restores or closely approaches the sensitivity target on the held-out local test splits (90.0%, 90.1%, and 89.5%), without refitting the model. Site C remains clinically problematic despite acceptable discrimination, because its critical-band prevalence is only 5.2%, PPV is 10.7%, and mean net benefit is negative across thresholds 0.05–0.50. This result strengthens the deployment boundary: federation can transfer a signal, but each site still needs local threshold governance and alert-capacity assessment.

5.8. Held-Out Subgroup Safety Diagnostics

Table 14. Subgroup diagnostics for the primary federated logistic model. Values are held-out test proportions at the fixed validation-selected threshold.

Subgroup	n	Critical n	AUROC	Sens.	Spec.	PPV	Alert Rate
Female	30,839	8134	0.867	0.874	0.670	0.487	0.474
Male	24,966	8747	0.885	0.924	0.622	0.569	0.569
18–34	15,991	3019	0.900	0.780	0.846	0.540	0.273
35–49	12,494	3140	0.882	0.857	0.730	0.516	0.417
50–64	13,323	4496	0.871	0.924	0.587	0.533	0.585
65+	13,996	6225	0.808	0.963	0.298	0.524	0.818

reports descriptive subgroup diagnostics for the primary federated logistic model. The validation-selected threshold was fixed before subgroup evaluation, and the same threshold was used for every subgroup.

The model meets the global 90% sensitivity target, but subgroup behavior is not uniform. Sensitivity is 87.4% for female encounters and 92.4% for male encounters. Age-band variation is larger: sensitivity is 78.0% in ages 18–34 and 96.3% in ages 65+, while the alert rate rises from 27.3% to 81.8%. Observed critical-band prevalence also rises with age, so these differences should not be interpreted as fairness certification or unfairness proof. They identify the subgroup constraints that a prospective study should pre-specify and monitor.

5.9. Feature-Boundary Sensitivity Analysis and Boundary of Five-Class Claims

To test whether broader advanced features could support full five-class ESI assignment without target-conditioned preprocessing, we reran a HistGradientBoosting sensitivity analysis after removing target-conditioned imputation and fitting imputation inside each training fold. Across three split seeds, the mean five-class accuracy was 48.4%, mean macro-F1 was 29.8%, mean critical-band sensitivity was 45.2%, and mean critical-band AUROC was 74.0%. A shuffled-label control collapsed as expected, with macro-F1 11.9% and critical-band AUROC 52.2%.

This analysis changes the evidentiary boundary of the paper. We do not use legacy high-performing five-class advanced-feature results as headline claims. The defensible primary result is the triage-time binary pre-triage federated experiment above. Full five-class ESI assignment remains a future task, but it needs stronger features available at triage, richer chief-complaint text, or independent validation data.

5.10. Open MIMIC-IV-ED Demo Schema Smoke Test

The full MIMIC-IV-ED database contains more than 400,000 ED admissions, but access requires PhysioNet registration, identity verification, human-subjects training, and a data use agreement [40]. To add an external-schema check that can be reproduced without credentialed access, we used the open MIMIC-IV-ED Demo, which contains a 100-patient subset with the same ED table structure [7]. This analysis was designed as a smoke test only, evaluating whether a minimal critical-band signal transfers across schemas when restricted to shared variables: age, sex, arrival mode, initial vital signs, and coarse chief-complaint indicators.

Table 15. External-schema critical-band smoke test using a minimal shared-feature logistic model. The model was trained on Kaggle and evaluated without refitting on MIMIC-IV-ED Demo.

Dataset	n	Crit. n	AUROC	AP	Accuracy	Sens.	Spec.	PPV
Kaggle held-out	111,606	33,761	0.801	0.634	0.745	0.716	0.757	0.561
MIMIC-IV-ED Demo	207	115	0.695	0.731	0.662	0.791	0.500	0.664

Note: The MIMIC-IV-ED Demo row has critical sensitivity 79.1% (95% CI: 70.8–85.6), specificity 50.0% (95% CI: 40.0–60.0), and alert rate 66.2% (95% CI: 59.5–72.3).

reports the Kaggle held-out and MIMIC-IV-ED Demo smoke-test results.

The MIMIC-IV-ED Demo result is useful because the model retains discrimination above chance and prioritizes critical-band sensitivity under an external schema. Its small sample size, lack of acuity-5 examples after filtering, and high critical-band prevalence prevent any external-validation claim. We interpret it as schema-compatibility evidence only.

5.11. Communication Efficiency

The primary binary federated model is a compact logistic classifier with 240 input features. Each client transmits and receives one parameter vector per communication round, so communication is negligible for hospital networks: roughly kilobytes rather than megabytes per client per round. The more expensive components of a real deployment would be governance, identity management, preprocessing consistency, monitoring, and audit logging, rather than bandwidth.

6. Discussion

6.1. What the Framework Shows

Federation did not outperform centralized learning in this study. Its value is different: Zendal makes the distributed triage trade-off measurable under a strict pre-triage feature boundary. In the primary experiment, federated logistic training across three site-aligned clients nearly matched the centralized logistic reference for ESI-2/ESI-1 escalation: AUROC 0.878 versus 0.879, AP 0.767 versus 0.767, and 90.0% sensitivity in both settings. The small loss in specificity and PPV is the measurable cost of not pooling raw rows in this retrospective setup.

The operating-point sweep and decision-curve analysis give this safety claim an operational interpretation. Federated logistic escalation produces a positive mean net benefit across thresholds of 0.05–0.50, whereas flag-none is zero and flag-all is negative on average. The sensitivity-target sweep shows the practical cost of tightening the safety constraint: relative to the 90% target, the 95% target prevents 871 additional false negatives in the held-out split but adds 6083 alerts. This matters because a high-sensitivity triage model can be clinically unusable if it simply alerts on nearly everyone. Zendal reports alert rate and PPV in addition to sensitivity, so that the cost of a safety-oriented operating point is visible.

The reliability analysis separates thresholded escalation from risk prediction. Class weighting is appropriate for protecting sensitivity, but it shifts probability scores upward. The federated logistic model reaches the desired operating-point sensitivity, yet its 10-bin ECE is 0.160 before recalibration, worse than the centralized reference. Validation-fitted Platt scaling lowers the federated ECE to 0.011 and Brier score to 0.126, showing that the reliability problem is partially correctable. This does not make the model deployment-ready: calibration was still retrospective, global, and fitted within the same public dataset. A clinical system should expose only locally recalibrated and prospectively monitored scores.

The per-site and leave-one-site-out diagnostics expose the strongest remaining risk. Site A has the best discrimination and PPV, Site B is lower, and Site C has a very low critical-band prevalence. When each site is withheld from model training and source-threshold selection, the federated source-site model still transfers discrimination (AUROC 0.846–0.876), but the source-selected threshold misses the intended 90% sensitivity target on Sites A and B and produces PPV near 10% with negative mean net benefit on Site C. Threshold-only local recalibration improves sensitivity on the local test splits without model refitting, but it cannot solve the low-prevalence alert-burden problem by itself. A real cross-hospital deployment would need local threshold governance, site-specific alert-capacity planning, and monitoring for prevalence shift.

The subgroup analysis adds a second deployment caveat. The global threshold reaches the intended 90% sensitivity target overall, but younger adults have lower sensitivity than older adults. At the same time, the observed critical-band prevalence and alert rate rise sharply with age. These patterns do not prove fairness failure; they show that subgroup endpoints must be specified prospectively. For triage, equal opportunity for ESI-2/ESI-1 detection is a more clinically meaningful target than demographic parity, because baseline acuity and resource needs differ across age groups.

The feature-boundary sensitivity analysis prevents the manuscript from overclaiming full five-class ESI assignment. Once target-conditioned preprocessing is removed from the broader advanced-feature path, five-class performance is much weaker than the binary pre-triage escalation result. This does not invalidate the Zendal framework; it clarifies the evidence. The current data support retrospective no-raw-row-exchange critical-band escalation research using pre-triage variables. They do not support a deployment-ready five-class ESI replacement.

The MIMIC-IV-ED Demo smoke test adds a narrow external-schema check. A minimal shared-feature model trained on Kaggle retained AUROC 0.695 and 79.1% sensitivity on the open MIMIC demo, despite different schema conventions and a higher critical-band prevalence. This result is not external validation. It reduces one specific concern: that the pre-triage variables cannot be mapped to another ED data model at all. The next validation step should use the full credentialed MIMIC-IV-ED cohort or independent hospital data with local threshold recalibration and pre-specified alert-capacity constraints.

6.2. Security, Privacy, and Systems Implications

Federated learning reduces direct data movement, but it is not inherently private [4]. Deployment-grade extensions would need client-side DP-SGD [26,27,29], robust aggregation [6], and secure transport. Secure aggregation can further reduce server visibility into individual client updates [31], but it is not enabled in the primary experiment. The current threat model is therefore honest-but-curious rather than fully malicious.

The primary logistic model has a very small communication footprint because only a 240-feature linear parameter vector is exchanged. Bandwidth is unlikely to be the limiting factor for this version of the framework. The harder systems work is operational: identity management, secure aggregation, preprocessing version control, threshold governance, drift monitoring, incident response, and audit logging.

6.3. Clinical Deployment Boundary

Zendal should be read as a research framework, not a clinical product. A deployment study would need independent hospital systems, local workflow integration, prospective monitoring, clinician-in-the-loop procedures, drift detection, audit logging, and institution-specific governance review. Model predictions should support, not replace, clinical judgment. The selected threshold should also be treated as a local governance decision: a hospital with different crowding, staffing, or critical-care capacity may prefer a different sensitivity–specificity–alert-rate trade-off. The operating-point sweep provides the kind of evidence such a governance decision would need, but a real site would have to repeat threshold selection with local validation data and alert-capacity constraints. If probabilities are shown to clinicians, they should be recalibrated and monitored locally rather than treated as calibrated risk estimates from this retrospective experiment.

Fairness and interpretability require further work. The held-out subgroup analysis reports clinically meaningful safety endpoints by age and gender, including sensitivity, specificity, PPV, and alert rate, but it remains descriptive because all rows came from one public dataset and no subgroup operating constraints were prospectively specified. A prospective triage study should define fairness around clinically meaningful targets, such as equal opportunity for the ESI-2/ESI-1 band, and should evaluate those targets across institutions and subgroups. Post hoc explanations are not treated as clinical interpretability evidence in this retrospective evaluation.

The next-stage clinical evaluation should be reported as a live AI decision-support study, including intended users, workflow integration, model-output presentation, override procedures, safety monitoring, human factors, subgroup endpoints, audit processes, and institutional accountability. These elements fall outside the present retrospective experiment but are central to a DECIDE-AI-aligned evaluation.

6.4. Limitations

This study used one public dataset for the main Zendal evaluation. The federated clients were site-aligned partitions derived from hospital-department identifiers, not independently governed hospitals with distinct EHR systems. The leave-one-site-out stress test is stronger than ordinary per-site reporting because the held-out site contributes no rows to source training or source-threshold selection, but it still remains within-dataset transportability evidence rather than independent external validation. The MIMIC-IV-ED Demo smoke test uses an external open dataset, but it contains only 100 patients and 207 stays with non-missing acuity, lacks acuity-5 examples, and includes only two acuity-4 examples. Results may change under different case mixes, coding practices, triage protocols, missingness patterns, and data-collection infrastructure.

The endpoint is ESI-2/ESI-1 assignment, not an independently adjudicated clinical deterioration outcome. This creates label circularity: the model learns patterns associated with the triage label assigned in routine care, and those labels can contain noise, inter-rater variability, local practice patterns, and historical triage bias. The present results therefore support critical-band ESI escalation research. They do not prove improved detection of mortality, ICU transfer, urgent intervention, or other independent clinical outcomes. A prospective validation study should evaluate both ESI agreement and patient-centered outcomes.

The public release does not provide a usable longitudinal patient identifier. A raw field named pid exists, but it has only two distinct values across 560,486 raw rows and is not suitable for patient-level splitting. The present split is therefore encounter-level within each site-aligned partition. Repeated encounters from the same real patient, if present in the original data, could appear across train, validation, and test splits and inflate apparent generalizability. This limitation should be resolved in any institutional validation using patient-level or temporal splitting.

The primary feature set is tabular and does not include free-text chief complaints, triage notes, imaging, laboratory results, or time-series physiology. The binary endpoint is clinically useful for escalation support but does not replace full ESI assignment. The federated experiment uses logistic models rather than more expressive neural architectures; this is deliberate for transparency and strict feature timing, but it limits representational capacity. Probability calibration is imperfect, especially for the class-weighted federated variants, so raw model scores should not be interpreted as absolute patient-level risks. Finally, the experiments were run in a controlled research environment rather than a production federated infrastructure with identity management, secure aggregation, access control, and real audit trails.

7. Conclusions

We have presented Zendal as a reproducible within-dataset federated evaluation framework for critical emergency triage escalation under a strict pre-triage feature boundary. On the primary binary endpoint, ESI-2/ESI-1 assignment, federated logistic training across three site-aligned clients nearly matched centralized logistic training without moving raw patient records: AUROC 0.878 versus 0.879, AP 0.767 versus 0.767, and 90.0% sensitivity in both settings at validation-selected thresholds. The operating-point sweep quantified the safety–capacity trade-off across 85%, 90%, and 95% validation sensitivity targets, and the federated model produced a positive decision-curve net benefit across clinically plausible thresholds. Post hoc Platt scaling improved federated calibration, but probability scores still require site-local recalibration and prospective monitoring before direct risk interpretation. Leave-one-site-out transfer testing showed that discrimination can transfer to withheld site partitions, but threshold performance, PPV, and net benefit remain site-dependent, especially at the low-prevalence site.

The main contribution is a transparent and auditable workflow for evaluating no-raw-row-exchange critical-band ESI escalation under a pre-triage feature boundary. The current evidence supports retrospective ESI critical-band research—not deployment, not independent clinical-outcome prediction, and not full five-class ESI replacement. The next evidence step is independent multi-institution validation with patient-level or temporal splitting, prospective clinical evaluation, local threshold governance, alert-capacity planning, and pre-specified subgroup safety endpoints.