4.1. Experimental Setup
This section evaluates the proposed U-STAR-PIML framework for unmanned fixed-wing aircraft using the JSBSim C172x model solely as a surrogate flight-dynamics simulation benchmark. The evaluation focuses on model behavior under ID conditions, three effective prescribed shifts, one stored-noise diagnostic control, recursive prediction, dynamic consistency, one-step predictive uncertainty, and ablation-based component analysis. The section is organized around the experimental setup, data source, evaluation protocol, and model comparison design, thereby providing a reproducible validation protocol for the methodology described in
Section 3.
Flight trajectories are generated with the JSBSim C172x surrogate model, which provides a reproducible six-degree-of-freedom simulation environment with a consistent interface and controlled randomization conditions. The data are produced with JSBSim version 1.2.4, installed through the Python 3.10 package jsbsim, using the bundled c172x aircraft definition. No aircraft-model files are modified; mass, inertia, aerodynamic, and air-density variations are applied through JSBSim properties at run time. The generated trajectories are used exclusively to evaluate unmanned fixed-wing aircraft dynamics prediction; they are not intended to reproduce piloted-aircraft missions or human-in-the-loop operation.
Each flight episode is generated independently and stores the clean state sequence, control inputs, wind disturbance information, noisy observations, and a scalar observation-availability indicator. The reported E5 training and evaluation configuration uses the clean state, control, and wind sequences for one-step and recursive prediction. The observation-related fields remain available for dataset diagnostics and future state-estimation protocols. This simulation setting provides a controlled environment for evaluating unmanned fixed-wing dynamics prediction while preserving the principal rigid-body flight dynamics structure used in recent fixed-wing UAV modeling studies [
13,
34,
38].
Data generation uses a sampling interval of s, corresponding to a 50 Hz integration rate, and each formal simulation episode lasts 40 s. The model input history length is , corresponding to s of historical information, and the one-step prediction interval is s. The main training configuration uses a batch size of 256 and a random seed of 42. The same seed drives the per-episode data-generation randomization, making the episode split and all sampled parameters reproducible.
Each episode is initialized near a level-flight trim condition at an altitude of approximately 1 km. The initial state is drawn around this trim by adding independent Gaussian perturbations to the body-axis velocity (standard deviation 0.5 m s
−1 on
u, clipped to
m s
−1, and
m s
−1 on
v and
w), the angular rates (
rad s
−1), the attitude angles (
rad on
and
, with
clipped to
rad, and
rad on
), and the down position (1 m). The control commands are normalized flight-control-system inputs in
: a trim throttle sampled in
is combined with small sinusoidal excitation, and one excitation profile is drawn per episode from an elevator doublet, an aileron doublet, a rudder doublet, a throttle step, and a multi-axis chirp, with fixed moderate amplitudes chosen to keep the C172x response stable while preserving excitation diversity. The wind disturbance follows a three-axis Ornstein–Uhlenbeck process whose per-episode mean and standard deviation are sampled from the ranges in
Table 3 and whose correlation time is drawn from
s. Because these settings, together with the JSBSim version and aircraft model, fully determine the generated data, the complete per-subset randomization ranges are listed in
Table 3, and the remaining fixed generator constants are available from the corresponding author upon reasonable request.
External real-flight validation. The external evaluation uses the public IDF-DS fixed-wing UAV telemetry dataset [
39]. We selected the Volantex Ranger 2400 subset recorded with a Holybro Pixhawk 6X running PX4 and evaluated the frozen E5 model without adaptation. A performance-independent deterministic protocol selected 10 independent source logs, yielding 29,500 valid one-step samples at 50 Hz, with a 50-sample history and a
s prediction interval. Normalized controls were reconstructed from PX4
actuator_servos and
actuator_motors records;
actuator_outputs pulse-width modulation (PWM) records were excluded. Horizontal wind used the onboard PX4
airspeed_wind estimate, and the unavailable vertical component was set to zero. No real-flight data were used for training, fine-tuning, or test-time adaptation, and recursive real-flight rollout was not evaluated. One candidate log was excluded before model evaluation because its actuator-channel behavior was incompatible with the fixed control-surface mapping used by the validation protocol; model performance was not used for this decision. Because the Ranger 2400 differs from the simulated C172x, this is a combined simulator-to-real and cross-airframe transfer diagnostic rather than same-airframe operational validation.
The dataset comprises ID subsets, three effective OOD subsets (wind, parameter, and delay), and one stored-noise diagnostic subset, with 98 formal flight episodes in total. The ID data include 40 training episodes, 8 validation episodes, and 10 ID test episodes. The ID test episodes follow the same randomization range as the training episodes and are reserved exclusively for evaluation. The three effective OOD subsets are generated by enlarging one model-input or transition-dynamics factor while keeping the remaining factors close to the nominal randomization range. Specifically,
ood_wind contains 10 episodes and expands the wind-process range from the ID wind mean interval
m/s and wind standard-deviation interval
m/s to the wind mean interval
m/s and wind standard-deviation interval
m/s. The
ood_param subset contains 10 episodes with mass and inertia scale factors in
, aerodynamic scale factors in
, and air-density scale factors in
. The dataset internally named
ood_noise contains 10 episodes and increases the observation-noise multiplier to
in the stored observation channels only. Because these channels are not consumed by the reported model, this subset is referred to throughout this manuscript as the stored-noise diagnostic rather than as an effective OOD robustness test. The
ood_delay subset contains 10 episodes and uses control-input delays of 3–12 simulation steps, corresponding to
–
s. All subsets are split at the episode level to prevent leakage from temporally adjacent samples. The dataset configuration is summarized in
Table 3.
Two complementary prediction modes are considered. In one-step prediction, each example uses a clean simulator-state history together with the aligned control and wind histories, and the target is the clean next state . Each example is independently rebuilt from the recorded sequence, so its history is unaffected by previous model predictions. This oracle-state protocol assumes that a deployment system would provide an equivalent state estimate; replacing clean simulator states with noisy or intermittent onboard estimates is outside the present evaluation.
In recursive rollout, the first clean states provide a s warm start. Thereafter, the deterministic predicted mean is appended to the state-history buffer, while controls and winds follow the recorded reference sequence. Future clean states are used only to compute errors, and the one-step uncertainty output is not sampled during rollout.
The model receives the exact wind vector used in the JSBSim simulation at every prediction step in both protocols, corresponding to perfect knowledge of the simulated wind, rather than an estimated or forecast wind field. This idealization excludes wind-estimation error because an airborne system would have to infer local wind from onboard measurements. The results should therefore be interpreted as an upper bound with respect to wind information. Evaluating wind inputs that are perturbed, delayed, or withheld is left for future work.
The baseline recursive training configuration uses 8 rollout steps. The standard evaluation horizons are s, s, s, and s, with extended horizons of s and s reported in the long-horizon analyses. The operator-based long-horizon variant uses 50 rollout training steps. The operator rollout fine-tuning variant uses 250 rollout steps, corresponding to s of recursive supervision. The 10 epochs reported for E8 denote additional rollout fine tuning after initialization from the converged 45-epoch E7 checkpoint, rather than the complete E8 training budget.
Because and are not selected by the evaluated configuration, the stored-noise diagnostic multiplier changes only the stored noisy-observation columns. It does not perturb the network inputs, clean targets, or reported clean-state metrics. Accordingly, the stored-noise diagnostic condition evaluates a dataset-storage control rather than robustness to noisy state inputs. The one-step protocol emphasizes local clean-state transition accuracy, whereas the recursive protocol evaluates deterministic error accumulation and free-rollout stability over longer horizons.
The evaluation metrics are selected to cover state accuracy, dynamic consistency, long-horizon stability, maneuver-sensitive behavior, one-step interval behavior, and computational cost. State prediction is evaluated using the root-mean-square error (RMSE) and grouped RMSE over physically related state variables. Derivative consistency is evaluated using derivative RMSE. Recursive prediction is evaluated using horizon-wise RMSE. High-maneuver or fast-state response is assessed using high-maneuver RMSE or fast-state RMSE. The one-step uncertainty output is evaluated using negative log-likelihood (NLL), prediction interval coverage probability (PICP), mean prediction interval width (MPIW), and calibration error. Computational efficiency is characterized by the parameter count, one-step inference time, and recursive rollout time per step. Numerical values are reported only in the corresponding result subsections.
The primary experiment is denoted as
E5: Full U-STAR-PIML, which corresponds to the complete staged U-STAR-PIML framework. This model uses a six-stage training schedule with 60, 80, 40, 40, 40, and 20 epochs, respectively, resulting in 280 training epochs in total. The comparison and ablation experiments are named sequentially from
E1 to
E8 to compare outcomes associated with the physics branch, temporal residual branch, state-dependent residual gating, one-step uncertainty output, staged training, and operator-based temporal extensions. Because several configurations differ simultaneously in architecture, optimization schedule, and total training budget, these experiments constitute partially coupled comparisons rather than strictly matched ablations. The compared experiments and their training schedules are summarized in
Table 4.
Run-to-run reproducibility was assessed using five predefined training seeds (42, 142, 242, 342, and 442), while the episode-level training, validation, ID-test, and OOD partitions were held fixed using split seed 42. E2–E8 were independently trained and evaluated for each seed under their original method-specific optimization schedules, yielding 35 complete runs. For each metric, the analysis reports the sample mean, sample standard deviation (SD; , denominator ), and two-sided 95% Student’s t confidence interval with four degrees of freedom. Because the method-specific epoch schedules were retained, this repeated-run analysis evaluates sensitivity to training stochasticity rather than providing a strictly matched-budget architecture-only comparison. The experiments therefore remain partially coupled comparisons rather than strictly matched ablations.
4.2. Overall Prediction Performance Across Evaluation Conditions
This subsection reports the overall prediction performance of the complete U-STAR-PIML model, denoted as
E5: Full U-STAR-PIML in
Table 4. The evaluation compares the ID test subset with three effective model-input or dynamics shifts—wind, parameter, and delay—and one stored-noise diagnostic subset. The purpose of this subsection is to provide an aggregate performance overview across these test conditions while preserving the implementation boundary of the stored-noise diagnostic subset. Detailed analyses of dynamic consistency, recursive error accumulation, one-step interval behavior, and ablation behavior are reserved for
Section 4.3,
Section 4.4 and
Section 4.5.
The ID test subset follows the same randomization range as the training data, while its episodes are excluded from model training. The wind-OOD subset strengthens the wind disturbance process. The parameter-OOD subset enlarges perturbations in mass, inertia, aerodynamic coefficients, and air density. The delay-OOD subset introduces larger control-input delays. The stored-noise diagnostic subset contains an increased observation-noise level only in the stored observation columns, which are not consumed by the reported clean-state model. Consequently, wind, parameter, and delay form the effective stress conditions, whereas the stored-noise diagnostic subset is retained as a diagnostic control.
The ID test case serves as the nominal generalization reference for the complete model. Under this condition, one-step prediction assesses whether the learned state-transition map remains accurate on unseen episodes sampled from the training distribution, while recursive rollout evaluates whether the same local model can be used repeatedly without immediate loss of trajectory-level coherence. The ID row in
Table 5 therefore provides the reference level for interpreting the three effective OOD shifts and the separate diagnostic control. A representative in-distribution prediction result is shown in
Figure 3, which is used to visualize the nominal prediction behavior of the full model.
The representative ID result indicates that the complete model preserves the dominant temporal evolution of the flight states under nominal randomization. This behavior is consistent with the role of the ID test set as a reference case for evaluating generalization to unseen episodes within the training distribution. Since this subsection focuses on overall prediction performance, detailed derivative-level and long-horizon error-propagation analyses are deferred to
Section 4.3.
The effective OOD rows in
Table 5 compare the model response to three prescribed shifts. The wind case directly changes the relative airflow experienced by the aircraft and can therefore affect velocity, attitude, and position errors through the aerodynamic response chain. The parameter case evaluates shifted physical properties and aerodynamic scales, while the delay case tests temporally shifted control effects. The stored-noise diagnostic subset contains stronger observation noise only in columns that are not consumed by the reported clean-state configuration. It therefore does not constitute an input-noise shift for the trained network and is retained only as a diagnostic control. Detailed derivative behavior, horizon-wise recursive trends, uncertainty calibration, and component-level ablations are analyzed in later subsections. A representative wind-OOD prediction result is shown in
Figure 4, providing a visual comparison with the nominal ID behavior.
The wind-OOD representative result shows a more challenging prediction condition than the ID case. The strengthened wind process mainly affects the aerodynamic response and therefore tends to increase discrepancies in velocity- and angular-rate-related states. This visual comparison supports the need for a separate dynamic-consistency and recursive-rollout analysis in the following subsection.
The aggregate cross-case comparison is further summarized in
Figure 5.
Figure 5 shows that wind produces the largest degradation among the three effective prescribed shifts under the present known-wind protocol. Relative to ID, wind OOD increases body-velocity, angular-rate, and rollout-position RMSE by approximately
,
, and
times, respectively, with rollout-position RMSE reaching
m. The pattern is consistent with a disturbance entering through relative airflow, first perturbing velocity and angular-rate prediction, then propagating through attitude kinematics and recursive position integration. For model-based simulation or a prediction module queried repeatedly, this result matters because accurate local transitions can still accumulate material navigation-position drift under persistent aerodynamic disturbance. The implication is bounded to the present known-wind simulation protocol: it does not establish robustness to real-world wind-estimation error or flight conditions. Parameter- and delay-OOD values remain closer to ID, while the stored-noise diagnostic changes only unused observation columns and therefore verifies the data path rather than sensor-noise robustness.
Although the aggregate results identify wind as the most challenging effective prescribed shift, the overall metrics and representative state plots alone do not explain how local dynamic errors propagate into long-horizon trajectory drift. Therefore, the next subsection examines derivative consistency, high-maneuver response, and recursive rollout behavior in greater detail.
4.3. Dynamic Consistency and Recursive Rollout Analysis
Although
Section 4.2 provides an aggregate comparison across the evaluated conditions, grouped RMSE values alone do not reveal how local dynamic errors propagate during recursive prediction. This subsection therefore evaluates dynamic consistency and rollout stability using derivative comparison, high-maneuver response, three-dimensional trajectory prediction, and horizon-wise rollout error.
The derivative comparison in
Figure 6 uses wind-OOD episode 58, the same fixed-order representative episode shown in
Figure 4, to assess local dynamic consistency at the level of state rates rather than state values alone. This evaluation indicates whether the learned transition model captures the local trend of flight dynamics before errors are accumulated through recursive prediction.
Derivative comparison focuses on the predicted rate of change and therefore complements the state-level prediction results reported in the previous subsection. The reference derivative is not read directly from the simulator: it is estimated from the recorded clean state sequence by finite differences, using a second-order central difference at interior samples and a first-order one-sided difference at the episode endpoints, with the yaw angle unwrapped before differentiation so that the branch cut does not create spurious spikes. Because the differencing is applied to the clean simulator states rather than to noisy observations, and because the sampling interval s is short relative to the dominant rigid-body time scales, the amplification of measurement noise that normally accompanies numerical differentiation is limited here; the residual discretization error nevertheless affects both the reference and the supervised targets identically, and it should be kept in mind when interpreting the derivative-level agreement. Under wind-OOD conditions, derivatives associated with velocity and angular-rate states are generally expected to be more sensitive because wind disturbances directly affect relative airflow and aerodynamic response. The purpose of this analysis is to characterize whether the model preserves the local dynamic tendency without expanding the discussion to each individual state component.
High-maneuver recursive behavior was evaluated in all 10 wind-OOD episodes (episodes 58–67). Within each episode, post-warm-start samples at or above the maneuver-score 75th percentile were identified independently. Qualifying intervals separated by no more than 0.10 s were merged, and merged intervals lasting at least 0.20 s were retained. The procedure yielded 157 segments, 3915 samples, and 75.16 s in total.
Figure 7 shows the highest-mean-score segment, from episode 63 at 38.22–38.78 s (0.56 s, 29 samples, mean score 0.979). This segment is illustrative; the dataset-level assessment uses all retained segments.
In the displayed segment, U-STAR-PIML captures the overall temporal trends of the five fast states but shows state-dependent discrepancies. The amplitudes of v, p, and q are visibly underestimated, while the decreasing trend in w is reproduced with a smoother and locally biased response. The prediction of r approaches the reference more closely toward the latter part of the segment. No visible numerical divergence or high-frequency oscillation occurs in this example. Because the segment lies late in a long recursive rollout, the displayed discrepancies include accumulated feedback effects. These local state-wise patterns should not be generalized from the single segment; the aggregate statistics across all detected maneuvers provide the dataset-level assessment.
Across the 157 retained segments from the 10 wind-OOD episodes, the segment-level aggregate fast-state RMSE was (sample SD across segments). State-wise physical-unit errors confirm that the degradation is nonuniform, with the vertical-velocity and pitch-rate channels showing the largest errors within their respective state groups. These fast-state discrepancies can propagate through recursive feedback, but the single displayed segment does not establish a causal mechanism or a common pattern across all maneuvers.
Recursive rollout is a stricter evaluation setting than one-step prediction because predicted states are fed back into the history window and used as inputs for subsequent predictions.
Figure 8 visualizes this effect through a three-dimensional trajectory comparison, where velocity and attitude errors can accumulate through the kinematic integration chain and appear as spatial drift.
The three-dimensional trajectory result illustrates accumulated spatial drift rather than isolated one-step discrepancies. Moderate local errors become more visible when each predicted state determines the next history window, so the trajectory view gives a geometric interpretation of horizon-wise rollout error. This distinction is relevant to model-based simulation, trajectory-planning prediction horizons, and prediction-oriented control modules, which repeatedly query a transition model. It does not, however, constitute a closed-loop control or navigation-accuracy validation.
To test whether the position-reference definition affects this conclusion, we repeated the rollout-position evaluation using the direct JSBSim local-position reference, defined as the simulator-provided local position relative to the initial position rather than a quantity reconstructed by velocity integration.
Table 6 compares the original kinematically reconstructed reference with this independent reference for E5 and E8. The direct-reference evaluation changes the reported RMSE by at most approximately
, while the reference disagreement itself is only
m for ID and
m for wind OOD. The E8 ranking is retained for ID, and the E5 ranking is retained for wind OOD under both reference definitions. This control experiment therefore indicates that the comparative rollout-position conclusion is insensitive to the position-reference definition under the evaluated conditions.
The horizon-wise rollout error in
Figure 9 quantifies how recursive prediction error evolves with prediction horizon. Short horizons mainly reflect local transition accuracy, whereas longer horizons expose error accumulation and free-running stability. This separation is important when a learned model is used for state propagation or a finite prediction window, because one-step accuracy alone cannot guarantee faithful long-horizon trajectories.
In combination with
Section 4.2, the horizon-wise trend shows why wind OOD deserves attention in long-horizon prediction. The curve-based comparison separates short-term local behavior from accumulated error and complements the trajectory visualization. The result can inform the choice of prediction horizon in simulation or planning, but it should not be read as evidence that the model improves closed-loop performance.
Table 7 reports the pooled horizon-wise evaluator metric from the five E5 result files. On the ID test set, the aggregate increases monotonically from
at
s to
at 20 s, with a marked acceleration beyond 5 s as recursively generated states increasingly determine the history buffer. Wind OOD is separated from the other conditions at every horizon: its error is approximately
times the ID value at
s and
times the ID value at 20 s. The declining ratio does not indicate recovery; the absolute gap expands from approximately
to
as both trajectories accumulate error. The parameter, stored-noise diagnostic, and delay conditions remain much closer to ID: their 20 s values differ from the ID aggregate by approximately
,
, and
, respectively, without a consistent degradation ordering across shorter horizons. Because this evaluator metric combines heterogeneous state dimensions, its absolute magnitude should be interpreted together with the physical-unit grouped position RMSE in
Table 5, rather than as a single physically dimensioned error. As in
Section 4.2, the stored-noise diagnostic condition remains a control because the stored noisy observations are not consumed by the evaluated model.
Overall, the dynamic-consistency and rollout analyses show that recursive deployment introduces an error-accumulation problem beyond local one-step prediction. This matters for any simulator or prediction-based decision system that repeatedly feeds model outputs back as inputs. The next subsection turns to one-step predictive uncertainty; uncertainty propagation through recursive rollout is not evaluated and remains an open question.
4.4. One-Step Predictive Uncertainty and Interval Behavior
While
Section 4.2 and
Section 4.3 focus on deterministic prediction accuracy and recursive error accumulation, the proposed model also outputs state-dependent one-step predictive uncertainty. This subsection evaluates the resulting Gaussian intervals across the ID, effective OOD, and diagnostic conditions. The assessment combines NLL, PICP, MPIW, calibration error, interval visualization, and nominal-versus-observed coverage diagrams.
Figure 10 visualizes the one-step predicted means together with state-dependent Gaussian prediction intervals. It provides a sample-level view of the relation among the target, predictive mean, and interval, complementing the aggregate coverage statistics reported below.
The interval visualization should be interpreted together with PICP and MPIW because coverage alone does not distinguish well-calibrated intervals from conservative ones. This distinction is especially relevant here, as the quantitative results show substantial over-coverage for most test subsets.
Figure 11 further evaluates calibration by comparing nominal and observed coverage for the wind-OOD case. The observed curve remains above the ideal diagonal across the displayed confidence levels, while the gap narrows toward the high-confidence region.
At nominal coverage levels of 90% and 95%, the wind-OOD evaluation achieves observed coverages of 95.40% and 96.68%, respectively. These points lie above the ideal diagonal by 5.40 and 1.68 percentage points. The smaller gap at 95% indicates closer agreement at the upper confidence level, although the intervals remain conservative rather than exactly calibrated.
Table 8 summarizes the one-step uncertainty results. Under ID, the 95% interval coverage is approximately 100%, indicating conservative intervals rather than exact calibration.
Wind OOD produces the clearest probabilistic response and the largest interval widening. Its 95% empirical coverage moves from approximately 100% under ID to 96.68%, closer to the nominal level, but this change does not indicate more accurate mean prediction; it reflects wider intervals and movement away from coverage saturation under stronger wind disturbance.
The parameter- and delay-OOD cases, together with the stored-noise diagnostic, remain close to the conservative ID behavior. The stored-noise similarity is expected because the modified observation columns are not consumed by the model, so it verifies the data-path boundary rather than uncertainty or sensor-noise robustness.
Overall, the one-step uncertainty output exhibits high empirical coverage and the largest interval widening under wind OOD. Most conditions show conservative over-coverage rather than exact calibration. From an engineering perspective, these state-dependent intervals are best treated as local monitoring cues accompanying the next-state estimate; their conservative behavior may reduce overconfident one-step predictions. The evidence does not establish a trajectory-level risk envelope, certified safety margin, or automatic fallback trigger, and coverage must be interpreted jointly with interval width and deterministic error. The next subsection examines the model components through comparison and ablation.
4.5. Model Comparison and Ablation Study
Section 4.2,
Section 4.3 and
Section 4.4 evaluate E5 in terms of prediction accuracy, recursive stability, and one-step interval behavior. The E1–E8 comparison in this subsection compares outcomes associated with physics guidance, temporal residual learning, gated fusion, staged uncertainty-head training, operator encoding, and rollout fine-tuning under the common ID-test protocol. Because the configurations are not matched in every architectural and training-budget dimension, the comparison supports association and trade-off analysis rather than causal attribution to a single component. The comparison also distinguishes local prediction accuracy from recursive stability, probabilistic output, and computational cost.
Figure 12a,b show the representative seed-42 comparison. E1 has a one-step RMSE of
, whereas the learned variants remain near the
range; its representative rollout-position error is also far larger than that of the learned models (for example,
m for E1 versus
m for E3). Across the one-step, rollout-position, and 20 s metrics, the learned variants therefore show approximately 94–
lower errors than E1 in this representative comparison. E1 was not included in the five-seed analysis, so this contrast is a descriptive single-run baseline comparison rather than evidence of superiority over a fully identified high-fidelity physics model. The compact prior preserves physical structure but lacks sufficient fidelity for the unresolved aerodynamic, disturbance, and simulator-specific dynamics in this benchmark, supporting learned residual correction without implying conclusions about certified flight modeling.
In the representative seed-42 run, E3 retains similar local accuracy to E2 while reducing rollout-position RMSE from to m. Its 20 s RMSE is also lower ( versus ), although this seed-42 minimum is not a repeated-run ranking. These observations motivate the residual branch for recursive consistency, while the five-seed comparison below provides the primary evidence for overall ranking.
E4 preserves the local accuracy of E3 but does not improve either recursive metric in the ID-only ablation. The comparison therefore provides no direct evidence of a deterministic rollout gain from gating, although the gate may serve a different role under varying or OOD conditions.
Table 9 does not isolate that contribution.
The learned-gate characterization provides an empirical description of this role. The final global physics coefficient was , which did not collapse to zero but was substantially lower than its initialization of . The physics branch therefore remained active, although its scalar contribution was down-weighted. The six-state mean residual-contribution fractions, computed from the mean absolute branch contributions rather than from alone, were for ID, for wind OOD, and for parameter OOD. Except for u, the effective physics and residual contributions were broadly comparable in most dynamic channels. Thus, should not be interpreted as the percentage contribution of the physics branch. Under ID, the mean values for were , and parameter OOD showed the same near-fully open pattern, .
Wind OOD activated stronger and uneven state-dependent residual modulation across the ten evaluated episodes. The mean gate values were
,
,
,
,
, and
. The corresponding standard deviations were
,
,
,
,
, and
, and the minima were
,
,
,
,
, and
, respectively. The most pronounced variation occurred in
w, followed by
q,
v, and
r. The representative heatmap in
Figure 13 illustrates pronounced
w and
q modulation around 9–15 s, without being used to generalize beyond that episode. Relative to the low-wind interval, the high-wind mean gate changes were
(
u),
(
v),
(
w),
(
p),
(
q), and
(
r). A lower
indicates suppression of the residual correction rather than a direct decrease in physics trust. Spearman correlations between
and wind magnitude were
(
u),
(
v),
(
w),
(
p),
(
q), and
(
r). The corresponding correlations with raw physics mismatch were
,
,
,
,
, and
. The stronger positive associations for
w and
q are consistent with state-specific sensitivity to local discrepancy, but they do not establish causality or a simple monotonic transfer of trust away from physics.
In the representative seed-42 run, E5 combines a low one-step RMSE (), low rollout-position RMSE ( m), and a conservative 95% interval coverage of . Its position score is second to E8 in that run, while its broader 20 s all-state score reflects a different objective. Because no matched E5-without-uncertainty-head or end-to-end E5 variant is available, these differences cannot be attributed to a single training component.
Table 9.
ID-test model comparison and ablation results.
Table 9.
ID-test model comparison and ablation results.
| Experiment | Model Variant | One-Step RMSE | Rollout Pos. RMSE | 20 s RMSE | PICP 95 | Runtime |
|---|
| E1 | Physics-only baseline | 0.111880 | 431.79 | 113.291 | – | 0.0331 |
| E2 | Data-driven baseline | 0.005538 | 25.71 | 3.553 | – | 0.0403 |
| E3 | Physics-residual model | 0.005566 | 20.85 | 2.640 | – | 0.0301 |
| E4 | State-dependent gated fusion model | 0.005557 | 21.00 | 2.852 | – | 0.0335 |
| E5 | Full U-STAR-PIML | 0.005513 | 18.82 | 3.546 | 1.0000 | 0.0266 |
| E6 | Operator data model | 0.005591 | 19.42 | 2.723 | – | 0.0434 |
| E7 | Operator residual model | 0.005651 | 22.25 | 5.633 | – | 0.0315 |
| E8 | Operator rollout model | 0.005659 | 16.92 | 2.726 | – | 0.0513 |
In the representative seed-42 run, the operator variants show metric-dependent effects: E6 improves both recursive metrics relative to E2, whereas adding the operator residual formulation in E7 does not preserve that behavior. These observations are limited to the current architectures, optimization schedules, and representative run and are not treated as overall repeated-run rankings.
Figure 12.
Representative seed-42 ID-test comparison of sequential variants: (
a) one-step RMSE; (
b) rollout-position and 20 s all-state RMSE; and (
c) runtime on an NVIDIA RTX 4070 graphics processing unit (GPU) with batch size 256. Panels (
a,
b) use logarithmic scales. Stars mark seed-42 minima at least 3% below the second-lowest value; black outlines mark E5, and hatching marks E6–E8. Five-seed variability of the learned variants is reported separately in
Table 10 and
Figure 14.
Figure 12.
Representative seed-42 ID-test comparison of sequential variants: (
a) one-step RMSE; (
b) rollout-position and 20 s all-state RMSE; and (
c) runtime on an NVIDIA RTX 4070 graphics processing unit (GPU) with batch size 256. Panels (
a,
b) use logarithmic scales. Stars mark seed-42 minima at least 3% below the second-lowest value; black outlines mark E5, and hatching marks E6–E8. Five-seed variability of the learned variants is reported separately in
Table 10 and
Figure 14.
In the representative seed-42 run, E8 attains a rollout-position RMSE of m, illustrating that suppressing position drift and reducing all-state long-horizon error are related but nonequivalent objectives. The five-seed analysis does not support a universal operator advantage, and the representative timing profile cannot attribute any difference causally to rollout fine-tuning.
Table 10.
Five-seed reproducibility of the learned variants on the ID-test set.
Table 10.
Five-seed reproducibility of the learned variants on the ID-test set.
| Experiment | One-Step RMSE | Rollout-Position RMSE (m) | 20 s All-State RMSE |
|---|
| E2 | | | |
| E3 | | | |
| E4 | | | |
| E5 | | | |
| E6 | | | |
| E7 | | | |
| E8 | | | |
Figure 13.
State-wise residual-gate response of E5 under the representative wind-OOD episode 58. The six rows correspond to the dynamic states u, v, w, p, q, and r. Values near one indicate that the learned residual correction is passed with little attenuation, whereas lower values indicate stronger suppression of the residual contribution. The most pronounced modulation occurs in the vertical-velocity and pitch-rate channels during the strong transient region. The gate should not be interpreted as a normalized physics-versus-data mixing coefficient.
Figure 13.
State-wise residual-gate response of E5 under the representative wind-OOD episode 58. The six rows correspond to the dynamic states u, v, w, p, q, and r. Values near one indicate that the learned residual correction is passed with little attenuation, whereas lower values indicate stronger suppression of the residual contribution. The most pronounced modulation occurs in the vertical-velocity and pitch-rate channels during the strong transient region. The gate should not be interpreted as a normalized physics-versus-data mixing coefficient.
Figure 12c shows the representative seed-42 runtime profile. E5 requires
ms per one-step forward pass on an NVIDIA RTX 4070 with batch size 256. This desktop-GPU measurement suggests that neural inference is unlikely to dominate computation in the reported setting, but it is not evidence of real-time onboard capability. Embedded-hardware tests, worst-case latency, memory, energy, and closed-loop scheduling measurements would be required for that claim; the profile is therefore retained as a representative profiling result rather than a stable runtime ranking.
Figure 14.
Five-seed reproducibility of the learned variants under their original method-specific training schedules. Markers denote sample means, and error bars denote 95% Student’s t confidence intervals over seeds 42, 142, 242, 342, and 442. The episode split is fixed across runs. Panel (a) uses a labeled local vertical scale to resolve the small one-step differences; panels (b,c) use linear scales beginning at zero. The analysis quantifies run-to-run variability and does not constitute a matched-budget architecture-only comparison.
Figure 14.
Five-seed reproducibility of the learned variants under their original method-specific training schedules. Markers denote sample means, and error bars denote 95% Student’s t confidence intervals over seeds 42, 142, 242, 342, and 442. The episode split is fixed across runs. Panel (a) uses a labeled local vertical scale to resolve the small one-step differences; panels (b,c) use linear scales beginning at zero. The analysis quantifies run-to-run variability and does not constitute a matched-budget architecture-only comparison.
Five-Seed Reproducibility Analysis
Table 10 and
Figure 14 summarize the repeated-run results computed from the 35 completed runs. E1 is excluded because it was retained only as the original single-run compact-physics reference and was not retrained across the five seeds.
Across the five runs, E5 achieves the lowest mean one-step RMSE, , compared with for E2. E5 produces the lower one-step RMSE in all five paired seeds. The mean paired difference, E2 minus E5, is (95% paired Student’s t confidence interval: –), corresponding to approximately a relative reduction. The repeated-run analysis therefore supports a consistent but quantitatively small local-accuracy advantage rather than a large practical separation. For model selection, this reproducible local difference should be weighed against the intended prediction horizon: short-horizon applications may value stable one-step transitions, whereas longer-horizon prediction must also account for position drift and run-to-run variability.
E5 also has the lowest five-seed mean rollout-position RMSE, m. However, the lowest-error variant changes from E8 to E4, E5, E5, and E2 across seeds 42, 142, 242, 342, and 442, respectively, and the confidence intervals are broad. The repeated-run results therefore do not support a universal winner for recursive position accuracy. For the 20 s all-state metric, E2 has the lowest mean, , while E3 and E5 remain close at and , respectively. The lowest-error variant is E3, E7, E2, E2, and E2 across the same five seeds, so the analysis does not establish a stable long-horizon winner among the leading variants. The operator-based variants E6–E8 also exhibit larger run-to-run dispersion in the 20 s metric than E3 and E5 under the current training protocols.
Overall, E1 remains a single-run reference showing that the isolated compact physics prior is insufficient for accurate simulator-dynamics prediction. Among E2–E8, the five-seed analysis confirms a reproducible but small one-step advantage for E5. E5 has the lowest mean rollout-position error, and E2 has the lowest mean 20 s all-state error, but both recursive rankings vary across seeds. The repeated-run evidence therefore supports objective-dependent and seed-sensitive trade-offs rather than a universal model winner.