Multi-Scale Feature Fusion and Ensemble Learning for Liquid Rocket Engine Fault Diagnosis

Abstract

Limited labeled data and overlapping multisensor signatures make liquid rocket engine fault classification challenging, particularly when feature fusion introduces redundant information. This study proposes a multi-scale feature fusion framework with a calibrated convex ensemble (CCE). A 684-dimensional multi-window statistical representation is fused with a 32-dimensional temporal bottleneck learned through multi-scale convolution, channel attention, and bidirectional long short-term memory. Random-forest-based recursive feature elimination selects features within training partitions, while calibrated predictions from four heterogeneous classifiers are aggregated using simplex-constrained weights. Evaluation focuses on 1106 simulated sequences representing nine known faults in XJTU-REF; the three healthy sequences are excluded from the primary task. Five repetitions of five-fold nested cross-validation yield an accuracy of 90.16 ± 1.04%, a macro-F1 score of 89.84 ± 1.08%, and a Matthews correlation coefficient of 0.891 ± 0.012 (mean ± standard deviation across outer-test folds). Accuracy exceeds that of the random-forest baseline using mean features from the final 20% of each sequence by 2.45 percentage points. Ablations support the complementary roles of the two representations and feature selection, while calibration reduces expected calibration error from 7.18% to 3.76% relative to the uncalibrated ensemble. Leave-one-unit-out accuracy is 86.26 ± 1.21% across three simulated units, indicating lower performance on unseen units than under sample-level validation. These findings support complementary feature fusion with dimensionality control and calibrated aggregation for closed-set fault classification, while healthy-state recognition and real-engine applicability require further validation.

1. Introduction

Reliable fault classification is an important component of liquid rocket engine (LRE) diagnostic analysis, providing a basis for identifying suspect subsystems and guiding subsequent engineering assessment. The coupled behavior of engine subsystems produces multisensor responses in which persistent operating-level deviations coexist with localized temporal changes. Consequently, diagnostically useful information may be distributed across different variables and temporal scales rather than concentrated in a single measurement or observation interval. Different fault mechanisms can also produce similar pressure and flow responses, making their separation dependent on subtle differences in signal magnitude, fluctuation, and temporal evolution. Learning these distinctions from a limited number of labeled engine sequences therefore requires a careful balance between representational richness and model complexity.

Existing LRE-specific studies have explored several approaches to this problem, including model-based filtering [1], small-sample Bayesian classification [2], and neural sequence representations [3]. Model-based methods exploit prescribed system dynamics, but their diagnostic interpretation remains dependent on the adequacy of the underlying model and its assumptions. Bayesian methods provide a framework for learning under limited observations, whereas neural sequence models offer flexible representations of nonlinear temporal relationships. These approaches address important aspects of LRE diagnosis, but they do not eliminate the trade-offs associated with representation design and finite-sample learning. In particular, a compact representation may omit useful fault information, while a more expressive representation can introduce additional estimation uncertainty. The challenge addressed in this study is to exploit complementary signal characteristics without allowing the resulting model complexity to exceed what the available labeled data can support.

A first challenge concerns the incomplete characterization of fault signatures by a single representation. Statistical descriptors summarize operating levels, variability, distributional shape, and local changes, but they do not explicitly preserve the complete temporal ordering of a sequence. Restricting these descriptors to one observation window can further obscure differences between persistent deviations and localized responses. Learned temporal representations can capture ordering and contextual dependencies, but a temporal-only model must infer useful discriminative structure from the available training sequences. Combining these two forms of information is therefore attractive, but their complementarity does not guarantee that direct concatenation will improve classification. Correlated statistical descriptors and weakly informative learned coordinates can enlarge the feature space without providing a corresponding increase in usable information. For small-sample LRE classification, feature fusion must consequently be accompanied by explicit dimensionality control.

A second challenge concerns the reliability of the final decision. Classification accuracy alone does not establish whether predicted probabilities appropriately reflect the frequency of correct decisions. This distinction is particularly relevant when fault categories have overlapping signatures. Moreover, heterogeneous classifiers produce outputs with different probabilistic characteristics, so direct soft voting can combine predictions that are not equally calibrated. Probability-calibration studies have demonstrated the importance of evaluating and correcting such discrepancies [4]. A suitable decision mechanism should therefore address both discrimination and probability quality, rather than assuming that a higher classification score necessarily implies more reliable confidence estimates. It should also estimate ensemble weights from training-derived predictions instead of relying solely on manually specified voting preferences.

A third challenge is to distinguish improvements in model performance from effects introduced by model selection and data partitioning. Feature extraction, feature selection, calibration, and ensemble-weight estimation are all data-dependent operations. Their placement within the validation procedure determines whether the resulting performance estimates reflect genuinely held-out observations. In addition, sample-level and engine-unit-level evaluations answer different questions: a random stratified split can assess classification of additional sequences from represented units, whereas holding out an entire engine unit evaluates transfer to a unit excluded from model development. These distinctions are essential when assessing a multistage framework with numerous tunable components and only a small number of simulated engine units.

To address these challenges, this paper proposes a multi-scale feature fusion framework with a calibrated convex ensemble (CCE). The statistical branch constructs a 684-dimensional representation from multiple temporal windows, preserving complementary descriptions of operating levels, distributional characteristics, and local variation. In parallel, a temporal branch combines multi-scale convolution, squeeze-and-excitation channel attention, and bidirectional recurrence to produce a compact 32-dimensional bottleneck. The resulting 716-dimensional representation is refined through random-forest-based recursive feature elimination (RF-RFE), following the recursive feature-selection principle [5]. The retained dimension and model settings are selected within the inner training procedure. Finally, Random Forest, Support Vector Machine, Gradient Boosting, and K-Nearest Neighbors predictions are calibrated and combined using nonnegative weights constrained to the probability simplex. The design thus links complementary representation, finite-sample complexity control, and probability-level decision fusion within a common training-partition framework.

The framework is evaluated on XJTU-REF, which provides simulated multisensor sequences generated from an open-cycle LRE model [6]. The dataset contains 1106 sequences across nine fault conditions but only three healthy sequences. Accordingly, the primary task is closed-set classification among the nine modeled faults, conditional on the input belonging to this fault set. Healthy-state recognition and the probability that an arbitrary operational sequence is faulty are outside the evaluated task. Repeated nested cross-validation and leave-one-unit-out evaluation are reported separately to distinguish sample-level performance from transfer across simulated engine units.

The main contributions of this study are as follows:

• A problem-specific dual-view representation. We combine multi-window statistical descriptors with a compact learned temporal representation to characterize persistent operating deviations and temporal fault patterns within the same sequence. Single-view and fused configurations provide explicit comparisons for assessing whether these representations contribute complementary classification information.
• Training-partition-controlled feature fusion and selection. We incorporate RF-RFE into the inner model-selection procedure so that the retained representation is determined without accessing the outer-test data. Comparing direct concatenation with selected fusion separates the effect of adding features from the effect of controlling their finite-sample complexity.
• A calibrated heterogeneous decision formulation. We calibrate training-derived cross-fitted outputs from four complementary classifiers and estimate their ensemble weights by minimizing a regularized class-balanced log-loss over the probability simplex. This formulation replaces manually assigned voting weights with a defined optimization objective and permits separate assessment of classification performance and probability quality.
• An evaluation that separates component effects and generalization targets. We combine repeated nested validation, component ablations, paired comparisons, and individual engine-unit holdouts to examine representation, selection, and decision-stage contributions. The resulting evidence is interpreted specifically for nine-fault classification on the simulated benchmark, with sample-level results distinguished from cross-unit transfer.

2. Related Work and Scope of the Contribution

2.1. LRE-Specific Evidence

Fault diagnosis for liquid rocket engines (LREs) has been pursued through model-based, knowledge-based, and data-driven paradigms. Model-based approaches, such as Kalman filtering and particle filtering, offer interpretable residuals but suffer from high dependency on accurate physical models and poor generalizability [7,8,9]. Knowledge-driven expert systems encode human rules or fuzzy logic but struggle with coupled fault relationships and inference efficiency [10,11]. Traditional signal processing techniques, e.g., frequency analysis, parity-space methods, and wavelet denoising, have been applied but require extensive domain expertise and perform poorly under noisy or non-stationary conditions [12,13,14]. With advances in AI, intelligent data-driven methods have gained traction. Conventional machine learning models (SVM, ANN, random forests) combined with hand-crafted features have been used for turbopump and steady-state fault detection [15,16,17,18,19,20], while deep learning architectures such as CNNs, LSTMs, and autoencoders enable end-to-end feature learning and classification [21,22,23,24].

To overcome the critical bottleneck of limited fault-labeled data, researchers have developed data resampling and generation techniques (SMOTE, GANs, VAEs, digital twins) [25,26,27,28,29,30,31], feature-level enhancements via attention and regularization [32,33,34,35,36], and algorithm-level solutions including cost-sensitive learning and ensemble classifiers [37,38,39]. Transfer learning leverages source-domain knowledge for new engines with sparse target data [40], while meta-learning and zero-shot learning are emerging for few-shot and unseen fault scenarios [41,42]. Despite these progresses, challenges remain in developing comprehensive frameworks, understanding deep model theory, and effectively integrating domain knowledge (e.g., physical causality and failure rules) with data-driven models to ensure interpretability and trustworthiness, issues that recent XAI methods like SHAP and LIME have begun to address but lack systematic evaluation.

2.2. General Methodological Foundations

Squeeze-and-excitation reweights learned channels, while RFE is a general feature-selection procedure [5]. Neither establishes physical sensor causality. Probability calibration has its own evaluation objectives [4]; a classifier can improve NLL without materially changing its predicted labels. Selection stability requires an explicit resampling definition and attention to subset size [43]. Modern time-series methods such as InceptionTime and ROCKET provide useful additional comparators [44,45]. The positioning of the study is shown in Table 1.

Table 1. Positioning of the study. Differences describe scope, not an audited absence of capabilities in earlier work.

Study or Methodological Family	Relevant Emphasis	Scope of the Present Comparison
Cha et al. [1]	Model-based startup-process diagnosis	A different diagnostic formulation; no cross-paper accuracy ranking.
Liu et al. [2]	Bayesian estimation for sparse LRE data	A small-data comparator requiring matched implementation.
Zhang et al. [3]	Attention-based temporal LRE classification	Demonstrates that CNN–recurrent attention is not individually novel.
InceptionTime/ROCKET [44,45]	General time-series classification	Additional matched baselines are needed.
Present formulation	Statistical/temporal views, nested selection, calibrated aggregation	Fault-only simulated benchmark; not normal/fault monitoring or demonstrated simulation-to-real transfer.

3. Proposed Methodology

3.1. Overview of the Proposed Framework

Figure 1 illustrates the complete learning and evaluation pipeline of the proposed framework. Given a multivariate liquid rocket engine sequence, sensor-wise normalization is first estimated exclusively from the current training partition. The normalized sequence is then processed through two complementary representation branches. The statistical branch extracts a 684-dimensional descriptor from four nested temporal windows to characterize operating levels, distributional properties, and local variations. In parallel, the temporal branch learns a 32-dimensional bottleneck representation through multi-scale convolution, latent-channel attention, bidirectional recurrence, and an auxiliary classification objective. The two representations are concatenated into a 716-dimensional dual-view feature vector, which is robustly scaled and reduced through Random-Forest-based recursive feature elimination (RF-RFE). The retained feature dimension and all remaining hyperparameters are selected entirely within the inner cross-validation loop.

Figure 1. Overall architecture of the proposed leakage-controlled dual-view fault-diagnosis framework.

The selected representation is subsequently supplied to four heterogeneous base classifiers: Random Forest (RF), radial-basis-function Support Vector Machine (RBF-SVM), Gradient Boosting (GB), and distance-weighted K-Nearest Neighbors (KNN). Their cross-fitted score vectors are individually calibrated and combined through simplex-constrained convex weight optimization. Every data-dependent operation, including normalization, representation learning, robust scaling, feature selection, probability calibration, and ensemble-weight estimation, is refitted using only the corresponding training partition. Repeated nested cross-validation evaluates sample-level generalization, whereas leave-one-unit-out evaluation measures transfer to an unseen simulated engine unit. The framework therefore addresses three related questions: whether the statistical and temporal views contain complementary diagnostic information, whether nested dimensionality control reduces the finite-sample variance introduced by feature fusion, and whether calibrated ensemble learning improves the reliability of the final predictive distribution.

3.2. Problem Formulation and Notation

We first define the principal notation in Table 2. Let the complete benchmark be represented by

$${\mathcal{D}}_{\mathrm{all}}={\left\{\left({\mathit{X}}_i,y_i,g_i\right)\right\}}_{i=1}^{N_{\mathrm{all}}},{\mathit{X}}_i\in {\mathbb{R}}^{T\times D},$$

(1)

where ${\mathit{X}}_i=\left[x_{itd}\right]$ is the ith multisensor sequence, $T=501$ is the number of time samples, $D=19$ is the number of monitored variables, $y_i$ is the operating-condition label, and $g_i$ identifies the simulated engine unit from which the sequence is generated.

Table 2. Principal notation used in the proposed methodology.

Symbol	Meaning	Symbol	Meaning
$N,T,D,C$	Numbers of fault samples, time points, sensors, and fault classes	$g_i$	Engine-unit or group identifier
$\mathcal{W}$	Set of nested statistical windows	$\mathcal{K}$	Set of convolutional kernel sizes
${\mathit{f}}^{\mathrm{stat}}$	Multi-window statistical representation	${\mathit{f}}^{\mathrm{temp}}$	Learned temporal bottleneck representation
$\mathit{z}$	Concatenated dual-view representation	$\widehat{\mathit{z}}$	Robustly scaled fused representation
${\mathcal{S}}_q^{\left(r\right)}$	Feature subset of size q selected in inner fold r	$\mathit{x}$	Selected representation supplied to the classifiers
${\mathit{s}}_m$	Raw score vector of base learner m	${\tilde{\mathit{p}}}_m$	Calibrated probability vector of base learner m
$\mathit{w}$	Nonnegative ensemble-weight vector	${\Delta }_M$	Probability simplex for M ensemble weights

Because the benchmark contains only three healthy observations, the primary analysis is conducted on the fault-only subset

$$\mathcal{D}=\left\{\left({\mathit{X}}_i,y_i,g_i\right)\in {\mathcal{D}}_{\mathrm{all}}:y_i\ne \mathrm{Healthy}\right\}={\left\{\left({\mathit{X}}_i,y_i,g_i\right)\right\}}_{i=1}^N,$$

(2)

with

$$y_i\in \{1,\dots ,C\},C=9.$$

(3)

The ten-class task including the healthy condition is retained only as an exploratory analysis and is not used to support claims concerning healthy-state recognition.

The diagnostic model maps an input sequence to the probability simplex

$$h:{\mathbb{R}}^{T\times D}\to {\Delta }_C,{\Delta }_C=\left\{\mathit{p}\in {\mathbb{R}}_{+}^C:{\displaystyle \sum _{c=1}^C}{p}_c=1\right\}.$$

(4)

The predicted class is

$${\widehat{y}}_i=\arg \underset{c\in \{1,\dots ,C\}}{\max }{h}_c\left({\mathit{X}}_i\right).$$

(5)

For a generic training index set $\mathcal{A}$, the supervised temporal extractor is trained by minimizing the class-balanced empirical cross-entropy

$${\widehat{\mathcal{R}}}_{\alpha }\left(\theta ;\mathcal{A}\right)=-{\displaystyle \frac{1}{\left|\mathcal{A}\right|}}\sum _{i\in \mathcal{A}}{\alpha }_{y_i}^{\left(\mathcal{A}\right)}\log p_{\theta }\left(y_i\mid {\mathit{X}}_i\right),$$

(6)

where

$${\alpha }_c^{\left(\mathcal{A}\right)}={\displaystyle \frac{\left|\mathcal{A}\right|}{C{N}_c^{\left(\mathcal{A}\right)}}},N_c^{\left(\mathcal{A}\right)}=\sum _{i\in \mathcal{A}}\mathbb{I}(y_i=c).$$

(7)

The downstream RF, RBF-SVM, GB, and KNN models retain their respective training objectives. Class weighting is applied to a downstream learner only when it is supported by that learner and included in the corresponding candidate configuration. The class-balanced negative log-likelihood is also used in the probability-level ensemble-weight optimization introduced in Section 3.7.

Assumption 1

(Evaluation isolation). For every outer split, the outer-test partition is excluded from sensor normalization, temporal-network training, early stopping, feature scaling, RF-RFE, hyperparameter selection, probability calibration, ensemble-weight estimation, and all other data-dependent operations. The outer-test partition is accessed only after the complete model-development procedure has been finalized on the corresponding outer-training partition.

3.3. Training-Partition Sensor Normalization

Sensor normalization is performed using statistics estimated from the current fitting partition rather than independently within each sample. Let $\mathcal{A}$ denote the training indices available to a particular fitting operation. For sensor d, the training-partition mean and variance are

$$\begin{array}{cc}\hfill {\mu }_d^{\left(\mathcal{A}\right)}& ={\displaystyle \frac{1}{\left|\mathcal{A}\right|T}}\sum _{i\in \mathcal{A}}{\displaystyle \sum _{t=1}^T}{x}_{itd},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\left({\sigma }_d^{\left(\mathcal{A}\right)}\right)}^2& ={\displaystyle \frac{1}{\left|\mathcal{A}\right|T-1}}\sum _{i\in \mathcal{A}}{\displaystyle \sum _{t=1}^T}{\left(x_{itd}-{\mu }_d^{\left(\mathcal{A}\right)}\right)}^2.\hfill \end{array}$$

(9)

Any training, validation, or test sequence transformed by this fitted normalizer is represented as

$${\tilde{x}}_{itd}={\displaystyle \frac{x_{itd}-{\mu }_d^{\left(\mathcal{A}\right)}}{\sqrt{{\left({\sigma }_d^{\left(\mathcal{A}\right)}\right)}^2+\epsilon }}},$$

(10)

where $\epsilon >0$ is a small numerical constant.

This training-partition transformation preserves between-sample deviations relative to the fitted operating distribution. Such deviations are diagnostically meaningful for pressure, flow, power, mixture-ratio, and temperature-related variables. In contrast, fitting a separate scaler to each individual sequence would force every sensor trajectory to have approximately zero mean and unit variance, thereby attenuating absolute operating-level differences that the statistical representation is intended to retain.

3.4. Multi-Scale Statistical View

The statistical view describes operating levels, distributional shape, and local temporal variation over nested horizons. The window set is

$$\mathcal{W}=\left\{(0,1),(0.5,1),(0.7,1),(0.85,1)\right\}.$$

(11)

For $w=(a_w,b_w)\in \mathcal{W}$, define

$${\mathcal{T}}_w=\left\{\lfloor a_{w}T\rfloor +1,\dots ,\lfloor b_{w}T\rfloor \right\},L_w=\left|{\mathcal{T}}_w\right|,$$

(12)

and

$${\mathcal{T}}_w^{-}={\mathcal{T}}_w\setminus \left\{\max {\mathcal{T}}_w\right\}.$$

(13)

At 500 Hz, the four windows correspond approximately to the complete one-second record and its final 0.50, 0.30, and 0.15 s segments.

For sample i, sensor d, and window w, the local mean and standard deviation are

$$\begin{array}{cc}\hfill {\mu }_{id}^{\left(w\right)}& ={\displaystyle \frac{1}{L_w}}\sum _{t\in {\mathcal{T}}_w}{\tilde{x}}_{itd},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill s_{id}^{\left(w\right)}& =\sqrt{{\displaystyle \frac{1}{L_w-1}}\sum _{t\in {\mathcal{T}}_w}{\left({\tilde{x}}_{itd}-{\mu }_{id}^{\left(w\right)}\right)}^2+\epsilon }.\hfill \end{array}$$

(15)

The skewness and excess kurtosis are

$$\begin{array}{cc}\hfill {\gamma }_{1,id}^{\left(w\right)}& ={\displaystyle \frac{1}{L_w}}\sum _{t\in {\mathcal{T}}_w}{\left({\displaystyle \frac{{\tilde{x}}_{itd}-{\mu }_{id}^{\left(w\right)}}{s_{id}^{\left(w\right)}}}\right)}^3,\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\gamma }_{2,id}^{\left(w\right)}& ={\displaystyle \frac{1}{L_w}}\sum _{t\in {\mathcal{T}}_w}{\left({\displaystyle \frac{{\tilde{x}}_{itd}-{\mu }_{id}^{\left(w\right)}}{s_{id}^{\left(w\right)}}}\right)}^4-3.\hfill \end{array}$$

(17)

Let

$$\Delta {\tilde{x}}_{itd}={\tilde{x}}_{i,t+1,d}-{\tilde{x}}_{itd},t\in {\mathcal{T}}_w^{-}.$$

(18)

The mean and standard deviation of the first-order differences are

$$\begin{array}{cc}\hfill {\overline{\Delta x}}_{id}^{\left(w\right)}& ={\displaystyle \frac{1}{L_w-1}}\sum _{t\in {\mathcal{T}}_w^{-}}\Delta {\tilde{x}}_{itd},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill s_{\Delta ,id}^{\left(w\right)}& =\sqrt{{\displaystyle \frac{1}{L_w-2}}\sum _{t\in {\mathcal{T}}_w^{-}}{\left(\Delta {\tilde{x}}_{itd}-{\overline{\Delta x}}_{id}^{\left(w\right)}\right)}^2+\epsilon }.\hfill \end{array}$$

(20)

Together with the minimum, maximum, and median, the nine-dimensional sensor-window descriptor is

$${\mathit{\varphi }}_{id}^{\left(w\right)}={\left[{\mu }_{id}^{\left(w\right)},s_{id}^{\left(w\right)},x_{\min ,id}^{\left(w\right)},x_{\max ,id}^{\left(w\right)},x_{\mathrm{med},id}^{\left(w\right)},{\gamma }_{1,id}^{\left(w\right)},{\gamma }_{2,id}^{\left(w\right)},{\overline{\Delta x}}_{id}^{\left(w\right)},s_{\Delta ,id}^{\left(w\right)}\right]}^{\top }\in {\mathbb{R}}^9.$$

(21)

The complete statistical representation is

$${\mathit{f}}_i^{\mathrm{stat}}=\underset{w\in \mathcal{W}}{⨁}{\displaystyle \underset{d=1}{\overset{D}{⨁}}}{\mathit{\varphi }}_{id}^{\left(w\right)}\in {\mathbb{R}}^{9\left|\mathcal{W}\right|D}={\mathbb{R}}^{684},$$

(22)

where ⨁ denotes vector concatenation.

The nested windows encode a controlled bias–variance trade-off. Long windows provide relatively stable estimates of persistent operating shifts, whereas short windows increase sensitivity to localized late-stage transients. The different horizons are therefore treated as complementary statistical views of the same sequence rather than as independent observations.

3.5. Learned Temporal View

The statistical representation does not retain the complete ordering of samples within each window; first-difference descriptors retain only limited local-order information. A learned temporal branch is therefore introduced to model local temporal motifs, longer-range dependencies, and interactions between latent dynamic channels.

3.5.1. Multi-Scale Convolutional Representation

Let

$$\mathcal{K}=\{3,7,15\}$$

(23)

be the set of convolutional kernel sizes. For sample i and branch $k\in \mathcal{K}$,

$${\mathit{H}}_i^{\left(k\right)}=Pool\left[BN\left(\rho \left({\tilde{\mathit{X}}}_i\ast {\mathit{W}}^{\left(k\right)}+{\mathit{b}}^{\left(k\right)}\right)\right)\right],$$

(24)

where $\rho \left(u\right)=\max (0,u)$, each branch contains $F=32$ filters, ∗ denotes same-padded one-dimensional convolution along the temporal dimension, and max pooling has width and stride two. At 500 Hz, the three kernels span approximately 6, 14, and 30 ms, respectively.

The branch outputs are concatenated along the latent-channel dimension:

$${\mathit{H}}_i^{\mathrm{ms}}=Concat\left({\mathit{H}}_i^{\left(3\right)},{\mathit{H}}_i^{\left(7\right)},{\mathit{H}}_i^{\left(15\right)}\right)\in {\mathbb{R}}^{T^{\prime }\times 96},T^{\prime }=⌊{\displaystyle \frac{T}{2}}⌋.$$

(25)

3.5.2. Latent-Channel Attention

The attention module reweights the 96 convolutional latent channels rather than the 19 original sensor variables. Temporal squeeze pooling produces

$${\mathit{q}}_i={\displaystyle \frac{1}{T^{\prime }}}{\displaystyle \sum _{t=1}^{T^{\prime }}}{\mathit{H}}_{i,t,:}^{\mathrm{ms}}\in {\mathbb{R}}^{96}.$$

(26)

The latent-channel weights are

$${\mathit{a}}_i=sigm\left[{\mathit{W}}_2\rho \left({\mathit{W}}_1{\mathit{q}}_i+{\mathit{b}}_1\right)+{\mathit{b}}_2\right]\in {(0,1)}^{96},$$

(27)

where

$${\mathit{W}}_1\in {\mathbb{R}}^{24\times 96},{\mathit{W}}_2\in {\mathbb{R}}^{96\times 24},$$

(28)

corresponding to a reduction ratio of four. The attended representation is

$${\mathit{H}}_{i,t,:}^{\mathrm{att}}={\mathit{H}}_{i,t,:}^{\mathrm{ms}}\odot {\mathit{a}}_i,t=1,\dots ,T^{\prime },$$

(29)

where ⊙ denotes element-wise multiplication.

Because the attention coefficients act after convolutional feature generation, they should not be interpreted as direct physical importance scores for individual sensors. Sensor-level interpretation requires a separate attribution analysis with respect to the original input variables.

3.5.3. Bidirectional Recurrence and Bottleneck Projection

A bidirectional LSTM with 32 hidden units per direction processes ${\mathit{H}}_i^{\mathrm{att}}$. Its pooled representation is

$${\mathit{u}}_i^{\mathrm{bi}}={\displaystyle \frac{1}{T^{\prime }}}{\displaystyle \sum _{t=1}^{T^{\prime }}}\left[{\overrightarrow{\mathit{h}}}_{it};{\overleftarrow{\mathit{h}}}_{it}\right]\in {\mathbb{R}}^{64}.$$

(30)

A complementary unidirectional LSTM with 32 hidden units is applied directly to the normalized sensor sequence:

$${\mathit{u}}_i^{\mathrm{uni}}=LastHidden\left[{LSTM}_{32}\left({\tilde{\mathit{X}}}_i\right)\right]\in {\mathbb{R}}^{32}.$$

(31)

The two recurrent representations are concatenated and projected through a two-layer bottleneck:

$$\begin{array}{cc}\hfill {\mathit{r}}_i& =\rho \left[{\mathit{W}}_{b1}\left({\mathit{u}}_i^{\mathrm{bi}}\oplus {\mathit{u}}_i^{\mathrm{uni}}\right)+{\mathit{b}}_{b1}\right]\in {\mathbb{R}}^{64},\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill {\mathit{f}}_i^{\mathrm{temp}}& =\rho \left[{\mathit{W}}_{b2}Dropout\left({\mathit{r}}_i;0.3\right)+{\mathit{b}}_{b2}\right]\in {\mathbb{R}}^{32}.\hfill \end{array}$$

(33)

The 32-dimensional vector ${\mathit{f}}_i^{\mathrm{temp}}$ is the temporal representation used in the subsequent feature-fusion stage.

During representation learning, an auxiliary classification head is attached:

$$p_{\theta }\left(c\mid {\mathit{X}}_i\right)=softmax{\left({\mathit{W}}_c{\mathit{f}}_i^{\mathrm{temp}}+{\mathit{b}}_c\right)}_c.$$

(34)

The temporal extractor is optimized using Equation (6). During inner-loop model development, the epoch number is selected using an early-stopping subset drawn exclusively from the corresponding inner-training partition. The inner-validation fold used for configuration scoring is not used for early stopping. Once the epoch number has been selected, the auxiliary head is discarded and Equation (33) is used as the temporal feature map.

3.6. Feature Fusion, Robust Scaling, and Nested Selection

The statistical and temporal views are concatenated as

$${\mathit{z}}_i={\mathit{f}}_i^{\mathrm{stat}}\oplus {\mathit{f}}_i^{\mathrm{temp}}\in {\mathbb{R}}^{684+32}={\mathbb{R}}^{716}.$$

(35)

For every fused coordinate j, a robust transformation is fitted from the current training partition $\mathcal{A}$:

$${\widehat{z}}_{ij}={\displaystyle \frac{z_{ij}-{median}_{r\in \mathcal{A}}\left(z_{rj}\right)}{{IQR}_{r\in \mathcal{A}}\left(z_{rj}\right)+\epsilon }}.$$

(36)

The same fitted median and interquartile range are subsequently applied to the associated validation or test partition.

RF-RFE recursively removes weak fused coordinates. For tree b and split node n, let $v_{bn}$ denote the feature used at that node, ${\pi }_{bn}$ the proportion of training samples reaching the node, and $G_{bn}$ its Gini impurity. The weighted impurity reduction is

$$\Delta G_{bn}={\pi }_{bn}{G}_{bn}-{\pi }_{bn}^L{G}_{bn}^L-{\pi }_{bn}^R{G}_{bn}^R.$$

(37)

The forest importance assigned to feature j is

$${\mathcal{I}}_j={\displaystyle \frac{1}{B_{\mathrm{tree}}}}{\displaystyle \sum _{b=1}^{B_{\mathrm{tree}}}}\sum _{n:v_{bn}=j}\Delta G_{bn}.$$

(38)

Starting from

$${\mathcal{S}}^{\left(0\right)}=\{1,\dots ,716\},$$

(39)

one elimination operation is

$${\mathcal{S}}^{(\ell +1)}={\mathcal{S}}^{(\ell )}\setminus \left\{\arg \underset{j\in {\mathcal{S}}^{(\ell )}}{\min }{\mathcal{I}}_j\right\}.$$

(40)

In implementation, multiple lowest-ranked variables may be removed per iteration to reduce computational cost, provided that the removal schedule is fixed before accessing the corresponding validation fold.

For the complete 716-D representation, the retained dimension is selected from

$$\mathcal{Q}=\{15,30,45,60,90,120\}.$$

(41)

For inner fold r, let ${\mathcal{S}}_q^{\left(r\right)}$ denote the q-feature subset obtained exclusively from that fold’s inner-training partition. The selected dimension satisfies

$$q^{★}=\arg \underset{q\in \mathcal{Q}}{\max }{\displaystyle \frac{1}{K_{\mathrm{in}}}}{\displaystyle \sum _{r=1}^{K_{\mathrm{in}}}}{MacroF1}_r\left({\mathcal{S}}_q^{\left(r\right)}\right).$$

(42)

For an ablation with fewer input coordinates, infeasible candidate dimensions are excluded. Ties are resolved by lower mean inner negative log-likelihood and then lower mean expected calibration error. In the complete search procedure, q is treated as one component of the candidate configuration $\theta \in \mathsf{\Theta }$ and is therefore selected jointly with the remaining hyperparameters.

For an aligned statistical-feature dictionary, feature-selection stability across $B_{\mathrm{stab}}$ training resamples is summarized by the mean pairwise Jaccard index

$$Stab={\displaystyle \frac{2}{B_{\mathrm{stab}}\left(B_{\mathrm{stab}}-1\right)}}\sum _{1\le b<b^{\prime }\le B_{\mathrm{stab}}}{\displaystyle \frac{\left|{\mathcal{S}}^{\left(b\right)}\cap {\mathcal{S}}^{\left(b^{\prime }\right)}\right|}{\left|{\mathcal{S}}^{\left(b\right)}\cup {\mathcal{S}}^{\left(b^{\prime }\right)}\right|}}.$$

(43)

This diagnostic is computed only from training-derived subsets of the fixed 684-coordinate statistical dictionary. Independently trained temporal coordinates cannot be assumed to have aligned identities; their retention is summarized at block level. Subset-size effects and a chance-adjusted statistic are discussed in Section 5.3. No numerical stability estimate is claimed without the corresponding selection archive.

The selected representation supplied to every base classifier is

$${\mathit{x}}_i={\widehat{\mathit{z}}}_{i,{\mathcal{S}}_{q^{★}}}\in {\mathbb{R}}^{q^{★}}.$$

(44)

Proposition 1

(Information monotonicity of the fused view). Let

$$U={\mathit{f}}^{\mathrm{stat}},V={\mathit{f}}^{\mathrm{temp}},$$

(45)

and define

$${\eta }_c\left(U\right)=Pr(Y=c\mid U),{\eta }_c(U,V)=Pr(Y=c\mid U,V).$$

(46)

Under zero–one loss,

$${\mathcal{R}}_U^{★}-{\mathcal{R}}_{U,V}^{★}=\mathbb{E}\left[\underset{c}{\max }{\eta }_c(U,V)-\underset{c}{\max }{\eta }_c\left(U\right)\right]\ge 0.$$

(47)

Proof. 

The Bayes risks based on U and $(U,V)$ are

$$\begin{array}{cc}\hfill {\mathcal{R}}_U^{★}& =1-\mathbb{E}\left[\underset{c}{\max }{\eta }_c\left(U\right)\right],\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\mathcal{R}}_{U,V}^{★}& =1-\mathbb{E}\left[\underset{c}{\max }{\eta }_c(U,V)\right].\hfill \end{array}$$

(49)

Conditioning on U gives

$$\begin{array}{cc}\hfill \mathbb{E}\left[\underset{c}{\max }{\eta }_c(U,V)\mid U\right]& \ge \underset{c}{\max }\mathbb{E}\left[{\eta }_c(U,V)\mid U\right]\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill & =\underset{c}{\max }{\eta }_c\left(U\right),\hfill \end{array}$$

(51)

where the inequality follows from the convexity of the maximum operator and the equality follows from the tower property. Taking expectations yields Equation (47). □

Proposition 1 establishes that observing the temporal view cannot increase the Bayes-optimal classification risk because a theoretically optimal classifier can always ignore the additional view. However, it does not imply that arbitrary empirical concatenation must improve finite-sample performance. With limited samples, redundant or weak coordinates can increase estimation variance and degrade a misspecified classifier. This distinction provides the theoretical motivation for nested scaling and RF-RFE after dual-view concatenation.

3.7. Calibrated Convex Ensemble

The selected representation is supplied to

$$M=4$$

(52)

heterogeneous base learners: RF, RBF-SVM, GB, and distance-weighted KNN. These learners provide different inductive biases. RF and GB construct piecewise nonlinear tree partitions, RBF-SVM produces a smooth kernel-induced decision boundary, and KNN preserves local neighborhood structure.

Let

$${\mathit{s}}_m\left({\mathit{x}}_i\right)\in {\mathbb{R}}^C$$

(53)

denote the raw score or uncalibrated probability vector produced by learner m. A calibration mapping

$${\mathcal{C}}_m:{\mathbb{R}}^C\to {\Delta }_C$$

(54)

produces

$${\tilde{\mathit{p}}}_m\left({\mathit{x}}_i\right)={\mathcal{C}}_m\left[{\mathit{s}}_m\left({\mathit{x}}_i\right)\right]\in {\Delta }_C.$$

(55)

Temperature scaling is considered when a compatible logit vector is available; Platt-type scaling and isotonic regression are considered for classifiers providing decision scores or empirical probabilities. The calibration family is selected in the inner loop using cross-fitted probabilities and validation NLL, and the calibrated multiclass vector is renormalized when required [4].

Let

$${\tilde{p}}_{imc}={\left[{\tilde{\mathit{p}}}_m\left({\mathit{x}}_i\right)\right]}_c$$

(56)

be the calibrated probability assigned by learner m to class c for cross-fitted observation i. The ensemble weights are estimated by

$$\begin{array}{cc}\hfill {\mathit{w}}^{★}=\arg \underset{\mathit{w}\in {\Delta }_M}{\min }\{& -{\displaystyle \frac{1}{N_{\mathrm{OOF}}}}{\displaystyle \sum _{i=1}^{N_{\mathrm{OOF}}}}{\alpha }_{y_i}^{\left(\mathrm{OOF}\right)}\log \left[{\displaystyle \sum _{m=1}^M}{w}_m{\tilde{p}}_{im,y_i}+\epsilon \right]\hfill \\ \hfill & +{\displaystyle \frac{{\lambda }_w}{2}}{\parallel \mathit{w}\parallel }_2^2\},{\Delta }_M=\left\{\mathit{w}\in {\mathbb{R}}_{+}^M:{\displaystyle \sum _{m=1}^M}{w}_m=1\right\}.\hfill \end{array}$$

(57)

The probability-simplex constraint prevents negative voting coefficients and makes the final output a valid convex combination of calibrated probability vectors. The regularization parameter ${\lambda }_w\ge 0$ controls concentration of the ensemble on a single learner.

For a new sequence $\mathit{X}$, the final predictive distribution is

$$\widehat{\mathit{p}}\left(\mathit{X}\right)={\displaystyle \sum _{m=1}^M}{w}_m^{★}{\mathcal{C}}_m^{★}\left[{\mathit{s}}_m\left(\mathit{x}\right)\right]\in {\Delta }_C,$$

(58)

and the predicted class is

$$\widehat{y}=\arg \underset{c}{\max }{\widehat{p}}_c\left(\mathit{X}\right).$$

(59)

A sample-level uncertainty summary is obtained from predictive entropy:

$$u\left(\mathit{X}\right)=-{\displaystyle \sum _{c=1}^C}{\widehat{p}}_c\left(\mathit{X}\right)\log \left[{\widehat{p}}_c\left(\mathit{X}\right)+\epsilon \right].$$

(60)

The name CCE describes the calibrated-component construction; it does not assert that an arbitrary mixture of calibrated models is itself perfectly calibrated. Probability quality must be checked on untouched predictions. Class-balanced log-loss also changes the relative influence of the benchmark class priors and does not establish calibrated probabilities for the prevalence of faults in operation. The convexity result below concerns only the final weight problem with fixed probability inputs, not the complete neural pipeline.

Proposition 2

(Convexity of ensemble-weight estimation). For fixed calibrated probabilities, the objective in Equation (57) is convex on ${\Delta }_M$. If ${\lambda }_w>0$, the objective is strongly convex and possesses a unique global minimizer.

Proof. 

For observation i, define

$${\mathit{a}}_i={\left[{\tilde{p}}_{i1,y_i},\dots ,{\tilde{p}}_{iM,y_i}\right]}^{\top }\in {\mathbb{R}}_{+}^M.$$

(61)

The Hessian of the negative logarithmic term is

$${\nabla }_{\mathit{w}}^2\left[-\log \left({\mathit{a}}_i^{\top }\mathit{w}+\epsilon \right)\right]={\displaystyle \frac{{\mathit{a}}_i{\mathit{a}}_i^{\top }}{{\left({\mathit{a}}_i^{\top }\mathit{w}+\epsilon \right)}^2}}⪰0.$$

(62)

Thus, every sample term is convex, and a nonnegative class-weighted sum preserves convexity. The quadratic regularizer has Hessian ${\lambda }_w{\mathit{I}}_M$, so the complete objective is strongly convex when ${\lambda }_w>0$. Because ${\Delta }_M$ is nonempty, compact, and convex, a minimizer exists; strong convexity makes the minimizer unique. □

Corollary 1

(Dominance over the best calibrated base probability model). When ${\lambda }_w=0$, the optimized cross-fitted class-balanced NLL is no larger than the corresponding NLL of the best individually calibrated base learner:

$${\mathcal{L}}_{\mathrm{NLL}}\left({\mathit{w}}^{★}\right)\le \underset{m\in \{1,\dots ,M\}}{\min }{\mathcal{L}}_{\mathrm{NLL}}\left({\mathit{e}}_m\right),$$

(63)

where ${\mathit{e}}_m$ is the mth vertex of ${\Delta }_M$.

Proof. 

Every simplex vertex ${\mathit{e}}_m$ is feasible. Minimizing the unregularized objective over the complete simplex therefore cannot produce a value larger than the minimum over the subset of simplex vertices. □

The corollary concerns the cross-fitted NLL objective used to estimate the weights. It does not guarantee that the optimized ensemble must outperform every base classifier in outer-test accuracy or macro-F1, because these metrics are not the direct optimization objective.

3.8. Leakage-Controlled Training and Inference

Let $\mathsf{\Theta }$ denote the candidate set containing temporal-network settings, the retained RFE dimension, base-learner hyperparameters, calibration family, and ${\lambda }_w$. The selection criterion is lexicographic: configurations are ranked first by decreasing mean inner macro-F1, then by increasing mean inner NLL, and finally by increasing mean inner ECE.

Two fitting operators are used in Algorithms 1 and 2. The operator $FitPipeline({\mathcal{D}}_{\mathcal{A}},\theta ,E)$ fits sensor normalization, the temporal extractor for E epochs, statistical extraction, robust scaling, RF-RFE, and all base learners using only ${\mathcal{D}}_{\mathcal{A}}$. The operator CrossFitScores repeatedly invokes the complete fitting operator on cross-fitting training folds and predicts their held-out folds. Within every cross-fitting split, all data-dependent transformations and early-stopping choices must be fitted using only that split’s training indices. The held-out score rows are base-learner out-of-fold scores for a specified configuration, not independent outer-test estimates. Configuration selection remains inside the containing training partition.

Algorithm 1 Outer Evaluation of the Leakage-Controlled Dual-View Framework

Require:  Primary dataset $\mathcal{D}={\left\{({\mathit{X}}_i,y_i,g_i)\right\}}_{i=1}^N$; outer splitter $\mathcal{O}$; inner splitter $\mathcal{V}$; candidate set $\mathsf{\Theta }$; number of calibration cross-fitting folds $K_{\mathrm{cf}}$ Ensure:  Outer-test probability vectors $\left\{{\widehat{\mathit{p}}}_i\right\}$; predicted classes $\left\{{\widehat{y}}_i\right\}$; selected configurations; selected feature subsets; calibrators; ensemble weights; and uncertainty values   1: for each outer split $({\mathcal{I}}_{\mathrm{tr}},{\mathcal{I}}_{\mathrm{te}})\in \mathcal{O}$ do   2:       $({\theta }^{★},E^{★})\leftarrow \mathrm{S}\mathrm{ELECT}\mathrm{C}\mathrm{ONFIGURATION}({\mathcal{D}}_{{\mathcal{I}}_{\mathrm{tr}}},\mathcal{V},\mathsf{\Theta },K_{\mathrm{cf}})$   3:       ${\mathit{S}}_{\mathrm{tr}}^{\mathrm{OOF}}\leftarrow \mathrm{C}\mathrm{ROSS}\mathrm{F}\mathrm{IT}\mathrm{B}\mathrm{ASE}\mathrm{S}\mathrm{CORES}({\mathcal{D}}_{{\mathcal{I}}_{\mathrm{tr}}},{\theta }^{★},E^{★},K_{\mathrm{cf}})$   4:       for $m=1,\dots ,M$ do   5:             Fit the final calibrator ${\mathcal{C}}_m^{★}$ using column block m of ${\mathit{S}}_{\mathrm{tr}}^{\mathrm{OOF}}$ and the labels in ${\mathcal{I}}_{\mathrm{tr}}$   6:       end for   7:       Calibrate ${\mathit{S}}_{\mathrm{tr}}^{\mathrm{OOF}}$ to obtain ${\tilde{\mathit{P}}}_{\mathrm{tr}}^{\mathrm{OOF}}$   8:       Solve Equation (57) to obtain ${\mathit{w}}^{★}$   9:       ${\mathsf{\Pi }}^{★}\leftarrow \mathrm{F}\mathrm{IT}\mathrm{P}\mathrm{IPELINE}({\mathcal{D}}_{{\mathcal{I}}_{\mathrm{tr}}},{\theta }^{★},E^{★})$ 10:       ${\mathit{S}}_{\mathrm{te}}\leftarrow \mathrm{P}\mathrm{REDICT}\mathrm{B}\mathrm{ASE}\mathrm{S}\mathrm{CORES}({\mathsf{\Pi }}^{★},\{{\mathit{X}}_i:i\in {\mathcal{I}}_{\mathrm{te}}\})$ 11:       ${\widehat{\mathit{P}}}_{\mathrm{te}}\leftarrow \mathrm{C}\mathrm{ALIBRATE}\mathrm{A}\mathrm{ND}\mathrm{C}\mathrm{OMBINE}({\mathit{S}}_{\mathrm{te}},{\left\{{\mathcal{C}}_m^{★}\right\}}_{m=1}^M,{\mathit{w}}^{★})$ 12:       Store outer-test probabilities, predicted classes, predictive entropies, selected features, and fitted weights 13: end for 14: Aggregate all outer-test predictions and compute accuracy, macro-F1, balanced accuracy, MCC, NLL, Brier score, and ECE 15: Perform corrected paired comparisons using identical outer folds 16: Repeat the complete procedure using LOUO outer splits defined by $g_i$

The cross-fitting layer prevents calibration and weight fitting from relying only on base-learner in-sample scores. A calibrator fitted to these out-of-fold score rows and then applied to the same rows produces a meta-training representation, not independently held-out calibrated predictions. Fitting weights on that representation is permitted inside the containing training partition; model selection and performance assessment must still use untouched inner-validation and outer-test observations, respectively. To claim calibrated out-of-fold predictions at the meta level, a further cross-fit of the calibrator is needed. Its complexity must be included in the logged fitting budget.

In every invocation of CrossFitBaseScores, the early-stopping epoch is selected using a subset of that cross-fit’s training rows only, rather than reusing an epoch chosen with access to its held-out rows. The argument E supplies a training cap or a selection rule; it must not carry held-out-dependent information. A final fit on the containing training partition may use epochs selected from that partition’s permitted inner folds. Algorithms 1 and 2 specify these isolation requirements; the execution archive is required to establish that the reported results actually followed them.

Algorithm 2 Inner Configuration Selection with Cross-Fitted Calibration

1: function SelectConfiguration(${\mathcal{D}}_{\mathcal{A}},\mathcal{V},\mathsf{\Theta },K_{\mathrm{cf}}$)   2:       for each candidate configuration $\theta \in \mathsf{\Theta }$ do   3:             Initialize ${\mathcal{R}}_{\theta }\leftarrow \varnothing$ and ${\mathcal{E}}_{\theta }\leftarrow \varnothing$   4:             for each inner split $({\mathcal{I}}_a,{\mathcal{I}}_b)\in \mathcal{V}\left(\mathcal{A}\right)$ do   5:                   Split ${\mathcal{I}}_a$ into ${\mathcal{I}}_a^{\mathrm{fit}}$ and ${\mathcal{I}}_a^{\mathrm{es}}$ using stratification   6:                   Fit the temporal branch on ${\mathcal{I}}_a^{\mathrm{fit}}$ and select the early-stopping epoch $E_{\theta ,a}$ using only ${\mathcal{I}}_a^{\mathrm{es}}$   7:                   Append $E_{\theta ,a}$ to ${\mathcal{E}}_{\theta }$   8:                   ${\mathit{S}}_a^{\mathrm{OOF}}\leftarrow \mathrm{C}\mathrm{ROSS}\mathrm{F}\mathrm{IT}\mathrm{B}\mathrm{ASE}\mathrm{S}\mathrm{CORES}({\mathcal{D}}_{{\mathcal{I}}_a},\theta ,E_{\theta ,a},K_{\mathrm{cf}})$   9:                   for $m=1,\dots ,M$ do 10:                         Fit ${\mathcal{C}}_{m,\theta ,a}$ using cross-fitted scores from ${\mathcal{I}}_a$ 11:                   end for 12:                   Calibrate ${\mathit{S}}_a^{\mathrm{OOF}}$ and solve Equation (57) to obtain ${\mathit{w}}_{\theta ,a}$ 13:                   ${\mathsf{\Pi }}_{\theta ,a}\leftarrow \mathrm{F}\mathrm{IT}\mathrm{P}\mathrm{IPELINE}({\mathcal{D}}_{{\mathcal{I}}_a},\theta ,E_{\theta ,a})$ 14:                   ${\mathit{S}}_b\leftarrow \mathrm{P}\mathrm{REDICT}\mathrm{B}\mathrm{ASE}\mathrm{S}\mathrm{CORES}({\mathsf{\Pi }}_{\theta ,a},\{{\mathit{X}}_i:i\in {\mathcal{I}}_b\})$ 15:                   ${\widehat{\mathit{P}}}_b\leftarrow \mathrm{C}\mathrm{ALIBRATE}\mathrm{A}\mathrm{ND}\mathrm{C}\mathrm{OMBINE}({\mathit{S}}_b,{\left\{{\mathcal{C}}_{m,\theta ,a}\right\}}_{m=1}^M,{\mathit{w}}_{\theta ,a})$ 16:                   Record inner macro-F1, NLL, ECE, selected feature subset, and retained dimension in ${\mathcal{R}}_{\theta }$ 17:             end for 18:       end for 19:      Select ${\theta }^{★}$ by the lexicographic criterion: highest mean inner macro-F1, then lowest mean NLL, then lowest mean ECE 20:       $E^{★}\leftarrow median\left({\mathcal{E}}_{{\theta }^{★}}\right)$ 21:       return $({\theta }^{★},E^{★})$ 22: end function

For repeated stratified nested evaluation, the outer splitter consists of five repetitions of five-fold stratified cross-validation. For cross-unit evaluation, the outer splitter contains three LOUO partitions:

$${\mathcal{I}}_{\mathrm{te}}^{\left(u\right)}=\{i:g_i=u\},{\mathcal{I}}_{\mathrm{tr}}^{\left(u\right)}=\{i:g_i\ne u\},u\in \{1,2,3\}.$$

(64)

All inner selection and calibration procedures are repeated independently inside each LOUO training partition.

3.9. Computational Complexity

Let

$$R=\left|\mathcal{W}\right|=4,Q=9,F=32,h=32,p=716.$$

(65)

Multi-window statistical extraction requires $\mathcal{O}\left(NRTD\right)$ time and $\mathcal{O}\left(NRDQ\right)$ storage for the extracted descriptors.

The three convolutional branches require approximately $\mathcal{O}\left(NTDF{\sum }_{k\in \mathcal{K}}k\right)$ operations. The bidirectional recurrent path requires $\mathcal{O}\left(N{T}^{\prime }h(h+3F)\right)$, while the direct unidirectional path requires

$$\mathcal{O}\left(NTh(h+D)\right).$$

(66)

The two dense bottleneck layers contribute a lower-order cost relative to the recurrent operations.

Let $J_{\mathrm{RFE}}$ denote the number of RF-RFE fitting iterations. If one forest with $B_{\mathrm{tree}}$ trees is fitted at each iteration, the feature selection stage has an approximate cost of

$$\mathcal{O}\left(J_{\mathrm{RFE}}{B}_{\mathrm{tree}}N{p}_{\mathrm{try}}\log N\right),$$

(67)

where $p_{\mathrm{try}}$ is the number of candidate coordinates examined at a tree split. Removing a fixed fraction of the lowest-ranked variables per iteration reduces $J_{\mathrm{RFE}}$ relative to one-feature-at-a-time elimination.

Let ${\mathcal{C}}_{\mathrm{pipe}}$ denote the cost of fitting one complete candidate pipeline. With $J_{\mathrm{out}}$ outer folds, $J_{\mathrm{in}}$ inner folds, $\left|\mathsf{\Theta }\right|$ candidate configurations, and $K_{\mathrm{cf}}$ calibration cross-fitting folds, the dominant nested training cost is approximately

$$\mathcal{O}\left[J_{\mathrm{out}}\left(\left|\mathsf{\Theta }\right|J_{\mathrm{in}}(K_{\mathrm{cf}}+1)+K_{\mathrm{cf}}+1\right){\mathcal{C}}_{\mathrm{pipe}}\right].$$

(68)

This cost is incurred during offline model development. Once the fitted pipeline is fixed, inference requires one statistical extraction, one temporal forward pass, one selected-feature transformation, four base-model predictions, and a weighted probability combination.

Although the computational cost is manageable for the present sequence length, the current formulation requires the complete observation window and employs bidirectional recurrence. It is therefore designed for retrospective or near-offline diagnosis after acquisition of the one-second sequence. Strictly causal online early-warning performance is not claimed. Such an extension would require replacing the bidirectional and full-window components with causal counterparts and explicitly evaluating detection delay.

3.10. Contribution and Scope

The framework combines complementary representations, fold-local complexity control, and probability-level aggregation for this simulated small-sample task. The propositions make the optimization and information arguments explicit, but are not claimed as new statistical theorems. Information monotonicity does not prove finite-sample improvement; convexity does not prove superior test accuracy; and a nested protocol is good evaluation practice rather than a novel classifier. A stronger empirical contribution requires the matched baselines and component-wise evidence specified below.

4. Experiments

4.1. Dataset and Class Counts

XJTU-REF is a component-model-based, open-cycle LRE simulation benchmark [6]. The benchmark contains 19 measured or simulator-derived variables, sampled at 500 Hz, with 501 time points per sequence. The interval between the first and last samples is $(501-1)/500=1$ s. One sequence, not each time point or each overlapping statistical window, is the classification sample.

Table 3 summarizes the class distribution. The dataset contains 1109 sequences, of which 1106 belong to the nine fault classes. The largest-to-smallest fault-class ratio is $147/114\approx 1.29$, indicating modest imbalance within the primary task. By contrast, the healthy class contains only three sequences, or approximately 0.27% of the dataset. The primary evaluation therefore addresses closed-set classification among the nine faults rather than healthy/fault detection.

Table 3. Sequence inventory of XJTU-REF; primary fault subset: 1106 sequences.

Fault Code	Fault Type	Samples
Ofu	Oxygen Secondary Line Clogging	114
Ojet	Oxidizer Injector Clogging	120
Op	Oxidizer Pump Efficiency Loss	120
Ov	Oxidizer Main Valve Fault	120
Rp	Fuel Pump Efficiency Loss	120
Rv	Fuel Main Valve Fault	120
Rxl	Gas Generator Fuel Leakage	147
ss	Combustion Chamber Ablation	120
Txl	Turbine Gas Leakage	125
H	Normal (Healthy)	3
Total		1109

The 19 sensor measurements include: turbine power ($N_t$), turbine heat value ($R{T}_t$), turbopump speed (n), oxidizer flow rate ($Q_o$), gas generator oxidizer flow ($Q_{op}$), oxidizer pump power ($N_{po}$), fuel pump pressure ($P_f$), oxidizer pump pressure ($P_o$), fuel flow rate ($Q_f$), gas generator fuel flow ($Q_{fp}$), fuel pump power ($N_{pf}$), gas generator mixture ratio ($K_p$), gas generator pressure ($P_p$), gas generator outlet flow ($Q_p$), gas generator heat value ($R{T}_p$), combustion chamber mixture ratio ($K_c$), combustion chamber pressure ($P_c$), combustion chamber outlet flow ($Q_c$), and combustion chamber heat value ($R{T}_c$).

4.2. Model Configuration

Table 4 summarizes the structural settings and feature-selection candidates. The statistical branch produces 684 features, while the temporal branch compresses the combined 96-dimensional recurrent representation into a 32-dimensional bottleneck. Thus, the feature vector entering RF-RFE has 716 coordinates; the intermediate 96-dimensional representation is not the temporal output used for fusion. The RF-RFE candidate set defines admissible retained dimensions rather than a single feature count shared by all fitted models.

Table 4. Model architecture, feature-selection candidates, and validation settings. Retained feature dimensions are candidates for inner-loop selection, not observed selection frequencies.

Component	Configuration
Input	501 time points and 19 signals per sequence, sampled at 500 Hz.
Statistical representation	Windows $(0,1)$, $(0.5,1)$, $(0.7,1)$, and $(0.85,1)$; nine statistics per sensor and window; $19\times 4\times 9=684$ features.
Multi-scale convolution	Kernel sizes $\{3,7,15\}$; 32 filters per branch; 96 concatenated channels; max-pooling width and stride of two.
Channel attention	Transformation $96\to 24\to 96$ over the convolutional latent channels.
Recurrent representation	BiLSTM with 32 hidden units per direction, producing a 64-dimensional pooled vector; a parallel direct LSTM with 32 hidden units; 96-dimensional concatenated output.
Bottleneck and fusion	Dense projection $96\to 64\to 32$ with dropout 0.3; fused dimension $684+32=716$.
Feature selection	RF-RFE retained-dimension candidates $\{15,30,45,60,90,120\}$, with admissible values restricted to the available input dimension.
Decision stage	RF, RBF-SVM, GB, and distance-weighted KNN; probability calibration followed by regularized simplex-constrained weighting. Uniform-weight control: 0.25 per learner.
Model selection	Inner macro-F1, with NLL and then ECE as tie-breakers; data-dependent operations confined to the corresponding training partition.
Validation	Five repetitions of five-fold outer stratified validation with four-fold inner selection; three separate LOUO evaluations.

The convolutional attention mechanism reweights learned channels rather than assigning importance directly to the 19 physical signals. Likewise, the retained feature dimension describes model complexity, not the stability of feature identities across folds. Exact numerical replication remains limited by the lack of a complete execution archive documenting trial histories, training seeds, fold-specific settings, and the software/hardware environment.

4.3. Validation Protocol

The primary evaluation uses five repetitions of five-fold outer stratified cross-validation, with four-fold inner model selection. Each sequence is one classification sample; its time points and overlapping statistical windows are not treated as independent observations. Sensor normalization, temporal-network training, early stopping, robust scaling, RF-RFE, classifier tuning, calibration, and ensemble-weight estimation are restricted to the corresponding training partition. Inner-validation observations are used for configuration scoring, not early stopping.

Sample-stratified validation estimates performance on additional sequences from the represented simulated-unit mixture. LOUO evaluation instead holds out one complete unit and develops the model on the other two, excluding the target unit from every fitting and selection step. Sample-level inner folds are distinct from unit-level validation because only two development units are available in each LOUO experiment. The individual unit results are reported separately to characterize variation across the three holdout tasks.

A supplementary ten-condition 80/20 hold-out experiment is used for classwise error analysis in Section 4.12. Its 222 test sequences include one healthy sequence. Because its label space, fitted configuration, and validation protocol differ from the primary nine-fault evaluation, its results are not pooled with the repeated nested-CV results or used in the primary paired comparisons.

4.4. Evaluated Comparators

Table 5 defines the six configurations used to assess the representation, selection, and decision stages. The comparisons use the same nine-fault subset, sensor inputs, sequence length, outer-test partitions, and evaluation metrics. All data-dependent transformations are fitted within the corresponding training partition. Model-specific representations differ by design: statistical RF uses window descriptors, whereas the temporal-only model uses the convolutional–recurrent network and its classification head.

Table 5. Definitions of the evaluated configurations. Their classification and probabilistic results are presented in Table 6 and Table 7.

Configuration	Representation and Decision Rule
RF + final-20% mean	One mean feature per sensor from the final 20% of each sequence, followed by RF.
RF + multi-scale statistics	The 684-dimensional statistical view followed by RF.
Temporal extractor only	The multi-scale convolutional–recurrent network with its auxiliary classification head.
Naive concatenation + RF	The 716-dimensional fused representation followed by RF without RFE.
Fusion + nested RFE + RF	The selected fused representation followed by a single RF.
Full CCE	The selected fused representation with calibrated RF/SVM/GB/KNN probabilities and optimized convex weights.

The temporal-only comparator evaluates the specified feature-extraction network; it is not a stand-alone vanilla CNN, LSTM, or BiLSTM. Accordingly, the comparisons establish differences among the configurations in Table 5, rather than a ranking against unevaluated time-series models. Shared partitions and training-only fitting control test access but do not establish equal computational or search budgets.

4.5. Multiclass Evaluation Metrics

Let $\mathit{M}\in {\mathbb{N}}^{C\times C}$ be the confusion matrix, where $M_{ab}$ is the number of samples with true class a predicted as class b. Define

$$\begin{array}{cc}\hfill Accuracy& ={\displaystyle \frac{{\sum }_{c=1}^C{M}_{cc}}{{\sum }_{a=1}^C{\sum }_{b=1}^C{M}_{ab}}},\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill {Precision}_c& ={\displaystyle \frac{M_{cc}}{{\sum }_{a=1}^C{M}_{ac}}},\hfill \end{array}$$

(70)

$$\begin{array}{cc}\hfill {Recall}_c& ={\displaystyle \frac{M_{cc}}{{\sum }_{b=1}^C{M}_{cb}}},\hfill \end{array}$$

(71)

$$\begin{array}{cc}\hfill {F1}_c& ={\displaystyle \frac{2{Precision}_c{Recall}_c}{{Precision}_c+{Recall}_c}},\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill MacroF1& ={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{F1}_c,\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill BalancedAccuracy& ={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{Recall}_c.\hfill \end{array}$$

(74)

For the multiclass MCC, let $t_c={\sum }_b{M}_{cb}$, $p_c={\sum }_a{M}_{ac}$, $s={\sum }_{a,b}{M}_{ab}$, and $r={\sum }_c{M}_{cc}$. Then

$$MCC={\displaystyle \frac{rs-{\sum }_{c=1}^C{p}_c{t}_c}{\sqrt{\left(s^2-{\sum }_{c=1}^C{p}_c^2\right)\left(s^2-{\sum }_{c=1}^C{t}_c^2\right)}}}.$$

(75)

For probabilistic predictions ${\widehat{p}}_{ic}$,

$$\begin{array}{cc}\hfill NLL& =-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\log ({\widehat{p}}_{i,y_i}+\epsilon ),\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill Brier& ={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{c=1}^C}{\left({\widehat{p}}_{ic}-\mathbb{I}[y_i=c]\right)}^2.\hfill \end{array}$$

(77)

Partition predictions into confidence bins ${\left\{{\mathcal{B}}_b\right\}}_{b=1}^{B_{\mathrm{cal}}}$. Expected calibration error is

$$ECE={\displaystyle \sum _{b=1}^{B_{\mathrm{cal}}}}{\displaystyle \frac{|{\mathcal{B}}_b|}{N}}\left|acc\left({\mathcal{B}}_b\right)-conf\left({\mathcal{B}}_b\right)\right|.$$

(78)

Accuracy measures overall correctness, macro-F1 and balanced accuracy weight classes equally, MCC summarizes the full confusion structure, and NLL/Brier/ECE assess probability quality.

Accuracy weights individual sequences, whereas macro-F1 and balanced accuracy give each class equal weight. MCC accounts for the multiclass confusion structure but remains sensitive to class prevalence. These metrics are reported together because similar aggregate scores do not imply comparable recall for every fault. Class weights, where used, are computed from the current training partition only. Undefined precision or F1 is assigned zero, and class support is retained in classwise reports.

4.6. Uncertainty and Statistical Comparisons

Repeated-validation results are summarized by the mean and standard deviation across 25 outer-test folds. The standard deviation describes resampling dispersion rather than uncertainty across independent engines. Paired method comparisons use the corrected resampled test to approximate the dependence induced by overlapping training partitions. For the CCE-minus-comparator macro-F1 difference $d_j$ in outer fold j, with $J=25$,

$$\begin{array}{cccc}\hfill \overline{d}& ={\displaystyle \frac{1}{J}}{\displaystyle \sum _{j=1}^J}{d}_j,\hfill & \hfill s_d^2& ={\displaystyle \frac{1}{J-1}}{\displaystyle \sum _{j=1}^J}{(d_j-\overline{d})}^2,\hfill \end{array}$$

(79)

$$\begin{array}{cccc}\hfill {SE}_{\mathrm{corr}}& =s_d\sqrt{{\displaystyle \frac{1}{J}}+{\displaystyle \frac{n_{\mathrm{te}}}{n_{\mathrm{tr}}}}},\hfill & \hfill t_{\mathrm{corr}}& ={\displaystyle \frac{\overline{d}}{{SE}_{\mathrm{corr}}}},\nu =J-1.\hfill \end{array}$$

(80)

The five-fold approximation uses $n_{\mathrm{te}}/n_{\mathrm{tr}}\approx 1/4$. A corrected 95% interval for the paired difference is $\overline{d}\pm t_{0.975,\nu }{SE}_{\mathrm{corr}}$, and the standardized paired effect is $d_z=\overline{d}/s_d$.

Holm adjustment controls multiplicity across the five baseline macro-F1 comparisons [46]. A comparison is treated as statistically significant when its adjusted $p<0.05$ and its corrected interval excludes zero. The intervals are pointwise, not simultaneous family-wise intervals. Component ablations are analyzed descriptively; their marginal means and standard deviations do not determine the variance of paired differences. LOUO results are also descriptive, with an equal-unit mean and sample standard deviation across the three unit scores. The corrected test does not eliminate all within-unit dependence or establish population-level cross-engine significance.

For the supplementary fixed-split classwise analysis, recall is calculated as the true-positive count divided by class support, and a Wilson 95% interval summarizes its conditional binomial uncertainty. These intervals do not incorporate training variability, model-selection uncertainty, or unit clustering. Repeated held-out predictions for the same sequence are not treated as new independent observations.

4.7. Repeated Nested Cross-Validation

Table 6 and Table 7 summarize classification and probabilistic performance, respectively, on the nine-fault classification task. Evaluation comprises five repetitions of five-fold nested cross-validation, and each entry is the mean and standard deviation across the 25 outer-test folds. These summaries describe fold-level performance rather than metrics calculated from pooled predictions. The comparisons assess the statistical and temporal views, their direct and selected fusion, and the final calibrated convex ensemble (CCE).

Table 6. Classification performance on the nine-fault task under repeated nested cross-validation. Values are mean ± standard deviation across 25 outer-test folds; higher values are better. Bold values identify the best mean for each metric among the evaluated methods.

Method	Accuracy (%)	Macro-F1 (%)	Balanced Acc. (%)	MCC
RF + final-20% mean	87.71 ± 1.42	87.23 ± 1.50	87.39 ± 1.46	0.861 ± 0.017
RF + multi-scale statistical view	88.31 ± 1.31	87.96 ± 1.38	88.05 ± 1.34	0.869 ± 0.016
Temporal extractor only	86.72 ± 1.63	86.20 ± 1.69	86.38 ± 1.66	0.850 ± 0.020
Naive concatenation + RF	87.04 ± 1.55	86.54 ± 1.60	86.73 ± 1.58	0.854 ± 0.019
Fusion + nested RFE + single RF	89.08 ± 1.18	88.73 ± 1.23	88.88 ± 1.20	0.879 ± 0.014
Proposed CCE	90.16 ± 1.04	89.84 ± 1.08	89.96 ± 1.05	0.891 ± 0.012

Table 7. Probabilistic performance on the nine-fault task under repeated nested cross-validation. Values are mean ± standard deviation across 25 outer-test folds; lower values are better. Bold values identify the lowest mean for each metric among the evaluated methods.

Method	NLL	Brier	ECE (%)
RF + final-20% mean	0.486 ± 0.052	0.159 ± 0.018	7.31 ± 1.06
RF + multi-scale statistical view	0.451 ± 0.049	0.149 ± 0.016	6.48 ± 0.91
Temporal extractor only	0.517 ± 0.060	0.170 ± 0.020	7.84 ± 1.14
Naive concatenation + RF	0.501 ± 0.057	0.165 ± 0.019	7.52 ± 1.09
Fusion + nested RFE + single RF	0.397 ± 0.043	0.133 ± 0.014	5.21 ± 0.78
Proposed CCE	0.342 ± 0.037	0.118 ± 0.012	3.76 ± 0.65

As shown in Table 6, CCE achieves the highest mean values for all four classification metrics: accuracy of (90.16 ± 1.04)%, macro-F1 of (89.84 ± 1.08)%, balanced accuracy of (89.96 ± 1.05)%, and MCC of 0.891 ± 0.012. Relative to RF using the final-20% mean, the corresponding improvements are 2.45, 2.61, and 2.57 percentage points for accuracy, macro-F1, and balanced accuracy, and 0.030 for MCC. The simultaneous increases in accuracy and the two equally class-weighted metrics show that the advantage persists under different class-aggregation rules. They do not, however, imply that every fault class improves by the same amount.

Figure 2 complements Table 6 by showing the dispersion of macro-F1 across the outer folds. CCE combines the highest mean macro-F1 with the lowest standard deviation among the six configurations. Compared with the final-window RF baseline, the macro-F1 standard deviation decreases from 1.50 to 1.08 percentage points; the corresponding accuracy standard deviation decreases from 1.42 to 1.04 percentage points. This pattern indicates lower score dispersion under the specified resampling procedure. Because outer folds share training observations, the error bars are not confidence intervals, and their overlap is not used to judge statistical significance.

Figure 2. Mean outer-fold macro-F1 for the six evaluated configurations. Error bars indicate ±1 standard deviation across the 25 outer-test folds, not confidence intervals. Statistical comparisons use paired fold differences rather than error-bar overlap.

The probabilistic results in Table 7 show a consistent advantage for CCE. Its NLL, Brier score, and ECE are 0.342 ± 0.037, 0.118 ± 0.012, and (3.76 ± 0.65)%, respectively. Compared with the final-window RF baseline, NLL decreases by 0.144 and the Brier score by 0.041, corresponding to relative reductions of 29.63% and 25.79%. ECE decreases from 7.31% to 3.76%, an absolute reduction of 3.55 percentage points and a relative reduction of 48.56%. The lower NLL and Brier score indicate improvements in probability-based predictive performance, while the lower ECE indicates closer agreement between confidence and empirical correctness under the adopted calibration measure. Together with Table 6, these results show that the benefit is not restricted to the final hard-label decision.

4.8. Effects of Representation, Selection, and Decision Fusion

The comparisons in Table 6 reveal that the choice of representation and the way it is combined are both important. Replacing the final-window mean with the multi-scale statistical view increases RF accuracy from 87.71% to 88.31% and macro-F1 from 87.23% to 87.96%, gains of 0.60 and 0.73 percentage points, respectively. This comparison evaluates the richer statistical representation as a whole: it changes both temporal coverage and the descriptor set, rather than isolating the effect of the number of windows. The temporal-only model achieves 86.20% macro-F1, 1.76 percentage points below the statistical-view RF. Thus, the statistical configuration is the stronger stand-alone comparator in this experiment, but this ordering does not determine whether temporal information is useful within a fused model.

Direct concatenation does not realize the potential benefit of the two views. In Table 6, naive fusion reduces macro-F1 from 87.96% for statistical-view RF to 86.54%, a loss of 1.42 percentage points. Table 7 shows a corresponding increase in NLL from 0.451 to 0.501 and in ECE from 6.48% to 7.52%. By contrast, fusion followed by nested RFE and a single RF achieves 88.73% macro-F1, recovering 2.19 percentage points relative to naive concatenation, while NLL decreases to 0.397. The macro-F1 standard deviation also decreases from 1.60 to 1.23 percentage points. This pattern is consistent with the role of feature selection in limiting the influence of redundant or weakly informative coordinates in the 716-dimensional fused representation. It supports selected fusion rather than concatenation alone as the useful representation strategy for the evaluated configurations; it does not measure the stability of the selected feature identities.

The decision-stage comparison provides a further improvement beyond selected-feature RF. As shown in Table 6 and Table 7, CCE increases accuracy from 89.08% to 90.16% and macro-F1 from 88.73% to 89.84%, while reducing NLL from 0.397 to 0.342, the Brier score from 0.133 to 0.118, and ECE from 5.21% to 3.76%. This comparison combines probability calibration and heterogeneous aggregation and therefore characterizes the final decision stage jointly.

4.9. Paired Statistical Comparisons

Table 8 presents paired comparisons of CCE with the five alternative configurations using outer-fold macro-F1. Positive differences favor CCE. The corrected resampled test in Equation (80) approximately accounts for dependence arising from overlapping training partitions, and Holm adjustment is applied across the five comparisons. The standardized paired effect $d_z=\overline{d}/s_d$ expresses the mean improvement relative to the standard deviation of its paired fold differences.

Table 8. Paired macro-F1 comparisons between CCE and the five alternative configurations. $\Delta$ denotes CCE minus the comparator, in percentage points (pp). Confidence intervals use the corrected resampled variance; p-values are Holm-adjusted across the five comparisons. The paired standardized effect is $d_z=\overline{d}/s_d$.

Comparator	$\mathbf{\Delta }$ Macro-F1 (pp)	Corrected 95% CI	Holm-Adjusted p	${\mathit{d}}_{\mathit{z}}$
Single-window mean RF	+2.61	[1.14, 4.08]	0.0036	1.98
Multi-scale statistical RF	+1.88	[0.66, 3.10]	0.0082	1.71
Temporal-only model	+3.64	[1.92, 5.36]	0.0011	2.35
Naive fusion + RF	+3.30	[1.64, 4.96]	0.0016	2.21
Fusion + nested RFE + RF	+1.11	[0.20, 2.02]	0.0191	1.35

All five comparisons in Table 8 have positive corrected 95% confidence intervals and Holm-adjusted p-values below 0.05. Against final-window RF, the mean macro-F1 gain is 2.61 percentage points, with a corrected interval of $[1.14,4.08]$, $p=0.0036$, and $d_z=1.98$. The gain over statistical-view RF is smaller but remains positive: 1.88 percentage points, a corrected interval of $[0.66,3.10]$, $p=0.0082$, and $d_z=1.71$. These comparisons support the full framework over the two statistical-only configurations within the specified repeated-validation design.

The largest mean differences in Table 8 occur against the temporal-only model and naive fusion. Their macro-F1 gains are 3.64 and 3.30 percentage points, with corrected intervals of $[1.92,5.36]$ and $[1.64,4.96]$, adjusted p-values of 0.0011 and 0.0016, and paired effects of 2.35 and 2.21, respectively. The smallest gain is against fusion with nested RFE and a single RF: 1.11 percentage points, with a corrected interval of $[0.20,2.02]$, $p=0.0191$, and $d_z=1.35$. This final contrast is important because it evaluates the combined decision-stage change beyond an already selected fused representation, rather than comparing the full model only with a low-information baseline.

The intervals in Table 8 are corrected pointwise intervals for the individual contrasts, not simultaneous family-wise confidence intervals; multiplicity control is reflected in the Holm-adjusted p-values. The conclusions concern paired method differences under the sample-level resampling design. The correction does not turn 25 overlapping folds into 25 independent engines, and these results do not establish population-level cross-engine or simulation-to-real significance.

4.10. Cross-Unit Generalization

Table 9 and Figure 3 evaluate transfer to an engine unit excluded from model development. In each leave-one-unit-out (LOUO) experiment, two complete simulated units are used for development and the third is held out for evaluation. This protocol differs from sample-stratified validation, in which sequences from the same units can occur in both training and test partitions.

Table 9. Leave-one-unit-out performance. Each model is developed on two simulated engine units and evaluated on the remaining unit. The final row gives the unweighted mean and sample standard deviation across the three held-out-unit scores.

Held-Out Unit	Accuracy (%)	Macro-F1 (%)	MCC	ECE (%)
Unit 1	87.36	86.91	0.858	5.62
Unit 2	84.97	84.31	0.831	6.94
Unit 3	86.45	85.88	0.846	6.21
Mean ± standard deviation	86.26 ± 1.21	85.70 ± 1.31	0.845 ± 0.014	6.26 ± 0.66

Figure 3. Accuracy and macro-F1 for each held-out simulated engine unit. Each point represents one unit-level test score; dispersion across units is summarized in Table 9.

Table 9 gives an unweighted mean accuracy of (86.26 ± 1.21)%, macro-F1 of (85.70 ± 1.31)%, MCC of 0.845 ± 0.014, and ECE of (6.26 ± 0.66)% across the three held-out units. These standard deviations describe variation between the three unit-level scores. The means give each engine unit equal weight and are therefore distinct from metrics calculated by pooling all held-out sequences.

The individual results in Table 9 reveal variation that would be obscured by the three-unit mean alone. Unit 1 achieves the highest accuracy and macro-F1, 87.36% and 86.91%, whereas Unit 2 achieves the lowest, 84.97% and 84.31%. Unit 3 lies between them at 86.45% and 85.88%. Thus, the accuracy and macro-F1 ranges are 2.39 and 2.60 percentage points, respectively. The same ordering is present in MCC (0.858, 0.846, and 0.831 for Units 1, 3, and 2), while Unit 2 also has the highest ECE, 6.94%. Figure 3 makes the common unit-level pattern in accuracy and macro-F1 explicit.

Compared with the sample-level CCE results in Table 6, mean LOUO accuracy decreases by 3.90 percentage points and macro-F1 by 4.14 percentage points; MCC falls from 0.891 to 0.845. Meanwhile, ECE increases from 3.76% in Table 7 to 6.26% in Table 9, a deterioration of 2.50 percentage points. The cross-unit evaluation therefore shows lower classification performance and a larger confidence–accuracy discrepancy than the sample-level evaluation. The training-set size and aggregation rule also differ between the two protocols, so these numerical gaps cannot be interpreted as isolated estimates of unit-distribution shift.

The unit-level results do not identify a physical cause for Unit 2’s lower scores. Differences in fault severity, operating levels, or within-class variability are possible explanations, but the aggregate metrics do not distinguish among them or establish increased Ojet–Ov confusion in that unit. The LOUO results demonstrate performance on the three simulated holdout tasks, not transfer to a population of real engines. In particular, they do not cover the instrumentation, aging, and operating variability associated with hot-fire or flight data.

4.11. Component Ablations and Calibration

Table 10 compares the full framework with configurations that remove or replace individual representation, selection, calibration, and aggregation components. The column $\Delta$ F1 is the ablated configuration’s mean macro-F1 minus that of the full model, so a negative value indicates lower performance. The analysis distinguishes changes in classification metrics from changes in probability quality.

Table 10. Component ablations and calibration analysis using identical outer-test folds. Values are mean ± standard deviation across 25 outer folds. Δ F1 denotes the ablated configuration minus the full framework, in percentage points (pp). Higher accuracy and macro-F1 and lower NLL and ECE indicate better performance. Bold text in the table indicates the best value.

Configuration	Acc. (%)	Macro-F1 (%)	Δ F1 (pp)	NLL	ECE (%)
Full proposed framework	90.16 ± 1.04	89.84 ± 1.08	—	0.342 ± 0.037	3.76 ± 0.65
Without multi-scale statistical view	88.43 ± 1.36	88.02 ± 1.41	−1.82	0.439 ± 0.049	5.91 ± 0.89
Without short windows $(0.70,1)$ and $(0.85,1)$	89.23 ± 1.18	88.88 ± 1.22	−0.96	0.392 ± 0.044	4.75 ± 0.77
Without temporal view	88.29 ± 1.32	87.91 ± 1.37	−1.93	0.448 ± 0.050	6.14 ± 0.94
Without latent-channel attention	89.42 ± 1.13	89.03 ± 1.18	−0.81	0.374 ± 0.041	4.42 ± 0.72
Without bidirectional recurrence	89.07 ± 1.21	88.62 ± 1.27	−1.22	0.401 ± 0.045	4.93 ± 0.81
Without nested RFE	88.71 ± 1.35	88.22 ± 1.41	−1.62	0.423 ± 0.048	5.36 ± 0.86
Without probability calibration	90.08 ± 1.06	89.76 ± 1.10	−0.08	0.421 ± 0.046	7.18 ± 1.01
Uniform ensemble weights	89.70 ± 1.10	89.32 ± 1.14	−0.52	0.365 ± 0.039	4.09 ± 0.68
Single best base learner	89.08 ± 1.18	88.73 ± 1.23	−1.11	0.397 ± 0.043	5.21 ± 0.78

The two representation branches have the largest branch-removal losses in Table 10. Removing the temporal view reduces macro-F1 from 89.84% to 87.91%, a decrease of 1.93 percentage points; removing the statistical view reduces it to 88.02%, a decrease of 1.82 percentage points. NLL also increases from 0.342 to 0.448 and 0.439, respectively. These comparisons are consistent with complementary contributions from the two views within the full framework. The 0.11-percentage-point difference between their macro-F1 losses does not establish that either view is statistically more important. The stand-alone temporal model in Table 6 and the branch-removal configuration in Table 10 are different fitted configurations and should not be treated as an identical single-view baseline.

The short-window ablation in Table 10 provides a more targeted assessment of temporal coverage. Removing the $(0.70,1)$ and $(0.85,1)$ windows decreases macro-F1 by 0.96 percentage points, to 88.88%, while NLL increases from 0.342 to 0.392 and ECE from 3.76% to 4.75%. This loss is smaller than that associated with removing the entire statistical branch. The comparison is consistent with an incremental contribution from shorter terminal windows beyond the longer-window summaries, rather than evidence that the short windows alone explain the benefit of statistical features.

Within the temporal representation, removing latent-channel attention reduces macro-F1 by 0.81 percentage points, and removing bidirectional recurrence reduces it by 1.22 percentage points (Table 10). Their NLL values increase to 0.374 and 0.401, respectively. Both configurations perform below the full model, consistent with a contribution from channel reweighting and temporal context. These comparisons concern the network representation: attention acts on convolutional latent channels and does not directly quantify the physical importance of individual sensors.

Feature selection has a substantial descriptive effect in Table 10. Without nested RFE, macro-F1 falls to 88.22%, a decrease of 1.62 percentage points, while NLL rises to 0.423 and ECE to 5.36%. Together with the naive-fusion comparison in Table 6 and Table 7, this pattern supports controlling the complexity of the fused representation before downstream classification. The 1.62-point ablation loss and the 2.19-point gain from adding RFE to naive-fusion RF are different contrasts: the former concerns the ensemble pipeline, whereas the latter concerns a single-RF configuration. Neither value by itself measures whether the same features are selected across folds.

Probability calibration primarily affects probability quality rather than mean hard-label performance. In Table 10, removing calibration changes accuracy from 90.16% to 90.08% and macro-F1 from 89.84% to 89.76%, both differences of only 0.08 percentage points. In contrast, calibration reduces NLL from 0.421 to 0.342, an absolute reduction of 0.079 and a relative reduction of 18.76% using the uncalibrated configuration as the denominator. ECE decreases from 7.18% to 3.76%, a reduction of 3.42 percentage points, or 47.63% relative to the uncalibrated value. The result supports calibration as a probability-reliability component; the small differences in accuracy and macro-F1 do not establish either a significant classification gain or statistical equivalence.

The aggregation comparisons in Table 10 separate the full ensemble from uniform voting and a single-learner decision. Uniform weights yield 89.32% macro-F1, 0.52 percentage points below CCE, with NLL of 0.365 and ECE of 4.09%. The single best base learner yields 88.73% macro-F1, 1.11 percentage points below CCE, with NLL of 0.397 and ECE of 5.21%. Uniform voting therefore exceeds the single-learner configuration by 0.59 percentage points in macro-F1, while optimized weighting provides a further 0.52-point increase in the mean. This ordering distinguishes the observed benefit of aggregation from the additional benefit of weight optimization, without attributing the gain to measured classifier-error diversity, which is not quantified by these metrics.

Overall, Table 10 associates the largest mean classification losses with removal of a representation branch or RFE, whereas calibration has its clearest effect on probability quality. These are conditional comparisons of fitted configurations, not an additive decomposition of total performance. Removing a branch also changes the available feature dimension, and architectural replacements can change model capacity. The table provides mean and standard deviation summaries, not component-wise paired confidence intervals or adjusted p-values. Accordingly, the component comparisons remain descriptive; formal significance conclusions are confined to the contrasts in Table 8.

4.12. Supplementary Hold-Out Classwise Analysis

Table 11 presents the classification report for a weighted ensemble evaluated using a supplementary ten-condition 80/20 hold-out split. The test set contains 222 sequences, including one healthy sequence, and 200 are classified correctly, giving an accuracy of 90.09%. This analysis characterizes class-specific errors under that fixed split; it is separate from the nine-fault nested evaluation in Table 6. The two experiments differ in label space, fitted configuration, and resampling, so their classwise and overall metrics are not interchangeable.

Table 11. Classwise performance of the weighted ensemble on the supplementary ten-condition hold-out test set. Precision and F1 are rounded to two decimals; recall and its Wilson 95% interval are calculated from the true-positive count and class support. These are fixed-split results, not classwise nested-CV estimates.

Class	Precision	F1	TP/Support	Recall (%)	95% CI (%)
Ofu	0.96	0.98	23/23	100.00	[85.69, 100.00]
Ojet	0.62	0.62	15/24	62.50	[42.71, 78.84]
Op	0.92	0.96	24/24	100.00	[86.20, 100.00]
Ov	0.60	0.61	15/24	62.50	[42.71, 78.84]
Rp	1.00	0.96	22/24	91.67	[74.15, 97.68]
Rv	1.00	1.00	24/24	100.00	[86.20, 100.00]
Rxl	1.00	1.00	29/29	100.00	[88.30, 100.00]
ss	1.00	1.00	24/24	100.00	[86.20, 100.00]
Txl	1.00	0.98	24/25	96.00	[80.46, 99.29]
Healthy	0.00	0.00	0/1	0.00	[0.00, 79.35]

The results in Table 11 are strongly class-dependent. Seven fault classes have F1 scores between 0.96 and 1.00, but their recall values are not uniformly within this range. In particular, Rp has precision 1.00 and F1 0.96, whereas its recall is $22/24=91.67\%$. Ofu, Op, Rv, Rxl, and ss have no observed false negatives in this split. Their finite supports nevertheless yield Wilson lower bounds below 100%, emphasizing that perfect test-set recall does not establish error-free recognition in the underlying population.

Ojet and Ov each have 15 correct classifications among 24 test sequences, corresponding to recall of 62.50% and a Wilson interval of $[42.71,78.84]\%$. Their precision values of 0.62 and 0.60 accompany F1 scores of 0.62 and 0.61, respectively. Together, the two classes contribute 18 of the 22 errors, or 81.82%; the remaining seven fault classes achieve 170 correct classifications among 173 sequences, or 98.27%. This error concentration identifies the two oxidizer-path faults as the principal weakness of the ten-condition experiment. It does not establish that all errors are exchanged between Ojet and Ov, because classwise recall does not specify the predicted destinations of misclassified sequences.

The healthy row contains one misclassified test sequence and therefore does not provide a reliable estimate of healthy-state recognition. Its recall interval, $[0,79.35]\%$, illustrates the lack of precision at this sample size. The healthy observation is retained to describe the full hold-out test set, not to validate normal/fault detection. Neither this row nor the nine-fault classification results establish healthy-state specificity or an operational false-alarm rate.

5. Discussion

5.1. Practical Scope and Healthy-State Recognition

The results in Table 6 demonstrate the classification performance of the proposed framework within a predefined set of nine simulated faults. This task differs from detecting a departure from nominal operation. As shown in Table 3, only three healthy sequences are available, which is insufficient to characterize nominal variability across engine units, operating conditions, and instrumentation. Excluding the healthy class therefore defines the scope of the evaluation rather than resolving the scarcity of nominal data. The present results support fault classification, but do not establish healthy-state specificity or an operational false-alarm rate.

The probability-quality improvements in Table 10 also apply within this fault-only task. Because the classifier distributes probability among the nine known classes, a healthy sequence, an unseen fault, or a compound fault can receive a known-fault label with high confidence. Calibration improves the interpretation of confidence within the evaluated distribution; it does not provide a mechanism for recognizing conditions outside the modeled class set. An extension toward health monitoring could place a separately validated one-class or residual-based anomaly detector before the fault classifier. Such a system would require broader nominal data and end-to-end assessment of false alarms, missed detections, fault classification, and rejection of unfamiliar conditions.

The temporal formulation imposes a further application constraint. The BiLSTM uses both temporal directions within a complete one-second record, so its predictions are available only after that record has been acquired. The framework is consequently evaluated as a full-window classifier rather than a causal early-warning system. Online use would require a causal representation and evaluation of the trade-off between detection delay and classification performance.

5.2. Ojet–Ov Classification Difficulty and Sensor Interpretation

Table 11 identifies oxidizer injector clogging (Ojet) and oxidizer main valve fault (Ov) as the principal weaknesses of the supplementary ten-condition hold-out experiment. Both classes have 15 correct predictions among 24 sequences, corresponding to a recall of 62.50%, while their F1 scores are 0.62 and 0.61. In contrast, the other seven fault classes achieve F1 scores of 0.96–1.00 in that split. The common recall indicates comparable false-negative proportions, although it does not imply that all errors are exchanged between Ojet and Ov. These classwise observations characterize the supplementary experiment and are distinct from the aggregate nine-fault nested results.

A plausible explanation for this difficulty is the limited observability of fault location through shared oxidizer-side responses. Both faults modify the oxidizer flow path and can produce overlapping changes in oxidizer flow $Q_o$, oxidizer pump pressure $P_o$, combustion chamber pressure $P_c$, and mixture ratio $K_c$. Coupling with the gas generator and turbomachinery can further distribute these changes across $Q_{op}$, $P_p$, $K_p$, and rotational speed n. The resulting signatures depend on fault severity, transient timing, and operating point, so a global pressure or flow deviation need not uniquely identify the affected component. The available variables do not include a dedicated valve-position channel or pressure measurements bracketing each restriction, which limits the direct information available for distinguishing these fault locations.

Table 12 summarizes the potential physical interpretation of the sensor–feature groups. Full-window and terminal means describe persistent operating shifts, while variation and first-difference statistics describe fluctuations and local changes. These descriptors can nevertheless overlap when two faults generate similar downstream responses. In particular, the mean first difference summarizes net change over a window rather than the full transient shape. The temporal branch may retain additional information about response ordering, but neither its latent coordinates nor its channel-attention weights directly identify a physical sensor. The mapping therefore provides a qualitative interpretation of the representation, not an empirical ranking of sensor importance or a causal attribution of errors.

Table 12. Potential physical interpretation of sensor–feature groups. The relationships describe diagnostic information represented by each group rather than measured feature-importance rankings.

Sensor Group	Feature and Window	Potential Interpretation
$Q_o$, $P_o$	Mean, median, and extrema over full and terminal windows	Oxidizer-path operating shifts associated with injector, valve, or pump faults; global responses can overlap.
$P_c$, $K_c$, $Q_c$, $R{T}_c$	Level, variation, and first differences over full and short windows	Downstream combustion response, which is not specific to one upstream component.
$Q_{op}$, $P_p$, $K_p$	Terminal levels and transient variation	Gas-generator coupling, with responses dependent on operating point and fault severity.
n, $N_{po}$, $N_t$	Operating levels and first-difference spread	Coupled turbomachinery response that can correlate with pressure and flow changes.
$Q_f$, $P_f$, $N_{pf}$	Corresponding multi-window statistical descriptors	Fuel-side response for comparison with oxidizer-side changes; coupling limits independent interpretation.
Temporal bottleneck	Whole-sequence learned representation	Transient ordering and context; latent coordinates have no fixed one-to-one correspondence with physical sensors.

The remaining classwise difficulty suggests that further gains may depend on more informative measurements and physically structured representations, not only on a more complex classifier. Analyses that compare faults at matched severity and operating conditions, together with localized pressure or valve-state information, could clarify whether their separation is limited primarily by sensing or by representation learning.

5.3. Feature Selection, Stability, and Interpretability

The results in Table 6 show that adding features alone is insufficient to improve classification: naive concatenation achieves a macro-F1 of 86.54%, compared with 87.96% for the statistical-view RF, whereas fusion with nested RF-RFE and a single RF reaches 88.73%. Removing RF-RFE from the full framework also reduces mean macro-F1 by 1.62 percentage points (Table 10). This pattern is consistent with the need to control redundancy and dimensionality in the 716-dimensional representation. It supports the predictive role of feature selection, but does not establish that the same subset is retained across training partitions.

Predictive performance and selection stability are distinct properties. Correlated features from overlapping windows can substitute for one another, so changes in individual coordinates need not imply a comparable change in the resulting predictions. Conversely, repeatable selection does not by itself establish physical causality. The 684 statistical coordinates have fixed sensor–statistic–window identities, making them suitable for cross-fold comparisons through selection frequencies, subset overlap, and chance-adjusted stability measures [43]. Such comparisons also depend on subset size; interpretation at the sensor, window, or statistic level requires accounting for the number of candidate features in each group.

The learned temporal coordinates require a different interpretation because independently trained bottlenecks need not share a common coordinate basis. Their retained counts or fractions are meaningful at the representation-block level, whereas matching coordinate indices across networks does not identify the same physical feature. The present findings therefore support feature selection as a component of the classification pipeline, while quantitative selection stability and consistent sensor-level attribution remain unestablished.

5.4. Cross-Unit Generalization and Simulation-to-Real Transfer

The difference between sample-level and unit-level evaluation is important for interpreting the practical relevance of the results. Mean macro-F1 falls from 89.84% in repeated nested validation to 85.70% in LOUO, while mean ECE increases from 3.76% to 6.26% (Table 6, Table 7 and Table 9). These changes indicate lower discrimination and poorer calibration when evaluated on an unseen simulated unit. However, LOUO also changes the training-set size, so the performance gap does not isolate the effect of unit shift. With only three units, the individual results provide a limited assessment of cross-unit variation rather than a characterization of an engine population.

Transfer to real engine data introduces additional uncertainties beyond those represented by the simulated-unit holdouts. Sensor bias and drift, colored noise, bandwidth limits, missing measurements, saturation, and sampling misalignment can alter both statistical descriptors and learned temporal patterns. Extrema, high-order moments, and first differences may be particularly sensitive to measurement disturbances, while the convolutional–recurrent branch may capture simulator-specific transient structure. Changes in operating conditions and unmodeled system dynamics can further modify these relationships. Consequently, calibration obtained from simulated data need not remain valid on real trajectories, even if some discriminative information transfers.

Further validation should distinguish sensitivity to controlled perturbations from transfer across actual engine tests. Sensor-informed perturbation experiments can assess the effects of noise, bias, missingness, and timing, while operating-condition and fault-severity holdouts can examine additional sources of variation. These tests complement rather than replace independent hot-fire evaluation. Any adaptation or recalibration on real data requires separate development and test cohorts with engine-unit and test-run boundaries preserved. Evaluation of both classwise discrimination and probability quality under these conditions is necessary to determine the extent to which the benchmark results translate into practical diagnostic performance.

6. Conclusions

This study proposed a multi-scale feature fusion framework with a calibrated convex ensemble for closed-set classification of nine liquid rocket engine faults. The framework combines multi-window statistical descriptors and learned temporal features with nested RF-RFE and calibrated heterogeneous classifiers. On XJTU-REF, repeated nested cross-validation achieved an accuracy of 90.16 ± 1.04%, a macro-F1 score of 89.84 ± 1.08%, and an MCC of 0.891 ± 0.012, improving accuracy and macro-F1 over the final-20%-mean RF baseline by 2.45 and 2.61 percentage points, respectively. The ablation results support the complementary roles of the two representations and feature selection, while calibration primarily improved probability quality, reducing ECE from 7.18% to 3.76%.

Leave-one-unit-out evaluation yielded a mean accuracy of 86.26 ± 1.21% and a macro-F1 score of 85.70 ± 1.31%, indicating lower performance on unseen simulated units than under sample-level validation. With only three healthy observations, three simulated engine units, and no hot-fire validation, these findings do not establish healthy-state recognition, unknown-fault detection, or readiness for operational deployment. Future work will prioritize validation on real engine data, robustness to sensor noise and operating variations, and improved discrimination between oxidizer injector clogging and oxidizer main valve faults.