Expert-Rule-Augmented Machine Learning for Autonomous Controllability Evaluation of Power Equipment with Missing Data

Abstract

To address the challenges of quantifying expert experience, handling missing data, and managing class imbalance in evaluating the autonomous controllability of power equipment, this paper proposes a quantitative evaluation method that integrates expert prior rules with machine learning. First, building upon a five-dimensional evaluation indicator system, expert decision logic—including dimension-average threshold judgments, multi-dimensional weakness-based cumulative downgrading mechanisms, and key sub-item interaction rules—is formalized into a 15-dimensional rule prior feature vector, which is concatenated with the original 21-dimensional raw indicators to construct a RAW + RULE augmented feature space. Second, a KNN algorithm is employed for missing value imputation, while cost-sensitive learning combined with the SMOTE is adopted in a dual-path parallel scheme to address class imbalance. Six machine learning models are evaluated and compared via 30 repeated stratified cross-validations on a real-world dataset of 97 high-voltage bushing suppliers. Experimental results show that, on complete datasets, the RAW + RULE configuration with the Random Forest model achieves a mean test accuracy of 0.936 and a Kappa of 0.938, substantially outperforming the pure raw-feature model (accuracy 0.769, Kappa 0.766). Under weighted random missingness ranging from 10% to 50%, the RAW + RULE configuration demonstrates superior robustness, with ensemble tree models maintaining mean accuracies of 0.614–0.636 even at a 50% missing rate. This study provides a practically deployable technical solution and methodological reference for the quantitative assessment of autonomous controllability levels and early security warning in the power equipment supply chain.

1. Introduction

As a strategic infrastructure supporting global energy security and national economic operation, the level of the autonomous controllability of power equipment is directly related to the security and stability of national industrial chains. In the context of the reshaping of the global industrial competition landscape and the increasingly prominent constraints of key core technologies, scientifically evaluating and enhancing the autonomous controllability capability of power equipment has become an urgent demand to safeguard national strategic security [1,2]. National strategic documents such as “Made in China 2025” have explicitly proposed development goals for the localization rate of power equipment, posing higher requirements for the comprehensiveness and accuracy of evaluation work. However, the traditional evaluation of equipment localization has long relied on the single metric of “localization rate” [3], typically adopting static statistical methods based on the origin of components or value proportions. This evaluation paradigm presents significant limitations: on the one hand, it is difficult to penetrate the surface to reflect the true mastery of core technologies, posing the risk of concealing the hidden danger of “core technology hollowing out”; on the other hand, this metric severs the sustainability of supply chain resilience and technological evolution, lacks consideration of industrial survival capabilities under extreme supply cutoff scenarios, and is inadequate to comprehensively map the overall autonomous controllable foundation of enterprises. Therefore, how to promote the evolution of the evaluation system from superficial “localization” to deep “autonomous controllability” [4], as well as how to construct a quantitative evaluation model with scientificity, multi-dimensionality, and operability, is a critical bottleneck that urgently needs to be addressed by current academia and industry.

In terms of macro policies and the theoretical exploration of autonomous controllability evaluation, international regulations mainly rely on government procurement and industrial security policies. For example, the US Federal Acquisition Regulation (FAR) established an incremental access threshold based on origin and cost proportion [5], the European Union formulated a “strategic autonomy” framework in key technology areas through the European Industrial Strategy [6], and Japan established a supply chain resilience evaluation mechanism via the Economic Security Promotion Act [7]. Domestically, autonomous controllability evaluation has evolved from physical composition measurement to multi-dimensional capability deconstruction. The Interim Provisions on the Calculation Method of the Localization Rate of Electromechanical Products issued by the early State Economic Commission laid the value statistical framework [8]. With the deepening of theoretical research, scholars such as Ni et al. proposed a multi-dimensional autonomous controllability evaluation system covering intellectual property, technical capability, development initiative, supply chain security, and domestic qualification [9,10]. Methodologically, existing studies predominantly rely on subjective weighting models such as the Analytic Hierarchy Process (AHP) [11], fuzzy comprehensive evaluation [12], and the Delphi method [13], with a minority introducing objective evaluation algorithms like the entropy weight method [14] and TOPSIS [15]. Nevertheless, existing achievements are mostly concentrated in the fields of information technology [16], aerospace [17], and nuclear power [18]. However, the evaluation results of these methods are highly dependent on expert-assigned weights, and the models themselves cannot learn complex associations among indicators from data; once some indicators are missing, reliable conclusions become difficult to obtain. At present, a systematic, data-driven evaluation methodology for the autonomous controllability of power equipment remains absent.

In recent years, informed machine learning has offered new perspectives for addressing the above challenges. In a systematic review, von Rueden et al. [19] categorized the integration pathways of prior knowledge into four types—feature augmentation, hypothesis space constraints, learning algorithm modification, and final output correction—noting that feature engineering is the most commonly used yet the most difficult to formalize. Sreedharan et al. [20], from the perspective of deep networks, summarized three knowledge injection strategies—input-layer augmentation, loss function constraints, and architectural constraints—and found that input-layer injection is the most straightforward under small-sample conditions. Wysocka et al. [21] injected clinical rules, causal networks, and empirical formulas into different stages of the machine learning pipeline in the medical domain, demonstrating that multi-stage integration can simultaneously improve both accuracy and interpretability. Bonechi et al. [22] validated, through expert-guided data structuring and feature construction in COVID-19 incidence prediction, that knowledge-guided representations can enhance the alignment between model outputs and expert reasoning. In the energy domain, Incremona et al. [23] introduced external variables such as temperature and calendar information to nonlinearly enhance base prediction models before aggregation, which not only improved the accuracy of electricity load forecasting but also maintained robust performance when some expert models were missing. However, the knowledge fusion mechanisms in the above works primarily involve incorporating external variables as auxiliary inputs or encoding physical laws as model constraints, and are oriented toward time-series forecasting or continuous-value estimation scenarios. None of them have addressed the problem of explicitly encoding expert grading decision logic as rule-based features for participation in discrete ordinal classification.

At the data quality level, power equipment evaluation simultaneously faces the dual constraints of missing data and class imbalance. KNN imputation, owing to its non-parametric nature and local adaptability, has demonstrated advantages in energy data reconstruction under missingness: Wang et al. [24] proposed a seasonal KNN method that incorporates temporal structural priors and outperformed conventional methods even at a 70% missing rate; Pierrot et al. [25] combined KNN with the spatial topology priors of wind farms, using Laplacian eigenmaps to guide neighbor selection, which significantly improved the reconstruction accuracy of missing power generation data. However, these studies are primarily oriented toward time-series scenarios, and research on imputation for cross-sectional data with strong inter-indicator correlations, as found in evaluation systems, remains limited. Regarding class imbalance, SMOTE [26] generates synthetic samples through linear interpolation between minority-class instances and is currently the most widely adopted oversampling method; Fernández et al. [27], in their 15th-anniversary review, noted that hybrid strategies combining SMOTE with noise filtering and ensemble learning can effectively improve synthetic sample quality. When missingness and imbalance co-occur, the MI-MOTE method proposed by Shin et al. [28] merges multiple imputation and minority oversampling into a unified preprocessing step, demonstrating that the conventional sequential pipeline of “first impute, then resample” introduces additional data distortion to minority-class samples. Nevertheless, the aforementioned works rarely involve industrial grading scenarios where expert rules play a substantial role, and few studies have jointly addressed rule-based feature fusion, missing value imputation, and class balancing within a single small-sample evaluation task.

To bridge these gaps, this paper proposes a quantitative evaluation method for the autonomous controllability of power equipment that incorporates prior rules and accounts for data missingness, validating its effectiveness using a high-voltage bushing supplier dataset as a case study. Building upon the available five-dimensional evaluation indicator system and its scoring logic [9,10], the novel contributions of this work are twofold: (1) At the feature representation level, expert decision logic is formalized into a 15-dimensional rule-based prior feature vector φ(x), which is concatenated with the original 21-dimensional indicator vector to construct a RAW + RULE augmented feature space, enabling domain priors to participate in machine learning classification in a structured manner; (2) At the engineering applicability level, KNN-based missing value imputation and class balancing training strategies (cost-sensitive learning and SMOTE in a dual-path parallel scheme) are integrated into a unified evaluation pipeline, enabling the method to operate reliably under incomplete data conditions. This study provides a practically deployable technical solution and methodological reference for the quantitative assessment of autonomous controllability levels and early security warning in the power equipment supply chain.

2. Evaluation Indicator System and Method for Autonomous Controllability of Power Equipment

2.1. Evaluation Indicator System for Autonomous Controllability of Power Equipment

With reference to the five-dimensional autonomous controllability evaluation theory [9], this paper constructs a hierarchical evaluation indicator system for the autonomous controllability of power equipment, encompassing five dimensions: domestic qualification, intellectual property, technical capability, supply chain security, and independent development. Specifically, the domestic qualification dimension verifies the controllable entity identity of suppliers from the perspectives of capital ownership and legal entity attributes. The intellectual property dimension quantifies the substantive mastery of technological sovereignty based on the independent ownership of patents, engineering drawings, and software copyrights. The technical capability dimension focuses on the engineering implementation level, evaluating an enterprise’s practical execution ability to transform autonomous knowledge into stable products. The supply chain security dimension emphasizes the resilience of the supply system against external supply disruption risks, with particular attention to the localization guarantee level of key raw materials and core equipment. The independent development prospectively measures an enterprise’s capacity for sustained investment in technology introduction and assimilation, re-innovation, and independent research and development (R&D).

Secondary quantitative indicators are established under each dimension, totaling 21 items, as shown

Table 1. Overview of dimensions in the evaluation index system for the autonomous controllability of power equipment.

Dimension	Feature
Domestic Qualification	Company registered address, capital control type, legal independence, domestic capital investment intensity
Intellectual Property	Number of core technology patents, drawing localization rate, software copyright localization rate
Technical Capability	Core talent reserve, domestic equipment scale coefficient, nationalization effectiveness index, equipment defect rate
Supply Chain Security	Frequency of domestic production, mass production level, overall manufacturing capability, key raw material assurance, equipment inventory
Independent Development	Technology accessibility, technology digestion and absorption rate, technology re-innovation rate, proportion of R&D funds, annual number of R&D projects

. Each indicator corresponds to a score mapping function that normalizes the original collected values into standard scores within the interval of [0, 100], thereby eliminating dimensional differences. Depending on the nature of the indicators, the mapping functions adopt various forms, such as graded scoring, linear proportional scaling, or composite product functions. An overview of the indicators under each dimension is presented in Table 1, and the detailed scoring logic can be found in Ref. [10].

Based on the evaluation indicator system for the autonomous controllability of power equipment proposed in Refs. [9,10], domain experts score the specific evaluated objects item by item, and the results are subsequently aggregated. Let x = (s₁, s₂, …, s₂₁)^T ∈ ℝ²¹ denote the 21-dimensional raw indicator vector of a sample, and y ∈ {1, 2, 3, 4} denote the comprehensive autonomous controllability grade assessed by experts, where a smaller value indicates a higher degree of autonomous controllability for the supplier.

2.2. Expert Rule-Based Prior Features

The rule features in this paper are derived from the systematic synthesis of expert interviews and evaluation practices. Rather than adopting closed-form scoring formulas, representative evaluation heuristics are extracted from the consistently observed judgment tendencies across multiple experts and further formalized into computable features informed by the data structure. Specifically, some experts tend to weight the overall performance of suppliers across all five dimensions, others impose additional penalties on pronounced weaknesses in critical dimensions, and still others focus primarily on specific sub-items such as supply chain security and intellectual property. These consensus judgment patterns are systematically extracted and explicitly encoded as rule prior features φ(x), which are concatenated with the original 21-dimensional indicator vector to form an augmented feature vector $\tilde{x}$ = [x; φ(x)]. The expert rule system summarized in this paper encompasses three categories: dimension average rules, downgrading rules, and key sub-item refinement rules, generating a total of 15 rule prior features.

(1)

Dimension average rules

There are 6 features in total. They include the average score of the evaluation indicators under each dimension, sequentially denoted as dim_k, k ∈ {DQ, IP, TC, SCS, ID}. In addition, the mean of these five-dimension average scores, denoted as D, is taken as the comprehensive quantitative baseline, which is mapped to the autonomous controllability grade L_base through piecewise thresholds:

$$\begin{array}{l}{\dim }_k={\displaystyle \sum _{i\in {\mathcal{I}}_k}{s}_i},\hspace{1em}k\in \{\mathrm{DQ}, \mathrm{IP}, \mathrm{TC}, \mathrm{SCS}, \mathrm{ID}\}\\ D={\displaystyle \frac{1}{5}}{\displaystyle \sum _k{\dim }_k}\\ \mathrm{L}\_\mathrm{base}=\left\{\begin{array}{l}1, D\ge 80\\ 2, 60\le D<80\\ 3, 40\le D<60\\ 4, D<40\end{array}\right.\end{array}$$

(1)

where i denotes the index set of indicators belonging to dimension k. By introducing three decision thresholds of 80, 60, and 40, L_base discretizes the continuous comprehensive score into four-level ordinal labels, thereby priorly encoding the comprehensive capability baseline of the enterprise in the feature space.

(2)

Downgrading rules

There are 4 features in total. The downgrading rules identify situations where a supplier has substantial weaknesses in specific dimensions but has not yet triggered the absolute veto threshold, applying a cumulative downward adjustment to the base grade L_base. The trigger conditions are dg_supply, dg_ip, dg_qual in {0, 1}, respectively, and the cumulative number of downgrades is n_dg = dg_supply + dg_ip + dg_qual in {0, 1, 2, 3}. The final grade is given by

$$\begin{array}{l}\mathrm{dg}\_{\mathrm{supply}=1 [\dim }_{\mathrm{SCS}}<40],\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\\ {\mathrm{dg}\_\mathrm{ip}=1 [\dim }_{\mathrm{IP}}<60],\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\\ {\mathrm{dg}\_\mathrm{qual}=1 [\dim }_{\mathrm{DQ}}<60],\\ \mathrm{n}\_\mathrm{dg}=\mathrm{dg}\_\mathrm{supply}+\mathrm{dg}\_\mathrm{ip}+\mathrm{dg}\_\mathrm{qual}\{0,1,2,3\}\\ \mathrm{L}\_\mathrm{final}=\max (1,\mathrm{L}\_\mathrm{base}\text{-}\mathrm{n}\_\mathrm{dg})\end{array}$$

(2)

The lower bound of L_final is Grade 1, ensuring that the downgrading rules do not generate invalid grades. The three downgrading conditions specifically target three typical scenarios: weak supply chain resilience, insufficient accumulation of intellectual property, and obvious gaps in domestic qualifications. The value definitions and trigger thresholds for each condition are detailed in

Table 2. Definition of downgrading rule features.

Feature	Value	Definition
dg_supply	0/1	Supply chain security score (dim_SCS) is below 40. Indicates insufficient supply chain resilience that breaches the warning line but remains above the fundamental veto threshold. Triggers a one-level downgrade.
dg_ip	0/1	Intellectual property score (dim_IP) is below 60. Represents severe inadequacies in patents, localized drawings, and software copyrights, indicating a systemic gap in technological sovereignty. Triggers a one-level downgrade.
dg_qual	0/1	Domestic qualification score (dim_DQ) is below 60. Reflects partial non-compliance in registration, capital control, or legal entity attributes that do not trigger the absolute veto criteria. Triggers a one-level downgrade.
L_final	1/2/3/4	Computed as L_final = max(1, L_base − n_dg), where n_dg is the cumulative number of triggered downgrades. Ranges from 1 to 4. This ordinal feature integrates the baseline grade with multi-dimensional penalties.

.

(3)

Key sub-item rules

There are 5 features in total. Relying solely on dimension average scores may mask internal structural differences. For instance, within the supply chain security dimension, among enterprises with identical comprehensive scores, one might rely on a single raw material while another exhibits balanced performance; their actual supply disruption risks differ significantly. To capture such differences, key sub-items and their interaction features within the aforementioned dimensions are extracted as supplements, with specific definitions provided in

Table 3. Definition of key sub-item granular rule features.

Feature	Value	Definition
sc_material	numeric	Product of raw material localization rate (P), inventory coverage factor (I), and supplier diversification factor (M). Any sub-item below 40 is forcibly zeroed.
sc_equipment	numeric	Key equipment availability score. If the number of core equipment days (K) is less than 15, a supply chain veto is triggered.
sc_core_prod	numeric	Normalized product of comprehensive manufacturing capability score and key material security score (÷100). Captures the composite state of supply chain “manufacturing end” and “raw material end” simultaneously.
ip_noPatent	0/1	Binary indicator for absence of core technology patents (0 = no core patents held by the enterprise).
ip_noDrawing	0/1	Binary indicator for absence of domestically designed drawings (0 = enterprise holds no domestic design drawings).

:

$$\begin{array}{l}\mathrm{sc}\_\mathrm{material}=\mathrm{P}\times \mathrm{I}\times \mathrm{M}\cdot 1\left[\min \right(\mathrm{P}, \mathrm{I}, \mathrm{M})>40]\\ \mathrm{sc}\_\mathrm{equipment}=K\cdot 1[K>15]\\ \mathrm{sc}\_\mathrm{core}\_\mathrm{prod}={\displaystyle \frac{{\mathrm{s}}_{\mathrm{mfg}}\times {\mathrm{s}}_{\mathrm{mat}}}{100}}\\ \mathrm{ip}\_\mathrm{noPatent}=1{ [\mathrm{s}}_{\mathrm{patent}}=0, \mathrm{ip}\_\mathrm{noPatent}=0]\\ {\mathrm{ip}\_\mathrm{noDrawing}=1 [\mathrm{s}}_{\mathrm{draw}}=0, \mathrm{ip}\_\mathrm{noDrawing}=0]\end{array}$$

(3)

where P, I, and M denote the product of raw material localization rate, inventory coverage factor, and supplier diversification factor, respectively; moreover, S_mat, S_mfg, S_patent, and S_draw denote the normalized scores for key material security, comprehensive manufacturing capability, number of core technology patents, and number of domestically designed drawings, respectively [9,10].

Synthesizing the above four types of rules, this paper generates a 15-dimensional rule prior feature vector φ(x)∈ℝ¹⁵, which is concatenated with the original 21-dimensional indicator vector to obtain the augmented feature vector:

$$\tilde{x}=\left[x, \varphi (x)\right]\in {ℝ}^{36}$$

(4)

2.3. Imputation of Autonomous Controllability Evaluation Indicators Based on Similarity Criteria

During the autonomous controllability evaluation of power equipment suppliers, influenced by the protection of corporate trade secrets or the lack of historical records, some indicators often exhibit numerical missingness. Directly discarding these missing values would lead to evaluation deviations. Therefore, this study adopts a K-Nearest Neighbors (KNN) imputation method based on the similarity of known sample features. Suppose an incomplete data sample is denoted as x_i ∈ ℝ^d, with its missing feature set being M_i ∈ {1, …, d}, and the known feature index set being K_i ∈ {1, …, d}\M_i. The specific imputation process is illustrated in Figure 1.

First, for each sample x_j^Train in the autonomous controllability dataset, its known feature sub-vector x_j^Train [K_i] is extracted to calculate the Euclidean distance to the incomplete sample based on the known features:

$${\mathrm{dist}}_{ij}=\sqrt{{\displaystyle \sum }_{k\in K_i}{\left(x_{i,k}-x_{j,k}^{\mathrm{train}}\right)}^2}$$

(5)

The dataset samples are sorted according to the distance dist_ij, and the k samples with the minimum distances are selected to form the neighborhood set N_i. The value of the missing feature m ∈ M_i is then imputed as the mean of the corresponding features from the neighborhood samples:

$$x_{i,m}={\displaystyle \frac{1}{\left|N_i\right|}}{\displaystyle \sum }_{j\in N_i}{x}_{j,m}^{\mathrm{train}}$$

(6)

Samples with similarities in known features can statistically reference each other for their missing feature values. Compared to simple mean or interpolation methods, this approach fully utilizes the structural information of the existing dataset, enabling the imputed samples to maintain consistency with the overall data distribution, thereby enhancing the reliability of the evaluation.

2.4. Evaluation Method for Autonomous Controllability of Power Equipment

The evaluation of the autonomous controllability of power equipment can be formalized as a supervised multi-class classification task; that is, given a training set {(x_i, y_i)}, a mapping function f: ℝ²¹→{1, 2, 3, 4} is learned to possess sufficient generalization capability for unseen samples. During the grading process, experts comprehensively utilize multi-source information, such as industry experience, field investigations of enterprises, and qualification document verification. Their grading conclusions integrate profound domain knowledge and subjective judgments. Due to individual differences in grading standards among experts, the internal logic of the mapping function f(·) remains opaque to external researchers. Therefore, this paper treats f(·) as an unknown function to be learned, approximating this mapping from a limited number of labeled samples through machine learning.

This paper selects six mainstream machine learning classification algorithms to conduct comparative experiments, specifically including Logistic Regression (L2 regularization), Support Vector Machine (SVM, RBF kernel), Decision Tree, Random Forest, Extra Trees, and Gradient Boosting. Shallow constraints are applied to all tree-based models to suppress the tendency of overfitting under small sample conditions.

Table 4. Hyperparameter search space, final settings, and selection criteria of the evaluated models.

Model	Tuned Hyperparameters	Candidate Values	Selection Criterion
LR	C	{0.01, 0.1, 1, 10}	Best mean macro-F1 (stratified CV, train)
SVM (RBF)	C, gamma	C ∈ {0.1, 1, 10, 100}; gamma ∈ {scale, 0.01, 0.1, 1}	Best mean macro-F1 (stratified CV, train)
DT	max_depth, min_samples_leaf	max_depth ∈ {3, 4, 5, 6}; min_samples_leaf ∈ {1, 2, 3, 4}	Best mean macro-F1 under complexity control
RF	n_estimators, max_depth, min_samples_leaf	n_estimators ∈ {100, 200, 300}; max_depth ∈ {4, 5, 6, None}; min_samples_leaf ∈ {1, 2, 3}	Best mean macro-F1 (stratified CV, train)
Extra Trees	n_estimators, max_depth, min_samples_leaf	n_estimators ∈ {100, 200, 300}; max_depth ∈ {4, 5, 6, None}; min_samples_leaf ∈ {1, 2, 3}	Best mean macro-F1 (stratified CV, train)
GBDT	n_estimators, learning_rate, max_depth, subsample	n_estimators ∈ {100, 200, 300}; learning_rate ∈ {0.03, 0.05, 0.1}; max_depth ∈ {2, 3, 4}; subsample ∈ {0.6, 0.8, 1.0}	Best mean macro-F1 (stratified CV, train)

lists the key tuning parameters, candidate ranges, and selection rationale for each model. All parameter comparisons are conducted exclusively within the training data using mean Macro-F1 under stratified cross-validation as the criterion; the independent test set is not involved in any part of the parameter selection process. The experiments are implemented in Python 3.10, with primary dependencies including scikit-learn 1.4.2 and imbalanced-learn 0.11.0.

To address the potential sample imbalance problem in the autonomous controllability evaluation dataset, this study designs two parallel training modes. The first is a weight compensation mechanism based on cost-sensitive learning, which automatically amplifies the influence of minority classes in gradient updates through a class balancing weight factor w_c = N/(C·n_c) (where N is the total number of samples, C is the number of classes, and n_c is the number of samples in that specific class), increasing the weight. The second mode employs the Synthetic Minority Over-sampling Technique (SMOTE) based on k = 3 nearest neighbors, generating synthetic samples through linear interpolation in the feature space to balance the training data of each grade prior to fitting. These two paths are verified in parallel during the training process, and the optimal scheme is selected as the final output based on the generalization performance in cross-validation.

Model performance is comprehensively evaluated using four metrics. The CV Macro-F1 is obtained from 3-fold stratified cross-validation on the training set, with macro-averaged F1 adopted as the intra-fold scoring metric, reflecting model stability and generalization during training. The Test Accuracy measures overall prediction correctness on the held-out test set. The Test Macro-F1 serves as the unweighted average of the F1 scores across all grades on the test set, exhibiting robustness to class imbalance scenarios. The Kappa coefficient accounts for the ordinal nature of the grade labels, imposing heavier penalties for cross-grade misjudgments. Due to the lack of statistical significance in a single experiment, this study repeated the experiment thirty times, randomly partitioning the training and test sets each time. The evaluation metrics mentioned above are the average values across these repeated experiments.

The specific execution steps of the evaluation method are as follows: ① The full dataset is split into a training set and an independent test set according to the stratified sampling principle. ② Within the training set, 3-fold stratified cross-validation is performed, with each fold serving in turn as the validation set and the remaining folds as the sub-training set, to evaluate the generalization performance of different models and preprocessing strategies. ③ Class imbalance handling is applied only to the sub-training set, with class-weight and SMOTE (k = 3 nearest neighbors) compared as two parallel strategies, and no synthetic samples are introduced into either the validation set or the test set. ④ The optimal hyperparameters and class balancing strategy for each model are determined based on the mean F1_macro on the validation set. ⑤ The model is refitted on the complete training set using the above optimal configuration, and the final performance is reported on the independent test set. For samples with missing indicators, the KNN imputer constructs its neighbor structure solely based on observed samples in the training set, and test samples do not participate in the fitting process of the imputer. The above procedure ensures that all estimation steps related to data distribution are strictly confined within the training data, thereby effectively controlling the risk of data leakage introduced during preprocessing.

3. Results and Analysis

3.1. Autonomous Controllability Dataset of Power Equipment

This paper takes 97 domestic and international high-voltage bushing suppliers as the research objects, with supplier codes sequentially denoted from BUSH01 to BUSH97. The evaluation of autonomous controllability in power equipment involves highly sensitive information such as the sourcing of key raw materials, ownership of core technical drawings, and patent rights, which substantially raises the barriers to data acquisition. The full-dimensional real-world business data from 97 suppliers represents the maximum feasible scale obtainable through long-cycle field investigations conducted in collaboration with the grid enterprise. Based on the collected enterprise qualification materials and production/operation data, the standardized scores of each supplier across 21 secondary quantitative indicators are calculated according to the indicator system and mapping rules described in Section 2.1, forming the original feature matrix x ∈ ℝ^{97 × 21}. Regarding the autonomous controllability evaluation, a review panel consisting of multiple domain experts with extensive industry experience is established to independently grade the 97 suppliers, and the final grades are confirmed through consensus by the expert panel. The grading results are distributed according to the four-level classification criteria defined in Section 2.3, and the calculation results of the indicator scores for each dimension (partial examples) are presented in

Table 5. Dataset of autonomous controllability for power equipment.

Feature	Supplier ID
	01	02	03	04	05	06	07	08	09	10	11	12	13	14	15	16	17
Company registered address	100	100	0	100	100	100	100	100	100	100	100	0	100	100	100	100	100
Capital control type	100	100	0	100	100	100	100	100	100	100	100	100	100	100	100	100	100
Legal independence	100	100	0	100	100	100	100	100	100	100	100	100	100	100	100	100	100
Domestic capital investment intensity	95	95	95	95	95	95	95	95	95	95	95	95	82	95	95	95	95
Number of core technology patents	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100
Drawing localization rate	100	100	50	100	50	100	75	100	75	75	100	100	75	50	75	50	100
Software copyright localization rate	100	0	0	100	0	0	75	100	100	75	100	100	100	100	100	100	100
Core talent reserve	80	80	80	80	80	80	80	80	80	80	80	80	80	80	80	80	80
Domestic equipment scale coefficient	80	60	60	60	80	60	60	80	60	60	80	80	60	60	60	80	60
Nationalization effectiveness index	98.3	96.6	97.1	95.3	93.6	99.4	91.5	98.9	91.5	100	99.8	99.7	93.3	98.3	91.7	98.3	98.3
Equipment defect rate	100	100	80	100	100	100	100	100	100	100	100	100	100	100	100	80	100
Domestic frequency	80	80	80	80	60	80	60	60	80	80	60	60	80	60	60	60	60
Scale of production	80	60	60	60	80	80	40	80	80	80	60	60	80	60	60	60	80
Overall manufacturing capability	67	67	50	50	50	79	50	50	79	67	50	50	67	50	50	67	67
Key raw material assurance	50	50	50	50	50	50	50	50	50	50	50	50	50	50	50	50	50
Equipment inventory	74	64	64	74	95	74	95	74	84	64	74	64	62.5	82.5	100	84	90
Technology accessibility	82.5	67.5	67.5	60	85	65	62.5	75	82.5	92.5	62.5	65	62.5	82.5	100	77.5	60
Technology digestion and absorption rate	60	60	60	40	100	100	60	20	60	60	60	80	20	80	40	40	60
Technology re-innovation rate	100	100	80	100	100	60	60	100	80	100	80	100	100	100	80	80	100
Proportion of R&D funds	95	95	95	95	95	95	95	95	95	95	95	95	82	95	95	95	95
Number of annual R&D projects	95	95	95	95	95	82	95	95	95	95	95	95	82	95	95	95	95
Autonomous Controllability Level	4	2	1	2	1	2	1	2	3	3	2	1	2	1	2	3	4

. Among the total of 97 samples, the number of samples distributed across autonomous controllability Grades 1 to 4 are 33, 36, 20, and 8, respectively. Subsets are constructed at a ratio of approximately 7:3 (approximately 68 training samples and 29 test samples per run) across 30 independent repeated partitions. The relatively low proportion of Grade 4 samples (8/97, approximately 8.2%) reflects the actual industrial distribution rather than sampling bias—under the current supply chain landscape, enterprises meeting the full criteria for autonomous controllability are inherently scarce.

3.2. Evaluation Results of Autonomous Controllability for Power Equipment Under Different Feature Dimensions

Figure 2 presents heatmaps of the mean CV Macro-F1, Test Accuracy, Test Macro-F1, and Kappa across six machine learning models under three feature configurations—RAW, RULE, and RAW + RULE—based on 30 independent repeated stratified cross-validations. The following conclusions can be drawn from a comprehensive analysis.

In terms of overall performance ranking, the RAW + RULE configuration consistently outperforms both the RULE and RAW configurations across all four metrics, with RULE ranking second and RAW performing the weakest; this ordering holds across all six models. Taking the best-performing RF model as an example, the RAW + RULE configuration achieves CV Macro-F1, Test Accuracy, Test Macro-F1, and Kappa of 0.909, 0.936, 0.910, and 0.938, respectively, representing improvements of 0.211, 0.167, 0.165, and 0.172 over the corresponding values under the RAW configuration (0.698, 0.769, 0.745, and 0.766), with substantial margins. The four metrics for RF under the RULE configuration are 0.864, 0.916, 0.908, and 0.920, respectively, showing only a marginal gap relative to RAW + RULE, which indicates that the rule features themselves carry strong discriminative information.

In terms of feature fusion gain, the performance improvement of RAW + RULE over pure RULE is modest but consistently present. Taking GBDT as an example, RAW + RULE achieves Test Accuracy, Test Macro-F1, and Kappa values of 0.927, 0.892, and 0.917, respectively, nearly identical to those of the RULE configuration (0.927, 0.893, and 0.919), with differences within 0.002; in contrast, in CV Macro-F1, RAW + RULE (0.865) marginally outperforms RULE (0.861). This phenomenon is consistent with the findings on the complete dataset, suggesting that under complete data conditions, rule features alone can provide sufficient discriminative information, with raw features offering limited supplementary benefit, although the fused configuration yields slightly superior stability.

Regarding the limitations of the RAW configuration, it exhibits clear disadvantages across all metrics, with the gap being most pronounced in CV Macro-F1. The DT model under the RAW configuration achieves a CV Macro-F1 of only 0.633 and a Test Accuracy of 0.696, whereas the same model under the RAW + RULE configuration improves to 0.852 and 0.883, respectively, representing gains of 0.219 and 0.187. This demonstrates that models relying solely on the original 21-dimensional indicators have limited generalization capability, and that the incorporation of expert rule features plays a substantive role in enhancing both model stability and generalization performance.

Figure 3 further corroborates this trend: nearly all models exhibit consistent performance improvements upon the introduction of rule features, with RF, GBDT, and ExtraTrees showing the most pronounced gains.

In terms of model selection, ensemble tree models (RF, ExtraTrees, and GBDT) consistently outperform DT, LR(L2), and SVM(RBF) across all three feature configurations. Among these, RF achieves optimal or near-optimal performance across all four metrics under the RAW + RULE configuration and is therefore recommended as the preferred model for this task. LR(L2) performs the weakest across all configurations, achieving a Test Accuracy of only 0.816 and a Kappa of 0.824 under RAW + RULE, indicating that linear models have limited capacity to accommodate the non-linear feature structure inherent in this task.

Regarding the consistency between CV and test metrics, the CV Macro-F1 and the test-set metrics (Test Accuracy and Test Macro-F1) exhibit an overall positive correlation across models, indicating that cross-validation results serve as reliable predictors of test-set performance and that overfitting risk is effectively controlled under the current parameter settings.

Figure 4 displays the mean feature importance distribution of the RF model under the RAW + RULE configuration, averaged across 30 independent repeated runs. The top three features are all rule-based (Dim_SCS, Dim_ID, and Dim_IP), with Dim_SCS exhibiting substantially higher importance than the others (approximately 0.175), indicating that the aggregated supply chain security score plays a decisive role in grade discrimination. Key sub-item rule features such as SC_coreProd and SC_equipment also rank within the top ten, demonstrating that rule features effectively concentrate decision-boundary-relevant information through weighted aggregation and sub-item interaction over the raw indicators. In contrast, most raw features—including company registered address, capital control type, and legal independence—contribute negligible importance, as the overwhelming majority of suppliers in the sample are Chinese enterprises whose scores on these basic qualification indicators are highly concentrated and offer little cross-supplier discriminability, rendering them ineffective for node splitting in tree-based models. A small number of raw features, such as technology accessibility and overall manufacturing capability, retain moderate importance, reflecting their relatively greater variability across suppliers. Overall, rule-based features dominate the feature importance distribution, corroborating the substantive contribution of domain prior knowledge to the discriminative power of the RAW + RULE framework.

3.3. Evaluation Results of Autonomous Controllability of Power Equipment Under Different Missing Rates

A statistical analysis of the missing value distribution across 21 raw indicators was conducted on the subset of supplier records with observed natural missingness in our dataset (see Figure 5). The results show that the normalized missing rates of individual indicators generally fall within the range of 0.02–0.08, yet exhibit a pronounced business-attribute-driven pattern: indicators related to technical details and supply chain assurance (e.g., key raw material security, domestic substitution effectiveness index, technology absorption rate, and drawing domestication rate) tend to have relatively higher missing rates, whereas foundational qualification indicators with greater public availability (e.g., company registered address, legal person independence, and capital control type) tend to have relatively lower missing rates. Based on this empirical missing distribution, weighted random sampling of missing indicators is performed according to each indicator’s actual missing rate, thereby enabling the simulated scenario to faithfully reflect the non-uniform missing distribution pattern observed in real business data.

Regarding statistical uncertainty estimation, the revised manuscript adopts 30 repeated stratified cross-validation runs for experimental results under all missing rate conditions, reporting mean Test Accuracy and its standard deviation in place of single-point estimates, thereby more objectively characterizing the range of model performance fluctuation under missing data conditions. Figure 6 presents the mean test accuracy of six machine learning models under three feature configurations—RAW, RULE, and RAW + RULE—across weighted random missing rates ranging from 10% to 50%, based on 30 repeated experiments. The RAW + RULE configuration achieves the highest performance across all missing rate levels. Taking GBDT and ExtraTrees as examples, their accuracies at a 10% missing rate are 0.851 and 0.840, respectively, and remain at 0.614 and 0.636 even at a 50% missing rate, both substantially higher than the corresponding values of the same models under the RAW configuration (GBDT: 0.681 at 10% missing rate and 0.513 at 50% missing rate; ExtraTrees: 0.667 at 10% missing rate and 0.534 at 50% missing rate). Compared with the MCAR setting in the original manuscript, the absolute performance of all models declines under the weighted random missingness simulation; however, the performance advantage of RAW + RULE over both RAW and RULE remains consistent across all missing rate levels, validating the robustness of the feature fusion strategy under more realistic missingness scenarios.

Under weighted random missing simulation, the RULE configuration demonstrates a certain advantage over RAW, though it remains below RAW + RULE overall. At a 10% missing rate, for instance, the accuracies of GBDT and ExtraTrees under the RULE configuration are 0.710 and 0.699, respectively, exceeding the RAW configuration values of 0.681 and 0.667; at a 50% missing rate, the corresponding RULE values of 0.526 and 0.539 still outperform the RAW values of 0.513 and 0.534. These results indicate that rule-based features possess a degree of independent robustness under missing conditions, and that RAW + RULE further amplifies this advantage through the complementary fusion of raw indicators and rule-based features.

As the missing rate increases, the accuracy of all three configurations declines monotonically, though with markedly different rates of degradation. The RAW configuration is most sensitive to missingness, with DT and LR(L2) dropping to 0.392 and 0.385, respectively, at a 50% missing rate; the RULE configuration degrades more gradually, with corresponding values of 0.430 and 0.432; and the RAW + RULE configuration exhibits the slowest degradation—even at the extreme missing rate of 50%, GBDT and ExtraTrees maintain accuracies of 0.614 and 0.636, respectively, demonstrating that the introduction of rule-based features provides a significant buffering effect against missing data interference. Ensemble tree models (ExtraTrees, RF, and GBDT) consistently outperform DT, SVM, and LR across all three feature configurations and missing rate levels, further confirming that the combination of the RAW + RULE feature fusion strategy with ensemble tree models represents the optimal technical solution for handling weighted random missing data scenarios.

4. Conclusions

To address challenges in the autonomous controllability evaluation of power equipment—such as the difficulty of quantifying expert experience, missing data, and class imbalance—this paper proposes a quantitative evaluation method that accounts for both prior rules and data deficiency, validated on a dataset of 97 high-voltage bushing suppliers. The main conclusions are as follows:

(1) A framework for fusing RAW + RULE features is proposed by formalizing expert decision logic into a 15-dimensional rule-based prior features (RULE), which is then concatenated with the 21 available raw metrics (RAW). The results from 30 repeated 3-fold stratified cross-validations demonstrate that the RAW + RULE configuration consistently outperforms pure RAW or RULE configurations across all models and metrics. Taking Random Forest (RF) as an example, the average test accuracy and Kappa coefficient under RAW + RULE reached 0.936 and 0.938, respectively, representing improvements of 0.167 and 0.172 over the RAW configuration. This confirms that the structured encoding of expert rules significantly enhances the discriminative power and stability of the evaluation model.

(2) A KNN imputation method based on sample feature similarity was integrated into the evaluation pipeline and tested under weighted random missing rates ranging from 10% to 50%. The RAW + RULE configuration exhibited the strongest robustness to missing data: even at a 50% missing rate, GBDT and ExtraTrees maintained average accuracies of 0.614 and 0.636, respectively, significantly outperforming the RAW configuration under identical conditions. This indicates that the introduction of rule-based features provides a significant buffering effect against the interference caused by missing data.

(3) Comparative analysis of six machine learning models on both complete and incomplete datasets shows that ensemble tree models are the optimal choice for this task, achieving the best balance between predictive accuracy, generalization stability, and robustness to data quality degradation.

This study focuses on the formalization of rule-based knowledge, data-fusion modeling, and empirical validation for autonomy and controllability evaluation in power equipment. However, the sample size is limited and the analysis is confined to a single equipment category; accordingly, the findings should be regarded as a stage-specific validation in a particular application setting. Moreover, the proposed method integrates domain-specific rule features within a standard machine learning framework and, while practically valuable, does not constitute a novel learning algorithm or theoretical framework. Future work will extend the analysis to larger, multi-category power-equipment datasets to assess generalizability and robustness and will further investigate the cross-scenario transferability of the rule-feature framework.