Feature Fusion Using Deep Learning Algorithms in Image Classification for Security Purposes by Random Weight Network

Abstract

Automated threat detection in X-ray security imagery is a critical yet challenging task, where conventional deep learning models often struggle with low accuracy and overfitting. This study addresses these limitations by introducing a novel framework based on feature fusion. The proposed method extracts features from multiple and diverse deep learning architectures and classifies them using a Random Weight Network (RWN), whose hyperparameters are optimized for maximum performance. The results show substantial improvements at each stage: while the best standalone deep learning model achieved a test accuracy of 83.55%, applying the RWN to a single feature set increased accuracy to 94.82%. Notably, the proposed feature fusion framework achieved a state-of-the-art test accuracy of 97.44%. These findings demonstrate that a modular approach combining multi-model feature fusion with an efficient classifier is a highly effective strategy for improving the accuracy and generalization capability of automated threat detection systems.

1. Introduction

The integrity of security screening in public spaces, such as airports, is a critical component of both national and international safety. Although numerous security measures are implemented, systems reliant on human operators remain vulnerable to error, potentially leading to severe security breaches with significant material and societal consequences. X-ray imaging systems play a central role in these security protocols, particularly for baggage inspection. However, the manual identification of concealed threats within complex environments, such as improvised explosive circuits hidden inside electronic devices like laptops, presents a formidable challenge. This specific task requires a high level of specialized expertise and is inherently prone to human oversight, thereby creating a significant vulnerability in security checkpoints.

In response to these challenges, automated detection systems driven by deep learning have been explored by researchers. However, early investigations that applied conventional deep learning models directly to this problem revealed significant limitations. These limitations include relatively low classification accuracy and a high tendency towards overfitting, largely attributed to the complexity and inherent variations within the X-ray dataset [1]. The overlapping nature of components in X-ray images increases intra-class variation, while the visual similarity between benign laptop circuits and threat items elevates inter-class confusion, making robust classification difficult. To overcome these deficiencies, this study proposes a novel framework centered on feature fusion combined with a Random Weight Network (RWN) for classification. The core hypothesis is that features extracted from multiple and diverse deep learning architectures can provide a richer, more discriminative representation of the input data. By fusing these features and employing an RWN, which is noted for its rapid training and resistance to overfitting, it is anticipated that a more accurate and generalizable classification model can be achieved. This approach addresses the key research questions regarding the performance enhancement that can be achieved through feature fusion and the optimal configuration of the RWN classifier, including the impact of hidden neuron count and activation function.

The main contributions of this work are systematically outlined as follows:

(a)

A Novel Feature Fusion Framework: This study proposes and validates a new framework that integrates features extracted from multiple deep learning models (e.g., ShuffleNet, InceptionV3) and employs a Random Weight Network (RWN) for classification. This multi-source feature fusion strategy marks a significant departure from conventional single-model approaches.

(b)

Significant Performance Improvement: A substantial improvement in classification performance is demonstrated. The proposed feature fusion methodology achieves a test accuracy of 97.44%. This result is markedly superior to both the 83.55% accuracy of the best-performing individual deep learning model, ShuffleNet, and the 94.82% accuracy from classification using features from a single model with an optimized RWN.

(c)

Comprehensive Empirical Analysis: A comprehensive empirical analysis of the RWN-based classifier is conducted. The investigation evaluates the influence of critical hyperparameters, including the number of hidden neurons and the choice of activation functions, providing a clear optimization guide for similar security applications.

(d)

Robustness and Generalization: The robustness and generalization capability of the proposed method are established through a comparative analysis against 11 state-of-the-art machine learning classifiers. The framework is shown to offer superior generalization and effective mitigation of overfitting.

(e)

Publicly Available Dataset: A challenging new dataset of X-ray images, featuring laptops with and without concealed circuits, has been created and made publicly available [1], thereby providing a valuable benchmark for future research in this domain.

Within this framework, the following research questions are posed to articulate the study’s core contributions and key capabilities:

(a)

How does it affect the classification performance using RWN on datasets whose features are extracted by deep learning models?

(b)

Can the combination of features extracted from different deep learning models significantly improve training and test accuracy in classification?

(c)

What are the performance implications of existing deep learning algorithms when applied to X-ray security datasets, and how can these be addressed through feature fusion techniques?

(d)

How does the use of an RWN influence classification performance when compared to standard deep learning models on X-ray datasets?

(e)

Do the combinations of merged features (e.g., N|M and M|N) have a significant effect on classification outcomes in RWN?

(f)

What is the impact of the number of hidden layer neurons on the performance of an RWN, and how can the risk of overfitting be minimized through optimal parameter selection?

(g)

How does the selection of activation functions (sigmoid, tangent sigmoid, or hardlim) affect the classification performance of an RWN, particularly in the context of combined datasets?

The organization of the study is as follows: In Section 1, an introduction to the study is provided, a literature review is presented, and the motivation and contribution of the study are outlined. Section 2 covers feature extraction from deep learning models, feature fusion, and dataset explanation. In Section 3, experiments are conducted, and the results obtained are analyzed. Section 4 discusses the findings, and the study is finally concluded in Section 5.

X-ray imaging technologies have been used in various aspects of daily life, as well as in fields such as crystallography, astronomy, and medicine, since the discovery of X-rays by Wilhelm Conrad Rontgen. These technologies encompass a wide range of purposes and methods, including traditional transmission methods, dual-energy techniques, and scattered X-ray methods [2]. In these technologies, rays emitted from an X-ray source are attenuated as they pass through objects. This decrease in intensity is utilized to calculate the density (d) and effective atomic number (Z_eff) of the materials [3]. Consequently, materials with higher density, which cause greater attenuation, appear brighter in X-ray images, while lower-density materials appear darker. X-ray technologies are widely used for various purposes, as evidenced by the information provided by X-ray devices. Applications range from inspecting welds in industrial settings and identifying bone fractures in medicine to detecting prohibited materials in security-sensitive locations like airports, courthouses, and shopping malls.

X-ray images are utilized for the detection of prohibited materials, aiming to minimize security risks at airports through the application of machine learning and image processing techniques. This involves identifying items passengers are forbidden to carry, whether on their person or in their luggage, by analyzing 3D or 2D X-ray images [4]. These applications are typically employed to assist personnel conducting baggage control or to automate the process. This section reviews the literature on X-ray image analysis and feature fusion using deep learning algorithms.

Previously, tasks such as classification in X-ray imaging were performed using manually extracted features, such as SIFT and PHOW, often within a Bag of Words (BoW) framework [5].

In later periods, the success of convolutional neural network (CNN) techniques led to their increased use in this field as well. Akçay et al. [6] implemented transfer learning in CNN using the fine-tuning paradigm. Jaccard et al. [7] detected the presence of threat materials in cargo containers using CNN on image patches. Mery et al. [8] compared methods such as Bag of Words, Sparse Representations, deep learning, and classical pattern recognition schemes. Jaccard et al. [9] also detected cars within cargo using CNN with data augmentation. Rogers et al. [10] used the original dual-energy images as separate channels in their CNN. They performed data augmentation with Threat Image Projection. Caldwell et al. [11], investigated transfer learning in different scenarios using deep networks, such as VGG. Morris et al. [12] focused on threat detection of traditional explosives using CNNs like VGG and Inception. In addition to these, newly emerged CNN models such as region-based CNNs [13] and single-shot models like YOLO (You Look Only Once) [14] have also been applied in X-ray imaging. Petrozziello and Jordanov [15] performed the detection of steel barrel holes using CNN and Stacked Autoencoder. Cheng et al. [16] used a YOLO-based method they called X-YOLO, which has feature fusion and attention mechanisms. Wu and Xu [17] used a hybrid Self-Supervised Learning in the pre-training phase to perform detection with a structure containing a transformer in the final stage of the Head-Tail Feature Pyramid head. Wang et al. [18] used a Yolov8-based method with a dual branch structure that includes Sobel convolution and convolution branches. Additionally, in the fusion part, they used a lightweight star operation module. In addition to classification, deep learning methods have also been employed for data augmentation. Yang et al. [19] performed data augmentation using generative adversarial networks (GANs), which utilize Fréchet Inception Distance scores and compared them (DCGAN, WGAN-GP). Kaminetzky and Mery [20] performed data augmentation using simulated 3D X-ray image models. Apart from these tasks, CNNs have also been used as feature extractors. Caldwell and Griffin [21] performed data augmentation with photographic images, using both photographs and X-ray images of the same object. Benedykciuk et al. [22] addressed the material recognition problem using a multiscale network structure consisting of five subnetworks and using image patches. Babalik and Babadag [23] used CNNs as feature extractors; in the next stages, they selected the features with a binary Sparrow Search Algorithm and classified them using Support Vector Machines (SVM) and k-nearest neighbors (KNN) as classifiers. Methods such as ensemble learning and feature fusion have also been used to improve the performance of deep learning models. Ayantayo et al. [24] proposed three different deep learning models, which used early-fusion, late-fusion, and late-ensemble learning strategies to resist overfitting. Zhang et al. [25] used multi-domain features, employing transfer learning and feature fusion. They used SVM in feature extraction and model selection stages, then fused these features and performed baby cry detection. Wu et al. [26] performed deep learning-based fault diagnosis for rolling bearings. In their method, they used a new multiscale feature fusion deep residual network containing multiple multiscale feature fusion blocks and a multiscale pooling layer. Liu et al. [27] implemented a multi-modal fusion approach that combines two different deep learning models trained on simple clinical information and group images, using logistic regression for breast nodule diagnosis. Patil and Kirange [28] proposed a method for detecting brain tumors by fusing deep network features, such as VGG Inception and a shallow CNN. Gill et al. [29] used deep learning methods including CNN, Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) for image-based classification of fruits using early fusion and late fusion strategies. Deng et al. [30] addressed side-channel attacks in information security using feature extraction with a multi-scale feature fusion mechanism. Al-Timemy et al. [31] utilized Xception and InceptionResNetV2 deep learning architectures to extract features from three different corneal maps. The extracted features from these models were fused into one pool to train conventional machine learning classifiers. Peng and Zhang [32] presented a deep learning network based on multiple feature fusion as well as ensemble learning approaches for the diagnosis and treatment of lung diseases. A deep supervised ensemble learning network was used to combine multiple inducers to improve lung lobe segmentation. Tu et al. [33] proposed a general framework for solving online packing problems using deep reinforcement learning hyper-heuristics. They used feature fusion to combine the visual information of real-time packing with distributional information of random parameters of the problem. Tan et al. [34] performed component identification using a deep learning network based on coarse-grained feature fusion. Medjahed et al. [35] fused CNNs trained on different modalities using machine learning (ML) algorithms in the classification phase. Ma et al. [36] proposed a deep dual-side learning ensemble model for Parkinson’s disease diagnosis by analyzing speech data. Their approach employs a weighted fusion mechanism to integrate multiple models. Alzubaidi et al. [37] detected shoulder abnormalities. They trained models using different body part images in the same domain and performed feature fusion with different machine learning classifiers. Agarwal et al. [38] used an approach that combined channel-based fusion and model-based fusion to classify chest x-ray images using ResNet50V3 and InceptionV3 models. Li et al. [39] designed a dual-channel feature fusion network for detecting distal radius fractures. They used Faster region-based CNN (RCNN)and ResNet50 on their channels. The feature fusion method includes an attention mechanism.

While a broader overview of deep learning for X-ray analysis is available in the literature [40], the foundation for the current study is the authors' prior work [1]. That paper details the creation of the dataset used herein and presents a comparative performance analysis of 11 different deep learning models.

The dataset presented in [1] is highly challenging due to two primary factors. Firstly, the overlapping nature of internal components in X-ray images leads to high intra-class variation. Secondly, high inter-class similarity, resulting from the visual resemblance between benign laptop circuits and threat circuits, reduces the distinction between the classes. Collectively, these issues caused the models evaluated in the previous work [1] to be prone to overfitting, a problem exacerbated by the dataset's limited size and high complexity. To overcome this issue and improve classification performance, it was proposed to use the RWN network for the fusion of the feature extraction capabilities of the pre-trained models in a way that is resistant to overfitting. This was anticipated to increase fusion success due to both the single-stage nature of the training and its resistance to overfitting.

Therefore, the motivation for this study is twofold: to address the identified gap in the literature for this specific security application and to overcome the performance limitations of conventional deep learning models that were observed in our prior work [1].

2. Feature Extraction Using Deep Learning

The fundamental difference between deep learning architectures and classical artificial neural networks is their ability to extract features from raw data. This is particularly popular in image processing, analysis, and classification. One significant reason for this popularity is that each pixel can be considered as a feature, leading to millions of features depending on the image resolution. The primary purpose of convolution filters used in image analysis or classification is to extract a feature map. Typically, due to the high number of extracted features, there is a pooling layer following the filter layers. The general scheme of convolution filters is illustrated in Figure 1, and pooling is depicted in Figure 2.

Different architectures have been proposed in the literature for various purposes, such as preventing the vanishing gradient problem by using a different number of convolution filters and pooling, improving the connectivity structure between layers, scalability, reducing computational costs, and achieving higher-performing models. In the scope of this study, ResNet, DarkNet, EfficientNet, DenseNet, MobileNet, InceptionV3, ShuffleNet, and Xception models were trained with the dataset, and features were extracted from these models. After obtaining the features, various machine learning methods such as kNN [41,42,43], SVM [42,43], and random forest [42,43] have been applied in the literature. In the scope of this study, feature vectors from 11 deep learning methods were obtained. An example of extracting and obtaining features after the training of the ResNet18 deep learning architecture is illustrated in Figure 3.

After training, data are passed through the network, and features are extracted from the outputs of intermediate layers, such as convolutional or pooling layers, prior to the final fully connected layer.

Table 1. Layers from which features are extracted and the number of extracted features from deep learning models.

Models	Feature-Extracted Layer
EfficientNet	globavgpool
ResNet18	pool5
ResNet50	avgpool
ResNet101	pool5
DarkNet19	conv19
DarkNet53	avg1
MobileNetV2	global_average_pooling
ShuffleNet	node_200
InceptionV3	avg_pool
DenseNet201	avg_pool
Xception	avg_pool

summarizes the models used, the specific layers from which features are extracted, and the corresponding number of extracted features.

Table 1 Facilitates the comparison of different deep learning models based on the layers chosen for feature extraction and the dimensionality of the extracted features. The number of features reflects the complexity and expressive power of each model in capturing information from the input data.

2.1. Late Feature Fusion and Random Weight Network

After extracting N features from one deep learning method and M features from another, two datasets are obtained. These datasets, combined in the forms of (N | M) and (M | N), are subsequently used for training and classification. The overall process of feature extraction, merging, training, and testing is illustrated in Figure 4.

In Figure 4, after feature extraction, the RWN is employed for the training and testing process. For the training of an RWN, the network must have a three-layered structure consisting of an input layer, a hidden layer, and an output layer [44,45,46,47]. The structure of the RWN architecture is shown in Figure 5.

In Figure 5, the features extracted from deep learning networks are provided to the input layer. The weights between the input layer and the hidden layer are randomly initialized. The Moore–Penrose approach is used between the hidden layer and the output layer. Unlike networks trained with iterative algorithms like backpropagation, an RWN calculates the output layer weights analytically in a single step. This non-iterative process results in significantly faster training time. In the current study, the RWN model and training algorithm proposed in [46,47] are used. In a feedforward artificial neural network with N neurons in the hidden layer and an activation function denoted as g(x), the output calculation $\left(O_j\right)$ for M training examples $(x_j,t_j)$ is performed using Equation (1).

$${\mathrm{O}}_{\mathrm{j}}=\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathsf{\beta }}_{\mathrm{i}}\mathrm{g}\left({{\mathrm{w}}_{\mathrm{i}}.\mathrm{x}}_{\mathrm{j}}+{\mathrm{b}}_{\mathrm{i}}\right),\mathrm{j}=1..\mathrm{M}$$

(1)

where ${x_j=\left[x_{j1},\dots {,x}_{jn}\right]}^T\in R^n$ represents the input vector, ${t_i=\left[t_{j1},\dots {,t}_{jm}\right]}^T\in R^m$ is the target label vector, ${w_i=\left[w_{i1},\dots {,w}_{in}\right]}^T$ is the weight vector between the input and hidden layers, ${\beta }_i={\left[{\beta }_{i1},\dots ,{\beta }_{im}\right]}^T$ is the weight vector between the hidden and output layers and $b_i$ denotes the bias of the $i-th$ hidden neuron. By considering the explanations mentioned above, the model can be written as follows.

$$\begin{gathered} H\beta =T \\ H=\left[\begin{array}{ccc}g\left(w_1.x_1+b_1\right)& \cdots & g\left(w_N.x_1+b_N\right)\\ ⋮& ⋮& ⋮\\ g\left(w_1.x_M+b_1\right)& \cdots & g\left(w_{\mathrm{N}}.x_M+b_N\right)\end{array}\right] \\ \beta =\left[\begin{array}{c}{\beta }_1^T\\ ⋮\\ {\beta }_{\mathrm{N}}^T\end{array}\right],T=\left[\begin{array}{c}{t}_1^T\\ ⋮\\ t_M^T\end{array}\right] \end{gathered}$$

(2)

In Equation (2), $H$ is the output matrix of the hidden layer. To train the feedforward artificial neural network, specific weights ${\beta }_i^{\prime }$ and $w_i^{\prime }$ are calculated using a gradient-based learning algorithm as follows:

$$‖H\left(w_1^{\prime },\dots ,w_N^{\prime },b_1^{\prime },\dots ,b_N^{\prime }\right)\beta -T‖=\underset{w_i,b_i,\beta }{\min }‖H\left(w_1,\dots ,w_N,\right)\beta -T‖$$

(3)

In the RWN, after the weights between the input layer and the hidden layer are randomly assigned, the weights ($\beta$) between the hidden layer and the output layer can be calculated as in Equation (4).

$$\beta =H^{+}T$$

(4)

where, $H^{+}$ is the Moore–Penrose generalized inverse matrix. In an RWN, if the number of training examples equals the number of hidden neurons, the hidden layer output matrix H becomes a square matrix, and its direct inverse can be computed without the need for the Moore–Penrose approach. However, since the number of training examples is generally greater than the number of neurons in the hidden layer, the Moore–Penrose approach is used to compute the inverse of the matrix. The reason for the fast training process of an RWN is shown in Equation (4), while iterative methods are used to determine the weights given in Equation (3). In the current study, the classification` performance of the RWN was measured using the accuracy value, which represents the ratio of correctly predicted samples by the model.

2.2. Dataset

The study focuses on the development of deep learning methods for the detection of concealed circuits within laptops and emphasizes feature fusion. In this context, there is a need for a dataset to train, test, and extract features from deep learning methods. To meet this requirement, a multi-step process was followed to generate the dataset. This process involved the following:

• Creating the circuits and/or elements to be concealed;
• Procuring a variety of laptops;
• Capturing X-ray images of the circuits after they were hidden inside the laptops.

Readily available Arduino boards were chosen as the example circuits for concealment. Due to the high prices of new laptops and the need for many different laptops, second-hand laptops were purchased from the market. With permission obtained from the Konya Airport Civil Administration Directorate, X-ray images of 60 laptops with different configurations were obtained using X-ray devices at the airport. X-ray images of laptops taken from different angles are provided in Figure 6. In Figure 6, the areas enclosed in red rectangles contain hidden circuits not belonging to the computer motherboard, and the rectangles are drawn manually.

As seen in Figure 6, the image of the laptop's motherboard and the embedded circuit is enclosed in a plastic box (standard practice at airports). Since the plastic transport box is irrelevant for classification, its image was removed from the background using a thresholding technique. This process also eliminated the bright yellow artifacts present in the images. In Figure 7, a clean X-ray image of the laptop is presented.

A total of 6395 X-ray images were captured. Among them, 2545 contain hidden circuits, while 3850 do not. To ensure balance in the data, the number of images without circuits was reduced to 2549, and a total of 5094 X-ray images were used in the experiments. Since the problem is treated as a classification problem, labels for the images are necessary during the training and testing processes with deep learning methods. The 5094 X-ray images were marked as normal or abnormal and stored in different folders. Due to variations in input dimensions in the literature analysis and the deep learning architectures used, the images were resized for each deep learning algorithm to match the input size. Although the width of the conveyor belt of the X-ray machine (number of columns in the image) is expressed as 704 pixels, the length varies due to the continuous flow behavior of the belt (this is not clearly seen in Figure 6 due to its white background). Therefore, as shown in Figure 7, the background and object images were segmented, and a clean image was obtained. This image was then adapted to fit the input of each deep learning method.

3. Experiments

All experiments were conducted on a workstation equipped with an AMD Ryzen 9 5950X 16-Core Processor and 64 GB of RAM. The MATLAB 2021a environment was used for all stages of the study, including feature extraction, feature fusion, and the training and evaluation of the classification models.

The feature extraction process requires deep learning models to first be trained on the dataset. A comprehensive performance analysis of 11 such architectures was previously presented in [1]. The results from that study showed that ShuffleNet achieved the highest test accuracy of 83.55%, followed by the InceptionV3 architecture at 81.31%. In this study, the aim was to achieve higher accuracy in classification using features extracted from these architectures. The number of features extracted from the methods in the [1] study is presented in

Table 2. The number of extracted features using deep learning models.

Models	Number of Extracted Features
EfficientNet	1280
ResNet18	512
ResNet50	2048
ResNet101	2048
DarkNet19	128
DarkNet53	1024
MobileNetV2	1280
ShuffleNet	544
InceptionV3	2048
DenseNet201	1920
Xception	2048

.

The features for the datasets obtained by combining the features given in Table 2 are provided in

Table 3. The number of combined features obtained from deep learning models.

Models	EfficientNet	ResNet18	ResNet50	ResNet101	DarkNet19	DarkNet53	MobileNetV2	ShuffleNet	InceptionV3	DenseNet201	Xception
EfficientNet	2560	1792	3328	3328	1408	2304	2560	1824	3328	3200	3328
ResNet18	1792	1024	2560	2560	640	1536	1792	1056	2560	2432	2560
ResNet50	3328	2560	4096	4096	2176	3072	3328	2592	4096	3968	4096
ResNet101	3328	2560	4096	4096	2176	3072	3328	2592	4096	3968	4096
DarkNet19	1408	640	2176	2176	256	1152	1408	672	2176	2048	2176
DarkNet53	2304	1536	3072	3072	1152	2048	2304	1568	3072	2944	3072
MobileNetV2	2560	1792	3328	3328	1408	2304	2560	1824	3328	3200	3328
ShuffleNet	1824	1056	2592	2592	672	1568	1824	1088	2592	2464	2592
InceptionV3	3328	2560	4096	4096	2176	3072	3328	2592	4096	3968	4096
DenseNet201	3200	2432	3968	3968	2048	2944	3200	2464	3968	3840	3968
Xception	3328	2560	4096	4096	2176	3072	3328	2592	4096	3968	4096

.

As seen in Table 3, besides using features from different architectures, the feature set from each architecture was concatenated with itself to investigate the effect of this repetition on classification performance. The datasets were used to conduct classification experiments with RWN, employing 10-fold cross-validation in each experiment. To account for the stochastic nature of the RWN, where input-to-hidden layer weights are randomly assigned, we repeated each 10-fold cross-validation experiment 30 times to obtain statistically robust performance measures. Our investigation focused on two key hyperparameters that govern RWN's behavior: the number of hidden layer neurons and the choice of activation function. The investigation began by evaluating the impact of the number of hidden neurons, testing the values as follows: 50, 100, 250, 500, 1000, 2000, and 4585. The value of 4585 was specifically chosen because it matches the number of training samples in each fold of our 10-fold cross-validation. When the number of hidden neurons equals the number of training samples, the hidden layer output matrix (H) becomes a square matrix, allowing its inverse to be calculated directly without requiring the Moore–Penrose pseudo-inverse method. Subsequently, the effect of the activation function, the second key hyperparameter, was investigated. For these experiments, the number of hidden neurons was fixed to the value that yielded the best average test accuracy from the previous stage. The activation functions evaluated include tangent sigmoid, sigmoid, sine, hard limit, triangular basis, and radial basis, all of which are commonly used with RWNs.

Firstly, the features given in Table 2 were reclassified using RWN and compared with the results of deep learning methods. Tangent sigmoid was used as the activation function in RWN, and the comparison results are presented in

Table 4. The comparison of deep learning models and RWN with different numbers of neurons in the hidden layer on the dataset.

Models	Models’ Result		RWN (50)		RWN (100)		RWN (250)		RWN (500)		RWN (1000)		RWN (2000)		RWN (4585)		SVM		TREE		KNN
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.9844	0.5913	0.8656	0.8615	0.8685	0.8602	0.8765	0.8573	0.8898	0.8518	0.9142	0.8411	0.9595	0.8150	1.0000	0.5688	0.8626	0.8624	0.9699	0.8013	0.8781	0.8528
ResNet18	0.9469	0.6588	0.8708	0.8696	0.8760	0.8725	0.8832	0.8743	0.8922	0.8724	0.9105	0.8629	0.9511	0.8244	1.0000	0.5481	0.8898	0.8787	0.9745	0.8129	0.8885	0.8657
ResNet50	0.9500	0.6258	0.8599	0.8562	0.8664	0.8619	0.8738	0.8625	0.8834	0.8585	0.9007	0.8466	0.9378	0.8008	1.0000	0.5487	0.8868	0.8751	0.9778	0.7972	0.8786	0.8557
ResNet101	0.9344	0.6101	0.8498	0.8482	0.8549	0.8516	0.8611	0.8491	0.8684	0.8456	0.8883	0.8286	0.9331	0.7784	1.0000	0.5253	0.8594	0.8573	0.9735	0.7683	0.8611	0.8341
DarkNet19	0.8969	0.7894	0.8809	0.8779	0.8854	0.8816	0.8934	0.8821	0.9039	0.8803	0.9242	0.8750	0.9663	0.8452	1.0000	0.5740	0.8947	0.8816	0.9723	0.8353	0.8988	0.8750
DarkNet53	0.7094	0.6698	0.7548	0.7509	0.7647	0.7571	0.7801	0.7601	0.7987	0.7572	0.8336	0.7445	0.8973	0.7072	0.9992	0.5654	0.7703	0.7635	0.9678	0.6837	0.8076	0.7479
MobileNetV2	0.9594	0.6023	0.8467	0.8446	0.8487	0.8441	0.8530	0.8407	0.8629	0.8350	0.8838	0.8219	0.9303	0.7784	1.0000	0.5273	0.8474	0.8445	0.9752	0.7780	0.8620	0.8276
ShuffleNet	0.9906	0.8355	0.9432	0.9424	0.9485	0.9465	0.9537	0.9482	0.9597	0.9477	0.9700	0.9467	0.9900	0.9393	1.0000	0.5745	0.9805	0.9362	0.9902	0.9124	0.9562	0.9476
InceptionV3	0.9219	0.8131	0.9290	0.9276	0.9319	0.9283	0.9390	0.9310	0.9474	0.9304	0.9604	0.9256	0.9831	0.9029	1.0000	0.5669	0.9326	0.9313	0.9853	0.8891	0.9375	0.9238
DenseNet201	0.8813	0.7772	0.9090	0.9072	0.9129	0.9094	0.9208	0.9111	0.9315	0.9107	0.9504	0.9067	0.9814	0.8849	1.0000	0.6113	0.8626	0.8624	0.9699	0.8013	0.8781	0.8528
Xception	0.9844	0.7126	0.9152	0.9126	0.9178	0.9126	0.9240	0.9112	0.9335	0.9072	0.9500	0.8996	0.9780	0.8739	1.0000	0.5733	0.8898	0.8787	0.9745	0.8129	0.8885	0.8657

.

Table 4 reveals a substantial improvement in test accuracy when features extracted from deep learning models are classified using an RWN. Specifically, while the best-performing standalone deep learning model (ShuffleNet) achieved an accuracy of 83.55%, this figure increased to 94.82% when using an RWN with 250 hidden neurons on the features extracted from the same ShuffleNet model. Additionally, the results indicate a trade-off related to the number of hidden neurons: increasing the neuron count improves training accuracy at the cost of decreasing test accuracy. This trend, which is indicative of overfitting, is illustrated for several model architectures in Figure 8.

A similar analysis using classical machine learning algorithms shows that features extracted from ShuffleNet consistently yield the best performance. Both SVM and KNN demonstrated strong generalization, achieving test accuracies of 93.62% and 94.76%, respectively, on the ShuffleNet features, results that are comparable to those of the RWN. In contrast, while the TREE model achieved high accuracy on the training set, its performance dropped significantly on the test set, clearly emphasizing its tendency to overfit. This comparison highlights the superior generalization capabilities of RWN, SVM, and KNN for this classification task.

A combined evaluation of Table 4 and Figure 8 indicates that increasing the number of hidden neurons leads to overfitting, where the network begins to memorize the training data rather than generalizing from it. The results show that setting the number of hidden neurons to 250 provides the best balance, yielding the highest average test accuracy and overall classification performance among the tested values. Given the dramatic decrease in test performance observed with 4585 neurons, a clear sign of severe overfitting, this value was excluded from subsequent experiments on the combined feature datasets. The results of the experiments using the remaining neuron counts on these combined datasets are presented in

Table 5. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 50).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8659	0.8618	0.9268	0.9249	0.9205	0.9185	0.9200	0.9183	0.9176	0.9152	0.8823	0.8796	0.9250	0.9238	0.9625	0.9609	0.9635	0.9624	0.9433	0.9418	0.9526	0.9513
ResNet18	0.9268	0.9256	0.8703	0.8684	0.9038	0.9009	0.9023	0.9006	0.9089	0.9066	0.8676	0.8645	0.9017	0.8994	0.9520	0.9506	0.9419	0.9404	0.9241	0.9218	0.9405	0.9385
ResNet50	0.9198	0.9187	0.9033	0.9005	0.8578	0.8555	0.8989	0.8958	0.9024	0.8999	0.8554	0.8518	0.9049	0.9028	0.9520	0.9500	0.9409	0.9388	0.9233	0.9210	0.9327	0.9303
ResNet101	0.9196	0.9180	0.9020	0.8997	0.8986	0.8961	0.8484	0.8458	0.8966	0.8948	0.8526	0.8495	0.9014	0.8985	0.9477	0.9459	0.9425	0.9409	0.9249	0.9233	0.9344	0.9332
DarkNet19	0.9171	0.9149	0.9083	0.9063	0.9027	0.8998	0.8960	0.8937	0.8808	0.8782	0.8764	0.8740	0.9046	0.9020	0.9405	0.9390	0.9125	0.9104	0.9027	0.9002	0.9208	0.9183
DarkNet53	0.8829	0.8814	0.8677	0.8642	0.8544	0.8523	0.8525	0.8492	0.8767	0.8741	0.7514	0.7469	0.8592	0.8576	0.9332	0.9313	0.9179	0.9160	0.8922	0.8906	0.9104	0.9084
MobileNetV2	0.9249	0.9238	0.9019	0.8997	0.9048	0.9023	0.9016	0.8995	0.9046	0.9015	0.8592	0.8574	0.8463	0.8437	0.9549	0.9536	0.9490	0.9476	0.9328	0.9317	0.9411	0.9400
ShuffleNet	0.9620	0.9613	0.9517	0.9499	0.9524	0.9515	0.9480	0.9467	0.9407	0.9388	0.9333	0.9312	0.9546	0.9535	0.9426	0.9410	0.9634	0.9621	0.9566	0.9544	0.9643	0.9629
InceptionV3	0.9634	0.9622	0.9420	0.9397	0.9410	0.9392	0.9424	0.9411	0.9117	0.9094	0.9174	0.9152	0.9493	0.9475	0.9634	0.9622	0.9285	0.9265	0.9536	0.9527	0.9604	0.9592
DenseNet201	0.9432	0.9420	0.9242	0.9220	0.9235	0.9212	0.9256	0.9231	0.9026	0.9001	0.8912	0.8886	0.9329	0.9311	0.9574	0.9561	0.9536	0.9520	0.9089	0.9067	0.9516	0.9504
Xception	0.9528	0.9517	0.9403	0.9387	0.9321	0.9306	0.9344	0.9322	0.9215	0.9192	0.9107	0.9091	0.9411	0.9401	0.9641	0.9630	0.9601	0.9589	0.9517	0.9500	0.9150	0.9119

,

Table 6. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 100).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8692	0.8596	0.9327	0.9293	0.9260	0.9226	0.9247	0.9217	0.9276	0.9237	0.8907	0.8868	0.9279	0.9256	0.9682	0.9658	0.9657	0.9639	0.9461	0.9428	0.9543	0.9518
ResNet18	0.9329	0.9300	0.8752	0.8717	0.9135	0.9077	0.9114	0.9081	0.9202	0.9165	0.8808	0.8764	0.9086	0.9051	0.9605	0.9578	0.9496	0.9477	0.9317	0.9284	0.9456	0.9423
ResNet50	0.9262	0.9232	0.9134	0.9086	0.8653	0.8609	0.9093	0.9047	0.9154	0.9111	0.8707	0.8641	0.9138	0.9094	0.9605	0.9584	0.9480	0.9449	0.9324	0.9289	0.9383	0.9347
ResNet101	0.9246	0.9213	0.9118	0.9076	0.9094	0.9043	0.8539	0.8490	0.9094	0.9049	0.8667	0.8612	0.9088	0.9050	0.9567	0.9546	0.9489	0.9465	0.9327	0.9296	0.9400	0.9364
DarkNet19	0.9269	0.9227	0.9206	0.9166	0.9155	0.9109	0.9091	0.9039	0.8855	0.8809	0.8839	0.8792	0.9159	0.9108	0.9533	0.9508	0.9250	0.9217	0.9137	0.9091	0.9320	0.9276
DarkNet53	0.8901	0.8866	0.8808	0.8764	0.8707	0.8646	0.8665	0.8616	0.8839	0.8793	0.7617	0.7548	0.8721	0.8670	0.9447	0.9419	0.9286	0.9256	0.9031	0.8992	0.9182	0.9145
MobileNetV2	0.9279	0.9249	0.9087	0.9049	0.9137	0.9095	0.9090	0.9048	0.9160	0.9120	0.8722	0.8676	0.8487	0.8438	0.9619	0.9596	0.9532	0.9512	0.9372	0.9354	0.9445	0.9416
ShuffleNet	0.9683	0.9664	0.9603	0.9579	0.9606	0.9585	0.9565	0.9541	0.9535	0.9505	0.9446	0.9418	0.9623	0.9602	0.9485	0.9460	0.9705	0.9685	0.9629	0.9610	0.9702	0.9680
InceptionV3	0.9658	0.9640	0.9496	0.9462	0.9481	0.9459	0.9493	0.9469	0.9251	0.9205	0.9291	0.9251	0.9531	0.9506	0.9706	0.9681	0.9313	0.9283	0.9570	0.9547	0.9629	0.9605
DenseNet201	0.9462	0.9427	0.9315	0.9277	0.9322	0.9293	0.9324	0.9293	0.9136	0.9087	0.9027	0.8995	0.9373	0.9342	0.9628	0.9604	0.9570	0.9546	0.9126	0.9087	0.9549	0.9525
Xception	0.9543	0.9520	0.9458	0.9430	0.9382	0.9351	0.9401	0.9364	0.9321	0.9280	0.9181	0.9147	0.9446	0.9415	0.9702	0.9680	0.9628	0.9608	0.9550	0.9525	0.9178	0.9113

,

Table 7. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 250).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8780	0.8559	0.9396	0.9318	0.9325	0.9248	0.9299	0.9226	0.9378	0.9289	0.8984	0.8901	0.9325	0.9249	0.9736	0.9694	0.9687	0.9639	0.9516	0.9438	0.9567	0.9510
ResNet18	0.9395	0.9319	0.8821	0.8735	0.9235	0.9134	0.9218	0.9111	0.9315	0.9221	0.8931	0.8823	0.9163	0.9069	0.9673	0.9620	0.9570	0.9499	0.9392	0.9317	0.9514	0.9454
ResNet50	0.9324	0.9249	0.9238	0.9127	0.8729	0.8619	0.9194	0.9088	0.9287	0.9189	0.8843	0.8716	0.9220	0.9124	0.9672	0.9625	0.9543	0.9475	0.9406	0.9334	0.9438	0.9365
ResNet101	0.9301	0.9218	0.9214	0.9114	0.9197	0.9085	0.8604	0.8495	0.9240	0.9127	0.8787	0.8680	0.9172	0.9079	0.9640	0.9589	0.9560	0.9500	0.9406	0.9333	0.9463	0.9383
DarkNet19	0.9376	0.9290	0.9317	0.9215	0.9288	0.9188	0.9239	0.9132	0.8931	0.8822	0.8944	0.8819	0.9281	0.9174	0.9641	0.9569	0.9388	0.9296	0.9257	0.9156	0.9426	0.9346
DarkNet53	0.8986	0.8898	0.8928	0.8826	0.8844	0.8724	0.8787	0.8674	0.8942	0.8831	0.7771	0.7579	0.8842	0.8733	0.9537	0.9485	0.9379	0.9316	0.9138	0.9054	0.9251	0.9170
MobileNetV2	0.9322	0.9251	0.9164	0.9073	0.9220	0.9124	0.9174	0.9079	0.9279	0.9170	0.8845	0.8737	0.8533	0.8406	0.9680	0.9638	0.9580	0.9519	0.9432	0.9372	0.9490	0.9418
ShuffleNet	0.9737	0.9694	0.9673	0.9624	0.9671	0.9621	0.9642	0.9588	0.9640	0.9571	0.9537	0.9477	0.9680	0.9634	0.9536	0.9478	0.9767	0.9724	0.9685	0.9641	0.9750	0.9708
InceptionV3	0.9687	0.9647	0.9570	0.9502	0.9544	0.9484	0.9559	0.9495	0.9387	0.9293	0.9381	0.9315	0.9580	0.9517	0.9768	0.9725	0.9385	0.9290	0.9618	0.9559	0.9662	0.9601
DenseNet201	0.9515	0.9442	0.9393	0.9307	0.9407	0.9321	0.9408	0.9331	0.9257	0.9159	0.9139	0.9059	0.9432	0.9370	0.9686	0.9640	0.9618	0.9559	0.9206	0.9097	0.9599	0.9539
Xception	0.9568	0.9494	0.9515	0.9446	0.9439	0.9360	0.9463	0.9387	0.9422	0.9340	0.9253	0.9175	0.9490	0.9411	0.9751	0.9705	0.9660	0.9604	0.9599	0.9539	0.9239	0.9100

,

Table 8. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 500).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8923	0.8512	0.9461	0.9319	0.9389	0.9230	0.9362	0.9213	0.9477	0.9292	0.9062	0.8891	0.9385	0.9233	0.9782	0.9709	0.9732	0.9639	0.9598	0.9430	0.9615	0.9481
ResNet18	0.9461	0.9319	0.8910	0.8711	0.9330	0.9126	0.9303	0.9113	0.9422	0.9236	0.9039	0.8823	0.9239	0.9054	0.9726	0.9634	0.9634	0.9503	0.9474	0.9320	0.9576	0.9444
ResNet50	0.9391	0.9225	0.9329	0.9126	0.8817	0.8581	0.9290	0.9081	0.9401	0.9198	0.8950	0.8704	0.9298	0.9113	0.9725	0.9635	0.9605	0.9465	0.9478	0.9323	0.9495	0.9346
ResNet101	0.9363	0.9211	0.9306	0.9122	0.9289	0.9072	0.8681	0.8441	0.9364	0.9143	0.8888	0.8663	0.9249	0.9061	0.9705	0.9609	0.9619	0.9495	0.9487	0.9340	0.9530	0.9365
DarkNet19	0.9479	0.9299	0.9421	0.9246	0.9402	0.9200	0.9362	0.9154	0.9037	0.8812	0.9057	0.8813	0.9386	0.9194	0.9713	0.9592	0.9496	0.9323	0.9367	0.9170	0.9516	0.9350
DarkNet53	0.9062	0.8880	0.9037	0.8832	0.8951	0.8713	0.8884	0.8660	0.9056	0.8824	0.7933	0.7553	0.8945	0.8729	0.9602	0.9505	0.9453	0.9320	0.9226	0.9064	0.9319	0.9155
MobileNetV2	0.9387	0.9239	0.9240	0.9051	0.9298	0.9111	0.9251	0.9063	0.9384	0.9192	0.8949	0.8727	0.8628	0.8354	0.9731	0.9646	0.9635	0.9515	0.9502	0.9367	0.9553	0.9394
ShuffleNet	0.9781	0.9711	0.9727	0.9641	0.9726	0.9641	0.9705	0.9607	0.9714	0.9596	0.9602	0.9500	0.9732	0.9643	0.9595	0.9481	0.9816	0.9743	0.9739	0.9653	0.9795	0.9719
InceptionV3	0.9733	0.9632	0.9636	0.9509	0.9606	0.9469	0.9618	0.9494	0.9495	0.9327	0.9452	0.9331	0.9636	0.9509	0.9816	0.9744	0.9469	0.9299	0.9680	0.9560	0.9707	0.9593
DenseNet201	0.9596	0.9440	0.9472	0.9323	0.9479	0.9319	0.9487	0.9344	0.9367	0.9175	0.9224	0.9063	0.9502	0.9379	0.9740	0.9655	0.9679	0.9562	0.9313	0.9104	0.9658	0.9546
Xception	0.9613	0.9486	0.9576	0.9445	0.9497	0.9346	0.9531	0.9369	0.9521	0.9352	0.9320	0.9148	0.9553	0.9397	0.9794	0.9718	0.9708	0.9591	0.9658	0.9543	0.9331	0.9068

,

Table 9. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 1000).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.9183	0.8410	0.9577	0.9278	0.9503	0.9178	0.9477	0.9167	0.9629	0.9287	0.9201	0.8767	0.9519	0.9181	0.9847	0.9706	0.9822	0.9601	0.9739	0.9410	0.9718	0.9433
ResNet18	0.9577	0.9271	0.9081	0.8615	0.9478	0.9051	0.9452	0.9051	0.9589	0.9212	0.9210	0.8746	0.9381	0.8985	0.9803	0.9627	0.9729	0.9485	0.9604	0.9285	0.9684	0.9397
ResNet50	0.9504	0.9164	0.9476	0.9070	0.8984	0.8445	0.9443	0.9012	0.9576	0.9183	0.9127	0.8614	0.9430	0.9035	0.9805	0.9634	0.9707	0.9423	0.9592	0.9266	0.9597	0.9265
ResNet101	0.9479	0.9137	0.9452	0.9054	0.9445	0.9001	0.8868	0.8254	0.9544	0.9129	0.9061	0.8544	0.9391	0.8990	0.9796	0.9603	0.9714	0.9468	0.9618	0.9310	0.9652	0.9320
DarkNet19	0.9631	0.9281	0.9587	0.9216	0.9575	0.9166	0.9544	0.9129	0.9246	0.8734	0.9268	0.8758	0.9554	0.9160	0.9817	0.9599	0.9649	0.9316	0.9547	0.9138	0.9670	0.9341
DarkNet53	0.9200	0.8768	0.9211	0.8760	0.9127	0.8601	0.9062	0.8535	0.9268	0.8763	0.8240	0.7409	0.9112	0.8622	0.9696	0.9493	0.9555	0.9285	0.9368	0.9005	0.9434	0.9091
MobileNetV2	0.9518	0.9185	0.9379	0.8986	0.9430	0.9040	0.9392	0.8987	0.9553	0.9162	0.9114	0.8620	0.8832	0.8203	0.9812	0.9635	0.9729	0.9481	0.9623	0.9344	0.9663	0.9334
ShuffleNet	0.9848	0.9708	0.9802	0.9628	0.9806	0.9635	0.9796	0.9607	0.9818	0.9588	0.9696	0.9489	0.9812	0.9636	0.9702	0.9474	0.9881	0.9743	0.9825	0.9660	0.9862	0.9714
InceptionV3	0.9822	0.9612	0.9730	0.9487	0.9707	0.9428	0.9715	0.9470	0.9649	0.9303	0.9556	0.9279	0.9728	0.9486	0.9879	0.9744	0.9607	0.9225	0.9790	0.9550	0.9802	0.9540
DenseNet201	0.9739	0.9411	0.9604	0.9288	0.9593	0.9268	0.9617	0.9305	0.9545	0.9143	0.9372	0.8994	0.9622	0.9343	0.9826	0.9660	0.9790	0.9553	0.9509	0.9044	0.9767	0.9507
Xception	0.9718	0.9422	0.9684	0.9403	0.9597	0.9271	0.9650	0.9329	0.9670	0.9331	0.9434	0.9074	0.9664	0.9333	0.9862	0.9712	0.9801	0.9538	0.9767	0.9515	0.9498	0.8948

and

Table 10. The performance of RWN on the combined dataset (The number of neurons in the hidden layer was set to 2000).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.9619	0.8147	0.9798	0.9074	0.9730	0.8910	0.9732	0.8888	0.9867	0.9141	0.9483	0.8343	0.9768	0.8953	0.9964	0.9683	0.9951	0.9516	0.9934	0.9306	0.9898	0.9218
ResNet18	0.9799	0.9078	0.9471	0.8233	0.9723	0.8815	0.9735	0.8816	0.9859	0.9060	0.9558	0.8421	0.9668	0.8712	0.9936	0.9584	0.9889	0.9355	0.9830	0.9121	0.9860	0.9254
ResNet50	0.9730	0.8903	0.9725	0.8806	0.9328	0.8021	0.9714	0.8722	0.9859	0.8996	0.9465	0.8206	0.9687	0.8737	0.9931	0.9587	0.9867	0.9264	0.9802	0.9048	0.9793	0.9034
ResNet101	0.9731	0.8872	0.9734	0.8821	0.9713	0.8725	0.9303	0.7768	0.9837	0.8936	0.9436	0.8080	0.9677	0.8684	0.9934	0.9558	0.9890	0.9319	0.9838	0.9149	0.9848	0.9141
DarkNet19	0.9866	0.9147	0.9860	0.9058	0.9859	0.9024	0.9838	0.8946	0.9667	0.8455	0.9681	0.8496	0.9845	0.8991	0.9954	0.9544	0.9882	0.9187	0.9839	0.8978	0.9900	0.9222
DarkNet53	0.9485	0.8357	0.9557	0.8422	0.9466	0.8216	0.9434	0.8077	0.9682	0.8476	0.8824	0.7082	0.9440	0.8195	0.9875	0.9405	0.9747	0.9029	0.9650	0.8659	0.9661	0.8764
MobileNetV2	0.9765	0.8962	0.9669	0.8727	0.9688	0.8732	0.9678	0.8704	0.9845	0.8974	0.9440	0.8199	0.9280	0.7756	0.9947	0.9582	0.9888	0.9342	0.9827	0.9177	0.9861	0.9133
ShuffleNet	0.9964	0.9684	0.9936	0.9576	0.9931	0.9576	0.9934	0.9552	0.9953	0.9539	0.9874	0.9412	0.9947	0.9589	0.9901	0.9393	0.9969	0.9716	0.9958	0.9632	0.9962	0.9684
InceptionV3	0.9951	0.9520	0.9890	0.9367	0.9867	0.9249	0.9890	0.9321	0.9882	0.9187	0.9747	0.9027	0.9889	0.9342	0.9968	0.9723	0.9833	0.9001	0.9944	0.9472	0.9939	0.9399
DenseNet201	0.9933	0.9307	0.9829	0.9120	0.9804	0.9041	0.9838	0.9138	0.9838	0.8984	0.9648	0.8649	0.9830	0.9183	0.9958	0.9636	0.9945	0.9483	0.9824	0.8834	0.9933	0.9385
Xception	0.9898	0.9250	0.9861	0.9249	0.9792	0.9038	0.9846	0.9137	0.9901	0.9225	0.9659	0.8768	0.9861	0.9130	0.9962	0.9687	0.9939	0.9390	0.9931	0.9391	0.9778	0.8677

.

The extensive feature combinations detailed in Table 3 were designed to systematically investigate the principles of an effective fusion strategy. Our analysis of the subsequent classification results (Table 5, Table 6, Table 7, Table 8, Table 9, Table 10,

Table 11. The performance of SVM on the combined dataset.

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8629	0.8626	0.9461	0.9358	0.9408	0.9301	0.9285	0.9248	0.9458	0.9356	0.8989	0.8958	0.9304	0.9268	0.9962	0.9592	0.9677	0.9658	0.9521	0.9462	0.9540	0.9521
ResNet18	0.9461	0.9358	0.8930	0.8791	0.9429	0.9223	0.9356	0.9201	0.9510	0.9323	0.9136	0.9007	0.9249	0.9130	0.9962	0.9519	0.9654	0.9568	0.9524	0.9405	0.9576	0.9474
ResNet50	0.9408	0.9301	0.9429	0.9223	0.8914	0.8763	0.9325	0.9179	0.9508	0.9299	0.9053	0.8891	0.9335	0.9195	0.9966	0.9549	0.9667	0.9566	0.9539	0.9403	0.9489	0.9403
ResNet101	0.9285	0.9248	0.9356	0.9201	0.9325	0.9179	0.8616	0.8557	0.9399	0.9278	0.8831	0.8751	0.9175	0.9101	0.9920	0.9490	0.9601	0.9564	0.9504	0.9425	0.9464	0.9431
DarkNet19	0.9458	0.9356	0.9510	0.9323	0.9508	0.9297	0.9398	0.9280	0.8948	0.8844	0.9056	0.8938	0.9383	0.9268	0.9976	0.9499	0.9546	0.9439	0.9500	0.9350	0.9523	0.9419
DarkNet53	0.8989	0.8958	0.9136	0.9007	0.9053	0.8891	0.8831	0.8751	0.9055	0.8938	0.7719	0.7631	0.8906	0.8838	0.9851	0.9388	0.9427	0.9391	0.9253	0.9191	0.9245	0.9201
MobileNetV2	0.9304	0.9268	0.9249	0.9130	0.9335	0.9195	0.9175	0.9101	0.9382	0.9268	0.8906	0.8838	0.8475	0.8430	0.9957	0.9539	0.9603	0.9566	0.9502	0.9444	0.9479	0.9423
ShuffleNet	0.9962	0.9592	0.9962	0.9519	0.9966	0.9549	0.9920	0.9490	0.9976	0.9499	0.9851	0.9388	0.9957	0.9539	0.9840	0.9307	0.9991	0.9668	0.9967	0.9584	0.9979	0.9617
InceptionV3	0.9677	0.9658	0.9654	0.9568	0.9667	0.9566	0.9601	0.9564	0.9546	0.9439	0.9427	0.9391	0.9603	0.9566	0.9991	0.9668	0.9328	0.9305	0.9668	0.9609	0.9648	0.9625
DenseNet201	0.9521	0.9462	0.9524	0.9405	0.9539	0.9403	0.9504	0.9425	0.9500	0.9350	0.9253	0.9191	0.9502	0.9444	0.9967	0.9584	0.9668	0.9609	0.9231	0.9170	0.9615	0.9543
Xception	0.9539	0.9521	0.9576	0.9474	0.9489	0.9403	0.9464	0.9431	0.9523	0.9417	0.9245	0.9201	0.9479	0.9423	0.9978	0.9617	0.9648	0.9625	0.9615	0.9543	0.9148	0.9140

,

Table 12. The performance of TREE on the combined dataset.

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.9709	0.8013	0.9872	0.8938	0.9881	0.8863	0.9866	0.8696	0.9855	0.8981	0.9810	0.8341	0.9869	0.8791	0.9944	0.9431	0.9927	0.9417	0.9929	0.9181	0.9896	0.9264
ResNet18	0.9872	0.8930	0.9759	0.8100	0.9870	0.8691	0.9860	0.8687	0.9853	0.8765	0.9817	0.8271	0.9833	0.8498	0.9927	0.9403	0.9916	0.9268	0.9904	0.8960	0.9910	0.9144
ResNet50	0.9881	0.8842	0.9873	0.8720	0.9791	0.7974	0.9868	0.8679	0.9872	0.8738	0.9815	0.8009	0.9873	0.8645	0.9926	0.9338	0.9923	0.9230	0.9901	0.8967	0.9906	0.8993
ResNet101	0.9866	0.8704	0.9859	0.8714	0.9868	0.8689	0.9747	0.7735	0.9842	0.8712	0.9806	0.8100	0.9858	0.8628	0.9930	0.9313	0.9907	0.9219	0.9906	0.8924	0.9892	0.9166
DarkNet19	0.9856	0.8965	0.9855	0.8806	0.9870	0.8700	0.9844	0.8734	0.9715	0.8255	0.9809	0.8204	0.9849	0.8767	0.9922	0.9335	0.9894	0.9179	0.9884	0.8895	0.9882	0.9117
DarkNet53	0.9811	0.8347	0.9821	0.8271	0.9813	0.8001	0.9804	0.8076	0.9809	0.8214	0.9698	0.6902	0.9793	0.8153	0.9908	0.9091	0.9895	0.9022	0.9858	0.8655	0.9854	0.8722
MobileNetV2	0.9868	0.8801	0.9833	0.8479	0.9872	0.8643	0.9859	0.8620	0.9847	0.8752	0.9796	0.8143	0.9751	0.7629	0.9932	0.9268	0.9911	0.9280	0.9883	0.9009	0.9908	0.9117
ShuffleNet	0.9944	0.9423	0.9926	0.9401	0.9928	0.9352	0.9930	0.9309	0.9921	0.9329	0.9906	0.9126	0.9932	0.9301	0.9894	0.9166	0.9954	0.9550	0.9933	0.9403	0.9950	0.9490
InceptionV3	0.9928	0.9407	0.9916	0.9248	0.9921	0.9240	0.9908	0.9199	0.9894	0.9174	0.9893	0.9042	0.9911	0.9264	0.9954	0.9552	0.9854	0.8881	0.9925	0.9287	0.9934	0.9321
DenseNet201	0.9929	0.9168	0.9904	0.8967	0.9899	0.8963	0.9903	0.8993	0.9882	0.8861	0.9863	0.8671	0.9884	0.9003	0.9933	0.9393	0.9923	0.9276	0.9826	0.8604	0.9918	0.9264
Xception	0.9896	0.9278	0.9913	0.9134	0.9908	0.8979	0.9891	0.9150	0.9881	0.9103	0.9855	0.8730	0.9908	0.9101	0.9950	0.9484	0.9933	0.9327	0.9918	0.9258	0.9829	0.8628

and

Table 13. The performance of KNN on the combined dataset.

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8785	0.8524	0.9398	0.9291	0.9357	0.9244	0.9307	0.9172	0.9361	0.9252	0.8992	0.8775	0.9335	0.9162	0.9759	0.9721	0.9662	0.9613	0.9523	0.9382	0.9579	0.9490
ResNet18	0.9398	0.9291	0.8881	0.8651	0.9258	0.9107	0.9259	0.9089	0.9316	0.9164	0.8984	0.8803	0.9197	0.9028	0.9696	0.9643	0.9589	0.9490	0.9413	0.9272	0.9527	0.9431
ResNet50	0.9357	0.9244	0.9258	0.9107	0.8784	0.8539	0.9210	0.9073	0.9240	0.9107	0.8911	0.8663	0.9273	0.9105	0.9705	0.9633	0.9544	0.9443	0.9435	0.9338	0.9431	0.9352
ResNet101	0.9307	0.9172	0.9259	0.9089	0.9210	0.9073	0.8608	0.8339	0.9170	0.8965	0.8897	0.8657	0.9176	0.9022	0.9690	0.9645	0.9569	0.9478	0.9447	0.9307	0.9475	0.9360
DarkNet19	0.9361	0.9252	0.9316	0.9164	0.9240	0.9107	0.9170	0.8965	0.8990	0.8769	0.8998	0.8816	0.9243	0.9081	0.9619	0.9517	0.9286	0.9152	0.9196	0.9038	0.9389	0.9248
DarkNet53	0.8992	0.8775	0.8984	0.8803	0.8911	0.8663	0.8897	0.8657	0.8998	0.8816	0.8070	0.7513	0.8898	0.8645	0.9571	0.9478	0.9416	0.9315	0.9201	0.9050	0.9273	0.9117
MobileNetV2	0.9335	0.9162	0.9197	0.9028	0.9273	0.9105	0.9176	0.9022	0.9243	0.9081	0.8898	0.8645	0.8610	0.8304	0.9693	0.9647	0.9578	0.9488	0.9463	0.9382	0.9494	0.9390
ShuffleNet	0.9759	0.9721	0.9696	0.9643	0.9705	0.9633	0.9690	0.9645	0.9619	0.9517	0.9571	0.9478	0.9693	0.9647	0.9563	0.9468	0.9787	0.9715	0.9723	0.9656	0.9764	0.9709
InceptionV3	0.9662	0.9613	0.9589	0.9490	0.9544	0.9443	0.9569	0.9478	0.9286	0.9152	0.9416	0.9315	0.9578	0.9488	0.9787	0.9715	0.9374	0.9248	0.9620	0.9560	0.9649	0.9590
DenseNet201	0.9523	0.9382	0.9413	0.9272	0.9435	0.9338	0.9447	0.9307	0.9196	0.9038	0.9201	0.9050	0.9463	0.9382	0.9723	0.9656	0.9620	0.9560	0.9200	0.9016	0.9603	0.9503
Xception	0.9579	0.9490	0.9527	0.9431	0.9431	0.9352	0.9475	0.9360	0.9389	0.9248	0.9273	0.9117	0.9494	0.9390	0.9764	0.9709	0.9649	0.9590	0.9603	0.9503	0.9207	0.9005

) revealed several key patterns. Firstly, experiments involving self-combination (e.g., ShuffleNet features fused with themselves) demonstrated no significant performance improvement over using the single feature set with an RWN. This critical finding indicates that merely increasing feature quantity is insufficient; feature diversity is a crucial driver of success. Secondly, the most substantial accuracy gains came from a synergistic fusion of features from the top-performing individual models, ShuffleNet and InceptionV3, where their distinct representational strengths complemented each other to create a more robust and discriminative feature space. This synergy proved more impactful than raw feature dimensionality alone, as this combination outperformed fusions with a higher total feature count. Finally, our tests also confirmed that the order of feature concatenation (e.g., N|M vs. M|N) had a negligible impact on the final classification outcome.

A holistic analysis of Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11, Table 12 and Table 13 reveals two key trends. First, duplicating the feature set by merging a dataset with itself does not yield a significant improvement in test performance. Second, as previously noted, increasing the number of neurons in the RWN's hidden layer consistently improves training performance, often at the expense of test performance. Furthermore, a critical finding is that fusing features from the individually best-performing deep learning models, notably ShuffleNet and InceptionV3, leads to the highest classification accuracies. This specific combination consistently produced the top results across different classifiers. Specifically, on the fused ShuffleNet-InceptionV3 feature set, several classifiers achieved high training accuracies, with SVM achieving 99.91%, TREE 99.54%, an RWN with 2000 neurons 99.69%, and KNN 97.87%. The highest test accuracy of 97.43% is achieved when the RWN hidden layer neuron count is set to 500 or 1000, and features extracted from Inception and ShuffleNet architectures are combined. As detailed in the overall performance comparison in

Table 14. Overall comparison of success with different neuron counts on combined datasets.

Metrics	The Number of Neurons in the Hidden Layer of RWN						SVM	TREE	KNN
	50	100	250	500	1000	2000
The Best Training Accuracy	0.9643	0.9706	0.9768	0.9816	0.9881	0.9969	0.9991	0.9954	0.9787
The Best Test Accuracy	0.9630	0.9685	0.9725	0.9744	0.9744	0.9723	0.9668	0.9550	0.9721
Average Training Accuracy	0.9185	0.9262	0.9342	0.9420	0.9551	0.9783	0.9450	0.9877	0.9358
Average Test Accuracy	0.9166	0.9227	0.9261	0.9259	0.9212	0.9014	0.9350	0.8881	0.9224

, other classifiers like SVM and KNN also achieved high performance on the fused feature sets, though the RWN provided a superior balance of accuracy and efficiency.

Having established the impact of the hidden neuron count, the investigation now shifts to evaluating the influence of the second key hyperparameter: the activation function. To conduct this analysis, the number of hidden neurons was fixed at 250. This value was chosen based on the results in Table 14, as it yielded the highest average test accuracy in the previous experiments. The performances of the different activation functions on the combined datasets are subsequently presented in

Table 15. The performance analysis of RWN on the combined dataset (Activation Function: Sigmoid).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8758	0.8593	0.9409	0.9340	0.9333	0.9268	0.9307	0.9231	0.9390	0.9306	0.9000	0.8919	0.9326	0.9251	0.9742	0.9703	0.9691	0.9649	0.9518	0.9445	0.9565	0.9512
ResNet18	0.9410	0.9336	0.8843	0.8755	0.9258	0.9154	0.9235	0.9140	0.9326	0.9240	0.8960	0.8856	0.9177	0.9088	0.9679	0.9630	0.9585	0.9524	0.9408	0.9338	0.9522	0.9461
ResNet50	0.9334	0.9265	0.9256	0.9148	0.8751	0.8640	0.9215	0.9113	0.9301	0.9194	0.8883	0.8754	0.9236	0.9149	0.9679	0.9630	0.9559	0.9492	0.9426	0.9343	0.9447	0.9382
ResNet101	0.9306	0.9230	0.9234	0.9138	0.9214	0.9109	0.8614	0.8512	0.9255	0.9147	0.8809	0.8706	0.9188	0.9108	0.9647	0.9596	0.9575	0.9516	0.9425	0.9343	0.9473	0.9396
DarkNet19	0.9391	0.9309	0.9326	0.9232	0.9303	0.9202	0.9253	0.9152	0.8936	0.8821	0.8947	0.8835	0.9288	0.9194	0.9643	0.9579	0.9398	0.9312	0.9268	0.9174	0.9432	0.9343
DarkNet53	0.9001	0.8917	0.8963	0.8870	0.8880	0.8767	0.8812	0.8703	0.8947	0.8830	0.7822	0.7612	0.8879	0.8771	0.9548	0.9495	0.9404	0.9344	0.9165	0.9094	0.9264	0.9190
MobileNetV2	0.9326	0.9259	0.9178	0.9092	0.9235	0.9150	0.9187	0.9109	0.9290	0.9194	0.8878	0.8776	0.8525	0.8406	0.9685	0.9642	0.9590	0.9536	0.9443	0.9384	0.9493	0.9424
ShuffleNet	0.9741	0.9703	0.9680	0.9635	0.9678	0.9632	0.9647	0.9594	0.9644	0.9580	0.9549	0.9487	0.9686	0.9644	0.9539	0.9483	0.9774	0.9740	0.9690	0.9643	0.9755	0.9712
InceptionV3	0.9691	0.9646	0.9585	0.9523	0.9561	0.9500	0.9573	0.9515	0.9397	0.9309	0.9404	0.9338	0.9592	0.9534	0.9773	0.9735	0.9394	0.9313	0.9626	0.9567	0.9665	0.9613
DenseNet201	0.9519	0.9447	0.9409	0.9337	0.9422	0.9353	0.9426	0.9352	0.9269	0.9182	0.9168	0.9095	0.9444	0.9382	0.9690	0.9645	0.9626	0.9567	0.9216	0.9119	0.9605	0.9550
Xception	0.9565	0.9512	0.9522	0.9461	0.9447	0.9384	0.9473	0.9395	0.9431	0.9355	0.9264	0.9188	0.9491	0.9423	0.9755	0.9715	0.9663	0.9619	0.9605	0.9552	0.9236	0.9121

,

Table 16. The performance analysis of RWN on the combined dataset (Activation Function: Sine).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8689	0.8606	0.8112	0.7824	0.8339	0.8053	0.8715	0.8491	0.5963	0.5044	0.8705	0.8487	0.9149	0.9027	0.5964	0.5033	0.9502	0.9397	0.9074	0.8910	0.9430	0.9332
ResNet18	0.8117	0.7830	0.7774	0.7450	0.6998	0.6547	0.7329	0.6924	0.5968	0.5036	0.7741	0.7424	0.7574	0.7210	0.5964	0.5056	0.7688	0.7338	0.7464	0.7062	0.7819	0.7505
ResNet50	0.8339	0.8080	0.6968	0.6503	0.7869	0.7552	0.7679	0.7323	0.5962	0.5028	0.7779	0.7440	0.7905	0.7581	0.5962	0.5037	0.8049	0.7751	0.7823	0.7488	0.8107	0.7804
ResNet101	0.8724	0.8499	0.7335	0.6948	0.7672	0.7305	0.8266	0.8018	0.5960	0.5049	0.8021	0.7690	0.8301	0.8029	0.5960	0.5027	0.8547	0.8304	0.8230	0.7948	0.8573	0.8327
DarkNet19	0.5966	0.5058	0.5958	0.5025	0.5960	0.5050	0.5964	0.5018	0.5966	0.5049	0.5963	0.5023	0.5963	0.5036	0.5961	0.5022	0.5964	0.5020	0.5964	0.5038	0.5961	0.5008
DarkNet53	0.8700	0.8471	0.7736	0.7382	0.7778	0.7425	0.8006	0.7673	0.5961	0.5053	0.7705	0.7375	0.8364	0.8109	0.5962	0.5019	0.8823	0.8595	0.8454	0.8197	0.8756	0.8526
MobileNetV2	0.9147	0.9023	0.7575	0.7223	0.7898	0.7576	0.8304	0.8042	0.5963	0.5020	0.8360	0.8108	0.8473	0.8333	0.5962	0.5034	0.9056	0.8887	0.8622	0.8389	0.9058	0.8904
ShuffleNet	0.5956	0.5037	0.5960	0.5024	0.5967	0.5015	0.5969	0.5029	0.5967	0.5007	0.5966	0.5025	0.5961	0.5028	0.5963	0.5031	0.5966	0.5049	0.5966	0.5023	0.5971	0.5009
InceptionV3	0.9510	0.9419	0.7677	0.7325	0.8064	0.7759	0.8551	0.8288	0.5963	0.5052	0.8826	0.8602	0.9053	0.8887	0.5961	0.5038	0.9317	0.9246	0.8777	0.8562	0.9235	0.9089
DenseNet201	0.9071	0.8900	0.7447	0.7068	0.7827	0.7484	0.8218	0.7941	0.5966	0.5038	0.8454	0.8205	0.8614	0.8394	0.5964	0.5016	0.8746	0.8525	0.8862	0.8705	0.8812	0.8603
Xception	0.9433	0.9344	0.7789	0.7446	0.8102	0.7816	0.8576	0.8342	0.5962	0.5040	0.8763	0.8533	0.9051	0.8903	0.5963	0.5018	0.9230	0.9089	0.8816	0.8608	0.9174	0.9082

,

Table 17. The performance analysis of RWN on the combined dataset (Activation Function: Tribas).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8785	0.8516	0.9241	0.9146	0.9171	0.9075	0.9166	0.9076	0.6526	0.5954	0.8792	0.8702	0.9281	0.9207	0.9329	0.9188	0.9650	0.9593	0.9287	0.9132	0.9545	0.9466
ResNet18	0.9243	0.9146	0.8674	0.8566	0.8916	0.8771	0.8874	0.8745	0.7262	0.6838	0.8509	0.8365	0.8986	0.8863	0.9122	0.8973	0.9359	0.9268	0.9214	0.9106	0.9369	0.9271
ResNet50	0.9174	0.9080	0.8918	0.8775	0.8544	0.8436	0.8847	0.8705	0.7277	0.6844	0.8395	0.8227	0.9016	0.8911	0.9124	0.8960	0.9357	0.9267	0.9211	0.9111	0.9278	0.9184
ResNet101	0.9164	0.9060	0.8876	0.8748	0.8853	0.8720	0.8461	0.8364	0.7006	0.6525	0.8368	0.8208	0.8964	0.8868	0.9031	0.8865	0.9363	0.9277	0.9205	0.9099	0.9294	0.9198
DarkNet19	0.6524	0.5961	0.7260	0.6832	0.7266	0.6846	0.7015	0.6560	0.6374	0.5758	0.7117	0.6656	0.7013	0.6535	0.7312	0.6925	0.6839	0.6333	0.6459	0.5855	0.6975	0.6476
DarkNet53	0.8795	0.8694	0.8511	0.8349	0.8393	0.8230	0.8374	0.8207	0.7120	0.6675	0.7533	0.7380	0.8539	0.8405	0.8885	0.8716	0.9107	0.9017	0.8872	0.8774	0.9040	0.8947
MobileNetV2	0.9284	0.9203	0.8986	0.8867	0.9020	0.8896	0.8967	0.8858	0.7002	0.6541	0.8540	0.8417	0.8495	0.8400	0.9260	0.9116	0.9511	0.9441	0.9358	0.9275	0.9439	0.9367
ShuffleNet	0.9330	0.9184	0.9129	0.8986	0.9131	0.8987	0.9036	0.8874	0.7318	0.6893	0.8881	0.8718	0.9259	0.9119	0.8999	0.8820	0.9333	0.9203	0.9273	0.9141	0.9364	0.9235
InceptionV3	0.9649	0.9588	0.9361	0.9274	0.9355	0.9275	0.9359	0.9274	0.6843	0.6355	0.9103	0.9013	0.9515	0.9451	0.9335	0.9193	0.9322	0.9243	0.9540	0.9454	0.9621	0.9557
DenseNet201	0.9289	0.9133	0.9213	0.9103	0.9207	0.9098	0.9210	0.9102	0.6457	0.5851	0.8877	0.8778	0.9358	0.9274	0.9269	0.9121	0.9538	0.9453	0.8680	0.8441	0.9536	0.9454
Xception	0.9544	0.9463	0.9370	0.9272	0.9280	0.9198	0.9294	0.9200	0.6975	0.6492	0.9038	0.8941	0.9437	0.9369	0.9356	0.9228	0.9620	0.9553	0.9537	0.9460	0.9185	0.9068

,

Table 18. The performance analysis of RWN on the combined dataset (Activation Function: Radbas).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8769	0.8558	0.9325	0.9236	0.9246	0.9152	0.9243	0.9155	0.6780	0.6254	0.8885	0.8790	0.9308	0.9216	0.9548	0.9449	0.9675	0.9623	0.9412	0.9288	0.9562	0.9494
ResNet18	0.9324	0.9239	0.8749	0.8654	0.9063	0.8929	0.9033	0.8913	0.7668	0.7301	0.8680	0.8540	0.9081	0.8976	0.9408	0.9287	0.9474	0.9390	0.9308	0.9212	0.9452	0.9369
ResNet50	0.9248	0.9155	0.9059	0.8938	0.8632	0.8506	0.9009	0.8881	0.7661	0.7298	0.8572	0.8406	0.9117	0.9007	0.9401	0.9285	0.9447	0.9365	0.9299	0.9197	0.9357	0.9261
ResNet101	0.9241	0.9148	0.9031	0.8906	0.9006	0.8870	0.8540	0.8426	0.7365	0.6960	0.8552	0.8400	0.9074	0.8967	0.9313	0.9192	0.9460	0.9377	0.9303	0.9210	0.9380	0.9276
DarkNet19	0.6774	0.6249	0.7655	0.7295	0.7665	0.7320	0.7363	0.6959	0.6518	0.5936	0.7489	0.7100	0.7361	0.6957	0.7730	0.7368	0.7181	0.6746	0.6669	0.6140	0.7328	0.6931
DarkNet53	0.8889	0.8792	0.8683	0.8542	0.8571	0.8404	0.8553	0.8397	0.7481	0.7111	0.7636	0.7465	0.8666	0.8543	0.9158	0.9030	0.9232	0.9139	0.8990	0.8893	0.9144	0.9055
MobileNetV2	0.9309	0.9224	0.9079	0.8979	0.9113	0.9002	0.9072	0.8958	0.7355	0.6954	0.8663	0.8536	0.8521	0.8399	0.9491	0.9389	0.9551	0.9486	0.9395	0.9326	0.9469	0.9388
ShuffleNet	0.9543	0.9454	0.9408	0.9280	0.9397	0.9292	0.9318	0.9192	0.7735	0.7399	0.9154	0.9023	0.9490	0.9384	0.9284	0.9150	0.9560	0.9463	0.9495	0.9397	0.9584	0.9494
InceptionV3	0.9676	0.9618	0.9472	0.9387	0.9447	0.9357	0.9461	0.9375	0.7170	0.6720	0.9232	0.9149	0.9552	0.9487	0.9562	0.9462	0.9357	0.9266	0.9580	0.9510	0.9649	0.9582
DenseNet201	0.9415	0.9286	0.9308	0.9214	0.9306	0.9200	0.9301	0.9202	0.6674	0.6140	0.8992	0.8900	0.9398	0.9320	0.9493	0.9388	0.9578	0.9515	0.8921	0.8724	0.9568	0.9490
Xception	0.9562	0.9493	0.9456	0.9366	0.9357	0.9271	0.9381	0.9288	0.7328	0.6923	0.9144	0.9052	0.9468	0.9385	0.9584	0.9494	0.9647	0.9587	0.9568	0.9495	0.9232	0.9079

and

Table 19. The performance analysis of RWN on the combined dataset (Activation Function: Hardlim).

Feature Extracted Models	EfficientNet		ResNet18		ResNet50		ResNet101		DarkNet19		DarkNet53		MobileNetV2		ShuffleNet		InceptionV3		DenseNet201		Xception
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
EfficientNet	0.8819	0.8521	0.9369	0.9301	0.9298	0.9242	0.9280	0.9203	0.9374	0.9281	0.8936	0.8874	0.9308	0.9239	0.9730	0.9685	0.9678	0.9629	0.9506	0.9419	0.9566	0.9518
ResNet18	0.9368	0.9302	0.8788	0.8724	0.9202	0.9110	0.9181	0.9098	0.9313	0.9218	0.8880	0.8799	0.9133	0.9057	0.9666	0.9620	0.9540	0.9479	0.9366	0.9296	0.9493	0.9435
ResNet50	0.9300	0.9235	0.9200	0.9113	0.8689	0.8614	0.9153	0.9072	0.9284	0.9183	0.8785	0.8693	0.9183	0.9115	0.9664	0.9617	0.9517	0.9463	0.9376	0.9305	0.9415	0.9363
ResNet101	0.9281	0.9210	0.9182	0.9093	0.9161	0.9072	0.8552	0.8475	0.9233	0.9118	0.8732	0.8633	0.9132	0.9048	0.9633	0.9577	0.9529	0.9471	0.9377	0.9316	0.9437	0.9370
DarkNet19	0.9375	0.9282	0.9311	0.9214	0.9283	0.9172	0.9230	0.9125	0.8936	0.8826	0.8940	0.8816	0.9275	0.9167	0.9634	0.9565	0.9372	0.9278	0.9249	0.9152	0.9415	0.9328
DarkNet53	0.8932	0.8866	0.8880	0.8795	0.8788	0.8684	0.8732	0.8635	0.8940	0.8815	0.7623	0.7508	0.8772	0.8689	0.9527	0.9478	0.9324	0.9275	0.9084	0.9019	0.9212	0.9160
MobileNetV2	0.9310	0.9236	0.9132	0.9062	0.9185	0.9118	0.9133	0.9057	0.9272	0.9165	0.8770	0.8688	0.8506	0.8424	0.9673	0.9631	0.9555	0.9506	0.9410	0.9352	0.9469	0.9414
ShuffleNet	0.9730	0.9687	0.9665	0.9615	0.9664	0.9615	0.9632	0.9577	0.9632	0.9569	0.9526	0.9467	0.9673	0.9627	0.9534	0.9473	0.9759	0.9712	0.9679	0.9631	0.9744	0.9704
InceptionV3	0.9678	0.9627	0.9539	0.9486	0.9515	0.9467	0.9528	0.9478	0.9374	0.9278	0.9324	0.9277	0.9556	0.9503	0.9760	0.9721	0.9331	0.9260	0.9603	0.9548	0.9650	0.9600
DenseNet201	0.9507	0.9420	0.9369	0.9299	0.9375	0.9309	0.9377	0.9309	0.9251	0.9147	0.9086	0.9022	0.9412	0.9355	0.9679	0.9629	0.9604	0.9547	0.9181	0.9057	0.9585	0.9524
Xception	0.9566	0.9506	0.9493	0.9439	0.9417	0.9367	0.9435	0.9369	0.9415	0.9327	0.9211	0.9156	0.9468	0.9411	0.9746	0.9700	0.9649	0.9602	0.9585	0.9524	0.9187	0.9095

. Note that the results for the Tangent Sigmoid function with 250 neurons, which were already presented in Table 7, are not duplicated in this section.

A comparative analysis of the results from Table 7 and Table 15, Table 16, Table 17, Table 18 and Table 19 reveals a clear distinction in the performance of the tested activation functions. The Sigmoid, Tangent Sigmoid, and Hardlimit functions consistently yielded strong and comparable results. In contrast, the Sine, Tribas, and Radbas functions were demonstrably less effective, with their average training and test accuracies remaining below 90%.

The best overall performance was unequivocally achieved using the Sigmoid activation function. On the fused Inception-ShuffleNet feature set, this configuration produced not only the highest training accuracy of 97.74% and a test accuracy of 97.40% but also the highest average training and test accuracies of 93.54% and 92.77%, respectively. A summary of these comparative results is presented in

Table 20. Comparisons of activation functions used in RWN on the combined dataset.

	Sig	Tansig	Sin	Tribas	Radbas	Hardlimit
The Best Training Accuracy	0.9774	0.9768	0.9510	0.9650	0.9676	0.9760
The Best Test Accuracy	0.9740	0.9725	0.9419	0.9593	0.9623	0.9721
The Average Training Accuracy	0.9354	0.9342	0.7568	0.8732	0.8889	0.9319
The Average Test Accuracy	0.9277	0.9261	0.7088	0.8552	0.8730	0.9246

.

The summary results in Table 20 highlight a clear performance hierarchy among the activation functions. Sigmoid, Tangent Sigmoid, and Hardlimit consistently emerge as the top-performing functions. Conversely, Sine, Tribas, and Radbas demonstrate markedly inferior performance, particularly with respect to their average test accuracies.

Figure 9 presents the time and disk space usage for tests conducted with individual feature sets, while Figure 10 illustrates the same metrics for tests performed with combined feature sets. The reported values are calculated as the averages of the consumption metrics across all tests for each classifier. When evaluating computational efficiency, the RWN demonstrates a strong and balanced time–performance profile. While its training time is longer than that of lazy learners like KNN, it is significantly faster than SVM. More critically, its testing time is remarkably short, outperforming the much slower KNN. SVM exhibits the longest training time of all classifiers but has a faster testing time compared to KNN. KNN, as a lazy learning algorithm, has negligible training time that is limited to loading instances into memory. However, its testing time is significantly longer due to the need to search for nearest neighbors during inference. TREE, on the other hand, is fast in both training and testing but, as demonstrated earlier, is highly prone to overfitting. In terms of disk usage, RWN is the most economical, requiring the least space, whereas SVM is the most demanding. Notably, when moving from individual to combined feature sets, the disk space consumption for KNN, TREE, and SVM increases significantly, while RWN's usage remains low and consistent.

Therefore, when considering the combined metrics of high classification accuracy, minimal disk space requirements, and a favorable balance of training/testing times, the RWN emerges as the most well-rounded and efficient classifier for this application. While SVM and KNN offer high accuracy potential, they demand greater computational and storage resources. TREE achieves a balanced trade-off between time and resource usage, but its classification performance does not match the other classifiers.

Having established the performance of the proposed RWN-based feature fusion framework, the final stage of our analysis compares these results against several state-of-the-art machine learning classifiers. Machine learning advancements have led to a variety of classifiers designed to solve complex problems with varying efficiency. To assess our proposed method, we compared its performance with state-of-the-art classifiers. Among these classifiers, CatBoost is a gradient boosting algorithm designed to handle categorical data effectively while mitigating overfitting [48]. Decision trees use a tree-like structure to model decisions and their possible outcomes. They are widely used for both classification and regression problems [49]. The Gaussian Naïve Bayes algorithm, based on Bayes' theorem, is a probabilistic classifier that calculates the likelihood of different classes. It gained popularity for its effectiveness in classification tasks [50]. Gradient boosting methods enhance model performance by iteratively combining weak learners to create a strong predictive model [51]. KNN is a straightforward yet effective algorithm for classification and regression. It assigns class labels by analyzing the nearest k data points in the feature space [52]. LightGBM is a gradient boosting framework optimized for handling large datasets efficiently through distributed learning [53]. Logistic regression is a statistical method used to predict the probability of a dependent variable belonging to a particular category [54]. Random Forests combine multiple decision trees to tackle complex classification problems, improving accuracy and robustness [55]. The Ridge Classifier is an extension of ridge regression; this algorithm is tailored for classification tasks. It excels by incorporating regularization to address overfitting [56]. SVM is an algorithm that finds the most appropriate hyperplane to separate data into different classes and is widely used for both linear and nonlinear problems [57]. XGBoost is a high-performance gradient boosting framework known for its speed and scalability [58]. In

Table 21. Performance analysis of the proposed method compared to state-of-the-art classifiers.

Metrics	RWN	CatBoost	Decision Tree	Gaussian Naïve Bayes	Gradient Boosting	KNN	LightGBM	Logistic Regression	Random Forest	Ridge Classifier	SVM	XGBoost
The Best Training Accuracy	0.9849	0.9549	0.9969	0.9437	0.9472	0.9802	1.0000	0.9734	0.9967	0.9634	0.9996	1.0000
The Best Test Accuracy	0.9843	0.951	0.9646	0.9646	0.9451	0.9823	0.9607	0.9548	0.9548	0.9608	0.9784	0.9568
Average Training Accuracy	0.9816	0.9535	0.9954	0.9427	0.9436	0.9787	1.0000	0.9719	0.9961	0.9619	0.9991	1.0000
Average Test Accuracy	0.9743	0.9446	0.9552	0.9425	0.934	0.9715	0.9484	0.9460	0.9425	0.9484	0.9668	0.9486
Mean Test F1-Score	0.9777	0.9451	0.9548	0.9442	0.9348	0.9717	0.9489	0.9464	0.9424	0.9491	0.9667	0.9491

, we present an analysis of the results, highlighting the strengths and generalization capabilities of the proposed approach.

The comparative results are presented in Table 21. It is crucial to note the experimental setup for this comparison: to provide a robust benchmark, the state-of-the-art classifiers were trained on the best-performing single feature set (ShuffleNet). Our proposed method, by contrast, was trained on the fused ShuffleNet-InceptionV3 feature set to specifically demonstrate the benefit of feature fusion.

The analysis clearly demonstrates the superiority of the proposed method. The RWN-based fusion approach not only achieves the highest average test accuracy of 97.43% but also exhibits the best generalization capability. This is evidenced by the minimal gap between its training and test accuracies, especially when compared to models like LightGBM and XGBoost, which, despite achieving perfect training scores, show a significant performance drop on the test set, indicating severe overfitting.

4. Results and Discussion

This study systematically investigated the performance of a novel feature fusion framework centered on a Random Weight Network (RWN) classifier. The findings directly address the core research questions posed in the Introduction, demonstrating a clear pathway to overcoming the limitations of conventional deep learning models in this challenging security domain.

The investigation first addressed the performance implications of substituting a standard deep learning classifier with an RWN. The results unequivocally demonstrate a significant performance uplift. For instance, on the features extracted from the best-performing standalone model, ShuffleNet, the test accuracy increased dramatically from 83.55% to 94.82% when an RWN with 250 hidden neurons was employed. This finding confirms that by decoupling feature extraction from classification, the inherent performance limitations of conventional models, namely low accuracy and a high propensity for overfitting on complex X-ray data, can be substantially mitigated.

Building upon this, the study validated its primary hypothesis regarding the efficacy of feature fusion. By combining features from different high-performing architectures, notably ShuffleNet and InceptionV3, the framework achieved a state-of-the-art test accuracy of 97.44%. This result provides a definitive affirmative answer to whether multi-model fusion can significantly enhance classification accuracy, clearly outperforming both standalone models and the RWN applied to single feature sets. This highlights that data diversity, achieved through fusing varied feature representations, is a key driver of performance. In contrast, simply duplicating an existing feature set by merging it with itself yields no significant improvement, reinforcing that the richness of the feature pool is what matters.

The performance of the proposed framework was also found to be critically dependent on the careful tuning of RWN’s hyperparameters. Addressing the impact of hidden layer size, the study revealed a clear trade-off: an excessive number of neurons led to overfitting, while an insufficient number resulted in ineffective learning. Optimal generalization was achieved through a balance, with 250 neurons providing the best average test accuracy across many scenarios, and 500 or 1000 neurons yielding the peak accuracy on the best fused dataset. Similarly, the choice of activation function proved significant. Sigmoid, Tangent Sigmoid, and Hardlimit functions consistently delivered superior and robust performance, with Sigmoid ultimately achieving the best overall results. Conversely, other implementation details, such as the order of feature concatenation (N|M vs. M|N), were found to have a negligible impact on the outcome.

In summary, this study confirms that a modular approach, which involves decoupling feature extraction, employing multi-model feature fusion, and utilizing a well-tuned RWN, is a highly effective strategy. This framework successfully answers the initial research challenges, demonstrating a clear pathway from the 83.55% accuracy of standalone models to the 97.44% achieved through the proposed methodology, thereby establishing a new performance benchmark in this security domain.

5. Conclusion and Future Works

This study successfully addressed the challenges of low accuracy and overfitting in deep learning-based X-ray image classification by proposing a novel framework based on feature fusion and a Random Weight Network (RWN) classifier. We demonstrated that by fusing features from diverse, high-performing deep learning architectures, specifically ShuffleNet and InceptionV3, and using a well-tuned RWN, the classification performance can be dramatically improved. The core contribution of this work is the significant increase in test accuracy, from a baseline of 83.55% for the best standalone model to a state-of-the-art 97.44% with the proposed fusion method. Our analysis confirmed that the success of this framework depends not only on the diversity of the fused features but also on the careful tuning of the RWN's hyperparameters, namely the number of hidden neurons and the choice of activation function. This modular approach provides a robust and computationally efficient alternative to end-to-end deep learning systems for this critical security task.

For future work, we plan to expand on these findings by investigating a broader range of feature combination methods and testing the performance of other advanced classifiers. Furthermore, we plan to explore transformer-based feature extraction techniques and deep fusion techniques. In this approach, instead of combining the features after they are fully extracted, the integration would happen at earlier or intermediate stages within a single, unified neural network.