Entropy-Guided Search Space Optimization for Efficient Neural Network Pruning

Abstract

Neural network pruning is essential for deploying deep learning models on resource-constrained devices by reducing computational and memory demands. In this paper, we propose a novel pruning framework, Entropy-Guided Search Space Optimization for Efficient Neural Network Pruning, which uses information entropy to assess the importance of convolutional layers. Specifically, we calculate the layer-wise entropy of pretrained weights, apply outlier detection to remove extreme values, and normalize the entropy values. These normalized values guide the selection of retention ratios, ensuring that layers with higher entropy retain more filters. By refining the subnetwork search space, our approach enhances the efficiency of the search process and improves overall subnetwork performance. The refined search space targets more promising regions, reducing computational overhead and resulting in higher-quality pruned networks. Through iterative optimization, the optimal subnetwork is identified and fine-tuned to produce the final pruned model. Experimental results on benchmark datasets show that our method significantly outperforms existing pruning methods, achieving substantial improvements in both accuracy and computational efficiency. This entropy-guided pruning strategy provides a robust and effective solution for neural network compression, suitable for a wide range of deep learning applications.

1. Introduction

Deep neural networks (DNNs) have demonstrated remarkable success in various computer vision tasks such as image classification, object detection, and video analysis [1,2,3,4,5,6,7]. However, the deployment of these networks on resource-constrained devices like mobile phones, drones, and wearable gadgets remains challenging due to their high computational and memory demands. These devices are active targets for intelligent application development, with core visual capabilities like image recognition and object detection. To address these challenges, numerous network compression and acceleration techniques have been developed, including quantization [8,9], low-rank approximation [10,11], knowledge distillation [12], and network pruning [13,14,15,16].

Network pruning, in particular, has gained significant attention as an effective method to reduce the size and complexity of deep neural networks without substantially compromising their performance [14,16,17,18,19,20,21,22,23,24,25]. Pruning methods can be broadly categorized into unstructured pruning and structured pruning. Unstructured pruning removes connections (weights) of neural networks, resulting in unstructured sparsity, which often requires specific hardware or library support for actual speedup. In contrast, structured pruning removes entire filters of neural networks and can achieve actual speedup and compression with standard hardware.

Despite the advancements in pruning techniques, several challenges remain. Existing methods often rely on heuristics or complex criteria to determine the importance of filters, which can lead to suboptimal pruning decisions. Furthermore, many methods do not effectively account for the distribution and variability of filter importance across different layers of the network.

To address these challenges, we introduce a novel pruning method that optimizes the subnetwork search space through entropy-guided adjustments, the entropy-guided principles have also been widely adopted [26]. By refining the search space, the proposed method enhances both accuracy and efficiency, enabling the identification of high-performing subnetworks with reduced computational demands. The proposed method systematically focuses on promising regions of the parameter space, resulting in significant improvements in the final pruned model’s performance.

The primary contributions of this paper are summarized as follows:

• We introduce a novel neural network pruning framework, Entropy-Guided Search Space Optimization (EGSSO), which refines the search space based on information entropy;
• We implement outlier detection and normalization techniques to stabilize entropy values and enhance pruning reliability;
• We demonstrate the effectiveness of EGSSO through extensive experiments on benchmark datasets, achieving notable gains in both accuracy and computational efficiency over existing pruning methods.

2. Related Works

Network pruning has been extensively studied to reduce the size and complexity of deep neural networks while maintaining acceptable performance. This section reviews various strategies, highlighting their key methodologies and limitations.

Early work on unstructured weight pruning removed individual weights from the network, leading to sparsity that often depended on specialized hardware or software for practical speedup. Several methods used L1/L2 norm pruning, median pruning, or Taylor expansion to estimate weight importance [8,15]. Filter pruning emerged as a structured alternative, removing entire filters to achieve genuine acceleration on standard hardware. Representative approaches include the following:

• Ranking and Pruning Filters: Criteria based on weight magnitude, activation statistics, or Average Percentage of Zero Activations (APoZ) [27].
• Reconstruction Error Minimization: Methods such as ThiNet [28] and Neuron Importance Score Propagation (NISP) [29] prune filters with minimal reconstruction error to preserve accuracy, though they can be computationally expensive.
• Similarity Measurement: Algorithms relying on the geometric median or clustering, including Filter Pruning via Geometric Median (FPGM) [30], effectively remove redundant filters but can introduce additional overhead.

Recent work introduced more advanced techniques:

• Discrimination-aware Pruning: Exploits additional loss functions so that pruned channels retain discriminative power, as shown in Discrimination-aware Channel Pruning (DCP) [31] and Discrimination-aware Kernel Pruning (DKP) [19].
• Random Channel Pruning: Provides a baseline by randomly sampling subnetworks [32] but often demands extensive searches and may yield suboptimal results.
• Structured Redundancy Reduction (SRR): Builds a graph for each layer to measure redundancy, pruning filters in layers deemed most redundant [33].
• Dependency Graph (DepGraph) [34]: Automatically models inter/intra-layer dependencies to facilitate group-level pruning, though it may be limited by complex architectures and hyperparameter sensitivity.
• CORING [35]: Employs tensor decomposition (HOSVD) to retain the multidimensional structure of filters. While it can reduce complexity, large-scale applications may still face fine-tuning challenges.

Despite progress, existing methods still exhibit certain limitations:

• Randomness in Subnetwork Search: Techniques such as random channel pruning can be inefficient, requiring extensive searching to find high-quality subnetworks [32].
• Lack of Layer Importance Consideration: Many methods overlook the variation in filter importance across different layers, risking the removal of crucial filters.
• Heuristic-based Decisions: Filters are often pruned based on heuristic criteria that may not accurately capture their overall contribution to performance.
• Computational Overhead: Methods that minimize reconstruction error or measure filter similarity can involve heavy computations, reducing pruning efficiency.

Neural network pruning has thus evolved through various trade-offs and strategies, yet the above shortcomings indicate a need for more robust, efficient pruning. The entropy-based approach proposed here seeks to address these gaps by leveraging information entropy and outlier detection to guide pruning decisions, delivering improved accuracy and efficiency for network compression. Recent works introduce optimization-based formulations to learn pruning decisions more explicitly. The differentiable pruning method [16] replaces discrete filter selection with continuous mask parameterization and optimizes pruning via optimal-transport-based soft top-k operators. These methods offer fine-grained sparsity control but require iterative Sinkhorn updates and backpropagation through relaxed masks, thereby increasing computational overhead. Bi-level optimization frameworks [23] formulate pruning as a nested optimization problem in which model weights are updated in the inner loop, while pruning parameters are optimized in the outer loop. Although such methods can produce high-quality subnetworks, they typically rely on implicit gradients or second-order approximations, resulting in substantial training cost. Our proposed EGSSO avoids continuous mask relaxation and bi-level optimization entirely. It leverages normalized information entropy to refine the pruning search space and performs lightweight subnetwork sampling, enabling efficient and stable pruning without additional gradient dependencies.

3. Methods

This section introduces our neural network pruning algorithm design based on the entropy-guided search space optimization (EGSSO). Traditional methods often rely on uniform or random search spaces, leading to suboptimal outcomes. In contrast, EGSSO calculates the entropy of weights in each convolutional layer to guide pruning: layers with higher entropy retain more filters, while lower-entropy layers are pruned more aggressively. Outlier detection and normalization further refine these entropy values, ensuring stability and directing the search toward promising subnetwork configurations. Figure 1 illustrates the EGSSO workflow, from baseline training to iterative pruning and evaluation. As shown in Algorithm 1 and visualized in Figure 1, EGSSO proceeds in four stages—(a) layer-wise information entropy; (b) outlier detection; and (c) subnetwork searching with entropy-guided bounds.

3.1. Information Entropy for Efficient Neural Network Pruning

Consider a deep convolutional neural network $\mathcal{N}$, where the l-th layer has a weight tensor ${\mathbf{W}}^l\in {\mathbb{R}}^{N_{l+1}\times N_l\times K_l\times K_l}$, with $N_l$ and $N_{l+1}$ denoting the number of input and output channels, and $K_l$ the kernel size. The convolution operation at layer l is defined as follows:

$${\mathbf{O}}_i^l={\mathbf{I}}^l\times {\mathbf{F}}_i^l\mathrm{for}1\le i\le N_{l+1},$$

(1)

where ${\mathbf{I}}^l$ is the input tensor, ${\mathbf{O}}_i^l$ is the output of the i-th filter, and ${\mathbf{F}}_i^l$ are its weights. Structured pruning seeks a subset of filters $\mathbf{F}$ that minimizes the loss $\mathcal{L}$ over the dataset $\mathcal{D}$, subject to a sparsity constraint:

$$\underset{\mathbf{F}}{min}\mathcal{L}(\mathbf{F};\mathcal{D})=\underset{\mathbf{F}}{min}{\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathcal{L}(\mathbf{F};({\mathbf{x}}_i,y_i))\mathrm{s}.\mathrm{t}.\mathrm{Card}\left(\mathbf{F}\right)\le \kappa ,$$

(2)

where $\kappa$ denotes the target number of remaining filters. As defined in Equation (3), the Information Entropy [36] measures the uncertainty in the weight distribution, and we computed the layer-wise entropy according to Equation (4). For a discrete random variable X with outcomes $\{x_1,x_2,\dots ,x_n\}$ and probabilities $\{p_1,p_2,\dots ,p_n\}$:

$$H\left(X\right)=-\sum _{i=1}^n{p}_i{log}_2\left(p_i\right),$$

(3)

In neural networks, a higher entropy weight distribution suggests richer, more critical information, while lower entropy implies less contribution to the network’s performance.

To compute layer-wise entropy, the filters in each convolutional layer were flattened and binned into 50 bins to form a probability density function. The entropy is then

$$H=\sum (-p_i{log}_2(p_i+ϵ)),$$

(4)

where $p_i$ is the probability in bin i, and $ϵ$ avoids the logarithm of zero. This entropy guides our pruning: layers with higher entropy values remained largely intact, whereas layers with lower entropy were pruned more aggressively. An empty dictionary was used to store each layer’s entropy, and layers not meant to be pruned were skipped. Outlier detection and normalization further stabilized these entropy measurements, preventing extreme values from distorting the pruning decisions.

By quantitatively assessing each layer’s information content, EGSSO focuses on retaining the filters most crucial to network performance, yielding a more efficient and effective pruning process.

3.2. Outlier Detection and Normalization of Information Entropy

The second step in our method involves detecting and replacing outliers in the entropy values, as they can distort the subsequent normalization process and degrade the performance of subnetwork search. To address this, we first calculated the mean (${\mu }_{\mathrm{entropy}}$) and standard deviation (${\sigma }_{\mathrm{entropy}}$) of the entropy values across all convolutional layers. We then used the z-score method as defined in Equation (5), to identify outliers [37,38], which measures how far a data point deviates from the mean in terms of standard deviations:

$$z_i={\displaystyle \frac{\left|H_i-{\mu }_{\mathrm{entropy}}\right|}{{\sigma }_{\mathrm{entropy}}}}$$

(5)

where $H_i$ is the entropy of the i-th convolutional layer, and ${\mu }_{\mathrm{entropy}}$ and ${\sigma }_{\mathrm{entropy}}$ are the mean and standard deviation of the entropy values, respectively.

Figure 2 visualizes the entropy values across 59 layers, with the horizontal axis showing the raw entropy values and the vertical axis listing the convolutional layers of the network. Colored rectangles indicate the outlier detection results at different z-score thresholds. The z-scores corresponding to these rectangles are labeled as $z=1.0$, $z=1.5$, $z=2.0$, and $z=2.5$, highlighting the detected outliers for each threshold.

Once the outliers are detected, we replaced each outlier with the nearest non-outlying value. By detecting and replacing outliers, we ensure a more stable and consistent entropy distribution, which is crucial for the subsequent normalization and subnetwork sampling steps. This enhances the robustness and effectiveness of our pruning strategy.

The third step in our method involves normalizing the entropy values to ensure that they are on a comparable scale. This normalization process uses the min–max normalization technique as defined in Equation (6), which scales the entropy values to the range $[0,1]$.

Min–max normalization was performed using the following:

$$H_i^{\prime }={\displaystyle \frac{H_i-H_{min}}{H_{max}-H_{min}}}$$

(6)

where $H_i$ represents the original entropy value of the i-th convolutional layer, $H_{min}$ is the minimum entropy value across all layers, and $H_{max}$ is the maximum entropy value. The normalized entropy value $H_i^{\prime }$ will be in the range $[0,1]$.

The primary reason for normalizing entropy values to the $[0,1]$ range is to facilitate the subsequent subnetwork search process. By having all entropy values within the same range, we can systematically determine appropriate pruning levels across different layers, ensuring a balanced and efficient pruning strategy.

3.3. Subnetwork Search Algorithm Design

This section describes the subnetwork sampling rules, explains how the search space is generated, and compares the search space of the proposed method with an existing approach. For each convolutional layer in the model, the retention ratio is determined based on the normalized entropy values as follows:

$$\begin{array}{cc}\hfill u_i& =min\left(1,H_i^{\prime }+{\delta }_u\right),\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill l_i& =max\left(r_{min},H_i^{\prime }-{\delta }_l\right),\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill r_i& =max\left(r_{min},\mathrm{random}(l_i,u_i)-{\delta }_r\right),\hfill \end{array}$$

(9)

where $H_i^{\prime }$ is the normalized entropy value of the i-th layer, $r_{min}$ is a minimum retention ratio parameter (typically 0.2), ${\delta }_u$ and ${\delta }_l$ are search space parameters (e.g., ${\delta }_u=0.2$, ${\delta }_l=0.4$), and ${\delta }_r$ is a reduction factor. This sampling strategy adjusts each layer’s retention ratio according to its entropy, aiming for more informed pruning decisions.

Figure 3 illustrates a comparison of the existing method and the proposed entropy-guided method on the detection network in the VisDrone dataset. Along the horizontal axis are 59 convolutional layers. The vertical axis shows the retention ratio search space from 0.0 to 1.0. Two bars are shown for each layer, one for the existing method and one for the proposed method.

In the existing method, the orange bar on the left extends uniformly from the minimum ratio of 0.2 up to 1.0, thus covering a broad but unstructured search space. While this allows for flexibility, it can include many suboptimal regions that slow down the search and can degrade overall subnetwork quality.

In contrast, the proposed method uses an entropy-based approach for each layer, depicted by the blue bar on the right. Layers with higher entropy values are assigned both upper and lower bounds at higher retention ratios, reflecting their richer information content and the need to preserve more filters. Conversely, layers with lower entropy are assigned proportionally smaller search intervals. As a result, the search range naturally adapts to each layer’s importance, and the dashed red line at 0.2 indicates the common lower limit below which neither bar can drop.

By restricting the search space to more relevant intervals, the proposed method increases the likelihood of discovering subnetwork configurations with stronger performance while also reducing unnecessary exploration of less promising ratios. Experiments show that this tailored strategy not only accelerates the search process but also leads to pruned models with better overall performance.

3.4. Theoretical Analysis of Outlier Detection in Subnetwork Search

• (1) Setup and Notation. Let $\{H_1,H_2,\dots ,H_n\}$ denote the entropy values of n convolutional layers. We define the following:

  $$H_{min}=\underset{1\le i\le n}{min}{H}_i,H_{max}=\underset{1\le i\le n}{max}{H}_i.$$

  (10)

  To ensure these entropy values are comparable across layers, we applied min–max normalization:

  $$H_i^{\prime }={\displaystyle \frac{H_i-H_{min}}{H_{max}-H_{min}}},i=1,\dots ,n,$$

  (11)

  where $H_i^{\prime }\in [0,1]$ ideally captures the relative importance of each layer. Larger values of $H_i^{\prime }$ indicate that a layer’s weight distribution is more diverse or informative.

• (2) Impact of Outliers on Normalization and Search. In practice, a single extreme outlier $H_{\mathrm{out}}\gg {max}_{j\ne \mathrm{out}}{H}_j$ can inflate the range $(H_{min},H_{max})$, forcing most layers’ entropy values $H_j$ to be much smaller by comparison. Consequently,

  $$H_j^{\prime }\approx {\displaystyle \frac{H_j-H_{min}}{H_{\mathrm{out}}-H_{min}}}(\ll 1),$$

  (12)

  meaning many $H_j^{\prime }$ values become compressed into a narrow interval, thereby losing important distinctions among layers.

Following our subnetwork search algorithm in last subsection, each layer i was assigned a pruning retention ratio $r_i$ based on $H_i^{\prime }$. If $H_i^{\prime }$ is artificially low due to an outlier $H_{\mathrm{out}}$, the resulting $r_i$ will also be disproportionately low, leading to excessive pruning of an otherwise critical layer. As shown in Figure 4, we visualized the outlier detection and replacement process. Removing extreme anomalous values not only eliminates distortion but also reduces the overall variance of the normalized entropy, demonstrating that outlier correction effectively stabilizes the entropy-guided search space.

• (3) Why Outlier Detection is Necessary. Without outlier correction, a single extreme $H_{\mathrm{out}}$ can skew the entire normalization scale, compromising the search space for pruning ratios. After outlier replacement, the effective range of entropies reflects a more realistic variability among layers. Consequently, the subsequent subnetwork search can assign retention ratios appropriately, preserving high-entropy (critical) layers and pruning low-entropy (less influential) ones more reliably.

In summary, detecting and replacing outliers is pivotal for maintaining the integrity of entropy-guided pruning, ensuring stable subnetwork sampling and preventing a few extreme values from dictating the network’s compression strategy.

3.5. Pruning Based on Norms

Once the pruning ratio $r_i$ for each layer is determined, we use the $L_1$ norm to select filters for retention. For a convolutional layer with weight tensor W, the $L_1$ norm of the k-th filter is defined as follows:

$$\parallel W_k{\parallel }_1=\sum _{i,j,c}\left|W_{k,i,j,c}\right|,$$

(13)

where $W_k$ represents the weights of the k-th filter, and $i,j,c$ are the spatial and channel indices of each filter.

After computing the $L_1$ norm for all filters in the i-th layer, we determined the number of filters to retain

$$n_i=⌈r_i·N_i⌉,$$

(14)

where $N_i$ is the total number of filters in that layer, and $r_i$ is the corresponding pruning ratio. We then sorted all filters by their $L_1$ norms and selected the top $n_i$ filters:

$$\mathrm{indices}=\mathrm{argsort}\left(\parallel W_k{\parallel }_1\right)[-n_i:],$$

(15)

We constructed a binary mask M to retain only the selected filters:

$$M_k=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}k\in \mathrm{indices},\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(16)

and applied it to the weight tensor W:

$$W_k^{\prime }=M_k·W_k.$$

(17)

$W_k^{\prime }$ denotes the pruned model parameters after channel pruning. We computed the FLOPs of the pruned and original models. If its retained computation matches the prescribed Target Remaining Rate within a small tolerance E, the pruning configuration is accepted, and we set the Target Remaining Rate as 0.4, 0.5, 0.6 and E as 0.02.

3.6. Pruning Algorithm Design Based on the Entropy-Guided Search Space Optimization

This section provides a comprehensive description of our proposed EGSSO algorithm, encompassing the following key components: entropy calculation, outlier detection, normalization, subnetwork sampling, and iterative evaluation.

Initially, we trained the neural network to establish a baseline performance. We then computed the information entropy of each convolutional layer’s weights, quantifying their uncertainty or randomness within the network. Layers with higher entropy values tend to contain more diverse and informative weights, while lower entropy layers are more likely to be redundant.

To ensure accurate entropy-based pruning, outlier detection was performed to identify and replace extreme values that could otherwise skew the normalization. This correction step stabilizes the entropy distribution and improves the reliability of subsequent pruning decisions.

Next, the entropy values were normalized via min–max scaling to a common range of [0, 1]. This provides a consistent basis for determining the retention ratios across layers. Specifically, layers with higher normalized entropy retained a larger proportion of filters, whereas layers with lower entropy were pruned more aggressively. The retention ratio for each layer was obtained by computing upper and lower limits and randomly sampling within these bounds, ensuring an optimal balance between model performance and parameter reduction.

If the subnetwork obtained through the above pruning scheme meets the FLOP (Floating Point Operation Per Second) requirement, it is deemed a successful candidate. We then evaluate its performance using adaptive batch normalization [32], which recalibrates network parameters to reflect the reduced number of filters. This step offers a quick way to assess the subnetwork’s potential accuracy without expensive fine-tuning. If the performance meets the desired criteria, the subnetwork is retained; otherwise, the process iterates, refining the retention ratios and pruning masks until an optimal subnetwork is found.

Finally, once the optimal subnetwork is identified, we perform fine-tuning to restore any lost accuracy. The training procedure is based on the neural network framework [39].

4. Experiments

4.1. Settings

To validate the effectiveness of the proposed method, extensive experiments were conducted on multiple datasets, including COCO [40] and VisDrone [41], using various previous neural network framework [39].

An accurate evaluation of subnetwork performance before fine-tuning plays a crucial role in the pruning pipeline. In this work, we follow the approach proposed by EagleEye [32], which employs adaptive batch normalization (BN) to recalibrate BN statistics for each pruned subnetwork. EagleEye’s quantitative correlation analysis indicates that subnetwork performance assessed by adaptive BN strongly correlates with the final fine-tuned accuracy, with a Pearson correlation coefficient [42] of up to 0.793. This higher correlation assures that the subnetwork selection criterion is more reliable, thereby improving the effectiveness and efficiency of our overall pruning procedure.

4.2. Evaluation Metrics

To evaluate the performance of our proposed pruning method, we utilize several widely accepted metrics: Mean Average Precision (mAP) and Floating Point Operations Per Second (FLOPs).

Mean Average Precision (mAP). First, the IoU is computed for each predicted bounding box with respect to the ground truth boxes. Precision and recall are then calculated:

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}},\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(18)

The mean average precision (mAP) is then computed by averaging the APs across all classes:

$$\mathrm{mAP}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\mathrm{AP}}_i$$

(19)

where N is the number of classes, and ${\mathrm{AP}}_i$ is the average precision for class i. We specifically use mAP@0.5, which refers to the mAP calculated at an IoU threshold of 0.5.

Floating Point Operations (FLOPs). The FLOPs for a convolutional layer can be computed using the following formula:

$$\mathrm{FLOPs}=H\times W\times C_{\mathrm{out}}\times (2\times C_{\mathrm{in}}\times K_h\times K_w-1)$$

(20)

where H and W are the height and width of the output feature map, $C_{in}$ and $C_{out}$ are the number of input and output channels, and $K_h$ and $K_w$ are the height and width of the convolution kernel. For a fully connected layer, the FLOPs can be calculated as follows:

$$\mathrm{FLOPs}=2\times I\times O-O$$

(21)

where I and O are the numbers of input and output units, respectively. The total FLOPs for a neural network is the sum of the FLOPs of all its layers.

4.3. Comparison of Search Results

In this section, we compare the subnetwork search results of the proposed method (EGSSO) against existing random-pruning methods [24,32] for different pruning retention ratios (0.4, 0.5, 0.6) on both COCO and VisDrone datasets. All results discussed here correspond to subnetworks before fine-tuning.

Figure 5 integrates all experimental results into a single four-row layout: (1) top row: COCO histograms at 0.4, 0.5, 0.6; (2) second row: VisDrone histograms at 0.4, 0.5, 0.6; (3) third row: COCO scatter plots at the same ratios; (4) bottom row: VisDrone scatter plots. In each histogram, the horizontal axis indicates the mAP range, while the vertical axis gives the frequency of subnetworks falling into each bin. Subnetworks from the proposed method (blue) tend to occupy higher mAP bins more frequently, whereas the existing method (red) concentrates more mass in lower ranges. For example, at a retention ratio of 0.4 on COCO (top-left histogram), EGSSO shows a notably heavier tail in the >$0.01$ mAP region compared to the existing method. A similar pattern is observed on VisDrone (second-row histograms), where EGSSO consistently yields a more favorable mAP distribution.

Turning to the scatter plots (Figure 6), we visualize the mAP of each sampled subnetwork (vertical axis) against its identifier (horizontal axis). Red points correspond to the existing method and blue points to EGSSO. We also highlight the top 10 subnetworks from each approach with green (existing) or gold (proposed) markers. Notably, EGSSO typically occupies higher mAP bands, with its top subnetworks reaching significantly above the red markers. This clear separation underscores EGSSO’s capability to search more effectively for high-performing subnetworks under various pruning ratios.

Table 1. Comparison of subnetwork performance (before fine-tuning) between existing random-pruning methods [24,32] and EGSSO.

Dataset	Remaining Rate	Max-Existing	Max-Proposed	Search-Count-Existing Max	Search-Count-First-Exceed	Count-Exceeding-Existing Max
COCO	40%	$0.01117\pm 0.0010$	$\mathbf{0}.\mathbf{0280}\pm \mathbf{0}.\mathbf{0034}$	985	12	226
	50%	$0.06477\pm 0.0055$	$\mathbf{0}.\mathbf{08257}\pm \mathbf{0}.\mathbf{0066}$	1898	389	5
	60%	$0.17230\pm 0.0095$	$\mathbf{0}.\mathbf{22591}\pm \mathbf{0}.\mathbf{0151}$	751	1000	8
VisDrone	40%	$0.08417\pm 0.0125$	$\mathbf{0}.\mathbf{32014}\pm \mathbf{0}.\mathbf{0066}$	1302	1	1941
	50%	$0.18968\pm 0.0125$	$\mathbf{0}.\mathbf{33850}\pm \mathbf{0}.\mathbf{0076}$	897	1	984
	60%	$0.30729\pm 0.0092$	$\mathbf{0}.\mathbf{413522}\pm \mathbf{0}.\mathbf{0036}$	82	1	927

Bold values indicate the best performance.

further quantifies these observations. Remaining Rate denotes the target pruning ratio (e.g., 40%, 50%, 60%). Max-Existing and Max-Proposed record the highest mAP reached by the existing method and EGSSO, respectively. Search-Count-Existing Max is the number of subnetworks the existing method sampled before achieving its maximum mAP. Search-Count-First-Exceed indicates how many subnetworks EGSSO sampled before surpassing the existing method’s best mAP for the first time. Count-Exceeding-Existing Max shows how many of EGSSO’s subnetworks exceeded the existing method’s top mAP.

Analyzing these metrics, the results show that EGSSO not only achieves higher maximum mAP in every scenario but also does so more efficiently. For instance, at 40% remaining rate on COCO, the existing method requires 985 searches to reach 0.01367 mAP, whereas EGSSO surpasses that value after only 11 searches and ultimately attains a max of 0.02566 mAP. Similar patterns emerge at 50% and 60% on both COCO and VisDrone, demonstrating EGSSO’s robust advantage in search efficiency and performance.

In summary, both the distributional evidence from Figure 5 histograms and Figure 6 scatter plots and the summarized metrics from Table 1 confirm that EGSSO outperforms the existing random-pruning methods across a range of pruning ratios. By leveraging layer-wise entropy to guide retention ratios, EGSSO effectively restricts the search space to more promising subnetworks, leading to consistently higher mAP and faster convergence in subnetwork discovery.

4.4. Performance Comparison of Fine-Tuned Subnetwork

After obtaining candidate subnetworks, it is crucial to evaluate their performance afterfine-tuning.

Table 2. Results of pruning YOLOv5m on COCO with the target remaining rate of 0.5.

Method	mAP@0.5 (%)	mAP@0.5:0.95 (%)	FLOPs (G)	Parameters (M)
YOLOv5m [39] (baseline)	63.1	44.5	51.3	21.4
YOLOv5s [39]	55.4	36.7	17.0	7.3
YOLOv4-Tiny [43]	42.6	23.6	7.8	6.5
YOLOX-S [44]	58.7	39.6	26.8	9.0
EagleEye [32]	58.7	40.2	26.8	10.4
PAGCP [14]	60.7	41.5	23.5	7.7
DSD [25]	61.14	40.94	26.2	7.3
Ours	62.1	44.4	24.8	6.18

Bold values indicate the best performance.

and

Table 3. Results of pruning YOLOv5L on COCO with the target remaining rates of 0.5 and 0.6.

Method	Target Remaining Rate (FLOPs)	mAP@0.5 (%)	mAP@0.5:0.95 (%)	Parameters (M)	GFLOPs (Remaining Rate%)
Baseline	-	66.9	48.2	47.06	115.6
HFP [45]	60%	63.1	43.6	16.09	67.6 (58.5)
	50%	63.5	43.4	12.36	57.2 (49.5)
EagleEye [32]	60%	63.8	45.0	23.95	66.9 (57.9)
	50%	63.7	44.6	23.72	55.9 (48.4)
TCFP [46]	60%	63.6	44.2	28.73	72.8 (63.0)
	50%	61.8	42.7	24.08	61.9 (53.5)
Ours	60%	64.5	45.3	21.36	67.4 (58.3)
	50%	64.3	45.0	17.03	56.3 (48.7)

Bold values indicate the best performance.

present the post-fine-tuning mAP performance of EGSSO and several existing methods on two YOLOv5 variants (YOLOv5m and YOLOv5L) evaluated on the COCO dataset. We report mAP@0.5 and mAP@0.5:0.95 to measure detection accuracy under different IoU thresholds, along with FLOPs and the number of parameters to reflect the computational and model-size efficiency.

As shown in Table 2, our EGSSO approach achieves competitive accuracy on YOLOv5m (mAP@0.5 of 62.1% and mAP@0.5:0.95 of 44.4%), outperforming several existing pruning or lightweight baselines such as EagleEye, PAGCP, and DSD while reducing FLOPs to 24.8 G and parameters to only 6.18 M. This balance between accuracy and model size indicates EGSSO’s potential for resource-constrained environments that still demand high detection accuracy. Since PAGCP belongs to gradient-based saliency pruning, where channel importance is inferred from performance-aware gradients, comparing against it provides a qualitative reference. EGSSO still achieves higher accuracy with fewer FLOPs, suggesting that entropy-guided search-space refinement can outperform gradient-based saliency estimation while maintaining stronger efficiency.

Similarly, for YOLOv5L (Table 3), EGSSO maintains superior mAP at different remaining rates (e.g., 64.5% at 60% rate) and exhibits fewer parameters and lower GFLOPs compared to other competitors. Notably, when the target remaining rate is set to 60%, EGSSO yields 67.4 GFLOPs (58.3% of the baseline) with a mAP@0.5 of 64.5%, surpassing methods like HFP [45] and TCFP [46] by a significant margin in terms of overall efficiency.

Overall, these fine-tuned results validate the robustness of EGSSO across different YOLOv5 variants. By leveraging entropy-guided pruning and refining the subnetwork search space, EGSSO consistently finds subnetworks that preserve accuracy while reducing computational cost. This highlights the method’s effectiveness in practical deployment scenarios, where models must balance performance, speed, and memory footprint.

4.5. Ablation Study

This section analyzes the effect of each component in Entropy-Guided Search Space Optimization (EGSSO), focusing on outlier detection under different z-score thresholds and the impact on subnetwork performance (before fine-tuning).

4.5.1. Motivation and Setup

The core idea of EGSSO is to guide pruning ratios according to the layer-wise entropy values, but outlier detection can further refine these entropy values by removing anomalous measurements that could distort the min–max normalization. To assess this, several z-score thresholds ($0.0,0.5,1.0,1.5,2.0,\dots ,5.0$) are tested in detecting outlier entropy values. A smaller threshold flags more entropy values as outliers, while a larger threshold is more lenient. Figure 7 reports how many entropy values are deemed outliers at each threshold on COCO (left) and VisDrone (right).

4.5.2. Threshold Effect on Outlier Counts

Figure 7 shows that, at a z-score threshold of 0.0 or 0.5, nearly all 59 layers contain entropy values flagged as outliers. The count drops rapidly with even moderate thresholds like 1.0 or 1.5, and tends to zero beyond 2.5. This suggests that a moderate threshold ($z=1.5$ or $z=2.0$) focuses on genuine anomalies without over-filtering normal entropy values.

4.5.3. Influence on Subnetwork Search

To examine how outlier detection contributes to final subnetwork quality (before fine-tuning), four configurations are compared: (1) existing random pruning, (2) entropy-guided without outlier detection, (3) entropy-guided with $z=1.5$, and (4) entropy-guided with $z=2.0$. Subnetworks are ranked by mAP, and the top 20 are displayed in Figure 8 for COCO (left) and VisDrone (right) at a pruning retention ratio of 0.4.

Figure 8 indicates that the existing random pruning has lower mAP overall. The entropy-guided approach without outlier detection performs better but still trails the versions that include outlier detection. Using $z=1.5$ or $z=2.0$ shifts the top-20 curves upward, showing higher mAP in the best subnetworks on both datasets. A slightly relaxed threshold ($z=2.0$) preserves more mildly atypical entropy values, sometimes leading to a higher first-ranked or fifth-ranked mAP.

In summary, these ablation results validate that outlier detection effectively refines the entropy distribution, boosting the subnetwork sampling quality. A moderate threshold strikes a good balance between ignoring extremely distorted layers and retaining legitimate, high-entropy ones. Thus, we include outlier detection in our final EGSSO pipeline, as it consistently leads to improved top subnetwork performance across datasets and pruning ratios.

5. Discussion

In this paper, EGSSO demonstrates significant improvements in pruning efficiency and accuracy preservation on object detection architectures. By substantially reducing FLOPs and parameter counts while maintaining competitive mAP performance, our entropy-guided search-space optimization proves to be an effective strategy for structured pruning. These results highlight the strength of leveraging information entropy and outlier correction to refine the search space and discover high-quality subnetworks efficiently.

Hardware-level evaluation is also an essential aspect of assessing the practical deployability of pruning methods. While this paper focuses on algorithmic efficiency in terms of FLOPs and parameter reduction, future work will extend our analysis to real edge platforms, including NVIDIA Jetson modules, ARM-based mobile SoCs, and dedicated AI accelerators. Systematic measurements of inference latency, memory footprint, and energy consumption on these devices will provide a more comprehensive understanding of EGSSO’s effectiveness in resource-constrained settings and further validate its suitability for real-world deployment.

Although this paper focuses on object detection models, EGSSO’s framework is architecture-agnostic. The entropy computation, outlier correction, and search-space refinement, making the method naturally applicable to other visual tasks. Future research will explore extending EGSSO to image classification architectures such as ResNet-50 [47], and lightweight semantic segmentation models such as DeepLab [48]. These investigations will help further verify the generality and scalability of EGSSO across a broader range of deep learning applications.

6. Conclusions

This paper introduces EGSSO, an entropy-guided pruning framework that integrates outlier detection to reduce parameters while preserving accuracy. By refining subnetwork sampling and iteratively optimizing the pruned model, EGSSO effectively focuses computational resources on the most informative parts of the network, resulting in a more efficient and accurate pruning process. Experimental results on multiple benchmarks show that EGSSO not only discovers high-performing subnetworks more frequently but also achieves these results with fewer search iterations and reduced pruning time cost. Fine-tuning on YOLOv5m and YOLOv5L further verifies EGSSO’s ability to maintain strong mAP with fewer FLOPs and parameters, underscoring its suitability for deployment in resource-constrained settings. Future extensions could investigate EGSSO’s applicability to other architectures, such as Transformers, and explore combinations with quantization or knowledge distillation for additional efficiency gains. These avenues may broaden EGSSO’s scope as a robust and scalable approach to neural network compression in real-world applications.