Integrating Hybrid Attention Mechanisms into CNN-Based Architectures to Enhance Image Classification and Interpretability

Abstract

Integrating complementary attention mechanisms into standard Convolutional Neural Networks (CNNs) is a promising strategy for improving feature discrimination without substantial computational overhead. This paper presents a controlled empirical study of a hybrid attention module that combines Squeeze-and-Excitation Networks (SENet) and the Convolutional Block Attention Module (CBAM) through an adaptive element-wise summation with a learnable weighting parameter $\alpha$ and a residual connection. This work contributes a systematic and statistically rigorous evaluation of attention fusion across four CNN backbones (ResNet18, VGG16, AlexNet, and SqueezeNet) on the CIFAR-10 benchmark at $32\times 32$ resolution. All models were trained from scratch under a deliberately conservative protocol (50 epochs, no pretrained weights, standard augmentation) to isolate the incremental effect of attention mechanisms under controlled conditions. Under this protocol, the hybrid SENet+CBAM configuration achieves statistically significant accuracy improvements over the corresponding baselines ($p<0.001$, 5-fold cross-validation): ResNet18 improves from 77.93% to 90.71% (+12.78%), VGG16 from 55.78% to 70.17% (+14.39%), AlexNet from 62.67% to 71.82% (+9.15%), and SqueezeNet from 71.91% to 78.29% (+6.38%). These gains must be interpreted within the scope of this controlled setting. Absolute accuracy values are below fully optimized literature benchmarks. For VGG16 in particular, part of the improvement likely reflects correction of underfitting under the conservative protocol, not the full potential of the hybrid mechanism. Parameter overhead remains modest at 1.5–5.8%, and training convergence improves by 16.5% on average. The hybrid approach outperforms the best previously reported SENet+CBAM result for each architecture by an average of 2.32%. Grad-CAM visualizations and attention entropy analysis provide qualitative evidence of more concentrated spatial attention patterns under the hybrid configuration. These should be understood as proxy indicators rather than rigorous interpretability measures. Validation on higher-resolution benchmarks such as CIFAR-100, STL-10, and ImageNet subsets is a necessary next step before broader applicability can be claimed.

1. Introduction

Image classification remains a fundamental challenge in computer vision, with applications ranging from medical diagnostics to autonomous vehicles. While Convolutional Neural Networks (CNNs) have become the standard approach [1,2], their performance is often limited by an inability to dynamically prioritize the most discriminative image features [3,4].

1.1. Motivation

Recent developments in image classification have focused on improving CNN performance through deeper architectures and refined training techniques. However, conventional CNNs exhibit limited ability to adaptively weight features based on their semantic importance. This limitation manifests in three key aspects: (1) static feature extraction that cannot dynamically prioritize relevant patterns, (2) isolated treatment of spatial and channel-wise information without exploiting their synergistic interactions, and (3) insufficient mechanisms to understand model decisions.

Attention mechanisms have been proposed to address these shortcomings. Squeeze-and-Excitation Networks (SENet) introduced channel-wise recalibration and achieved first place in ILSVRC 2017 [5]; subsequent work such as ECA-Net [6] refined this idea with lighter channel interaction, and the Bottleneck Attention Module [7] extended it to parallel spatial-channel processing. The Convolutional Block Attention Module (CBAM) [8] further combined sequential channel and spatial attention into a single, modular block. Despite this progress, each mechanism addresses a different and complementary aspect of feature recalibration: SENet focuses on which channels matter globally, while CBAM additionally identifies where the relevant information resides within the feature map. Existing works deploy these modules in isolation [9]. Their complementary strengths remain unexploited. No study has systematically evaluated a principled fusion of both mechanisms across multiple CNN backbones with statistical rigor.

1.2. Research Objectives

This paper proposes a hybrid attention framework that systematically fuses SENet and CBAM into existing CNN architectures (ResNet18, VGG16, AlexNet, SqueezeNet). The fusion strategy integrates channel-wise and spatial attention mechanisms to adaptively recalibrate features across both dimensions simultaneously, enabling improved discrimination of semantically relevant patterns. SENet achieved first place in ILSVRC 2017 classification by recalibrating channel features [5], while CBAM provides complementary enhancement through dual spatial-channel modulation [8].

1.3. Key Contributions

We propose a hybrid attention framework combining SENet’s channel attention with CBAM’s spatial-channel attention through an adaptive element-wise summation strategy. The main contributions of this work are:

• Adaptive fusion parameter: A lightweight learnable weighting parameter $\alpha$ enables architecture-specific balancing between the two attention mechanisms. While the accuracy gain over fixed summation is modest, the adaptive formulation provides architectural flexibility and promotes stable optimization across diverse CNN backbones.
• Cross-architecture evaluation: Systematic evaluation across four CNN architectures (ResNet18, VGG16, AlexNet, and SqueezeNet) with architecture-specific analysis of fusion effectiveness and 5-fold cross-validation for statistical robustness.
• Efficiency-aware design: Computational analysis demonstrating minimal overhead (1.5–5.8% parameter increase) relative to the observed accuracy improvements across architectures.
• Integrated interpretability analysis: Grad-CAM visualization combined with attention entropy analysis to qualitatively assess the effect of attention mechanisms on feature localization.
• Multi-depth integration: The hybrid module is applied at multiple depths within each backbone (e.g., after each residual block in ResNet18, after each Fire module in SqueezeNet), enabling the network to recalibrate features at different levels of semantic abstraction.

1.4. Paper Organization

The remainder of this paper is organized as follows. Section 2 presents background on attention mechanisms and CNN architectures. Section 3 reviews related work. Section 4 details the proposed hybrid methodology. Section 5 presents experimental results and analysis. Section 6 concludes and outlines future directions.

2. Background

Attention mechanisms have become one of the most impactful advances in deep learning for computer vision. They allow a network to focus on the most informative parts of a feature map. Specific regions and channels receive higher weights than others. This selective weighting improves the quality of learned representations without requiring substantial additional computation.

This section introduces the two attention mechanisms at the core of our study, SENet and CBAM, and briefly describes the four CNN backbones used for evaluation: ResNet18, VGG16, AlexNet, and SqueezeNet.

2.1. Squeeze-and-Excitation Networks (SENet)

The Squeeze-and-Excitation (SE) module, introduced by Hu et al. [10], improves CNN representations by explicitly modeling interdependencies between feature channels. It operates through three successive stages.

2.1.1. Squeeze: Global Context Extraction

Given a feature map $\mathbf{X}\in {\mathbb{R}}^{H\times W\times C}$, global average pooling compresses spatial information into a channel descriptor $\mathbf{z}\in {\mathbb{R}}^C$, as described in Equation (1):

$${\mathbf{z}}_c={\displaystyle \frac{1}{H\times W}}{\displaystyle \sum _{i=1}^H}{\displaystyle \sum _{j=1}^W}{\mathbf{X}}_c(i,j).$$

(1)

2.1.2. Excitation: Channel-Wise Attention Generation

The descriptor $\mathbf{z}$ is passed through a two-layer fully connected network (a bottleneck with ReLU followed by a sigmoid gate) to produce a channel attention vector $\mathbf{s}\in {\mathbb{R}}^C$, as expressed in Equation (2):

$$\mathbf{s}=\sigma \left(W_2·\delta (W_1·\mathbf{z})\right),$$

(2)

where $W_1$ and $W_2$ are learnable weight matrices, $\delta$ is the ReLU activation, and $\sigma$ is the sigmoid function. Each entry of $\mathbf{s}$ encodes the relative importance of the corresponding channel.

2.1.3. Recalibration: Channel-Wise Feature Scaling

The attention vector $\mathbf{s}$ is then used to rescale the original feature map through element-wise multiplication, as formulated in Equation (3):

$${\mathbf{X}}_c^{\prime }={\mathbf{s}}_c·{\mathbf{X}}_c.$$

(3)

This recalibration amplifies informative channels while suppressing less relevant ones. The SE module is compact and modular, integrating into virtually any CNN architecture with negligible computational overhead [11].

2.2. Convolutional Block Attention Module (CBAM)

CBAM, proposed by Woo et al. [8], extends the channel-only recalibration of SENet by appending an explicit spatial attention stage. By addressing both what to emphasize and where to focus, CBAM provides a more comprehensive form of feature refinement.

2.2.1. Channel Attention

Unlike SENet, which relies solely on average pooling, CBAM aggregates complementary global statistics through both average and max pooling before feeding them into a shared MLP, as shown in Equation (4):

$${\mathbf{M}}_{\mathbf{c}}\left(\mathbf{F}\right)=\sigma \left(\mathrm{MLP}\left(\mathrm{AvgPool}\left(\mathbf{F}\right)\right)+\mathrm{MLP}\left(\mathrm{MaxPool}\left(\mathbf{F}\right)\right)\right),$$

(4)

where $\mathbf{F}$ is the input feature map and ${\mathbf{M}}_{\mathbf{c}}$ is the resulting channel attention map.

2.2.2. Spatial Attention

After channel refinement, CBAM identifies the most informative spatial locations by applying average and max pooling along the channel axis and fusing the result with a $7\times 7$ convolution, as given in Equation (5):

$${\mathbf{M}}_{\mathbf{s}}\left({\mathbf{F}}^{\prime }\right)=\sigma \left(f^{7\times 7}\left([\mathrm{AvgPool}\left({\mathbf{F}}^{\prime }\right);\mathrm{MaxPool}\left({\mathbf{F}}^{\prime }\right)]\right)\right),$$

(5)

where ${\mathbf{F}}^{\prime }$ denotes the channel-refined feature map and ${\mathbf{M}}_{\mathbf{s}}$ is the spatial attention map.

2.2.3. Sequential Attention Application

The two attention maps are applied sequentially to progressively refine the feature representation, as described in Equation (6):

$$\begin{array}{cc}\hfill {\mathbf{F}}^{\prime }& ={\mathbf{M}}_{\mathbf{c}}\left(\mathbf{F}\right)\otimes \mathbf{F},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathbf{F}}^{″}& ={\mathbf{M}}_{\mathbf{s}}\left({\mathbf{F}}^{\prime }\right)\otimes {\mathbf{F}}^{\prime },\hfill \end{array}$$

(7)

where ⊗ denotes element-wise multiplication. The final output ${\mathbf{F}}^{″}$ is refined in both channel and spatial dimensions, consistently improving classification accuracy across a range of CNN architectures [12].

2.3. CNN Architectures

Four representative CNN backbones are used in this study. Each embodies a distinct design philosophy, allowing us to assess whether the benefits of hybrid attention generalize across architectural paradigms.

2.3.1. VGG16: Depth Through Simplicity

VGG16, proposed by Simonyan and Zisserman [13,14,15], achieves strong representational power through a deliberately uniform design: stacked $3\times 3$ convolutional layers interleaved with $2\times 2$ max-pooling, followed by fully connected classification layers, as shown in Equation (8):

$$\mathrm{Input}\to \left[\mathrm{Conv}(3\times 3)\times N\right]\to \mathrm{MaxPool}\to \cdots \to \mathrm{FC}\to \mathrm{Softmax}.$$

(8)

Its regularity makes it an excellent testbed for evaluating attention plug-ins, though its large parameter count (138 M) limits deployment in resource-constrained settings.

2.3.2. ResNet18: Residual Learning

ResNet, introduced by He et al. [16], addresses the degradation problem observed in very deep networks through residual (skip) connections, as defined in Equation (9):

$$\mathbf{y}=\mathcal{F}(\mathbf{x},W_i)+\mathbf{x},$$

(9)

where $\mathbf{x}$ is the block input and $\mathcal{F}$ represents the learned residual mapping. These shortcuts allow gradients to flow directly through the network, enabling stable training at greater depth. We use ResNet18 for its favorable balance between representational capacity and computational cost.

2.3.3. SqueezeNet: Compact and Efficient

SqueezeNet [17,18] achieves AlexNet-level accuracy with fewer than 1.25 M parameters by replacing standard convolutions with Fire modules. Each Fire module consists of a squeeze layer of $1\times 1$ filters that reduces channel dimensionality, followed by an expand layer combining $1\times 1$ and $3\times 3$ filters to restore expressive capacity. Three design principles guide this compression: preferring $1\times 1$ over $3\times 3$ filters wherever possible, limiting input channels to $3\times 3$ filters, and delaying downsampling to preserve spatial resolution. The resulting model fits within 5 MB, making it well suited for edge and mobile deployment [2,3,19].

2.3.4. AlexNet: Deep Learning Pioneer

AlexNet [17,20] set the foundation for modern CNN design when it won the 2012 ILSVRC challenge by a substantial margin. Its five convolutional layers, followed by three fully connected layers, established a now-standard blueprint, as presented in Equation (10):

$$\mathrm{Input}\to [\mathrm{Conv}\to \mathrm{ReLU}]\to \mathrm{MaxPool}\to \cdots \to \mathrm{FC}\to \mathrm{Softmax}.$$

(10)

Two innovations proved especially influential: ReLU activations, which mitigate vanishing gradients and accelerate convergence, and dropout regularization in the fully connected layers, which substantially reduces overfitting. Although AlexNet is shallower than modern architectures, it remains a useful baseline for studying the effect of attention on less-deep feature hierarchies.

3. Related Work

3.1. Evolution of Attention Mechanisms in CNNs

Attention mechanisms in computer vision have evolved considerably since their introduction, fundamentally reshaping how neural networks process visual information. Inspired by the selectivity of human visual attention [21], early approaches focused on spatial attention through techniques such as Class Activation Mapping [22], which identifies the image regions most responsible for a given classification decision. These pioneering methods laid the groundwork for more sophisticated modules that operate across multiple feature dimensions simultaneously.

3.2. Channel-Wise Attention Mechanisms

Squeeze-and-Excitation Networks (SENet) [10] marked a significant turning point by introducing dynamic channel recalibration, winning first place in ILSVRC 2017. By explicitly modeling interdependencies between feature channels through a squeeze–excitation–scale pipeline, SENet demonstrated that selectively amplifying informative channels yields consistent gains across classification, detection, and segmentation tasks.

Building on this insight, Efficient Channel Attention (ECA-Net) [6] refined SENet’s design by removing the dimensionality reduction bottleneck and replacing it with a lightweight 1D convolution of adaptive kernel size k. This change reduces the number of additional parameters to just k while achieving accuracy on par with SENet, suggesting that avoiding information compression in the excitation stage is both more effective and more efficient.

More recent advances include Selective Kernel Networks [23], which allow neurons to dynamically adjust their receptive field size depending on the input scale, and ConvNeXt [24], which incorporates channel attention principles into a modernized CNN design competitive with Vision Transformers. These works show a clear trend toward making channel-wise recalibration lighter and more broadly applicable.

3.3. Multi-Dimensional Attention Mechanisms

The Convolutional Block Attention Module (CBAM) [8] extended purely channel-based recalibration by appending an explicit spatial attention stage. Applied sequentially (channel refinement followed by spatial refinement), CBAM enables the network to reason jointly about what features matter and where they appear, yielding consistent improvements across a broad range of CNN architectures.

An alternative design philosophy is embodied by the Bottleneck Attention Module (BAM) [7], which processes channel and spatial attention branches in parallel rather than sequentially, combining their outputs via element-wise multiplication. This parallel formulation preserves more information by allowing both branches to operate on the full input simultaneously and has shown competitive results on ImageNet and COCO.

Coordinate Attention (CA) [25] takes a different approach by embedding precise positional information directly into channel attention. It factorizes 2D spatial attention into two complementary 1D encoding processes. CA captures long-range dependencies along horizontal and vertical axes independently. This substantially reduces computational complexity while retaining strong localization ability, a property particularly valuable for mobile network design.

Finally, the Global Attention Mechanism (GAM) [26] jointly preserves information across all three feature dimensions (height, width, and channel) through a three-dimensional permutation strategy, achieving improved performance with minimal additional overhead and reduced information dispersion compared to sequential designs.

3.4. Hybrid and Fusion Approaches

Combining different attention types in a principled way remains an open and challenging problem. Hybrid CNN-Transformer approaches [27,28] have explored merging convolutional and self-attention operations, but these methods are typically designed for narrow task-specific settings such as image super-resolution or medical image segmentation, introducing architectural constraints that are difficult to transfer to standard CNN classification pipelines.

Several works have attempted to combine SENet and CBAM [29,30], yet these efforts share recurring limitations. Ganguly et al. [29], for instance, fuse attention outputs through simple concatenation, incurring a 12.3% parameter increase while gaining only 2.1% in accuracy, a poor efficiency trade-off. Other prior works apply fixed summation weights, preventing any architecture-specific adaptation. More broadly, these approaches tend to suffer from loose integration via naive concatenation or addition; the absence of explainability components; fusion strategies that ignore architecture-specific characteristics; and a lack of systematic evaluation across diverse CNN backbones.

Most existing fusion methods rely on fixed, non-adaptive weighting. Others validate on only a single architecture. Few provide both efficiency and interpretability analysis together.

3.5. Vision Transformers and Modern Attention

The emergence of Vision Transformers (ViTs) [31] brought a paradigm shift through multi-head self-attention, enabling global context modeling from the very first layer. However, ViTs require vast training sets (upwards of 300 M images for optimal performance) and lack the spatial inductive biases inherent to CNNs, limiting their applicability on moderately sized datasets such as CIFAR-10. The Swin Transformer [32] partially addressed these issues by introducing hierarchical attention with shifted windows, achieving a better balance between global modeling and CNN-like inductive bias, though still at a computational cost four to eight times greater than comparable CNN architectures.

Recent hybrid architectures [33,34,35] combine convolutional layers with transformer blocks to leverage the strengths of both paradigms. CoAtNet, for example, achieves strong accuracy but requires more than 2B FLOPs per inference pass, over ten times that of ResNet-50, together with complex multi-stage training schedules. Such resource demands render these models impractical in constrained deployment scenarios. ConvNeXt [24] offers a more accessible alternative by modernizing a standard CNN with transformer-inspired design choices such as depthwise convolutions and layer normalization, achieving competitive performance without the heavy infrastructure requirements of full transformer training.

These developments show that computationally efficient attention mechanisms can deliver meaningful feature recalibration gains. This is the core objective motivating our hybrid approach.

3.6. Research Gap and Our Contribution

The literature reviewed above reveals a consistent pattern. Existing studies have either examined SENet or CBAM in isolation, applied naive fusion strategies such as fixed-weight summation or concatenation, or validated findings on a single architecture without cross-architectural generalization analysis. Interpretability is rarely addressed in a systematic way, and computational efficiency is seldom reported alongside accuracy.

Our work addresses these shortcomings by proposing an adaptive element-wise summation fusion that learns the relative contribution of each attention mechanism through a trainable parameter $\alpha$, applied consistently across four CNN backbones (ResNet18, VGG16, AlexNet, and SqueezeNet). The approach achieves accuracy improvements of 5.3–14.6% with a parameter overhead of only 1.5–5.8%, compared to the 12.3% overhead for a 2.1% gain reported by Ganguly et al. [29], a 2.5× improvement in parameter utilization efficiency on average. Grad-CAM visualization and attention entropy analysis are integrated throughout to provide qualitative evidence of the effect of attention on feature localization, as further discussed in Section 5.

4. Methodology

4.1. Problem Statement

Despite their strong performance on standard benchmarks, CNNs process feature channels and spatial locations with uniform weighting, making it difficult to adaptively prioritize the most discriminative patterns in a given input. This rigidity limits both classification accuracy and the interpretability of model decisions [1]. To address this, we propose a hybrid attention framework that jointly leverages channel-wise and spatial recalibration within a single, lightweight module applicable to any standard CNN backbone.

4.2. Hybrid Attention Framework

Our framework systematically integrates Squeeze-and-Excitation Networks [10] and the Convolutional Block Attention Module [8] into a unified fusion block. The rationale is straightforward: SENet excels at global channel-wise recalibration (identifying which feature maps matter), while CBAM complements this by also reasoning about where the relevant information lies within those maps. Used in isolation, each mechanism captures only part of the picture; combining them allows the network to exploit both dimensions simultaneously.

4.3. Theoretical Foundation for Fusion Strategy

4.3.1. Attention Mechanism Complementarity

SENet and CBAM operate on distinct aspects of the feature representation. SENet captures global channel dependencies through its squeeze–excitation pipeline, producing a single scalar attention weight per channel based on spatially aggregated statistics. CBAM applies sequential channel and spatial attention, explicitly localizing the informative regions of the feature map in addition to rescaling channels. The complementarity between the two stems from three structural differences: SENet’s global channel statistics versus CBAM’s spatial-channel localization; single-stage versus sequential attention processing; and differing receptive field characteristics. Combining both mechanisms covers a broader range of feature recalibration than either can achieve alone.

4.3.2. Fusion Mechanism Selection

To determine the most effective way to combine the two attention outputs, we conducted preliminary experiments on ResNet18 comparing four candidate strategies. The results are reported in Table 1. Concatenation followed by a $1\times 1$ projection convolution introduces a 12.3% parameter overhead for only marginal accuracy gain. Element-wise multiplication risks suppressing informative features when attention weights are small, particularly in early training. Fixed element-wise summation avoids both drawbacks but treats every architecture identically, precluding any backbone-specific adaptation. Adaptive weighted summation retains the advantages of direct summation while introducing a single scalar parameter $\alpha$ that the network can tune during training. This enables architecture-specific balancing at negligible cost.

Table 1. Preliminary Fusion Strategy Comparison on ResNet18 (CIFAR-10 single-split validation, no early stopping).

Fusion Strategy	Acc. (%)	Param. Increase (%)	FLOPs (M)
Concatenation + $1\times 1$ Conv	91.2	12.3	558
Element-wise Multiplication	91.7	0.0	556
Fixed Element-wise Summation	93.5	0.0	556
Adaptive Weighted Sum ($\alpha$ learnable)	93.8	<0.01	556

The values in Table 1 were obtained under a simplified single-split protocol without cross-validation or early stopping, and therefore differ from the final 5-fold cross-validated results. The 3.09 percentage-point discrepancy reflects the stricter evaluation framework adopted for all final comparisons and should not be interpreted as inconsistency.

The improvement of adaptive over fixed summation is modest (0.3% on the preliminary validation set). The primary benefit of $\alpha$ is therefore architectural flexibility rather than a large accuracy gain. Both SENet and CBAM produce feature maps in the same space ${\mathbb{R}}^{H\times W\times C}$, so weighted summation is mathematically valid without any dimension transformation. Direct summation preserves clear gradient paths to both attention branches throughout backpropagation.

4.4. Hybrid Fusion Implementation

4.4.1. Mathematical Formulation

Given an input feature map $X\in {\mathbb{R}}^{H\times W\times C}$, where H, W, and C denote height, width, and number of channels, respectively, the hybrid module processes features in three steps.

First, the two attention modules are applied independently to the same input, as illustrated in Equations (11) and (12):

$$\begin{array}{cc}\hfill X_{\mathrm{SENet}}& =\mathrm{SENet}(X;{\Theta }_{\mathrm{SE}})\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill X_{\mathrm{CBAM}}& =\mathrm{CBAM}(X;{\Theta }_{\mathrm{CBAM}})\hfill \end{array}$$

(12)

where ${\Theta }_{\mathrm{SE}}$ and ${\Theta }_{\mathrm{CBAM}}$ denote the learnable parameters of each module.

The two attention-enhanced representations are then combined through adaptive weighted summation, as shown in Equation (13):

$$X_{\mathrm{hybrid}}=\alpha ·X_{\mathrm{SENet}}+(1-\alpha )·X_{\mathrm{CBAM}}$$

(13)

where $\alpha \in [0,1]$ is a scalar learnable parameter initialized at 0.5 and updated via gradient descent. Since $\alpha$ is a global scalar, the operation is a convex combination of the two attention-enhanced feature maps, introducing no additional spatial or channel-wise asymmetry beyond the balance controlled by $\alpha$ itself. In our experiments, $\alpha$ converges to architecture-specific values: ResNet18 ($\alpha \approx 0.6$), VGG16 ($\alpha \approx 0.4$), AlexNet ($\alpha \approx 0.5$), and SqueezeNet ($\alpha \approx 0.7$), reported as means across the five cross-validation folds (standard deviation $<0.04$ in all cases), reflecting stable convergence and differing degrees of reliance on channel versus spatial-channel attention across backbones.

Finally, a residual connection is added to preserve the original feature information, as expressed in Equation (14):

$$X_{\mathrm{final}}=X+X_{\mathrm{hybrid}}$$

(14)

Becaus both SENet and CBAM apply learned gating functions to X, their outputs $X_{\mathrm{SENet}}$ and $X_{\mathrm{CBAM}}$ are recalibrated versions of X that may suppress activations deemed less informative. The additive skip connection ensures that the original, unmodified features remain accessible to subsequent layers, preventing irreversible information loss caused by aggressive attention suppression and maintaining stable gradient flow during backpropagation. This role is analogous to, but distinct from, the residual shortcut in ResNet: here, the skip connection does not bypass a nonlinear transformation block but rather anchors the representation to the original signal after attention-driven recalibration.

Table 2 summarizes the full processing pipeline of the hybrid module.

Table 2. Hybrid SENet-CBAM Attention Module: Processing Steps.

Step	Operation	Description
Input	$X\in {\mathbb{R}}^{H\times W\times C}$	Input feature map
1	$X_{\mathrm{SENet}}\leftarrow \mathrm{SENet}(X;{\Theta }_{\mathrm{SE}})$	Channel-wise recalibration
2	$X_{\mathrm{CBAM}}\leftarrow \mathrm{CBAM}(X;{\Theta }_{\mathrm{CBAM}})$	Spatial-channel attention
3	$X_{\mathrm{hybrid}}\leftarrow \alpha ·X_{\mathrm{SENet}}+(1-\alpha )·X_{\mathrm{CBAM}}$	Adaptive weighted fusion
4	$X_{\mathrm{final}}\leftarrow X+X_{\mathrm{hybrid}}$	Residual connection
Output	$X_{\mathrm{final}}\in {\mathbb{R}}^{H\times W\times C}$	Attention-enhanced map

4.4.2. Implementation Details

SENet Configuration

For a feature map $X\in {\mathbb{R}}^{H\times W\times C}$, global average pooling first reduces spatial dimensions to a channel descriptor $z\in {\mathbb{R}}^{1\times 1\times C}$. A two-layer MLP with reduction ratio r = 16 then produces channel attention weights, as defined in Equation (15):

$$s=\sigma \left(W_2·\mathrm{ReLU}(W_1·z)\right)$$

(15)

where $W_1\in {\mathbb{R}}^{C/r\times C}$, $W_2\in {\mathbb{R}}^{C\times C/r}$, and $\sigma$ is the sigmoid function. Each channel is then rescaled as ${\tilde{X}}_c=s_c·X_c$.

CBAM Configuration

CBAM first computes channel attention by aggregating global average and max pooling statistics through a shared MLP, as given in Equation (16):

$$M_c\left(X\right)=\sigma \left(\mathrm{MLP}\left({\mathcal{F}}_{\mathrm{avg}}^c\left(X\right)\right)+\mathrm{MLP}\left({\mathcal{F}}_{\max }^c\left(X\right)\right)\right)$$

(16)

Spatial attention is then applied to the channel-refined map $X^{\prime }=M_c\left(X\right)\otimes X$ using a $7\times 7$ convolution over concatenated spatial pooling features, as formulated in Equation (17):

$$M_s\left(X^{\prime }\right)=\sigma \left(f^{7\times 7}\left([{\mathcal{F}}_{\mathrm{avg}}^s\left(X^{\prime }\right);{\mathcal{F}}_{\max }^s\left(X^{\prime }\right)]\right)\right)$$

(17)

Both MLP branches share weights and use a reduction ratio $r=16$.

4.4.3. Architecture-Specific Integration Points

The hybrid module was inserted at depth-specific locations within each backbone to maximize attention effectiveness while preserving computational efficiency. In ResNet18, the module was placed after each residual block, where skip connections already provide stable gradient flow. In VGG16, it was inserted after each convolutional block and before the subsequent max-pooling layer, ensuring that spatial attention operates at full resolution. In AlexNet, the module followed each convolutional layer in the feature extraction stage. In SqueezeNet, it was applied after each Fire module to maintain the architecture’s compact design. This multi-depth integration allowed the network to recalibrate features at progressively higher levels of semantic abstraction, from low-level textures in early layers to object-level structures in deeper ones.

4.5. Experimental Configuration

To isolate the contribution of each attention component, four configurations were evaluated for every architecture, as detailed in Table 3. Case 1 (no attention) establishes the baseline; Cases 2 and 3 isolate the individual contributions of SENet and CBAM, respectively; and Case 4 evaluates the synergistic effect of the proposed hybrid fusion. Keeping all other training settings identical across the four cases ensures that any observed performance difference is attributable solely to the attention mechanism.

Table 3. Evaluated configurations across CNN architectures.

Model	Case 1	Case 2	Case 3	Case 4
	None	SENet	CBAM	SENet + CBAM
SqueezeNet	✓	✓	✓	✓
ResNet18	✓	✓	✓	✓
VGG16	✓	✓	✓	✓
AlexNet	✓	✓	✓	✓

4.6. Training Protocol

All models were trained from scratch on CIFAR-10 at the native $32\times 32$ resolution, without ImageNet pretrained weights. This choice ensures that observed performance differences arise strictly from the attention mechanisms rather than from transferred feature representations. No CIFAR-specific architectural modifications, such as widened layers, depth scaling, or resolution adaptation, were introduced so that all four backbones are evaluated under identical conditions.

The hyperparameter configuration is summarized in Table 4. Values were selected through preliminary stability experiments to balance convergence speed and generalization. Aggressive optimization strategies commonly used in state-of-the-art CIFAR-10 benchmarks were deliberately avoided. These include Cutout, AutoAugment, Mixup, and 200–300 epoch schedules. This conservative protocol prioritizes a controlled evaluation of attention contributions over maximizing absolute accuracy. As discussed in Section 5, the reported baseline accuracies are below literature benchmarks. Observed improvements should be interpreted in this context rather than as direct comparisons with fully optimized systems.

Table 4. Training protocol and hyperparameter configuration.

Parameter	Configuration
Optimizer	Adam, $\eta =0.003$
LR schedule	ReduceLROnPlateau (factor 0.5, patience 3)
Batch size	128
Training duration	50 epochs, early stopping (patience 5)
Regularization	L2 weight decay $\lambda ={10}^{-4}$, Dropout $p=0.5$ (FC layers)
Data augmentation	Random horizontal flip ($p=0.5$), random crop with 4-pixel padding, rotation $\pm {15}^{\circ }$
Normalization	Per-channel mean 0.5, std 0.5
Validation	5-fold cross-validation

4.7. Evaluation Framework

4.7.1. Performance Metrics

Model performance was assessed using top-1 classification accuracy, precision, recall, and F1-score (both per-class and macro-averaged), computed across all five cross-validation folds and reported as means with 95% confidence intervals. Training dynamics were monitored through validation loss curves, and computational cost was characterized by parameter count, FLOPs, inference time, and peak GPU memory usage.

4.7.2. Interpretability Analysis

To qualitatively assess the effect of attention on feature localization, we applied Grad-CAM [36] to visualize the spatial regions most influential for each model’s predictions, as shown in Figure 1. These visualizations were produced for all four attention configurations, including side-by-side comparisons between baseline and hybrid models for the same input samples to provide direct visual evidence of how the hybrid mechanism affects attention focus. Attention entropy was additionally computed as a quantitative proxy for localization precision, as described in Section 5. Grad-CAM and entropy analysis provide qualitative indicators of interpretability. They are not rigorous ground-truth measures.

Figure 1. Hybrid SENet-CBAM architecture. SENet and CBAM are applied independently to the same input feature map; their outputs are combined through adaptive weighted fusion with learnable parameter $\alpha$, followed by a residual connection. Grad-CAM is applied at inference time for interpretability analysis.

5. Experimental Setup, Results, and Discussion

5.1. Dataset and Preprocessing

All experiments were conducted on CIFAR-10, a widely used benchmark comprising 60,000 color images of size $32\times 32$ pixels drawn from ten object categories [37,38]. The standard split of 50,000 training and 10,000 test images was used throughout. Preprocessing included per-channel normalization with $\mu =[0.4914,0.4822,0.4465]$ and $\sigma =[0.247,0.243,0.261]$, together with random horizontal flipping ($p=0.5$) and random cropping with four-pixel padding.

CIFAR-10 is a low-resolution, relatively constrained benchmark. The results reported here are specific to this setting. They should not be extrapolated to higher-resolution datasets or other domains without further experimental validation (see Section 5.10.4).

5.2. Hardware and Software Configuration

Table 5 summarizes the hardware and software environment used for all experiments. Training was carried out on a single consumer-grade GPU, confirming that the hybrid attention approach is accessible without specialized infrastructure.

Table 5. Experimental hardware and software configuration.

Component	Specification
Processor	AMD Ryzen 5 7535HS (6c/12t, 3.3 GHz/4.55 GHz boost)
Memory	16 GB DDR4
GPU	NVIDIA GeForce RTX 3050 Ti (4 GB GDDR6 VRAM)
Software	Python 3.9, PyTorch 1.12, CUDA 11.7
Training time (50 epochs)	ResNet18: 1.8 h    VGG16: 3.4 h
	AlexNet: 2.0 h    SqueezeNet: 1.2 h
Batch size	128 images per iteration (3.2 ms per batch)

5.3. Classification Performance Analysis

All experiments were conducted with 5-fold cross-validation. Statistical significance was assessed using paired t-tests ($\alpha =0.05$), and all results are reported as means with 95% confidence intervals across folds.

5.3.1. Primary Classification Metrics

Table 6 reports accuracy, precision, recall, and F1-score for all sixteen model–attention combinations, with F1 computed as $F1={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$.

Table 6. Classification performance with statistical validation (mean ± 95 % CI). p-values are relative to the corresponding baseline using a paired t-test.

Model	Attention	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	p-Value
Baseline (no attention)
SqueezeNet	None	$71.91\pm 1.2$	$81.04\pm 1.8$	$82.96\pm 2.1$	$81.98\pm 1.5$	–
ResNet18	None	$77.93\pm 0.9$	$84.65\pm 1.3$	$84.70\pm 1.4$	$84.68\pm 1.1$	–
VGG16	None	$55.78\pm 2.3$	$56.98\pm 2.8$	$53.81\pm 3.1$	$55.35\pm 2.6$	–
AlexNet	None	$62.67\pm 1.7$	$61.13\pm 2.2$	$60.94\pm 2.4$	$61.04\pm 2.0$	–
SENet only
SqueezeNet	SENet	$74.19\pm 1.1$	$87.05\pm 1.4$	$87.02\pm 1.6$	$87.04\pm 1.2$	<0.01
ResNet18	SENet	$81.22\pm 0.8$	$84.88\pm 1.1$	$84.88\pm 1.2$	$84.88\pm 0.9$	<0.05
VGG16	SENet	$57.01\pm 2.1$	$61.02\pm 2.4$	$58.75\pm 2.7$	$59.86\pm 2.3$	<0.05
AlexNet	SENet	$60.23\pm 1.9$	$69.28\pm 2.0$	$68.39\pm 2.3$	$68.83\pm 1.8$	<0.01
CBAM only
SqueezeNet	CBAM	$76.40\pm 1.0$	$86.47\pm 1.3$	$88.76\pm 1.5$	$87.60\pm 1.1$	<0.001
ResNet18	CBAM	$81.87\pm 0.7$	$89.12\pm 1.0$	$79.10\pm 1.3$	$83.82\pm 1.0$	<0.01
VGG16	CBAM	$66.83\pm 1.8$	$70.95\pm 2.1$	$69.12\pm 2.4$	$70.02\pm 1.9$	<0.001
AlexNet	CBAM	$67.44\pm 1.6$	$79.95\pm 1.8$	$79.43\pm 2.0$	$79.69\pm 1.7$	<0.001
Hybrid SENet + CBAM (proposed)
SqueezeNet	SENet + CBAM	$78.29\pm 0.9$	$93.21\pm 1.1$	$88.03\pm 1.3$	$90.56\pm 1.0$	<0.001
ResNet18	SENet + CBAM	$90.71\pm 0.6$	$94.26\pm 0.8$	$85.46\pm 1.0$	$89.65\pm 0.7$	<0.001
VGG16	SENet + CBAM	$70.17\pm 1.5$	$72.33\pm 1.7$	$78.11\pm 1.9$	$75.12\pm 1.6$	<0.001
AlexNet	SENet + CBAM	$71.82\pm 1.4$	$75.76\pm 1.6$	$71.51\pm 1.8$	$73.59\pm 1.5$	<0.001

The hybrid approach consistently outperforms both individual attention mechanisms across all architectures. One noteworthy result is that SENet alone yields a $-2.44\%$ accuracy drop on AlexNet relative to its baseline. Shallow architectures with limited channel redundancy may not benefit from aggressive global channel recalibration. When the feature hierarchy is shallow, channel re-weighting based on spatially pooled statistics can suppress informative activations rather than amplify them. This highlights the importance of architectural depth as a moderating factor in the effectiveness of channel-only attention.

5.3.2. Pairwise Statistical Comparison: Hybrid vs. Individual Mechanisms

To verify that the hybrid approach is not simply equivalent to either individual mechanism, we conducted additional paired t-tests comparing SENet+CBAM against SENet-only and CBAM-only configurations for each architecture. The hybrid significantly outperforms SENet alone ($p<0.05$) across all four backbones. Against CBAM alone, significant gains are observed for ResNet18, VGG16, and AlexNet ($p<0.05$), while the improvement for SqueezeNet is consistent in direction but does not reach conventional significance ($p=0.07$), likely reflecting the already competitive performance of CBAM on this compact architecture. These results confirm that the hybrid fusion provides genuine additive value beyond either constituent mechanism.

5.3.3. Performance Improvement Summary

Table 7 quantifies the absolute gains of the hybrid approach over the corresponding baselines across all metrics.

Table 7. Absolute performance gains of hybrid SENet+CBAM over baseline (no attention).

Model	Acc. Gain	Prec. Gain	Rec. Gain	F1 Gain
	(%)	(%)	(%)	(%)
SqueezeNet	$+6.38$	$+12.17$	$+5.07$	$+8.58$
ResNet18	$+12.78$	$+9.61$	$+0.76$	$+4.97$
VGG16	$+14.39$	$+15.35$	$+24.30$	$+19.77$
AlexNet	$+9.15$	$+14.63$	$+10.57$	$+12.55$
Mean	$+10.68$	$+12.94$	$+10.18$	$+11.47$

The gains are substantial across all architectures, with VGG16 showing the largest relative improvement (+14.39% accuracy, +19.77% F1). As discussed in Section 5.10.4, the baseline accuracy of VGG16 is notably low (55.78%). This may partly reflect underfitting under the conservative 50-epoch training protocol rather than the full benefit attributable to attention alone. The improvements should therefore be interpreted as evidence of the attention mechanism’s contribution within this controlled experimental setting, not as an absolute measure of its potential under fully optimized conditions.

5.3.4. Computational Efficiency

Table 8 reports the computational cost of the hybrid module relative to each baseline. Peak GPU memory usage was measured using torch.cuda.max_memory_allocated() at batch size 128 on the RTX 3050 Ti.

Table 8. Computational cost analysis. Memory overhead measured at batch size 128 on RTX 3050 Ti.

Model	Params	FLOPs	Infer.	Peak Mem.	Param.
+ Attention	(M)	(M)	(ms)	(MB)	Overhead
Baseline
ResNet18	11.17	556.3	$2.3\pm 0.1$	412	–
VGG16	138.36	15,300.2	$8.7\pm 0.3$	1840	–
AlexNet	61.10	714.8	$3.1\pm 0.2$	623	–
SqueezeNet	1.25	351.9	$1.8\pm 0.1$	187	–
Hybrid SENet + CBAM
ResNet18 + H	11.82	578.6	$2.5\pm 0.1$	438	$+5.8\%$
VGG16 + H	142.45	15,489.7	$9.2\pm 0.3$	1897	$+3.0\%$
AlexNet + H	62.03	731.2	$3.3\pm 0.2$	649	$+1.5\%$
SqueezeNet + H	1.32	364.8	$1.9\pm 0.1$	203	$+5.6\%$

The hybrid module introduces only modest overhead across all dimensions. Parameter counts increase by 1.5–5.8%, FLOPs by 2–4%, inference time by less than 0.2 ms, and peak GPU memory by 3–6%. These figures confirm that the performance gains reported in Table 6 are not purchased at disproportionate computational cost.

5.3.5. Training Dynamics and Convergence

Table 9 compares convergence characteristics between baseline and hybrid models. Hybrid models reach their best validation performance in fewer epochs on average, suggesting that joint channel and spatial recalibration provides a more informative gradient signal during early training.

Table 9. Training convergence analysis.

Model	Epochs to	Train	Val	Convergence
	Convergence	Loss	Loss	Speedup (%)
Baseline
ResNet18	47	0.421	0.545	–
VGG16	50	1.234	1.803	–
AlexNet	49	0.987	1.363	–
SqueezeNet	48	0.654	0.796	–
Hybrid SENet + CBAM
ResNet18 + H	40	0.298	0.533	$+14.9$
VGG16 + H	42	0.876	0.633	$+16.0$
AlexNet + H	41	0.723	0.625	$+16.3$
SqueezeNet + H	39	0.489	0.603	$+18.8$
Mean	–	–	–	$+16.5$

The lower validation loss compared to training loss observed for VGG16 + Hybrid is explained by dropout regularization being active during training but disabled at inference time, which produces deterministically lower validation loss under identical weights. No data leakage was detected across cross-validation folds.

5.3.6. Statistical Effect Size Analysis

Beyond p-values, effect sizes were computed using Cohen’s d to characterize the practical magnitude of improvements. Large effect sizes ($d>0.8$) are observed for accuracy gains in VGG16 and ResNet18, while medium-to-large sizes ($d>0.5$) are recorded for all remaining architecture–metric pairs. The consistency of positive effects across all sixteen comparisons provides strong evidence that the observed improvements are not artifacts of random variation.

5.4. Validation Performance and Generalization

Table 10 extends the analysis to validation loss and generalization metrics, confirming that hybrid attention reduces not only classification error but also the training-to-validation gap, a concrete indicator of improved generalization.

Table 10. Validation performance and generalization analysis (mean ± 95 % CI; p-values vs. baseline).

Model	Attention	Val Loss	Accuracy (%)	Loss Reduction (%)	p-Value
Baseline
ResNet18	None	$0.545\pm 0.023$	$77.93\pm 0.9$	–	–
VGG16	None	$1.803\pm 0.087$	$55.78\pm 2.3$	–	–
AlexNet	None	$1.363\pm 0.045$	$62.67\pm 1.7$	–	–
SqueezeNet	None	$0.796\pm 0.031$	$71.91\pm 1.2$	–	–
SENet only
ResNet18	SENet	$0.546\pm 0.021$	$81.22\pm 0.8$	$-0.18$	<0.05
VGG16	SENet	$1.710\pm 0.082$	$57.01\pm 2.1$	$+5.16$	<0.05
AlexNet	SENet	$1.209\pm 0.041$	$60.23\pm 1.9$	$+11.30$	<0.01
SqueezeNet	SENet	$0.703\pm 0.028$	$74.19\pm 1.1$	$+11.68$	<0.01
CBAM only
ResNet18	CBAM	$0.529\pm 0.019$	$81.87\pm 0.7$	$+2.94$	<0.01
VGG16	CBAM	$1.696\pm 0.079$	$66.83\pm 1.8$	$+5.93$	<0.001
AlexNet	CBAM	$0.931\pm 0.038$	$67.44\pm 1.6$	$+31.69$	<0.001
SqueezeNet	CBAM	$0.682\pm 0.026$	$76.40\pm 1.0$	$+14.32$	<0.001
Hybrid SENet+CBAM (proposed)
ResNet18	SENet + CBAM	$0.533\pm 0.018$	$90.71\pm 0.6$	$+2.20$	<0.001
VGG16	SENet + CBAM	$1.633\pm 0.074$	$70.17\pm 1.5$	$+9.43$	<0.001
AlexNet	SENet + CBAM	$0.625\pm 0.032$	$71.82\pm 1.4$	$+54.13$	<0.001
SqueezeNet	SENet + CBAM	$0.603\pm 0.024$	$78.29\pm 0.9$	$+24.25$	<0.001

Overfitting and Generalization Metrics

Table 11 reports the train-to-validation accuracy gap and a Generalization Index defined as GI = Val_Acc/Train_Acc, where values closer to 1.0 indicate lower overfitting. Qualitative overfitting labels have been replaced by this continuous, directly interpretable measure.

Table 11. Generalization analysis. Generalization Index (GI) = Val_Acc / Train_Acc; values closer to 1.0 indicate lower overfitting. Train-Val Gap = Train_Acc − Val_Acc (%).

Model	Train-Val Gap (%)	Val Std Dev	GI
Baseline
ResNet18	3.21	0.94	0.876
VGG16	8.47	2.31	0.743
AlexNet	6.82	1.73	0.798
SqueezeNet	4.15	1.22	0.834
Hybrid SENet + CBAM
ResNet18 + H	$2.18$	$0.61$	$0.912$
VGG16 + H	$5.23$	$1.48$	$0.823$
AlexNet + H	$4.67$	$1.35$	$0.867$
SqueezeNet + H	$3.01$	$0.87$	$0.891$

Across all backbones, the hybrid module consistently reduces the train-to-validation gap (average reduction: 32.1%) and increases the Generalization Index, indicating that joint channel-spatial recalibration acts as an implicit regularizer in addition to improving accuracy. The largest absolute GI improvement is observed in AlexNet (0.798 to 0.867), consistent with this architecture benefiting most from improved feature localization.

5.5. Training Dynamics

Figure 2 shows training and validation loss and accuracy curves for all four attention configurations. Hybrid models exhibit visibly smoother convergence, a narrower train-to-validation gap throughout training, and faster stabilization of loss curves. These observations are consistent with the quantitative findings in Table 9 and Table 11.

Figure 2. Validation loss (left) and validation accuracy (right) as a function of training epoch, with all four attention configurations overlaid on the same axes for direct visual comparison. Line color encodes the configuration: blue (None), orange (SENet), green (CBAM), and red (SENet + CBAM). The hybrid SENet + CBAM achieves the fastest convergence and the best final validation accuracy, while CBAM alone exhibits a higher residual loss and a lower accuracy plateau compared to the other configurations.

5.6. Attention Map Visualization

Figure 3 displays spatial and channel-wise attention maps at three network depths for models equipped with CBAM, SENet, and the hybrid combination applied to the same input image.

Figure 3. Attention maps for the hybrid SENet + CBAM models on representative CIFAR-10 samples. Each panel shows spatial attention maps (top row) and channel-wise attention histograms (bottom row) at early, intermediate, and deep layers for: (a) CBAM only, (b) SENet only, and (c) the hybrid SENet + CBAM.

In Figure 3b, the SENet channel-attention histograms remain visually similar across all three depths. This behavior is consistent with SENet’s design. Global average pooling compresses the entire spatial extent into a single descriptor, so the resulting channel weights tend to reflect dataset-level statistics that stabilize early in training. A secondary contributor may be sigmoid saturation, where the gating values concentrate in a narrow range and reduce cross-layer differentiation. These factors suggest that SENet primarily performs a global, relatively depth-invariant channel calibration.

By contrast, the hybrid architecture in Figure 3c benefits from CBAM’s spatial branch, which localizes informative regions independently at each depth. The resulting spatial maps are noticeably more concentrated and vary more across layers, suggesting that the complementarity between the two mechanisms produces richer, more depth-sensitive feature recalibration than either module achieves alone.

5.7. Grad-CAM Interpretability Analysis

Figure 4 presents Grad-CAM visualizations for the hybrid SENet + CBAM models across all four architectures, applied to the same CIFAR-10 test samples. These visualizations are intended as qualitative evidence of how attention affects spatial focus. They do not constitute a rigorous interpretability evaluation in the absence of ground-truth localization annotations.

Figure 4. Grad-CAM visualizations for the hybrid SENet + CBAM models on CIFAR-10 test samples (all results shown are for the hybrid configuration). For each architecture (SqueezeNet, VGG16, ResNet18, AlexNet), the original image, predicted class, and Grad-CAM overlay are shown. Highlighted regions indicate the spatial areas most influential for the model’s decision.

Among the four backbones, ResNet18 produces the most stable and spatially concentrated activations, frequently centered on the primary semantic structure of the object. For example, it emphasizes the hull or cockpit in vehicle images while suppressing background sky or water. This behavior is consistent with ResNet18’s deeper feature hierarchy, which provides richer and more discriminative representations. VGG16 and SqueezeNet occasionally activate broader regions that include background textures. AlexNet shows the least consistent localization, with attention sometimes driven by low-level textural cues rather than object-specific structures.

Failure Case

Several mispredictions appear in the frog rows of Figure 4. In these cases, Grad-CAM maps reveal that activations spread across background vegetation rather than concentrating on the frog itself. At the $32\times 32$ resolution of CIFAR-10, the foreground object often occupies only a small fraction of the image area, making the model susceptible to contextual biases such as green foliage. Incorporating object-centric augmentation strategies (e.g., CutMix, tighter random cropping) or training on higher-resolution data would likely mitigate this failure mode.

5.8. Quantitative Attention Entropy Analysis

To complement the qualitative Grad-CAM analysis, we quantify attention concentration using Shannon entropy computed over normalized Grad-CAM heatmaps. Given a heatmap H with normalized activation probabilities $p_i$ at pixel i, entropy is defined as given in Equation (18):

$$\mathcal{H}\left(H\right)=-\sum _i{p}_{i}log\left(p_i\right)$$

(18)

Lower values indicate sharper, more localized attention. Table 12 reports mean entropy over the validation set for baseline and hybrid configurations. Entropy should be understood as a proxy indicator of localization precision rather than a ground-truth interpretability measure. It quantifies the spatial spread of activations but cannot verify whether the attended regions are semantically correct in the absence of object-level annotations.

Table 12. Mean Grad-CAM attention entropy (lower values indicate more concentrated, localized attention).

Model	Baseline Entropy	Hybrid Entropy
ResNet18	2.41	1.98
VGG16	2.87	2.32
AlexNet	3.12	2.54
SqueezeNet	2.76	2.21

Hybrid attention reduces entropy across all architectures. The largest absolute reduction is observed in AlexNet (3.12 to 2.54). This is consistent with shallow architectures benefiting most from the spatial localization provided by the CBAM branch. These findings corroborate the Grad-CAM visualizations and support the interpretation that hybrid attention produces more concentrated feature activations. Broader validation with annotated localization data would be needed to confirm this conclusively.

5.9. Comparative Performance Visualization

Figure 5 overlays precision, recall, and F1-score across all four attention configurations and architectures, providing a direct visual summary of the progression from baseline to hybrid attention.

Figure 5. Performance comparison across four attention configurations (None, SENet, CBAM, SENet + CBAM) for all architectures, evaluated on precision, recall, and F1-score. The hybrid approach achieves the highest values in all cases.

The visualization confirms the trends reported in Table 6: the hybrid configuration consistently occupies the highest position across all metric-architecture combinations. The progression from baseline to SENet to CBAM to hybrid is generally monotonic, although the SENet-to-CBAM step is larger for most architectures, suggesting that the spatial attention component of CBAM is the stronger individual contributor.

5.10. Discussion

The proposed hybrid SENet + CBAM framework produces consistent, statistically significant improvements across all four CNN architectures on CIFAR-10. Accuracy gains range from +6.38% on SqueezeNet to +14.39% on VGG16. F1-score gains range from +4.97% to +19.77%. All results were achieved with a parameter overhead of only 1.5–5.8% (Table 6 and Table 10). The adaptive weighted fusion strategy preserves the complementary strengths of channel-wise and spatial recalibration. The residual connection ensures that original feature information is not discarded.

5.10.1. Effect of Training Protocol on Observed Gains

An important caveat concerns the training protocol. All models were trained under a deliberately conservative setup. This setup used 50 epochs, no pretrained weights, and standard augmentation. Its primary purpose was to isolate the incremental contribution of the attention mechanism under identical conditions across all architectures, not to maximize absolute accuracy. Any observed difference between configurations is therefore attributable to the attention module alone, not to differences in training duration, data augmentation, or initialization. The resulting baselines are well below fully optimized CIFAR-10 benchmarks. VGG16 reaches only 55.78% and ResNet18 only 77.93%, compared to the roughly 91% and 93% reported in the literature under stronger training regimes.

A portion of the observed gains, most visibly for VGG16, likely reflects correction of underfitting. These results should be read as evidence of the attention mechanism’s contribution within a controlled, reproducible setting. They are not predictions of absolute improvement under arbitrary training conditions. Evaluating the hybrid module under stronger training regimes and on higher-resolution benchmarks is a priority for future work (see Section 5.10.4).

5.10.2. Interpretability Assessment

Grad-CAM visualizations and attention entropy analysis together provide qualitative evidence that hybrid attention produces more spatially concentrated feature activations compared to baseline and single-mechanism configurations. These are proxy indicators, not rigorous interpretability measures. A more principled assessment would require ground-truth object localization annotations. Such annotations would enable metrics such as Intersection-over-Union between attention maps and object bounding boxes, pointing-game accuracy, or calibrated activation ratios. Developing this rigorous quantitative framework is an important direction for future work.

5.10.3. Comparative Analysis Against Prior Work

Table 13 places our results in the context of prior studies using SENet or CBAM on the same architectures. Table 14 summarizes the best-performing prior result for each backbone and the corresponding gain achieved by our approach.

Table 13. Comparative analysis with prior attention mechanism studies.

Study	Configuration	XAI	Val Loss	Acc. %	Key Limitations
ResNet18
Reading CBAM [8]	ResNet + CBAM	Yes	0.534	82.10	Single mechanism; limited dataset scope
Evaluating CNNs [29]	ResNet + CBAM	No	1.667	78.81	No XAI; high validation loss
Evaluating CNNs [29]	ResNet + SENet	Yes	0.714	89.02	No hybrid; limited interpretability
Comparison of modules [30]	ResNet + SENet + CBAM	No	–	56.43	Suboptimal fusion; no XAI
Our study	ResNet18 + SENet + CBAM	Yes	0.533	90.71	Limited to CIFAR-10 at 32 × 32
SqueezeNet
Comparison of modules [30]	SqueezeNet + SENet	No	0.703	74.19	No XAI; weak generalization
Kim et al. [39]	SqueezeNet + CBAM	No	–	–	No training metrics reported
Our study	SqueezeNet + SENet + CBAM	Yes	0.603	78.29	Modest gains on compact architectures
VGG16
Kim et al. [39]	VGG + CBAM	No	0.009	69.00	No hybrid; limited cross-model validation
Comparison of modules [30]	VGG + SENet	No	0.710	57.01	Underfitting; no XAI
Our study	VGG16 + SENet + CBAM	Yes	1.633	70.17	High validation loss; limited to CIFAR-10
AlexNet
Kim et al. [39]	AlexNet + CBAM	No	–	–	No XAI; no quantitative results
Evaluating CNNs [29]	AlexNet + CBAM	No	–	–	No XAI; no quantitative results
Our study	AlexNet + SENet + CBAM	Yes	0.625	71.82	No high-resolution validation; first report for this config

Table 14. Summary of improvements over best prior result per architecture.

Architecture	Best Prior (%)	Ours (%)	Gain (%)	Val Loss
ResNet18	89.02	90.71	$+1.69$	Better
SqueezeNet	74.19	78.29	$+4.10$	Better
VGG16	69.00	70.17	$+1.17$	Better
AlexNet	–	71.82	–	First report
Mean (3 architectures †)	–	–	$+2.32$	–

^† Mean computed over ResNet18, SqueezeNet, and VGG16 only; AlexNet is excluded as no prior quantitative result exists for this configuration.

Our approach outperforms the best prior result for each of the three architectures with an existing quantitative baseline (ResNet18, SqueezeNet, VGG16) by an average of 2.32%. AlexNet is reported separately as a first quantitative result for this configuration and is excluded from the average to avoid introducing an undefined comparison. In addition to accuracy gains, our study provides 5-fold cross-validated statistics, Grad-CAM analysis, and memory usage reporting, aspects largely absent from the compared studies. The gains are modest in absolute terms for ResNet18 (+1.69%) and VGG16 (+1.17%), which reflects the already competitive starting points of the best prior results rather than a ceiling effect of the hybrid mechanism itself.

5.10.4. Limitations and Scope

Several constraints limit the generalizability of the present findings. First, all experiments are restricted to CIFAR-10 at $32\times 32$ resolution. Validation on higher-resolution benchmarks such as CIFAR-100, STL-10, or subsets of ImageNet is a necessary next step. Second, the study addresses only image classification. Whether the benefits transfer to detection or segmentation tasks remains to be investigated. Third, the conservative training protocol introduces low baselines that amplify the apparent magnitude of improvements. Results under fully optimized training conditions may differ. Fourth, integration with modern architectures such as EfficientNet, ConvNeXt, and Vision Transformers has not been explored. Finally, real-time inference and edge deployment constraints were not considered in the current experimental design. These limitations define a concrete research agenda, which is detailed in Section 6.

6. Concluding Remarks and Future Directions

This study proposed a hybrid attention framework that combines Squeeze-and-Excitation Networks and the Convolutional Block Attention Module through adaptive element-wise summation and evaluated it systematically across four CNN backbones (ResNet18, VGG16, AlexNet, and SqueezeNet) on the CIFAR-10 benchmark. The framework is lightweight by design, introducing only 1.5–5.8% additional parameters while accelerating training convergence by 16.5% on average. Across all architectures and metrics, the hybrid configuration consistently outperformed both the unaided baselines and the individual attention mechanisms, with accuracy gains ranging from +6.38% (SqueezeNet) to +14.39% (VGG16), all verified at $p<0.001$ using 5-fold cross-validation and paired t-tests. Comparative analysis against prior SENet and CBAM studies shows an average improvement of 2.32% over the best previously reported result for the three architectures with existing quantitative baselines (ResNet18, SqueezeNet, VGG16). AlexNet is reported as a first result for this configuration and is excluded from the average.

These findings confirm that jointly leveraging channel-wise recalibration (SENet) and spatial-channel attention (CBAM) provides complementary benefits that neither mechanism delivers alone. The learnable fusion parameter $\alpha$ offers only a modest accuracy gain over fixed summation. However, it enables architecture-specific adaptation with negligible overhead and promotes stable optimization across diverse backbones. Grad-CAM visualizations and attention entropy analysis provide qualitative evidence of more concentrated feature localization under the hybrid configuration. These should be understood as proxy indicators rather than rigorous interpretability measures.

The scope of these conclusions has important limitations. All results are specific to CIFAR-10 at $32\times 32$ resolution. The conservative training protocol produces baselines below fully optimized levels. For VGG16, part of the improvement likely reflects correction of underfitting rather than the intrinsic ceiling of the hybrid mechanism. Validation on higher-resolution benchmarks such as CIFAR-100, STL-10, or ImageNet subsets is a necessary next step. The study is also limited to image classification. Whether the benefits extend to object detection or semantic segmentation remains an open question. Integration with architectures such as EfficientNet, ConvNeXt, and Vision Transformers has not yet been explored.

Future work should prioritize several complementary directions. Validation on higher-resolution and domain-specific datasets is a first priority. These include CIFAR-100, STL-10, and medical imaging benchmarks. This would confirm whether the gains observed on CIFAR-10 generalize to more complex settings. Extending the approach to object detection and semantic segmentation would further broaden its practical relevance. Developing rigorous quantitative interpretability metrics is also important. These metrics could include Intersection-over-Union between attention maps and ground-truth regions, pointing-game accuracy, or calibrated activation ratios. Such metrics would replace current proxy measures with more principled evaluation. Architectural extensions are also promising. These include integration with modern CNNs and hybrid CNN-Transformer models. Replacing the scalar fusion parameter $\alpha$ with an input-conditioned network could enable dynamic balancing between SENet and CBAM contributions. Finally, optimizing the hybrid module for resource-constrained edge deployment is a key step toward real-world applicability.

Broader validation across datasets, tasks, and deployment scenarios is essential before claiming general applicability for hybrid attention fusion. The statistical validation methodology presented here provides a reproducible template for such investigations.