CFNet: Achieving Practical Speedup in Lightweight CNNs via Channel-Focused Design and Cross-Channel Mixing ^†

Abstract

Convolutional Neural Networks (CNNs) have achieved remarkable performance in computer vision tasks, but their deployment on resource-constrained devices remains challenging. While existing lightweight CNNs reduce FLOPs significantly, their practical inference speed is limited by memory access bottlenecks. We hence propose CFNet, an efficient architecture that bridges the gap between theoretical efficiency and practical speed through synergistic design of channel-focused convolution (CFConv) and channel mixed unit (CMU). CFConv dynamically selects informative channels via learnable GroupNorm scaling factors and reparameterization, reducing both FLOPs and memory access, while CMU enables cross-channel communication through a split-transform-and-mix strategy to mitigate information loss. Experiments on CIFAR/ImageNet classification and MS COCO object detection demonstrate CFNet’s superior performance. On ImageNet-1K, CFNet-A achieves 35.5% and 189.4% GPU throughput improvements over MobileNetV2 and MobileViTv1-XXS respectively, while delivering 1.76% and 4.09% accuracy gains. CFNet-E attains 83.5% top-1 accuracy, outperforming Swin-S by 0.47% with 44.6% higher GPU throughput and 43.6% lower CPU inference latency.

1. Introduction

Deep convolutional neural networks have demonstrated exceptional performance across various computer vision tasks, including image recognition [1], object detection [2], and semantic segmentation [3]. These advancements have enabled numerous applications in autonomous driving and visual search, where CNNs are essential for achieving high performance. However, such success relies heavily on intensive computational and storage resources, posing significant challenges for efficient deployment in resource-constrained environments such as mobile devices, embedded systems, edge computing platforms, and Internet of Things (IoT) devices. Consequently, recent trends in deep neural network design focus on exploring portable and efficient architectures while maintaining acceptable performance on mobile devices.

Mainstream lightweight CNNs focus on reducing FLOPs through techniques like depthwise separable convolutions, group convolutions, and channel compression [4,5,6,7,8,9]. However, these approaches exhibit a notable disconnect between theoretical efficiency gains and actual inference performances. As illustrated in Figure 1, FLOP reduction does not correlate linearly with inference latency reduction. Compared to the selected baseline ResNet series, many lightweight networks compress FLOPs by over 50%, yet inference latency decreases by only approximately 20% or less. This reveals that although current lightweight networks successfully reduce the number of parameters and FLOPs, approaches centered on theoretical computational optimization fail to deliver corresponding gains in practical inference speed. FasterNet [10] has shown that frequent memory access is the main bottleneck limiting practical speedup and has proposed partial convolution to reduce both redundant computation and memory access. However, this approach leads to accuracy drops due to limited information flow across channels.

Figure 1. The discrepancy between FLOP reduction and inference latency improvement in lightweight CNN models. Despite significant FLOPs compression, practical speedup remains limited due to memory access overhead. The enlarged view highlights the efficiency gap among low-FLOPs models.

To achieve efficient inference while maintaining accuracy, we propose a novel channel-focused convolution (CFConv). This lightweight operator selectively processes informative channels through learnable GroupNorm scaling factors, while employing reparameterization techniques to simultaneously reduce FLOPs and optimize memory access efficiency. However, the channel selection mechanism in CFConv inherently restricts cross-channel information flow, potentially causing accuracy degradation. To address this limitation, we further introduce a channel mixed unit (CMU) that implements a split-transform-and-mix strategy, explicitly facilitating efficient feature interaction across channels. By synergistically integrating CFConv and CMU, we construct CFNet—a highly efficient architecture that demonstrates superior trade-offs between computational cost, latency, and accuracy compared to existing approaches like MobileNet, ShuffleNetV2, and FasterNet, while maintaining competitive task performance.

We conduct extensive experiments on image classification and object detection to validate our approach. It is shown that our CFNet achieves better performance than existing methods with significantly lower latency. Specifically, on ImageNet-1K, our CFNet-A achieves GPU throughput improvements of 35.5% and 189.4% compared to MobileNetV2 and MobileViTv1-XXS, respectively, with accuracy gains of 1.76% and 4.09%. The contributions are as follows.

• We propose channel-focused convolution (CFConv), a lightweight convolution operator that selects informative channels using learnable Group Normalization scaling factors and incorporates reparameterization techniques to balance memory access and computational efficiency.
• We propose a channel mixed unit (CMU), which utilizes a split-transform-and-mix strategy that enables cross-channel information flow.
• We present a CFNet based on CFConv and CMU, achieving superior speed-accuracy trade-offs compared to existing lightweight networks through comprehensive experiments.

2. Related Work

2.1. Lightweight CNN Architectures

The development of lightweight CNNs has focused on reducing computational complexity while maintaining competitive accuracy. SqueezeNet [11] pioneered extreme model compression by introducing Fire modules that first squeeze input channels and then expand with a mix of convolutions, achieving AlexNet-level accuracy with 50 times fewer parameters. MobileNet [4] introduced depthwise separable convolutions, while subsequent MobileNet versions [5,6] incorporated inverted residuals and hardware-aware scaling strategies to build efficient architectures for mobile devices. ShuffleNet [7] explored group convolutions with channel shuffle operations and architectural optimizations, emphasizing practical guidelines for efficient design. GhostNet [9] proposed generating redundant feature maps through cheap linear operations, achieving significant parameter reduction. The EfficientNets [12,13] developed compound scaling methods that systematically balance network depth, width, and resolution for optimal efficiency–accuracy trade-offs. Nonetheless, most approaches primarily focused on reducing the number of parameters and FLOPs, yet they suffer from increased memory access overhead when scaling network width to compensate for accuracy degradation. FasterNet [10] represents a shift in focus by directly targeting memory bottlenecks through partial convolution that processes only a channel subset. However, its random channel selection may discard informative features. We address both efficiency dimensions by proposing CFConv with learnable channel selection to simultaneously reduce FLOPs and optimize memory access patterns.

2.2. Channel Interaction Mechanisms

Beyond optimizing FLOPs and the number of parameters, lightweight CNN design has increasingly focused on efficient channel interaction and spatial information exchange. Squeeze-and-Excitation networks [14] introduced explicit channel attention mechanisms but with additional computational overhead, while ECA-Net [15] proposed more efficient attention with reduced parameters through local cross-channel interactions. Beyond simple channel attention, SKNet [16] introduced a selective kernel mechanism with multi-branch soft attention, enabling dynamic receptive field selection through feature-level gating. ShuffleNet [7] employed channel shuffling to facilitate information exchange within group convolutions, and reparameterization techniques [17] have shown promise in improving training dynamics while maintaining inference efficiency. However, existing approaches struggle to balance computational efficiency with cross-channel communication, often relying on heuristic channel selection that may discard informative features or introduce memory access overhead. Our proposed CFConv addresses these limitations by leveraging learnable Group Normalization scaling factors to intelligently select channels, while CMU facilitates information flow through an efficient Split-Transform-and-Mix strategy.

2.3. Reparameterization Techniques

Structural reparameterization has emerged as an effective technique for decoupling training-time and inference-time architectures. RepVGG [17] pioneered this approach by demonstrating that multi-branch structures during training can be algebraically merged into a single convolution during inference without loss of representational capacity. This technique enables networks to benefit from rich training dynamics while maintaining deployment efficiency. Subsequent works have extended reparameterization to various contexts. RepMLP [18] applies structural reparameterization to MLP-based architectures. ACNet [19] explores asymmetric convolution blocks that strengthen kernel skeletons. DBB [20] introduces diverse branch blocks incorporating multiple convolution types, and MobileOne [21] demonstrates efficient mobile deployment through carefully designed branch structures. However, most existing reparameterization methods apply uniform transformations across all channels without considering channel importance variations. In contrast, our method incorporates reparameterization specifically into the informative channels selected through a learnable mechanism, thereby enhancing feature learning capacity while maintaining high inference efficiency.

3. Methodology

The persistent gap between theoretical efficiency gains and practical speedup in lightweight CNNs stems from overlooking memory access patterns—the true bottleneck in resource-constrained deployment. To bridge this divide, we introduce a comprehensive framework addressing efficiency from both computational and memory perspectives. We begin with the Channel Memory Access Ratio (CMAR), a principled metric that reveals why conventional architectures fail to achieve proportional speedup despite dramatic FLOP reduction. Building upon this analysis, we propose Channel-Focused Convolution (CFConv), which leverages Group Normalization scaling factors to intelligently process information-rich channels, optimizing both computational redundancy and memory access. To address the inevitable constraint on cross-channel information flow, we introduce the Channel Mixed Unit (CMU) with a split-transform-and-mix strategy that restores efficient inter-channel communication. These components are integrated into CFNet, an architecture that achieves better speed-accuracy trade-offs by addressing both computational and memory constraints. The detailed algorithmic procedures for both CFConv and CMU are provided in Appendix A.

3.1. Preliminary

Building on the observation that memory access bottlenecks limit practical speedup, we provide a mathematical framework to quantify this phenomenon. Unlike previous approaches that address memory access through architectural modifications, we analyze the fundamental trade-offs between computational intensity and memory bandwidth requirements. We introduce the Computation-to-Memory Access Ratio (CMAR) as a metric to analyze the efficiency bottlenecks in different convolution operations. This metric reflects the proportional relationship between FLOPs and memory access during model inference.

$$I={\displaystyle \frac{FLOPs}{MAC}}$$

(1)

where $MAC$ includes all data movement:

$$MAC=MA{C}_{S_{Input}}+MA{C}_{S_{Output}}+MA{C}_{S_{Weight}}$$

(2)

where $MA{C}_{S_{Input}}$, $MA{C}_{S_{Output}}$ and $MA{C}_{S_{Weight}}$ are the computational volumes of the input feature maps, output feature maps, and convolution kernels, respectively.

A higher CMAR indicates computation-intensive operations, while a lower CMAR signifies memory-bound operations. Since memory access time overhead typically exceeds CPU FLOPs in computer architecture, low CMAR scenarios significantly increase inference latency. This represents the primary bottleneck for lightweight models in practical deployment, particularly on devices running I/O-intensive tasks where efficiency is further compromised under memorybound conditions.

For regular convolution with input $\mathbf{I}\in {\mathbb{R}}^{c\times h\times w}$, filters $\mathbf{W}\in {\mathbb{R}}^{k\times k}$ and $\mathbf{O}\in {\mathbb{R}}^{c\times h\times w}$, the CMAR can be calculated as

$$I_{Conv}={\displaystyle \frac{h\times w\times k^2\times c}{h\times w\times 2c+k^2\times c^2}}$$

(3)

Depthwise separable convolution decomposes the regular convolution into depthwise convolutions (DWConv) and pointwise convolutions (PWConv). The CMAR of depthwise separable convolution can be calculated as

$$I_{DWConv}={\displaystyle \frac{h\times w\times k^2\times c+h\times w\times c^2}{h\times w\times 2c+k^2\times c+h\times w\times 2c+c^2}}$$

(4)

Although depthwise separable convolution reduces FLOPs by approximately 9× (assuming k = 3) compared to regular convolution. The separation into DWConv and PWConv decomposition significantly increases feature map accesses, with memory access volume roughly doubling due to the necessity of storing intermediate features between DWConv and PWConv. Consequently, the CMAR decreases by 18× compared to regular convolution, making operations more memory-bound and explaining why theoretical efficiency gains often fail to materialize in practice.

3.2. Channel-Focused Convolution (CFConv) as an Operator

To achieve computation-efficient inference with optimized memory usage, we propose channel-focused convolution (CFConv), which builds upon the partial convolution paradigm but introduces a principled channel selection mechanism. Unlike existing partial convolution approaches that rely on random or uniform channel selection, CFConv leverages the scaling factors in Group Normalization (GN) [22] layers as indicators of channel importance, enabling intelligent identification of information-rich channels for focused computation. Additionally, CFConv incorporates reparameterization techniques to reduce inference latency while maintaining training flexibility.

Specifically, as illustrated in Figure 2, for an input feature map with c channels, we first utilize PWConv with a channel reduction ratio $\lambda$ (typical $\lambda =1/2$ in our experiments) to squeeze feature channels for computational efficiency and information fusion. We then leverage Group Normalization to divide channels into r groups and normalize each group independently (group factor $r=2$ is a typical setting in our experiments). The normalization process for a GN layer [22] is

$$y_c={\gamma }_c·{\displaystyle \frac{x_c-{\mu }_g}{\sqrt{{\sigma }_g^2+ϵ}}}+{\beta }_c$$

(5)

where ${\mu }_g$ and ${\sigma }_g$ are the mean and standard deviation of the group g, respectively. ${\gamma }_c$ and ${\beta }_c$ are the trainable affine transformation.

Figure 2. The architecture of channel-focused convolution (CFConv).

Notably, we leverage the trainable parameters ${\gamma }_c$ in GN layers as indicators of channel importance. During training, larger ${\gamma }_c$ values indicate channels requiring stronger scaling to maintain discriminative power after normalization. We select the group $\widehat{c}$ with highest accumulated ${\gamma }_c$ for enhanced processing through multi-branch convolutions consisting of $1\times 1$, $3\times 3$ convolutions, and identity mapping. These three distinct branches significantly enhance the feature extraction capability for informative channels by capturing features at different scales and complexities. The remaining groups undergo identity transformation and are concatenated with the branch outputs. Finally, 1 × 1 PWC operations are applied to restore the output feature maps to match the input dimensionality.

3.2.1. Inference Optimization

While the multi-branch architecture effectively enhances learning capacity, it suffers from lower inference efficiency. Following RepVGG [17], we employ re-parameterization to merge the multi-branch convolutions into a single $3\times 3$ convolution during inference. Specifically, for the multi-branch structure with $3\times 3$, $1\times 1$ convolutions and identity mapping, each followed by Batch Normalization (BN), we first fuse the BN parameters into convolution weights as in RepVGG [17].

$$W_{i,:,:,:}^{\prime }={\displaystyle \frac{{\tilde{\gamma }}_i}{{\tilde{\sigma }}_i}}{W}_{i,:,:,:},b_i^{\prime }=-{\displaystyle \frac{{\tilde{\mu }}_i{\tilde{\gamma }}_i}{{\tilde{\sigma }}_i}}+{\tilde{\beta }}_i$$

(6)

where $\tilde{\gamma }$, $\tilde{\sigma }$, $\tilde{\mu }$, $\tilde{\beta }$ are the BN parameters. Then, leveraging the additive property of convolutions, the three branches are merged as

$$W_{merged}=W^{\prime \left(3\right)}+\mathrm{pad}\left(W^{\prime \left(1\right)}\right)+\mathrm{diag}\left(W^{\prime \left(0\right)}\right)$$

(7)

where $\mathrm{pad}(·)$ converts the $1\times 1$ kernel to $3\times 3$ by zero-padding corners, and $\mathrm{diag}(·)$ creates a $3\times 3$ kernel with 1 at the center for the identity branch. This transforms the complex training architecture into a simplified single-branch model for deployment, maintaining representational capacity while achieving computational efficiency.

3.2.2. Analysis on Complexities

During inference, our CFConv only considers $\widehat{c}$ channels as the representatives of the whole feature maps for computation. The CMAR of CFConv is given by

$$I_{\mathrm{CFConv}}={\displaystyle \frac{h\times w\times k^2\times {\widehat{c}}^2}{h\times w\times 2\widehat{c}+k^2\times {\widehat{c}}^2}}$$

(8)

With a typical partial ratio $\rho ={\displaystyle \frac{\widehat{c}}{c}}={\displaystyle \frac{1}{4}}$, CFConv reduces FLOPs to ${\displaystyle \frac{1}{16}}$ and memory access volume to ${\displaystyle \frac{1}{4}}$ of regular convolution, demonstrating significant efficiency gains. Correspondingly, CFConv achieves a 4× lower CMAR than regular convolution. More significantly, compared to depthwise separable convolution, CFConv achieves over 4× higher CMAR, demonstrating considerable theoretical advantages in computation-to-memory access efficiency.

3.3. Channel Mixed Unit

While CFConv effectively replaces various convolution operations while maintaining sufficient model accuracy, it undeniably suffers from information loss across feature channels, similar to other lightweight networks that employ partial convolution methods. To overcome the side effects brought by limited channel communication, we propose a novel channel mixed unit (CMU) to facilitate information flow across feature channels, as illustrated in Figure 3. CMU utilizes a Split-Transform-and-Mix strategy to enhance cross-channel information exchange while maintaining computational efficiency.

Figure 3. The architecture of channel mixed unit (CMU).

3.3.1. Split

The CMU module receives the concatenated outputs from CFConv’s multiple branches. We first split the $c/2$ channels into two distinct parts with different characteristics. The first part consists of the channels that have been processed by CFConv’s selective convolution operations, serving as “Rich Feature Channels” that contain enhanced and refined feature representations. The second part comprises all the original $c/2$ channels before selective processing, serving as “Full Channels” that preserve the complete feature information without selective processing.

3.3.2. Transform

For “Rich Feature Channels”, we utilize $1\times 1$ PWC operations to squeeze the channels of feature maps to $c/2$ channels. For “Full Channels”, we first utilize $1\times 1$ PWC operations to diffuse the channel information. The obtained channels are then subjected to pointwise convolutions and grouped convolutions, respectively, followed by the operation of element-wise addition on the two obtained feature maps.

3.3.3. Mix

After the transformation, we apply global average pooling to ${\mathbf{Y}}_{rich}$ and ${\mathbf{Y}}_{full}$ to collect global spatial information, yielding two weight tensors. These tensors are then concatenated along the channel dimension to form $\mathbf{S}\in {\mathbb{R}}^{c\times 1\times 1}$. Following the soft attention mechanism in SKNet [16], we apply channel-wise soft attention operation to generate feature importance vectors ${\alpha }_1,{\alpha }_2\in {\mathbb{R}}^{c/2}$ as   

$${\alpha }_1={\displaystyle \frac{e^{S_1}}{e^{S_1}+e^{S_2}}},{\alpha }_2={\displaystyle \frac{e^{S_2}}{e^{S_1}+e^{S_2}}},{\alpha }_1+{\alpha }_2=1$$

(9)

Under the guidance of the feature importance vectors ${\alpha }_1,{\alpha }_2$, the mixed features ${\mathbf{Y}}_{mix}$ can be obtained by combining the rich and full features as

$${\mathbf{Y}}_{mix}={\alpha }_1\odot {\mathbf{Y}}_{rich}+{\alpha }_2\odot {\mathbf{Y}}_{full}$$

(10)

where ${\mathbf{Y}}_{rich}$ and ${\mathbf{Y}}_{full}$ represent the transformed rich and full feature channels, respectively, and ⊙ denotes element-wise multiplication.

In brief, we adopt CMU to compensate for the limitation of CFConv, which loses part of the feature channel information flow during partial convolution operations. The rich features provide enhanced representations through selective processing, while the full features preserve complete information diversity.

3.4. CFNet as a General Backbone

Given our novel CFConv operator and Channel Mixed Unit as the primary building module, we further propose CFNet, a lightweight neural network that achieves much lower latency and high accuracy for image classification and object detection tasks. We aim to keep the architecture as simple as possible, without bells and whistles, to make it hardware-friendly in general.

We present the overall architecture in Figure 4. It begins with a Stem module for initial feature extraction, followed by four hierarchical stages. Each stage has a stack of CFNet Blocks that incorporate our proposed CFConv operations. We distribute the CFNet Blocks across the four stages in a 1:1:3:1 ratio. Taking CFNet-A as an example with 18 convolution blocks in total, this yields [3, 3, 9, 3] CFNet Blocks per stage respectively. After each stage, a subsampling layer (a 2 × 2 Conv with stride 2) performs spatial downsampling and channel number expanding.

Figure 4. Overall architecture of CFNet. The network comprises four stages with CFNet blocks in a 1:1:3:1 ratio. Each block integrates CFConv for selective channel processing and CMU (hybrid ratio $\eta$ = 1/4) for cross-channel communication. Subsampling layers perform spatial downsampling between stages.

Each CFNet Block has a CFConv layer integrated with efficient design principles. The channel dimensions are set as [32, 64, 128, 256] across the four stages, following the pyramid principle commonly adopted in modern architectures. To balance efficiency and performance, we deploy CMU selectively with a hybrid ratio $\eta$ ($\eta$ = 1/4 is a typical setting in our experiments) relative to CFConv operators and at each stage end. The network concludes with global average pooling and a fully-connected layer for classification.

4. Experimental Results

In this section, we evaluate the proposed neural network CFNet through experiments on image classification, object detection and instance segmentation. We provide various CFNet scales (A/B/C/D/E) with similar architectures but different depths to accommodate diverse computational budgets.

4.1. Experimental Settings

4.1.1. Tasks and Datasets

• Image Classification. The CIFAR-10, CIFAR-100 [23], and ImageNet-1k datasets [24] are used for image classification tasks. We train our models for 300 epochs using the AdamW optimizer. We set the batch size to 2048 for CFNet and all baseline methods. The cosine learning rate scheduler is used with a peak learning rate of 0.001 · batch size/1024 and 20-epoch linear warmup. Standard regularization and augmentation techniques are applied, including Weight Decay, Stochastic Depth, Label Smoothing, Mixup, Cutmix and RandAugment, with parameters tuned for different baselines. For ImageNet-1k, we train with 192 × 192 resolution for the first 280 epochs and 224 × 224 for the final 20 epochs to improve training efficiency.
• Object Detection and Instance Segmentation. COCO dataset [25] is used for object detection and instance segmentation tasks. CFNet-D and CFNet-E were utilized as the backbone within a standard Mask R-CNN framework. The model is pre-trained on ImageNet-1k and subsequently fine-tuned on COCO. Training is conducted with the AdamW optimizer, using a batch size of 16.

4.1.2. Hardware Configuration

A system equipped with four NVIDIA GeForce 1080Ti GPUs is used for experiments on CIFAR-10 and CIFAR-100 datasets. A more powerful setup featuring a 64-core Intel Xeon Gold 6226R CPU and four NVIDIA GeForce RTX 3090 GPUs, each with 24 GB of memory. This configuration was utilized for resource-intensive experiments, including model training and evaluation on ImageNet-1k and COCO datasets. Additionally, to evaluate the practical advantages of our method in real-world deployment scenarios, we conducted ARM Cortex-A simulation experiments using a Xilinx Zynq UltraScale+ MPSoC development board (Xilinx, Inc., San Jose, CA, USA) equipped with quad-core ARM Cortex-A53 processors running at 1.2 GHz and dual-core ARM Cortex-R5 processors operating at 500 MHz.

4.2. Image Classification on CIFAR

In the image classification task, we evaluate our CFNet against baseline methods on CIFAR-10/100 datasets using top-1 accuracy, the number of parameters, FLOPs and GPU throughput metrics. The baseline approaches include MobileNet [4], ShuffleNet [7], GhostNet [9], MobileViTv1 [26] CycleMLP [27], EdgeNeXt [28] and Swin Transformer [29].

As shown in Table 1, our CFNet variants (A/B/C) consistently outperform all baseline models across dual metrics of accuracy and throughput, demonstrating the superiority of the proposed architecture. Given the consistent performance trends across both datasets, we focus our analysis on CIFAR-100 for its finer-grained 100-class classification. Specifically, CFNet-C outperforms ResNet-50 by 0.43% in accuracy and 98.5% in throughput. Notably, CFNet-D shows a remarkable 111% throughput boost relative to ResNet-101, while achieving 0.12% accuracy improvement. Compared to the state-of-the-art ResNeXt-101, CFNet-D achieves over 3× higher throughput with only a marginal 0.09% accuracy trade-off.

Table 1. Image Classification Results on CIFAR-10 and CIFAR-100.

Model	Top-1 ACC (%)		Params (M)	FLOPs (G)	Throughput on GPU (fps)
	CIFAR-10	CIFAR-100
ResNet-18	93.33	74.19	11.68	1.81	2412
MobileNetV2	93.26	74.04	3.56	0.36	2874
ShuffleNetV2×1.5	93.58	74.28	3.52	0.35	2982
GhostNet	93.47	74.23	3.38	0.30	2712
CFNet-A	93.71	74.49	3.67	0.41	3265
ResNet-34	93.75	74.63	20.78	3.72	1418
MobileNetV2×1.4	93.82	74.67	6.16	0.60	1765
ShuffleNetV2×2	94.03	74.88	7.45	0.59	1822
GhostNet x1.3	93.92	74.84	7.42	0.48	1713
MobileViTv1-XS	93.87	74.79	4.13	0.82	1905
CFNet-B	94.23	75.08	7.72	0.81	2295
ResNet-50	94.28	75.15	25.61	4.11	716
CycleMLP-B1	94.63	75.52	15.20	2.10	987
MobileViTv1-S	94.56	75.23	5.62	2.02	1130
EdgeNeXt-S	94.64	75.57	5.60	1.26	1113
CFNet-C	94.67	75.58	15.67	1.63	1421
ResNet-101	94.71	75.76	44.46	7.62	438
CycleMLP-B2	95.19	75.97	26.80	3.92	421
Swin-T	95.13	75.92	28.30	4.51	583
ResNeXt-101	95.22	75.97	44.10	8.07	281
CFNet-D	95.12	75.88	32.54	4.14	924

Note: Rows highlighted in grey indicate our proposed method, CFNet. Results in bold represent the best performance.

Furthermore, to provide a comprehensive comparison, we plot the GPU throughput vs. Top-1 accuracy curve on CIFAR-100 in Figure 5. The results show that CFNet consistently outperforms all competing methods including ResNet, MobileNet, ShuffleNet, GhostNet, CycleMLP, and MobileViTv1 across the entire accuracy-throughput spectrum. This clearly demonstrates that CFNet achieves superior balance between accuracy and inference efficiency among all examined networks, making it a highly effective lightweight architecture for resource-constrained deployment scenarios.

Figure 5. Top1-Accuracy v.s. Throughput on GPU on CIFAR-100.

4.3. Image Classification on ImageNet

To verify CFNet’s effectiveness, we conduct experiments on ImageNet-1k classification. As shown in Table 2, CFNet achieves inference performance on both CPU and GPU platforms while maintaining superior accuracy compared to mainstream CNN, ViT, and MLP models. CFNet-A delivers 4× higher throughput than ResNet-18 with 2.19% accuracy improvement, and outperforms MobileNetV2 and MobileViTv1-XXS by 35.5% and 189.4% in throughput with 1.76% and 4.09% accuracy gains, respectively. Although CFNet-A and MobileNetV2 have similar FLOPs at 0.41 G and 0.36 G, respectively, the 35.5% throughput gap indicates that MobileNetV2’s depthwise separable convolutions create memory access bottlenecks. As analyzed in Section 3.1, depthwise decomposition reduces CMAR by 18× compared to regular convolution, whereas CFNet maintains 4× higher CMAR through intelligent channel selection, successfully bridging the gap between theoretical efficiency and practical hardware performance. CFNet-D achieves over 2× higher throughput than ConvNeXt-T with only 0.03% accuracy reduction. For CFNet-E, it achieves an accuracy of 83.50%, which is comparable to state-of-the-art models such as Swin-S and ConvNeXt-S, while delivering throughput improvements of 44.6% and 33.8% on GPU respectively and reducing inference latency by 43.6% and 23.2% on CPU respectively. These results demonstrate CFNet’s significant acceleration on both GPU and CPU platforms while maintaining scalability from lightweight to high-performance models through network depth scaling.

Table 2. Image Classification Results on ImageNet-1k.

Model	Top-1 ACC (%)	FLOPs (G)	Params (M)	Inference Speed
				GPU (fps)	CPU (ms)
ResNet-18	71.62	1.81	11.68	2113	31.80
MobileNetV2	72.05	0.36	3.56	5684	13.20
ShuffleNetV2-1.5×	72.60	0.35	3.52	5586	11.70
GhostNet	72.58	0.30	3.38	5419	12.40
MobileViTv1-XXS	69.72	0.44	1.32	2662	31.80
FasterNet-T0	71.90	0.34	3.92	7924	8.20
CFNet-A	73.81	0.41	3.67	7703	8.60
ResNet-34	75.15	3.72	20.78	1648	50.60
MobileNetV2-1.4×	74.72	0.60	6.16	3612	20.90
ShuffleNetV2-2×	74.92	0.59	7.45	3803	16.70
GhostNet-1.3×	75.77	0.48	7.42	3316	19.30
MobileViTv1-XS	74.83	1.02	2.37	1658	39.00
MobileViTv3-XS	75.71	0.96	2.55	1726	36.30
FasterNet-T1	75.18	0.85	7.64	4954	16.40
CFNet-B	76.73	0.81	7.72	4921	17.80
ResNet-50	78.81	4.11	25.61	1035	73.00
CycleMLP-B1	79.15	2.10	15.24	889	116.10
MobileViTv1-S	78.40	2.02	5.62	1101	56.20
EdgeNeXt-S	79.47	1.26	5.60	1197	56.80
PoolFormer-S12	77.21	1.82	11.92	1440	49.00
PVT-Small	79.83	3.83	24.58	1233	39.00
FasterNet-T2	78.79	1.90	15.01	2282	30.80
CFNet-C	80.03	1.63	15.67	2193	31.00
ResNet-101	81.13	7.62	44.46	338	121.60
CycleMLP-B2	81.67	3.92	26.84	530	178.10
Swin-T	81.32	4.51	28.31	583	122.30
ResNeXt-101	82.11	8.07	44.10	281	109.50
ConvNeXt-T	82.15	4.46	28.62	453	90.60
PoolFormer-S24	80.31	3.41	21.46	575	90.70
PVT-Medium	81.22	6.69	44.29	560	102.30
FasterNet-S	80.99	4.55	31.11	1142	67.90
CFNet-D	82.12	4.14	32.54	1074	68.80
ResNet-152	82.76	11.00	60.11	281	249.80
ConvNeXt-S	83.21	8.71	50.20	373	167.40
Swin-S	83.03	8.77	49.60	345	227.90
FasterNet-M	82.46	8.72	53.57	533	123.00
CFNet-E	83.50	8.83	53.61	499	128.50

Note: Rows highlighted in grey indicate our proposed method, CFNet. Results in bold represent the best performance.

Additionally, we include FasterNet, an efficient lightweight network utilizing partial convolution, as a competitive baseline. As shown in Figure 6, the proposed CFNet achieves nearly identical inference performance while delivering a significant accuracy improvement exceeding 1%. The accuracy gap reveals fundamental differences in channel selection strategies. FasterNet employs random or contiguous selection that is content-agnostic and may inadvertently discard informative features. In contrast, CFNet leverages learnable GroupNorm scaling factors to identify channels with high information content, ensuring that computational resources concentrate on semantically rich features. Furthermore, our CMU module explicitly facilitates cross-channel communication to restore information flow restricted by partial processing, enabling CFNet to preserve representational capacity without compromising efficiency.

Figure 6. Top1-Accuracy vs. Latency on CPU and Throughput on GPU on ImageNet-1k.

4.4. Object Detection and Instance Segmentation

To further evaluate the generalization ability of CFNet, we conduct experiments on the challenging COCO dataset [25] for object detection and instance segmentation.

We compare CFNet against ResNet, PoolFormer, and FasterNet as baseline methods. As shown in Table 3, the ${\mathbf{AP}}^b$ and the ${\mathbf{AP}}^m$ of CFNet-D are 40.2% and 36.8%, outperforming original ResNet-50 by 2.1% and 3.1% while reducing inference latency by 33%. Compared to the high-accuracy ViT framework PoolFormer-S24, our CFNet-D achieves only 0.3% lower ${\mathbf{AP}}^b$ while reducing inference latency by 67%. Against PoolFormer-S36, our CFNet-E delivers 0.4% higher ${\mathbf{AP}}^b$ with 40% lower inference latency.

Table 3. Object Detection and Instance Segmentation Results on COCO.

Model	Params (M)	FLOPs (G)	GPU Latency (ms)	Object Detection			Instance Segmentation
				${\mathbf{AP}}^{\mathit{b}}$	${\mathbf{AP}}_{50}^{\mathit{b}}$	${\mathbf{AP}}_{75}^{\mathit{b}}$	${\mathbf{AP}}^{\mathit{m}}$	${\mathbf{AP}}_{50}^{\mathit{m}}$	${\mathbf{AP}}_{75}^{\mathit{m}}$
ResNet-50	253	44.1	55.6	38.1	58.2	40.9	33.7	55.0	36.4
PoolFormerS24	234	41.2	114.2	40.5	61.8	43.1	36.8	59.1	39.3
FasterNet-S	258	49.0	35.1	38.9	60.2	42.6	35.6	58.1	38.7
CFNet-D	276	51.2	37.3	40.2	61.4	43.4	36.8	59.0	38.9
ResNet-101	329	63.2	68.3	40.4	61.2	44.2	36.3	57.8	38.7
PoolFormerS36	266	58.5	146.9	42.8	63.3	46.9	38.7	62.0	42.8
FasterNet-M	344	71.2	80.5	42.0	62.4	46.3	37.9	61.5	41.8
CFNet-E	368	77.2	88.4	43.2	63.5	46.9	38.6	61.8	42.5

Note: Rows highlighted in grey indicate our proposed method, CFNet. Results in bold represent the best performance.

From an overall view of Figure 7, the curve of our proposed model is above all other methods. Compared to the fast-inference FasterNet, our method achieves approximately 1% improvement in average precision with comparable inference speed. This further demonstrates that CFNet enhances model learning capability while maintaining computational efficiency.

Figure 7. Average Precision vs. GPU Latency on COCO.

4.5. Practical Deployment Analysis

To better demonstrate the practical advantages of our method in real-world deployment scenarios, we conduct comprehensive evaluation experiments on three hardware platforms: GPU, CPU, and ARM Cortex-A simulation using the ImageNet-1k dataset. During deployment, all networks are trained with knowledge distillation methods including KD, DKD, and ReviewKD. The selected lightweight networks for comparison include the baseline ResNet18, MobileNetV2, RepVGG-A0, FasterNet-A, and our proposed CFNet-A.

As shown in Table 4, our proposed CFNet-A demonstrates superior performance across different hardware platforms. On GPU, CFNet-A achieves approximately 1.7× higher throughput than ResNet18 while delivering nearly 3% higher accuracy (using ReviewKD as the distillation method). On CPU devices, CFNet-A matches FasterNet-A’s inference latency and is 0.7 ms faster than the second-best RepVGG-A0, benefiting from reduced memory access pressure that allows our design optimizations to be more effectively utilized. On ARM platforms, CFNet-A demonstrates exceptional efficiency with 57 ms lower latency compared to the baseline ResNet18. Although marginally 3 ms slower than FasterNet-A, CFNet-A’s substantial accuracy advantage makes it more suitable for edge deployment scenarios where both speed and accuracy are critical. These results confirm that CFNet-A delivers an optimal balance between accuracy and inference efficiency across diverse hardware platforms. These comprehensive cross-platform evaluations demonstrate that CFNet maintains its efficiency advantages across diverse deployment scenarios, while knowledge distillation further enhances its practical applicability in real-world applications.

Table 4. Performance Evaluation on Different Hardware Platforms.

Model	Inference Speed			Distillation Method	Top-1 ACC (%)	Quantization Loss (%)
	GPU (fps)	CPU (ms)	ARM (ms)
ResNet18	7749	7.9	184	KD	73.13	−0.08
				DKD	74.28	−0.17
				ReviewKD	73.22	−0.12
MobileNetV2	8974	6.2	179	KD	73.93	−0.86
				DKD	74.18	−0.97
				ReviewKD	74.32	−0.95
RepVGG-A0	11,416	4.5	138	KD	73.41	−0.66
				DKD	73.82	−0.71
				ReviewKD	73.99	−0.65
FasterNet-A	13,386	3.8	124	KD	73.42	−0.58
				DKD	73.53	−0.49
				ReviewKD	72.28	−0.55
CFNet-A	13,342	3.8	127	KD	74.92	−0.19
				DKD	75.18	−0.21
				ReviewKD	75.42	−0.28

Note: Rows highlighted in grey indicate our proposed method, CFNet. Results in bold represent the best performance.

4.6. Ablation Study

We conduct a brief ablation study on the value of channel reduction ratio $\lambda$, the hybrid ratio $\eta$ and the group factor r. We compare different variants in terms of Cifar100 top-1 accuracy, params, FLOPs and throughput on GPU, with results detailed in Table 5. Different hyperparameter settings exhibit varying impacts on model performance.

Table 5. Ablation on the channel reduction ratio $\lambda$, the hybrid ratio $\eta$ and the group factor r of CFNet.

Ablation	Variant	Top-1 ACC(%)	Params (M)	FLOPs (G)	Throughput on GPU (fps)
Channel reduction ratio ($\lambda$)	$\lambda =2/1$	74.77	7.67	5.41	1722
	$\lambda =1$	74.61	5.73	1.60	2687
	$\lambda =1/2$	74.49	3.67	0.41	3265
	$\lambda =1/4$	74.02	2.75	0.36	3371
CMU/CFNet ratio ($\eta$)	$\eta =1$	75.01	6.82	1.10	1778
	$\eta =1/2$	74.72	4.33	0.86	2277
	$\eta =1/4$	74.49	3.67	0.41	3265
	$\eta =0$	74.17	3.25	0.36	3428
Group factor (r)	$r=1$	75.21	8.86	2.38	1843
	$r=2$	74.49	3.67	0.41	3265
	$r=4$	73.76	3.24	0.31	3614

Note: Rows highlighted in grey are the default settings.

The results in Table 5 illuminate the performance trade-offs governed by these parameters. The channel reduction ratio $\lambda$ demonstrates a clear efficiency–accuracy trade-off, where $\lambda =1/2$ provides the optimal balance—significantly reducing computational cost while maintaining competitive accuracy compared to more aggressive compression. Similarly, the group factor r directly controls channel selection granularity, where $r=4$ achieves maximal throughput but with significant accuracy reduction, while $r=1$ improves accuracy at substantial computational cost. This establishes $r=2$ as a robust middle ground. Similarly, the hybrid ratio $\eta$ reveals that while removing CMU entirely maximizes speed, it severely degrades accuracy. The setting $\eta =1/4$ proves optimal, recovering most accuracy loss with minimal latency impact. These analyses provide valuable insights for different deployment scenarios, where configurations like $\lambda =1/4$, $\eta =0$, $r=4$ offer viable high-efficiency options for extremely resource-constrained edge environments.

Our ablation study validates that the default configuration with $\lambda =1/2$, $\eta =1/4$, $r=2$ achieves an optimal efficiency–accuracy balance through principled design. These choices create a robust operating point where CFNet maintains competitive advantages across various tasks and datasets, demonstrating that our architectural innovations rather than parameter tuning drive the performance gains.

5. Conclusions

In this paper, we have investigated the persistent efficiency bottleneck in lightweight CNNs where reduced FLOPs fail to translate into practical speedup due to memory access overhead. To overcome this fundamental limitation, we proposed channel-focused convolution (CFConv), an intelligent operator that leverages Group Normalization scaling factors to dynamically select informative channels, and channel mixed unit (CMU) that enables efficient cross-channel communication through a Split-Transform-and-Mix strategy. Integrating these innovations, we built CFNet—a hardware-friendly architecture that optimally balances computational efficiency and memory access.

While CFNet demonstrates strong performance across various vision tasks, several aspects merit further investigation. The current architecture, though efficient, may require specialized optimization for extremely memory-constrained edge devices where every kilobyte of memory usage matters. Additionally, the scalability of our channel-focused paradigm to hybrid CNN-Transformer architectures presents an intriguing research direction. The principled channel selection mechanism in CFConv could potentially complement attention mechanisms in transformers, offering opportunities to reduce computational overhead while maintaining representational capacity.

The extensive experiments with various baseline methods on image classification, object detection and instance segmentation have indicated the superiority of CFNet for striking a much better trade-off between performance and inference efficiency. We hope the proposed method provides valuable insights for efficient architectural design and hope it inspires future research towards more hardware-aware neural network architectures.