CML-RTDETR: A Lightweight Wheat Head Detection and Counting Algorithm Based on the Improved RT-DETR

Abstract

Wheat is one of the important grain crops, and spike counting is crucial for predicting spike yield. However, in complex farmland environments, the wheat body scale has huge differences, its color is highly similar to the background, and wheat ears often overlap with each other, which makes wheat ear detection work face a lot of challenges. At the same time, the increasing demand for high accuracy and fast response in wheat spike detection has led to the need for models to be lightweight function with reduced the hardware costs. Therefore, this study proposes a lightweight wheat ear detection model, CML-RTDETR, for efficient and accurate detection of wheat ears in real complex farmland environments. In the model construction, the lightweight network CSPDarknet is firstly introduced as the backbone network of CML-RTDETR to enhance the feature extraction efficiency. In addition, the FM module is cleverly introduced to modify the bottleneck layer in the C2f component, and hybrid feature extraction is realized by spatial and frequency domain splicing to enhance the feature extraction capability of wheat to be tested in complex scenes. Secondly, to improve the model’s detection capability for targets of different scales, a multi-scale feature enhancement pyramid (MFEP) is designed, consisting of GHSDConv, for efficiently obtaining low-level detail information and CSPDWOK for constructing a multi-scale semantic fusion structure. Finally, channel pruning based on Layer-Adaptive Magnitude Pruning (LAMP) scoring is performed to reduce model parameters and runtime memory. The experimental results on the GWHD2021 dataset show that the ${\mathrm{AP}}_{50}$ of CML-RTDETR reaches 90.5%, which is an improvement of 1.2% compared to the baseline RTDETR-R18 model. Meanwhile, the parameters and GFLOPs have been decreased to 11.03 M and 37.8 G, respectively, resulting in a reduction of 42% and 34%, respectively. Finally, the real-time frame rate reaches 73 fps, significantly achieving parameter simplification and speed improvement.

1. Introduction

Wheat (Triticum aestivum L.), as one of the most important food crops in the world [1,2], occupies a crucial position in our agricultural production and food security [3,4]. The formation of wheat yield in the field mainly depends on two key agronomic traits—the number of ears per unit area and the thousand grain weight [5,6]—of which the number of ears per unit area is the most decisive yield-constituting factor. Therefore, accurate counting of wheat ears in the field is of great theoretical and practical significance for dynamic monitoring of wheat growth, yield prediction and evaluation, and optimization of the breeding process.

In the traditional method of wheat spike detection and counting, manual surveying has long dominated the techniques employed [7,8,9]. However, there are many obvious drawbacks of manual surveying, such as low efficiency, making it difficult to ensure accurate results and introducing strong subjectivity to the prediction results; these factors make it difficult to meet the needs of modern agriculture for efficient and accurate wheat spike counting [10]. With the rapid development of computer technology, the method of counting wheat ears based on image processing and machine learning came into being, providing a new turnaround for the work of counting wheat ears. However, in practical applications, due to the complexity and variability of the field environment, there are large differences in the shape, color, and scale of wheat ears, which bring many challenges to the realization of high-throughput wheat ear detection and counting [3,11].

Satellite remote sensing technology, as a non-invasive monitoring approach, can efficiently obtain multi-spectral information of crops on a regional scale, providing data support for agricultural needs at different spatial and temporal resolutions, and it is increasingly widely used in the fields of fine crop management and yield modeling [12,13,14,15]. Many scholars have carried out in-depth research in this field and achieved remarkable results. For example, Wang et al. [16] proposed HyperSIGMA, a hyperspectral intelligent understanding model, to address the problems of weak cross-task generalization and poor cross-scene mobility in traditional hyperspectral image processing models, and they verified its excellent performance on multiple hyperspectral image datasets. Ileana et al. [17] combined remote sensing technology with the M5P model tree algorithm to estimate rapeseed oil yield. Chen et al. [18] used hyperspectral remote sensing technology to predict potato underground yield traits with the help of structural equation modeling, which provided a new method for precision agriculture. Zhao et al. [19] proposed a winter wheat extraction method based on multi-task learning and visual transformer for GF-2 remote sensing data, which effectively improves the accuracy of winter wheat extraction by fusing normalized vegetation index and surface temperature data. These research results have achieved remarkable results in agricultural crop characterization and promoted the development of precision agriculture. However, the application potential of satellite remote sensing in agricultural monitoring is constrained by its inherent technical limitations. Its low spatial resolution and temporal revisit frequency make it difficult to meet the demand for high spatial and temporal accuracy in fine agricultural management, which may lead to problems such as bias in characterization and identification during critical periods and difficulty in accurately counting important agronomic traits. In addition, the acquisition of multi-spectral information is highly dependent on favorable meteorological conditions [20,21], while wheat has large differences in individual morphology during the tasseling period, and the spatial distribution and visibility of wheat ears in the field change rapidly, which further reduce the feasibility of using multi-spectral data to achieve reliable and synchronized wheat ear counting. Therefore, the development of an efficient and automated high-throughput wheat spike counting technology has become a critical need to advance the wheat breeding process.

Visible light imaging systems carried by UAVs have received extensive attention in scientific research and engineering applications due to their significant advantages such as high mobility, high timeliness, and excellent spatial resolution [22,23,24,25,26]. With the continuous progress of computer vision technology, counting methods based on UAV images have gradually been applied in the automatic counting of various crops. At present, the mainstream counting methods are mainly categorized into two types: the rectangular box style detection model and the dense counting model based on density regression [27]. Among them, the dense counting model based on density regression shows better performance in handling the counting task in dense situations. However, the method can only output the numerical results of counting, and it cannot obtain the specific coordinates and location information of the target through the detection box as in the detection model, which limits its application in precision agriculture to a certain extent [28,29]. Therefore, the method based on detection counting has gradually become a research hotspot in precision agriculture. Numerous scholars have carried out in-depth research in this field and achieved fruitful results. For example, Lin et al. [30] used a new cascaded convolutional neural network to extract regions of interest from visible and multi-spectral images acquired by unmanned aerial systems, and they realized the detection and counting of grain ears under different field conditions. Guo et al. [31] proposed a deep learning algorithm based on natural image segmentation for automatically calculating the fruiting rate in response to the problem of spatial overlapping of rice grains that makes counting difficult. Li et al. [28] proposed a lightweight wheat ear detection model called RT-WEDT, which aims to improve the accuracy and efficiency of wheat ear detection and counting in a complex farmland environment. To enhance the model’s representation ability in complex agricultural scenarios, researchers have also introduced the attention mechanism as a core enhancement method: Firozeh [32] et al. used data rebalancing and the YOLOv8-SE attention module to solve the class imbalance and small-target problems in tomato flower/fruit/bud detection. Yan [33] et al., based on improved YOLOv11 (APYOLO) and DeepSORT, combined MSCA attention and EnMPDIoU loss to achieve apple detection and yield estimation in occluded and overlapping environments.

Although the above methods have shown good performance in detecting and counting various crops, in the actual field environment, due to the differences in the growth patterns of different wheat ears and the influence of factors such as wind and light, the wheat ear images often have problems such as occlusion and sticking, inconsistent scale, and being difficult to distinguish from the background, which bring great challenges to the work of counting wheat ears [34,35]. Previous studies have specifically conducted empirical exploration in dense occlusion scenarios: They proposed AO-DETR, introduced category-to-category one-to-one matching (CSA) and forward dense anchor boxes (LFDs) into the DETR-DINO framework, significantly alleviating the feature coupling and edge blurring caused by overlapping X-ray contraband features [36], as well as the lightweight YOLO-PDGT algorithm based on YOLOv8—under the synergy of a simplified neck, accelerated convolution, and enhanced small-target detection ability through triplet attention, achieving real-time detection and counting of unripe pomegranates [37]. In view of this, this study has proposed an innovative approach. First, hybrid feature extraction is realized by splicing the spatial and frequency domains, which improves the feature extraction capability of the target in the complex background. Second, the GHSDConv and CSPDWOK structures are designed to construct the multi-scale sequence feature fusion module MFEP, which aims to improve the model’s detection capability for targets of different scales. Finally, considering the growing demand for high accuracy and fast response in precision agriculture and the realistic need to reduce the hardware cost, this study adopts the channel pruning technique based on Layer-Adaptive Magnitude Pruning (LAMP) scoring based on a lightweight backbone network in order to reduce the model parameters and runtime memory. The main innovations of this paper are as follows:

(1) The FM module is introduced to transform the bottleneck layer in C2f to realize hybrid feature extraction by splicing spatial and frequency domains, which significantly improves the feature extraction capability of wheat to be tested in complex scenes.

(2) The multi-scale feature enhancement pyramid (MFEP) is designed by combining GHSDConv, which is used to efficiently obtain low-level detail information, and CSPDWOK, which constructs a multi-scale semantic fusion structure. This improves the model’s ability to detect targets at different scales while balancing the expression of small-target details and the context modeling of large targets.

(3) The channel pruning technique based on LAMP scoring is adopted to reduce the model parameters and runtime memory and realize the lightweight characteristic of the model under the premise of guaranteeing the model’s performance, which reduces the hardware cost and meets the demands of precision agriculture for high accuracy and fast responsiveness.

2. Dataset

In this paper, the 2021 version of the global wheat head detection (GWHD) [38] dataset was used as the training dataset for the model. The dataset consists of 6422 wheat head images (JPG format) taken in Asia, North America, Europe, and Oceania, and it contains 275,187 bounding box labels. It covers 60+ wheat varieties (including spring-type, winter-type, local varieties, and modern high-yield varieties). Samples were taken throughout the growth period from the time of wheat head emergence to the complete ripening stage, ensuring that the dataset contains different head shapes, colors, and densities. The imaging conditions were classified according to weather and light conditions: 45% for sunny days, 38% for cloudy days, and 17% for backlighting. This ensures the diversity of scenes and environments. The image specifications include a resolution of 1024 × 1024 pixels taken from heights ranging from 1.8 m to 3 m.

The global wheat sheaf dataset is rich in genetic and environmental diversity, which makes it ideal for use as a source of training data for neural networks and helps to improve the accuracy of wheat sheaf detection. According to the principle of machine learning data partitioning, if the dataset bounding box is in the order of ${10}^4$, considering that a larger validation set can provide more samples for the evaluation of the model configurations and thus be more conducive to the adjustment of the optimal parameters in training, the global wheat sheaf dataset was randomly partitioned into a training set, a validation set, and a test set in the ratio of 7:2:1.

Figure 1 illustrates the distribution of the center points (x, y), widths, and heights of the rectangular boxes. The center points (x, y) are relatively uniformly distributed across the image, indicating a lack of distinct target concentration areas and a more dispersed distribution. The widths and heights are predominantly concentrated within smaller numerical ranges, with the heights exhibiting a near-normal distribution around a mean value, while the widths have a broader distribution but are still predominantly small. This suggests that the dataset contains targets of varying sizes, with most being relatively small and randomly positioned without significant spatial clustering.

3. Methods

3.1. RT-DETR Network

Detection Transformer (DETR) is the first deep learning model to apply the Transformer architecture to the field of object detection. It reframes the task as a sequence prediction problem, drawing inspiration from the Transformer’s encoder–decoder structure and bipartite graph-based matching strategy, while abandoning the non-maximum suppression (NMS) post-processing step. Compared to the YOLO series of algorithms that incorporate NMS post-processing, DETR not only offers lower optimization complexity but also offers greater robustness. However, it suffers from issues such as a large number of parameters and slow convergence, making it challenging to achieve good real-time detection performance in complex tasks. RT-DETR is a novel real-time end-to-end object detection model developed based on DETR. Compared to the YOLO series, it has stronger global feature modeling capabilities that can effectively capture the relationships between objects. In complex scenarios such as detecting closely arranged wheat targets, it demonstrates stronger robustness and better detection performance.

As shown in Figure 2, the RT-DETR architecture mainly consists of three parts: a backbone network, a hybrid encoder, and a decoder. The backbone network uses a convolutional neural network to extract three output features of different scales, namely, 8, 16, and 32.The hybrid encoder decouples intra-scale interactions and cross-scale fusion to effectively process multi-scale features, with the features from the last three layers S3, S4, and S5 serving as encoder inputs. Specifically, the Attention-based Intra-scale Feature Interaction (AIFI) module is used to process high-level features, reducing computational load and improving speed without compromising performance. Meanwhile, the Cross-scale Feature Fusion Module (CCFM) based on convolutional neural networks is employed for multi-scale integration and interaction, converting multi-scale features into a sequence representation of image features. The decoder employs a multi-layer Transformer denoising decoder, which obtains initial object queries from an IoU-aware query selection mechanism and iteratively optimizes them. This mechanism dynamically adjusts queries based on the IoU metric to focus on regions most relevant to the detected targets, improving the quality of bidirectional matching samples and accelerating training convergence. Additionally, it allows for flexible adjustment of the inference speed by changing the number of decoder layers, without the need for retraining. In summary, the RT-DETR model holds great potential for development in the field of object detection in agriculture and complex scenarios.

3.2. General Architecture of the CML-RTDETR Model

The RT-DETR has demonstrated high performance in wheat ear detection. However, in complex agricultural environments, wheat plants vary greatly in size, have similar colors to the background, and ears often overlap, posing challenges for detection. To address this issue, we designed the CML-RTDETR model based on the RT-DETR, which includes a backbone network and multi-scale feature enhancement module. First, we introduce the lightweight network CSPDarknet as the backbone network to effectively improve feature extraction efficiency and reduce computational complexity. Second, we explain how we modified the bottleneck layer in C2f using the FM module and thus could achieve hybrid feature extraction through spatial and frequency domain concatenation, thereby enhancing the feature extraction capabilities for wheat in complex scenes. Third, to balance the expression of small-target details and the contextual modeling of large targets, we detail the design of a multi-scale feature enhancement pyramid (MFEP) composed of GHSDConv for efficiently obtaining low-level detail information and CSPDWOK for constructing a multi-scale semantic fusion structure, thereby improving the model’s detection capability for targets of different scales. Furthermore, a loss function, CML_loss, was designed the model for dense small-object detection in scenes with overlapping and occluded objects to mitigate the missing detection box problem caused by overlapping and occlusion. Finally, channel pruning is performed based on Layer-wise Adaptive Amplitude Pruning (LAMP) scoring to reduce model parameters and runtime memory. Figure 3 illustrates the overall structure of the CML-RTDETR network.

3.3. Backbone Network

3.3.1. CSPDarknet Backbone Network

The selection of an appropriate backbone network is particularly important in RT-DETR-based wheat detection research. The initial official RT-DETR model uses the HGNetV2 [39] as the backbone feature extraction network, which is not powerful enough in multi-scale feature fusion and is limited in terms of network depth and width for small-target detection, sacrificing certain feature extraction capability and accuracy. Therefore, this paper replaces the backbone network with CSPDarknet in YOLOv8, the core of which lies in the use of the cross-stage partial network (CSP) structure, which effectively reduces the amount of computation and improves the learning ability of the network. The specific CSPDarknet structure is shown in Figure 4. CSPDarknet is capable of extracting richer features through a more effective feature transfer and fusion mechanism, which allows for more accurate identification and localization of wheat.

3.3.2. C2f-FM Module

The global wheat dataset is usually characterized by the difficulties in distinguishing wheat from the background in complex backgrounds, large scale differences in the wheat to be tested, strong irregularities, and the presence of many small targets. The C2f structure retains the rich gradient information through the multi-residual connection, but when dealing with highly complex and textured images, its traditional bottleneck structure is limited by the local spatial domain convolution. The traditional bottleneck structure is limited by the limited sensory field of the local spatial domain convolution, which makes it difficult to effectively capture global high-frequency features and low-frequency semantic information. For this reason, this paper proposes a novel Fused Fourier Convolution Mixer (FM) module, which replaces the bottleneck structure in the original C2f, and constructs the C2f-FM module. Gao et al. proposed the Fused Fourier Convolution Mixer (FM) module for efficient global modeling by transforming the spatial domain features to the frequency domain using the discrete Fourier transform (DFT). The method not only retains the global feature perception capability but also avoids the high computational overhead in the self-attention mechanism. Since the background of the field wheat and the object to be measured have significant and stable feature distributions in the frequency domain, they have more obvious differences. The FM module deeply integrates the Fast Fourier Transform (FFT) with the hybrid convolution mechanism in the frequency domain–space domain, aiming to enhance the multi-scale feature characterization capability in complex scenes through the synergistic optimization of the global modeling in the frequency domain and the local feature enhancement. The right side of Figure 4 illustrates the basic principle of C2f-FM. Inspired by the FM module, in order to achieve deep perception of global feature information, as well as a more efficient network model and to reduce the amount of floating-point calculations, the FM module is integrated with the Fourier convolution mixer and C2f. Through spatial domain and frequency domain splicing to realize hybrid feature extraction to enhance the feature extraction ability of wheat to be tested in complex scenes, its specific processing flow is as follows.

Given the input feature map $X\in {\mathbb{R}}^{C\times H\times W}$, it is first mapped to the frequency domain by FFT as follows:

$$\mathcal{F}\left(X\right)=\mathrm{FFT}\left(X\right)\in {\mathbb{C}}^{C\times H\times W}$$

(1)

where $\mathcal{F}$ denotes the Fourier transform operation.

In the frequency domain, a learnable complex convolution kernel $K_{\mathrm{freq}}\in {\mathbb{C}}^{C\times C\times k\times k}$ is designed for cross-channel feature blending as follows:

$$\mathcal{F}\left(Y\right)=\mathcal{F}\left(X\right)\odot K_{\mathrm{freq}}+\mathcal{F}\left(b\right)$$

(2)

where ⊙ denotes frequency domain complex multiplication, and b is the bias term.

The frequency domain features $\mathcal{F}\left(Y\right)$ are recovered to the spatial domain by the Inverse Fast Fourier transform (iFFT) and gated fusion with the original input as follows:

$$Y_{\mathrm{spatial}}={\mathrm{Conv}}_{3\times 3}\left(X\right)$$

(3)

$$G=\sigma \left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{iFFT}\left(\mathcal{F}\left(Y\right)\right)\right)\right)$$

(4)

$$Y_{\mathrm{fused}}=G\otimes Y_{\mathrm{spatial}}+(1-G)\otimes \mathrm{iFFT}\left(\mathcal{F}\left(Y\right)\right)$$

(5)

where $\sigma$ is the sigmoid function, and ⊗ denotes channel-by-channel multiplication. The gating coefficients $G\in {[0,1]}^{C\times H\times W}$ dynamically adjust the contribution weights of the frequency domain and spatial features.

In order to reduce the complexity of the traditional frequency domain convolution $O\left(N^2\right)$, the FM adopts the chunked frequency domain processing strategy defined as follows:

$$\mathrm{Complexity}=C·{\displaystyle \frac{H}{p}}·{\displaystyle \frac{W}{p}}·(plogp)$$

(6)

where p is the chunk size (default 16 × 16).

3.4. Neck Network

3.4.1. Multi-Scale Feature Enhancement Pyramid

In the complex scenario of monitoring wheat in farmland, variations in light conditions, differences in planting density, and different growth stages jointly result in a significant size variation of wheat ears. Therefore, enhancing the model’s ability to detect multi-scale wheat ears is the key to achieving high-precision recognition. Although traditional feature pyramids can alleviate the scale differences to some extent, their lack of deep-level details or rough fusion strategies makes it difficult to simultaneously balance the fine representation of small targets and the context modeling of large targets, thereby limiting further improvement of multi-scale detection performance. To address this challenge, this paper designs a multi-scale feature enhancement pyramid (MFEP) in the neck network. By strengthening cross-layer feature interaction and detail retention, it significantly improves the model’s robustness to extreme scale changes. Its specific structure is shown in Figure 3.

This module is jointly composed of two core sub-modules: GHSDConv and CSPDWOK. The former efficiently extracts shallow details, and the latter accomplishes multi-scale semantic fusion. The collaboration of these two enhances the model’s ability to represent and discriminate different-scale wheat ears. Specifically, GHSDConv acts on the 1/4 resolution shallow feature P2, compressing spatial information losslessly to the channel dimension through a Space-to-Depth operation, generating low-redundancy channel features using Ghost convolution and then restoring the original resolution through Pixel-Shuffle. This allows it to output feature maps of the same scale as the main feature and that are rich in details, thereby retaining the complete information of small targets while reducing computational and memory costs; CSPDWOK, through three complementary receptive field paths of global dual-domain attention, deformable large kernel convolution, and 3 × 3 depth convolution, respectively captures long-range dependencies, medium–large-target structures, and local texture edges, and its complete lightweight fusion within the cross-stage partial framework effectively enhances the model’s ability to represent and discriminate wheat ears of different scales in complex scenes.

3.4.2. GHSDConv

In order to control the model’s computation while introducing shallow high-resolution features, this paper proposes a lightweight shallow feature compression module, GHSDConv, as shown in Figure 5. The module firstly performs a Space-to-Depth (SPD) operation on the high-resolution feature maps output from the second stage in the backbone network. The SPD operation rearranges the local structural information in the spatial dimension to the channel dimension so as to obtain the spatial scale matching with the deep features of the backbone without losing the detail information, which is conducive to the subsequent feature fusion. Subsequently, in order to reduce the redundancy of convolutional computation, a dual-stream GhostConv is used, which is a module that decouples the features into main paths and ghost paths, with the main paths firstly generating a portion of the main features through a small amount of standard convolution and the ghost paths generating the complementary features using depth-separable convolution; after fusion of the dual-stream paths, the later selects the 1 × 1 convolution kernel for applying standard convolutional operations to the rearranged tensor. The specific process can be expressed as

$$X_{sd}=\mathrm{Space}\mathrm{to}\mathrm{Depth}\left(X\right)=\mathrm{Concat}(X_{00},X_{10},X_{01},X_{11})$$

(7)

Among them,

$$\begin{array}{cc}\hfill X_{00}& =X[:,:,0::2,0::2]\hfill \\ \hfill X_{10}& =X[:,:,1::2,0::2]\hfill \\ \hfill X_{01}& =X[:,:,0::2,1::2]\hfill \\ \hfill X_{11}& =X[:,:,1::2,1::2]\hfill \end{array}$$

$$X_{\mathrm{main}}=\mathrm{SiLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{1\times 1}\left(X_{sd}\right)\right)\right)$$

(8)

$$X_{\mathrm{cheap}}=\mathrm{SiLU}\left(\mathrm{BN}\left({\mathrm{DWConv}}_{3\times 3}\left(X_{\mathrm{main}}\right)\right)\right)$$

(9)

$$Y={\mathrm{Conv}}_{1\times 1}\left(\mathrm{Concat}\left(X_{\mathrm{main}},X_{\mathrm{cheap}}\right)\right)$$

(10)

3.4.3. CSPDWOK

In order to enhance the model’s ability to express different sensory field features, this paper further designs a multi-path fusion structure—CSPDWOK—to fuse feature maps at different scales. The specific network structure is shown in Figure 6. The design of this module is inspired by the CSP structure, which splits the original channel into two paths, ok_branch and identity, and the data in the ok_branch path are processed by Depth-wise Separable Convolution (DSConv) and then transformed by OmniKernel; followng this, the data are spliced into two paths. The data are bipath-spliced and then passed through the Depth-wise Separable Convolution with SE attention. The OmniKernel structure incorporates global, large-scale, and local branches through a parallel three-branch design to model global, local, and medium-sense wild feature representations, respectively, and reduces resource overhead through a lightweight design. GlobalPath: Dual-domain attention is introduced to integrate spatial attention and channel attention to strengthen the model’s ability to model large-scale targets and the contextual semantics of the whole map. Large Kernel Path: The network adopts different shapes of large-scale DWConv (DepthwiseConv) convolution to enhance the model’s ability to model medium-scale structures. Small Path: The network extracts local detail features using point-by-point depth-wise convolution to enhance the model’s ability to represent small-target edge textures. The CSP structure is used in the ok_branch and identity splicing process, which effectively suppresses the problem of gradient vanishing and feature redundancy through partial passing of the feature channel with residual connection while further improving the computational efficiency of the model. The CSPDWOK module, as the core fusion component of the MFEP, makes the model capable of taking into account details and semantic expression when facing different scales of wheat targets and improving detection robustness through the parallel modeling of the information of multi-scale paths. The specific process can be expressed as

$$Y_1=\mathrm{DConv}\left(X\right)=\mathrm{SiLU}\left(\mathrm{BN}\left(X\ast W_{\mathrm{dw}}^{\left(1\right)}\right)\ast W_{\mathrm{pw}}^{\left(1\right)}\right)$$

(11)

$$(Y_{\mathrm{ok}},Y_{\mathrm{id}})=\mathrm{Split}\left(Y_1\right)=\left\{\begin{array}{c}{Y}_{\mathrm{ok}}\in {\mathbb{R}}^{\epsilon C\times H\times W}\hfill \\ Y_{\mathrm{id}}\in {\mathbb{R}}^{(1-\epsilon )C\times H\times W}\hfill \end{array}\right.$$

(12)

$$Z_{\mathrm{ok}}=\Omega \left(Y_{\mathrm{ok}}\right)=W_{\mathrm{out}}·\mathrm{ReLU}\left(\sum _{k\in \{1,3,3,1,3,3,1,1\}}{Y}_{\mathrm{ok}}\ast W_{\mathrm{dw}}^{\left(k\right)}+{\mathsf{\Gamma }}_{\mathrm{FCA-SCA}}\left(Y_{\mathrm{ok}}\right)\right)$$

(13)

where

$${\mathsf{\Gamma }}_{\mathrm{FCA-SCA}}\left(X\right)=\mathrm{ReLU}\left({\displaystyle \frac{{\mathcal{F}}^{-1}({\mathsf{\Phi }}_{\mathrm{fac}}\odot \mathcal{F}\left(X\right))}{\mathrm{FCA}}}\odot {\mathsf{\Phi }}_{\mathrm{sca}}\right)$$

(14)

$${\mathsf{\Phi }}_{\mathrm{fac}}=W_{\mathrm{fac}}·\mathrm{GAP}\left(X\right)$$

(15)

$${\mathsf{\Phi }}_{\mathrm{sca}}=W_{\mathrm{sca}}·\mathrm{GAP}\left(X\right)$$

(16)

$$Y_{\mathrm{fuse}}=\mathrm{Concat}(Z_{\mathrm{ok}},Y_{\mathrm{id}})\in {\mathbb{R}}^{C\times H\times W}$$

(17)

$$Z_{\mathrm{final}}=\mathrm{SE}\left(\mathrm{DW}-\mathrm{PWConv}\left(Y_{\mathrm{fuse}}\right)\right)$$

(18)

$$Z_{\mathrm{final}}=\left[\mathrm{SiLU}\left(\mathrm{BN}\left(Y_{\mathrm{fuse}}\ast W_{\mathrm{dw}}^{\left(2\right)}\right)\ast W_{\mathrm{pw}}^{\left(2\right)}\right)\right]\odot \sigma \left(W_2\delta \left(W_1\mathrm{GAP}\left(Y_{\mathrm{fuse}}\right)\right)\right)$$

(19)

where the input features are $X\in {\mathbb{R}}^{C\times H\times W}$, and the channel segmentation scale is $e\in (0,1)$.

3.5. The Improved Loss Function

The GIoU loss function used by the RT-DETR solves the problem of a zero gradient when there is no intersection of the two objectives by introducing the minimum outer rectangle of the prediction frame and the true frame to obtain the weight of the prediction frame and the true frame in the region of the minimum outer rectangle [40]. However, there is a failure problem of GIoU loss in the case where the prediction frame completely covers the real frame. To solve this problem, we designed a new loss function of CML_loss function to speed up the fitting speed of the model to the positional relationship between the prediction frame and the ground truth frame, especially in the scenes involving dense small-target that appear to be overlapped and occluded, so as to achieve more accurate target localization. The computational process of CML_loss is greatly simplified and better solves the problem of loss function failure when the aspect ratio is the same, especially in scenes involving overlapping occlusion, so as to realize more accurate target localization. Specifically, we introduce the MPDIoU and innerIoU, and we combine them with the NGWD through weighting to construct the CML_loss. The formula of CML_loss is as follows:

$$L_{\mathrm{inner-MPDIoU}}=1-\mathrm{MPDIoU}+\mathrm{IoU}-\mathrm{innerIoU}$$

(20)

$$CM{L}_{loss}=\alpha L_{\mathrm{inner-MPDIoU}}+(1-\alpha )NGWD(N_a-N_b)$$

(21)

where the Normalized Gaussian Wasserstein Distance (NGWD) is a new metric for calculating the similarity between boxes. It models boxes as Gaussian distributions and then uses the Wasserstein distance to measure the similarity between these two distributions, replacing the loU. The advantage of this distance is that even if two boxes do not overlap at all or overlap very little, it can still measure their similarity. In addition, the NGWD is insensitive to the scale of the target and is more stable for small targets. The formula for the NGWD Loss is as follows:

$$W_2^2(N_a,N_b)={∥\left({\left[\begin{array}{c}c{x}_a,c{y}_a,{\displaystyle \frac{w_a}{2}},{\displaystyle \frac{h_a}{2}}\end{array}\right]}^T,{\left[\begin{array}{c}c{x}_b,c{y}_b,{\displaystyle \frac{w_b}{2}},{\displaystyle \frac{h_b}{2}}\end{array}\right]}^T\right)∥}_2^2$$

(22)

$$NGWD(N_a,N_b)=exp\left(-{\displaystyle \frac{\sqrt{W_2^2(N_a,N_b)}}{C}}\right)$$

(23)

The innerIoU dynamically adjusts the loss weights through an auxiliary box IoU, where high-IoU samples are accelerated and optimized using small auxiliary boxes, while low-IoU samples are adjusted in the opposite direction to prevent overfitting, thereby achieving adaptive regression.

The MPDIoU (Multi-Point Distance Intersection over Union) is an improved bounding box regression loss function. Its core lies in optimizing the positioning of bounding boxes by directly minimizing the distances between the predicted boxes and the real boxes at key points. It comprehensively considers the overlapping area, center point distance, and width–height deviation. Compared to traditional IoU series loss functions, its calculation process is simpler, and its performance is better. This method combines the IoU with a point distance penalty term. The formula is

$$MPDIoU=IoU-{\displaystyle \frac{{(x_1^{prd}-x_1^{gt})}^2+{(y_1^{prd}-y_1^{gt})}^2}{w^2+h^2}}-{\displaystyle \frac{{(x_2^{prd}-x_2^{gt})}^2+{(y_2^{prd}-y_2^{gt})}^2}{w^2+h^2}}$$

(24)

where $(x_1^{gt},y_1^{gt}),(x_2^{gt},y_2^{gt})$ are the coordinates of the ground truth bounding box, $(x_1^{prd},y_1^{prd})$ and $(x_2^{prd},y_2^{prd})$ are the predicted bounding box coordinates, and the values $w,h$ are the width and height of the input image, respectively.

3.6. LAMP

Layer-Adaptive Magnitude Pruning (LAMP) achieves model lightweighting by quantifying the relative importance of channel weights. Its core lies in using mathematical modeling to measure the contribution of individual channels to feature extraction in the network. The LAMP score is defined as follows: For the weight tensor W of a layer in a neural network, when it is flattened into a one-dimensional vector and sorted by index (where index u corresponds to the weight $W\left[u\right]$, and for $u<v$, $W\left[u\right]\le W\left[v\right]$), the LAMP score of the u-th weight is given by

$$\mathrm{score}(u;W)={\displaystyle \frac{{\left(W\left[u\right]\right)}^2}{{\sum }_{v\ge u}{\left(W\left[v\right]\right)}^2}}$$

(25)

Here, the denominator ${\sum }_{v\ge u}{\left(W\left[v\right]\right)}^2$ represents the sum of squared magnitudes of all remaining connections in the same layer, and the numerator ${\left(W\left[u\right]\right)}^2$ represents the squared magnitude of the target connection’s weight.

The core principle of the LAMP pruning method is to calculate the LAMP score for the weights of each layer in a neural network. For a weight tensor of a certain layer, it is expanded into a one-dimensional vector and sorted by index. The ratio of each weight squared to the sum of the squares of the remaining weights in that layer is calculated. The lower the score, the more redundant the channel. This method uses a global pruning strategy to remove low-score channels while ensuring that each layer retains at least one valid connection to prevent layer collapse. It belongs to the category of structured pruning, which maintains the integrity of the network structure and avoids the issues of irregular connections and memory overhead caused by unstructured pruning. The specific algorithm flow is shown in Algorithm 1.

In the improved CML-RTDETR model, LAMP removes channels that contribute little to wheat spike feature extraction, reducing the number of model parameters and degree of computation while retaining high-weight channels related to key features such as spike morphology and texture. Combined with a fine-tuning mechanism for the pruned model, this approach achieves model lightweighting with only a slight loss in accuracy.

Algorithm 1 Channel pruning based on LAMP scores.
Input	Model parameters before pruning, global pruning rate
	1. Traverse all layers of the model to obtain the weight parameters of all convolutional layers
	2. For each convolutional layer l:
Steps	2.1. Extract the weight matrix $W_l$ of this layer and flatten it into a one-dimensional vector ${\widehat{W}}_l$
	2.2. Calculate the square of each weight: ${\widehat{W}}_l^2=\left\{w^2\right|w\in {\widehat{W}}_l\}$
	2.3. Sort the squares in descending order to obtain the sorted index idx
	2.4. Calculate the sum of squares of all weights in this layer $S={\sum }_{w\in {\widehat{W}}_l}{w}^2$
	2.5. Calculate the LAMP score for each weight: $\mathrm{score}(u;W_l)={\displaystyle \frac{{\left({\widehat{W}}_l\left[\mathrm{idx}\left[u\right]\right]\right)}^2}{{\sum }_{i=u}^{|{\widehat{W}}_l|-1}{\left({\widehat{W}}_l\left[\mathrm{idx}\left[i\right]\right]\right)}^2}}$
	3. Based on the pruning rate r, remove the $r\times$ number of channels with the lowest LAMP scores from each layer, ensuring at least one channel is retained in each layer.
	4. Reconstruct the model structure after pruning, retaining the weights of the unpruned channels.
Output	The pruned model parameters and structure.

4. Experimental Setting and Evaluation Metrics

4.1. Experimental Environment

The experiments in this paper were conducted on a computer with an Ubuntu operating system, equipped with an NVIDIA RTX 3090 GPU (24 GB of VRAM) and Intel (R) Xeon (R) Platinum 8358P CPU @ 2.60 GHz processor. The software utilized PyTorch 1.11.0, CUDA 11.3, and Python3.8 to realize the experiments. The training parameters were set as follows: the batch size was 16, the number of training rounds was 200, the learning rate was set to 0.0001, and the training process used the cosine annealing strategy and the AdamW optimizer, which remained unchanged in subsequent experiments.

4.2. Evaluation Metrics

In this paper, the standard COCOAP evaluation metrics were used to evaluate the model performance: ${\mathrm{AP}}_{50}$, ${\mathrm{AP}}_{75}$, ${\mathrm{AP}}_{50-95}$, ${\mathrm{AP}}_S$, ${\mathrm{AP}}_M$, and ${\mathrm{AP}}_L$. The AP is the area enclosed by the P-R curve, which is used to measure how good the model is at detecting a class. The formula for average precision is expressed as

$$AP={\int }_0^{1}P\left(r\right)dr$$

(26)

The IoU is the ratio of the intersection and concatenation of the predicted and real frames, and this ratio is used to measure the degree of overlap between the predicted and real frames. ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_{75}$ denote the average accuracy values under the IoU thresholds of 0.50 and 0.75, respectively, and ${\mathrm{AP}}_{50-95}$ denotes the average accuracy mean values under the IoU thresholds of 0.50–0.95 (with a step size of 0.05) for ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_{75}$. ${\mathrm{AP}}_{50-95}$ denotes an overall generalization of the model accuracy, ${\mathrm{AP}}_S$ denotes the value of ${\mathrm{AP}}_{50-95}$ in the small-target range, ${\mathrm{AP}}_M$ denotes the value of ${\mathrm{AP}}_{50-95}$ in the medium-target range, and ${\mathrm{AP}}_L$ denotes the value of ${\mathrm{AP}}_{50-95}$ in the large-target range. In this paper, the parameter count (parameters) and the giga floating-point operations per second (GFLOPs) were used to measure the complexity of the model, and the frame rate (frames per second, FPS) was used to measure the real-time performance of the model.

5. Results and Analysis

5.1. Comparison Experiment

5.1.1. Detection Performance Evaluation

In order to compare the detection effect of this paper’s model with the current mainstream target detection models for wheat ears in the field, the classical CNN-based detection models Faster-RCNN [41], SSD [42], Centernet [43], and Retinanet [44]; lightweight CNN networks YOLOv8n [45], YOLOv10n [46], YOLOv11n [47], and YOLOv12n [48]; and the Transformer-based network model RT-DETR series were selected in this paper.

Table 1. Comparison between this research program and mainstream model.

Model	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%	fps
Faster-RCNN	26.98	948.1	28.6	62.6	22.0	3.1	28.5	35.5	7
SSD	22.52	273.2	33.0	68.5	27.2	3.8	33.0	40.6	56
Centenet	31.15	109.7	49.6	87.7	50.5	13.4	49.3	58.6	50
Retinanet	34.65	163.5	46.4	84.5	45.8	11.1	46.0	55.7	53
YOLOv8n	2.86	8.1	52.0	90.0	53.9	16.9	51.5	61.4	118
YOLOv10n	2.16	6.5	51.6	89.6	53.3	16.8	51.1	60.5	110
YOLOv11n	2.46	6.3	52.0	89.7	54.0	16.1	51.5	61.5	112
YOLOv12n	2.39	5.8	52.1	89.9	54.0	15.5	51.6	61.9	82
RTDETR-R18	18.95	56.9	51.2	89.3	52.1	16.4	50.6	61.0	62
RTDETR-R34	29.66	88.8	52.1	90.3	53.2	18.0	51.5	61.4	54
RTDETR-R50M	34.57	96.1	52.2	90.2	53.8	17.9	51.4	62.1	52
RTDETR-L	30.50	103.4	52.1	90.3	53.2	17.7	51.5	61.4	45
RTDETR-R50	40.01	129.5	52.3	90.5	53.5	19.2	51.5	62.1	39
CML-RTDETR	11.03	37.8	52.7	90.5	54.2	19.0	52.0	62.9	73

Bold values indicate statistically significant results.

shows the results of the evaluation metrics of the CML-RTDETR model proposed in this paper compared against other mainstream target detection models on the global wheat dataset, from which it can be seen that the parameter size of the CML-RTDETR model is 11.03 M, GFLOPs is 37.8 G, ${\mathrm{AP}}_{50-95}$ is 52.7%, ${\mathrm{AP}}_{50}$ is 90.5%, and real-time frame rate is 73 fps.

Compared with the RT-DETR series models, the CML-RTDETR model demonstrates significant performance advantages in terms of parameter efficiency and real-time performance: Its parameter size is only 58% of the RTDETR-R18 model (18.95M) and 28% of the RTDETR-R50 model (40.01M), with GFLOPs reduced by 33% and 71%, respectively, compared to the aforementioned models. In terms of detection accuracy, the ${\mathrm{AP}}_{50}$ of CML-RTDETR is 90.5%, putting it on par with RTDETR-R50. In terms of real-time performance, its real-time frame rate of 73 fps is significantly higher than the 54 fps of RTDETR-R34 and the 39 fps of RTDETR-R50, representing a 17.7% improvement compared to the baseline model RTDETR-R18. This meets real-time detection requirements while maintaining the same accuracy, thereby achieving parameter simplification and speed improvement.

A comparative analysis was conducted between CML-RTDETR and the YOLO series networks with lower parameter densities, YOLOv8n to YOLOv12n, primarily based on the actual deployment requirements of agricultural scenarios. The parameter ranges of these models are from 2.16 M to 2.86 M, with the GFLOPs ranging from 5.8 G to 8.1 G. By streamlining network depth and width, these models achieve extremely low computational overhead, making them naturally suited for resource-constrained scenarios such as drone inspections and edge devices, thereby effectively reducing hardware costs. Additionally, CML-RTDETR breaks through the traditional bottleneck where lightweight models struggle to balance parameter scale and detection accuracy: Despite having a parameter scale four to five times larger than the YOLO series nano versions, it employs optimization techniques such as the FM module’s frequency domain-to-spatial domain feature fusion mechanism and the MFEP’s multi-scale feature enhancement strategy. These result in significantly improved detection robustness in complex background interference and target overlap scenarios compared to the comparison models. CML-RTDETR achieved an ${\mathrm{AP}}_{50}$ of 90.5%, outperforming YOLOv8n (90.0%),YOLOv10n (89.6%), YOLOv11n (89.7%), and YOLOv12n (89.9%) by 0.5%, 0.9%, 0.8%, and 0.6%, respectively.The small-object detection accuracy (${\mathrm{AP}}_s$) was 19.0%, significantly better than the 15.5% of YOLOv12n and the 16.1% of YOLOv11n. It led by 2.5 to 3.5 percentage points, demonstrating significantly superior small-object perception capabilities in complex large-scale environments compared to the YOLO series networks with lower parameter density. The lightweight design of CML-RTDETR focuses on improving parameter utilization efficiency rather than simply minimizing parameter scale, achieving an optimal balance between complexity and detection accuracy compared to other models and thereby showcasing its strong comprehensive performance advantages.

In order to specifically visualize and hold the performance difference between different detection models in terms of detection, we performed wheat ear detection inspection on the GWHD dataset. Figure 7 shows the inspection details; the red rectangular boxes denote the model prediction boxes, the blue rectangular boxes denote the wheat ears missed by the model, and the white rectangular boxes denote the redundant detection boxes generated by the model. As can be seen from Figure 7, all CNN models exhibited false negatives, while the RT-DETR series showed varying degrees of false positives. These results are due to the similarity between wheat ears and the background in complex field environments, making them difficult to distinguish. The CML-RTDETR model proposed in this paper could fully extract information from the global features of images, achieving the best overall detection performance in the examples shown in the figure and demonstrating the superiority of the proposed algorithm. In summary, the performance improvement of CML-RTDETR is attributed to the precise selection of redundant channels by the LAMP technique and the efficient fusion of multi-scale features by the CSPDWOK module. This enables the model to maintain its global feature modeling capability while better adapting to the real-time detection requirements of field scenarios, fully demonstrating its application potential in agricultural high-throughput monitoring scenarios.

5.1.2. Accuracy Aalysis in Different Scenes

In order to deeply explore the influence of environmental factors on the detection accuracy of the model, this paper manually screened the wheat images under four typical field scenes, namely, complex background, blurring, strong illumination, and overlapping occlusion.

The original real images and predicted images of each scene are shown in Figure 8. From the local magnified areas, it can be seen that even in the case of complex background textures, the detection boxes output by the model are highly consistent with the manually annotated true values, with negligible differences. Under the conditions of motion blur and leaf occlusion, the target edges are weakened, the foreground–background discrimination decreases, the model shows a certain decline in performance, there are missed detections, and they are mainly concentrated on the targets partially occluded at the tip of the wheat ears. In the strong-light condition, the overexposed areas enhance the local contrast, and the model mistakenly identified the highlights as the contour of the wheat ears, resulting in redundant detection boxes. However, in general, in the face of the complex field environment under natural conditions, the model’s detection performance was not significantly affected, and it has certain generalization ability and robustness.

5.1.3. Counting Performance Analysis

In order to further evaluate the counting performance of the model proposed in this paper, we chose to compare its counting performance with some classic density estimation-based counting models (MCNN [49], CSRNet [50], and the TasselNet [51] series) and the new models in crop counting in recent years (such as RapeNet [52], RPNet [27], RiceNet [53], and CMRNet [54]). The mean absolute error (MAE), root mean absolute error (rMAE), root mean squared error (RMSE), and the coefficient of determination ${\mathrm{R}}^2$ were used as evaluation metrics. Among them, the smaller the MAE value, the smaller the error between the real value and the predicted value. The smaller the RMSE value, the smaller the dispersion between the predicted value and the real value. ${\mathrm{R}}^2$ represents the fitting degree of the trend line, and the closer it is to 1, the higher the fitting degree is. All models in the comparison experiments were trained and tested on the global wheat dataset, and the comparison results of the different methods are shown in

Table 2. Performance comparison of different models.

Model	MAE	rMAE	RMSE	${\mathit{R}}^2$
MCNN	13.21	3.63	15.36	0.75
CSRNet	10.35	3.21	13.95	0.84
TasselNetV2	8.78	2.96	11.15	0.89
RapeNet	7.36	2.71	10.45	0.91
RPNet	5.91	2.43	8.19	0.92
RiceNet	5.62	2.37	7.98	0.91
CMRNet	3.27	1.80	5.37	0.93
CML-RTDETR	2.75	1.66	4.75	0.95

Bold values indicate statistically significant results.

. The fitting result graph of ${\mathrm{R}}^2$ is shown in Figure 9.

The experimental results show that our proposed method yielded an 18% advantage over the current state-of-the-art density regression counting network in the core metric of the MAE. This performance difference mainly stems from the complex background textures and significant illumination variations in wheat ear images, which make it difficult for models based on density maps to obtain stable feature representations, thereby reducing their counting accuracy and robustness. In contrast, the model in this paper, through the collaborative effect of the FM module and MFEP, can more effectively capture target features in complex scenarios, suppress the negative impacts brought by background and illumination intensity changes, and thereby improve the overall counting performance.

5.2. Ablation Experiments

5.2.1. Hyperparametric Experiment

To explore the impact of optimizers on the performance of the baseline model RTDETR-R18, which served as the foundation for subsequent model improvements and training, hyperparametric experiments were conducted by comparing four optimizers: RAdam, Adam, NAdam, and AdamW. The results are presented in

Table 3. Hyperparametric Experimental results of each optimizer.

Optimizer	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%
RAdam	18.95	56.9	45.5	86.4	43.0	13.0	45.4	53.2
Adam	18.95	56.9	45.1	86.2	41.9	13.0	44.9	52.0
NAdam	18.95	56.9	45.4	86.8	42.1	11.3	45.2	53.7
AdamW	18.95	56.9	51.2	89.3	52.1	16.4	50.6	61.0

Bold values indicate statistically significant results.

, covering key metrics such as parameters, GFLOPs, and various AP values.

As shown in Table 3, AdamW outperformed other optimizers, and the baseline model RTDETR-R18 performed best under AdamW, with an AP₅₀₋₉₅ of 51.2%, an AP₅₀ of 89.3%, and an AP₇₅ of 52.1%, producing results that were significantly higher than the corresponding metrics of RAdam, Adam, and NAdam.The latter three methods performed relatively poorly, with their AP₅₀₋₉₅ values ranging from 45.1% to 45.5% and AP₅₀ values ranging from 86.2% to 86.8%.

This sensitivity analysis of the optimizer hyperparameter demonstrates that the performance of RTDETR-R18 varies significantly with different optimizers. It reveals the model’s sensitivity to this hyperparameter, providing valuable insights for selecting the optimal optimizer in subsequent improvements and training to ensure robust performance.

5.2.2. Backbone Network Ablation Experiment

To validate the effectiveness of the proposed improved CSPDarknet backbone network, representative backbone networks were selected, including CNN-based HGNetV2, Fasternet [55], MobilenetV4 [56], and ConvNextv2 [57], as well as Transformer-based EfficientViT [58], EfficientFormerv2 [59], and Swin-Transformer [60], to replace the original backbone of RT-DETR for comparison experiments. The experimental results are shown in

Table 4. Backbone network ablation experimental results of each improved module.

Backbone	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%
HGNetV2	18.95	56.9	51.2	89.3	52.1	16.4	50.6	61.0
Fasternet	10.30	28.5	49.4	88.0	51.2	14.0	49.8	59.7
MobilenetV4	10.78	39.5	49.4	87.8	49.6	13.4	48.7	59.7
Convnextv2	11.73	31.9	50.5	88.7	51.2	14.0	49.8	61.6
EfficientViT	10.20	27.2	48.4	86.3	48.7	11.6	47.5	59.7
EfficientFormerv2	11.25	29.4	51.5	89.3	52.6	15.2	50.8	61.7
Swin-Tranformer	34.63	97.0	51.4	89.4	52.5	16.1	50.9	61.2
Improved CSPDarknet	15.79	50.5	52.2	90.2	53.7	17.4	51.5	62.0

Bold values indicate statistically significant results.

. As can be seen from Table 4, although the improved CSPDarknet backbone network designed in this paper has a slightly higher parameter count and computational complexity compared to lightweight backbone networks such as Fasternet and EfficientViT, it achieved significant improvements in all accuracy metrics. Its AP50-95 reached 52.2%, higher than all other backbone networks, with its AP50 at 90.2% and its AP75 at 53.7%. It also achieved the best performance in small-object detection accuracy (${\mathrm{AP}}_S$), medium-object detection accuracy (${\mathrm{AP}}_M$), and large-object detection accuracy (${\mathrm{AP}}_L$). When compared to the Transformer-based Swin-Transformer, although Swin-Transformer achieved similar performance on some metrics, the improved CSPDarknet has a parameter count of only 15.79 M, significantly lower than Swin-Transformer’s 34.63 M, and a computational complexity of 50.5 G, far smaller than Swin-Transformer’s 97.0 G. This results in significantly reduced computational costs while maintaining higher accuracy. In summary, the improved CSPDarknet achieves the optimal balance between complexity and detection accuracy compared to other state-of-the-art methods.

5.2.3. Network Ablation Experiment

In this paper, 631 wheat ears images selected from the global wheat ears dataset were used as the test set for backbone network ablation experiments, and the proposed modules were added one by one to verify the impact of the proposed modules on the performance of the model.

Table 5. Network ablation experimental results of each improved module.

Name	Model	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%
EXP1	RTDETR	18.95	56.9	51.2	89.3	52.1	16.4	50.6	61.0
EXP2	RTDETR+A	15.79	50.5	52.2	90.2	53.7	17.4	51.5	62.0
EXP3	RTDETR+A+B	16.11	53.8	52.7	90.5	54.7	18.0	52.0	63.2
EXP4	RTDETR+A+B+C	16.11	53.8	52.9	90.6	54.6	18.9	52.2	63.4
EXP5	RTDETR+A+B+C+D	11.03	37.8	52.7	90.5	54.2	19.0	52.0	62.9

Bold values indicate statistically significant results.

shows the test results of adding each module. A, B, C, and D represent the four modules—improved CSPDarknet, MFEP, CML_loss, and LAMP, respectively.

As can be seen from the table, after CML-RTDETR improved the original backbone network of RTDETR-improved CSPDarknet, the ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_{50-95}$ results increased by 0.9 and 1.0 percentage points, respectively, and the model’s overall number of parameters and floating-point arithmetic were reduced by 16% and 11%. On this basis, the number of parameters only increased by 0.32 M after adding the multi-scale feature enhancement pyramid, but the ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_{50-95}$ improved by 0.3 and 0.5 percentage points, respectively. In addition, there was a 0.6 percentage point improvement in the detection accuracy metrics for the ${\mathrm{AP}}_S$ on small targets, a 0.5 percentage point improvement in the ${\mathrm{AP}}_M$ on medium targets, and a 1.2 percentage point improvement in the ${\mathrm{AP}}_L$ on large targets, which verifies that the multi-scale feature enhancement pyramid proposed in this paper can improve the model’s detection ability on wheat targets of different scales. After replacing the CML_loss function, the ${\mathrm{AP}}_{50}$ and small-target ${\mathrm{AP}}_S$ were improved by 0.1 and 0.9 percentage points, respectively, while the number of parameters was kept constant. Since the model in this paper considered the loss in accuracy for small targets after lightweighting, the final model improved the detection accuracy index ${\mathrm{AP}}_{50}$ by 1.2 percentage points to 90.5% and the ${\mathrm{AP}}_{50-95}$ by 1.5 percentage points while reducing the number of parameters in the model. These results prove the effectiveness of the improved module in this paper. In order to verify the effectiveness of the improved module more intuitively, this paper utilized a heat map to embody the network’s ability to capture the target features of wheat ears, gradually adding each module adopted in this paper under the benchmark of RT-DETR and comparing the results with the original RT-DETR. The experimental results in Figure 10 show that the original RT-DETR focuses attention on wheat ears, the wheat ear pole diameter, and part of the background, whereas the CML-RTDETR favors attention more toward wheat ears and reduces the degree of attention on the background and wheat ear pole diameter.

5.2.4. Loss Function Ablation Experiment

In order to explore the effect of the NGWD and $L_{\mathrm{inner-MPDIoU}}$ weighting on the performance of the RT-DETR model, this paper conducted seven sets of experiments with different weight values ranging from 0 to 1. The results of the experiments are shown in

Table 6. Experimental results of loss functions with different weights.

Name	$\mathit{\alpha }$	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%
EXP 1	-	16.11	53.8	52.7	90.5	54.7	18.0	52.0	63.2
EXP 2	0	16.11	53.8	52.8	90.6	54.6	18.6	52.1	62.8
EXP 3	0.2	16.11	53.8	52.6	90.5	54.5	18.1	51.8	62.9
EXP 4	0.4	16.11	53.8	52.6	90.4	54.7	18.2	51.7	63.1
EXP 5	0.6	16.11	53.8	52.5	90.5	54.4	17.8	51.8	63.0
EXP 6	0.8	16.11	53.8	52.9	90.6	54.6	18.9	52.2	63.4
EXP 7	1.0	16.11	53.8	52.6	90.4	54.6	18.2	51.9	62.7

Bold values indicate statistically significant results.

. By comparing the training effect under different weights, the effect of $L_{\mathrm{inner-MPDIoU}}$ and NGWD on the model performance could be analyzed.

From the experimental results in the table, it can be seen that even without introducing the loss function we designed, this model was still able to maintain a high level of focus on the target area; however, when encountering background occlusion areas, there were cases where the model incorrectly identified the occlusion areas as the target areas. Different weight values were found to have different impacts on the performance of the model. When the weight value $\alpha$ of the loss functions of $L_{\mathrm{inner-MPDIoU}}$ and NGWD is 0.8, the performance of the model reaches its peak. Under this configuration, the model takes into account the needs of small-target detection and multi-scale target detection; the detection accuracy of each category was improved, and the AP₅₀ reached the highest value of 90.6. As can be seen from Figure 11, under this configuration, the model paid the closest attention to the target, and the interference from the background was the least. Therefore, in this paper, the weight value $\alpha$ of 0.8 was used to combine these loss functions to obtain the best model performance.

5.2.5. Model Pruning Rate Ablation Experiment

According to the ablation experiments and heat maps, the CML-RTDETR detection accuracy has been improved, but the model has redundant parameters. Therefore, a pruning operation based on the LAMP score was used to compress the model. During pruning, some output layers could not be pruned, so we used a skip-layer operation to prune the network and adjusted different pruning rates (the final model GFLOPs as a proportion of the original model) to achieve different degrees of pruning. In order to further explore the impact of pruning operations on model efficiency while ensuring detection accuracy, we conducted ablation experiments with different pruning rates (defined as the ratio of the final model’s GFLOPs to the original model’s GFLOPs) and comprehensively evaluated the changes in key metrics such as parameters, computational complexity, detection accuracy, and real-time performance.

Table 7. Performance of the model with different pruning rates.

Pruning Rate	Parameters	GFLOPs/G	${\mathrm{AP}}_{50-95}$/%	${\mathrm{AP}}_{50}$/%	${\mathrm{AP}}_{75}$/%	${\mathrm{AP}}_{\mathit{S}}$/%	${\mathrm{AP}}_{\mathit{M}}$/%	${\mathrm{AP}}_{\mathit{L}}$/%	fps
base	16.11	53.8	52.9	90.6	54.6	18.9	52.2	63.4	56
1.2	12.12	43.7	52.7	90.5	54.4	18.8	52.0	62.9	62
1.4	11.03	37.8	52.7	90.5	54.2	19.0	52.0	62.9	73
1.6	10.29	32.7	52.3	90.4	53.4	18.4	51.5	62.1	70
1.8	9.84	29.4	51.7	89.6	52.6	17.1	51.1	61.6	68
2.0	9.27	25.3	50.7	89.1	51.4	15.1	50.1	61.0	71
2.2	9.08	23.9	50.3	88.8	50.8	14.8	49.9	60.0	67

Bold values indicate statistically significant results.

shows the changes in the performance metrics under different pruning rates. It can be observed from the experimental data that as the pruning rate increased, the number of parameters and GFLOPs of the model showed a decreasing trend, which is consistent with the expectation of model lightweighting. Notably, the real-time performance (fps) of the model presented a non-linear variation: It increased from 56 fps of the base model to the highest 73 fps when the pruning rate was 1.4 and then decreased slightly with the further increase in the pruning rate. This indicates that an appropriate pruning rate can effectively reduce redundant computations and improve the inference speed, but excessive pruning may damage the integrity of the network structure, leading to a slight decline in real-time performance. In terms of detection accuracy, when the pruning rate was 1.2 and 1.4, key metrics such as ${\mathrm{AP}}_{50-95}$), ${\mathrm{AP}}_{50}$, and ${\mathrm{AP}}_S$ remained basically stable compared to the base model, with no significant loss. However, when the pruning rate reached 1.6 and above, indicators like ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_S$ showed a noticeable decline, which may be due to the excessive removal of channels related to important feature extraction, affecting the model’s ability to capture target information. Furthermore, as shown in Figure 12, analyzing the changes in the number of channels in each layer of the network before and after pruning, the proportion of output channels decreased with increasing pruning rate, while the backbone network (layer indices 1 to 8) showed no significant changes. This indicates that the shallow layers have low redundancy, while the deep layers exhibit more parameter redundancy, and pruning can remove these redundant parameters without significantly affecting the overall network performance.

Comprehensively considering the balance between detection accuracy and lightweight performance, especially the significant improvement in real-time performance (73 fps) at a pruning rate of 1.4 while maintaining high accuracy, this pruning rate was selected as the optimal choice for the final model. This ensures that the model can meet the requirements of real-time detection in complex farmland scenarios while maintaining accurate wheat ear detection and counting capabilities, laying a solid foundation for its practical application in agricultural high-throughput monitoring.

6. Conclusions

In order to realize accurate detection of wheat and facilitate the subsequent realization of lightweight accurate counting of wheat, this paper proposes an improved wheat detection and counting model based on RT-DETR, and the main conclusions are as follows: The lightweight network CSPDarknet was adopted as the backbone network of RT-DETR to improve the feature extraction efficiency, and the FM module was introduced to modify the bottleneck in C2f to realize hybrid feature extraction through spatial and frequency domain splicing to improve the feature extraction capability of wheat to be tested in complex scenes. Second, the GHSDConv and CSPDWOK structures were designed to construct the multi-scale sequence feature fusion module MFEP, which aims to improve the model’s detection capability for targets of different scales. In addition, a loss function named CML_loss has been created specifically for dense small-object detection in scenes with overlapping and occluded objects. It aims to mitigate the missing detection box problem caused by overlapping and occlusion. Finally, channel pruning based on LAMP scoring was performed to reduce the model parameters and runtime memory. The accuracy ${\mathrm{AP}}_{50}$ of the improved model reached 90.5, and the ${\mathrm{AP}}_{50-95}$, ${\mathrm{AP}}_{75}$, ${\mathrm{AP}}_S$, ${\mathrm{AP}}_M$, and ${\mathrm{AP}}_L$ were improved by 1.5%, 2.1%, 3.6%, 1.4%, and 1.9% over the initial values. Comparing the more advantageous lightweight CNN networks in recent years, the ${\mathrm{AP}}_{50}$ results of CML-RTDETR were 0.5 and 0.9 percentage points higher than YOLOv8n, and YOLO10n, respectively. Compared with the classical Retinanet and CenterNet, the ${\mathrm{AP}}_{50}$ was 6.0 percentage points and 2.8 percentage points higher, respectively. Since models such as the CNN lack shallow feature maps to recognize localized small-target wheat ears, and CML-RTDETR obtains multi-scale fusion features through GHSDConv, the detection accuracy ${\mathrm{AP}}_S$ values of most CNN models were much lower than that of CML-DETR on small targets. Compared with the Transfomer-based RT-DETR series, the number of parameter of the model proposed in this paper is only a half of theirs, which has a large advantage in terms of parameters and is equal to them in terms of ${\mathrm{AP}}_{50}$ metrics.

However, the actual field environment is much more complex: The diversity in morphology and texture brought about by variety differences, as well as the significant fluctuations in resolution caused by differences in collection equipment and shooting distance, all significantly reduce the robustness and generalization ability of the model. In the future, we will do the following: carry out systematic evaluations on an expanded dataset covering multiple varieties, multiple resolutions, and multiple climate zones to quantify the specific impacts of these factors on the detection performance and design and implement an adaptive illumination normalization module to suppress the illumination disturbances caused by strong light, shadows, and color bias, further improving the stability and accuracy of the model in complex lighting conditions.