GSD-YOLO: A Lightweight Decoupled Wheat Scab Spore Detection Network Based on Yolov7-Tiny

Abstract

Aimed at the problem of the difference between intra-class and inter-class pathogenic spores of Wheat Scab image being small and difficult to distinguish, in this paper, we propose a lightweight decoupled Wheat Scab spore detection network based on Yolov7-tiny (GSD-YOLO). Specifically, considering the limitations of the storage space and power consumption of actual field detection equipment, the original detection head is optimized as a decoupled head, and the GSConv lightweight module is embedded to reduce the parameters of the model and the number of calculations required. In addition, we utilize an improved Spore–Copy data augmentation strategy to improve the detection performance and generalization ability of the algorithm to fit the large numbers, morphology, and variety of wheat disease spores in the actual field and to improve the efficiency of constructing a large data set of diverse spores. The experimental results show that the mAP of the proposed algorithm reaches 98.0%, which is 3.9 percentage points higher than that of the original model. At the same time, the detection speed of the algorithm is 114 f/s, and the memory is 13.1 MB, which meets the application requirements of hardware deployment and real-time detection. It can provide some technical support to the prevention and grading of Wheat Scab in actual farmland.

1. Introduction

Wheat is one of the three major food crops in the world, and it is also the main crop, commodity grain, and reserve grain variety in China. Therefore, guaranteeing the production of wheat is of great significance to China’s food security. Disease is the most important factor affecting wheat production, and Wheat Scab, one of the major and frequently occurring diseases in the main wheat-producing areas, is also called wheat cancer [1,2].

In general epidemic years, Wheat Scab can cause a 10–20% yield loss of wheat; in a pandemic year, it can lead to no harvest. In addition, Wheat Scab breaks out rapidly, it is extremely difficult to cure the disease after it occurs, and pesticide pollution easily occurs during the control process. Therefore, the detection and prevention of Wheat Scab is of great significance to current wheat production.

Wheat Scab is a disease caused by a variety of Fusarium species. The diversity of Fusarium species on wheat ears in twenty-six regions of eight provinces in China was investigated, and it was found that Fusarium graminearum (Fg) and Fusarium asiaticum (Fa) were the two main pathogens causing Wheat Scab in wheat-producing areas in China [3]. The two Fusarium spore pathogens come in the form of hyphae, conidia, or ascus, which are saprophytic in wheat over summer and winter, and then spread and infect wheat ears through wind and rain, resulting in an outbreak of Wheat Scab [4].

The conventional monitoring methods for Wheat Scab can be divided into three kinds: manual detection, meteorological prediction, and pathogen source detection. The manual detection method refers to the visual investigation of wheat in the field by experienced professionals during the outbreak of disease to evaluate the severity of the disease. Although this method is simple and easy to use, it has the disadvantages of strong subjectivity and easy misjudgment. The meteorological prediction method uses a corresponding model to diagnose and grade the Wheat Scab disease based on meteorological factors. Although this method has a high accuracy, the key shortcoming of the prediction model is that it is difficult to determine the results and they need to be verified over a long period. The method of detecting the pathogen source refers to the manual collection of spore samples in the field and the observation of the type and quantity of scab spores under a microscope to determine the degree of disease. Although this method is accurate, more efficient than other disease detection methods, and can quickly classify and diagnose wheat scab, the Wheat Scab spore targets are too small and the number of them is too large, and intra-class and inter-class differences in pathogenic spores are difficult to distinguish, so the process is complicated and time-consuming.

Aimed at the problem that it is difficult to distinguish between the same and different species of two pathogenic spores of wheat fusarium spores, we propose a lightweight decoupled Wheat Scab spore detection network based on Yolov7-tiny (GSD-YOLO). The network adopts an improved Spore-Copy data augmentation strategy, and uses decoupled head and GSConv modules to achieve a balance between algorithm accuracy and speed. In addition, the algorithm also takes into account the limitations of the storage space and power consumption of the actual field detection equipment, which can efficiently and quickly detect the microscopic image of pathogenic spores of Wheat Scab, and can also provide some technical support to the prevention of scab disease.

2. Related Work

With the development of agricultural machine vision, domestic and foreign scholars have made some progress in target detection in the field of pathogen spore imaging. In traditional machine learning, Qi et al. used distance transform and an improved watershed algorithm to detect adhesion spores based on morphological features such as ovality and complexity, and the accuracy of their method reached 98.5%, but the feature extraction process was more complicated [5]. Wang et al. proposed an additive cross-kernel support vector machine (IKSVM) method for rice blast spore detection by extracting the HOG features of spore images, which achieved a detection rate of 98.2%, but the detection took too long [6]. In deep learning, Dai et al. assessed RGB images of Wheat Scab based on image processing and the Deeplabv3+ model and obtained an average accuracy of 0.969 [7]. Tahir et al. proposed new detection architectures based on a CNN and obtained a 94.8% accuracy on 40,800 images of five fungal spores [8]. Zhang et al. proposed a multi-layer fusion structure network based on FSNet and achieved an average detection accuracy of 91.6% on the fungal data set through measures such as anchor optimization and regional sampling strategy improvement [9]. In summary, the machine vision method must extract features from the image in advance to detect, but the process is not efficient and is time-consuming and laborious. In deep learning, the detection of Wheat Scab is mostly based on macroscopic disease detection, and there are few studies on the detection of internal microscopic spores. In comparison, our method for the prevention of Wheat Scab disease is faster and more efficient.

3. Materials and Methods

3.1. Spore Microscopic Image Acquisition

The pictures of Wheat Scab spores used in the experiment were taken by the Anhui Academy of Agricultural Sciences. The strains used in the experiment were the two main pathogenic spores of Wheat Scab in China, Fusarium asiaticum (strain number fa) and Fusarium graminearum (strain number fg). At the same time, to fit the actual farmland environment, other hybrid spores were randomly added. The two pathogenic spores are shown in Figure 1. It can be seen from the picture that the two pathogenic spores Fa and Fg are generally sickle-like, long, and narrow in size, with multiple transverse cell membranes in the body, and random mottling. There is no obvious distinction between the other two types of spores, and the morphology and size of single species spores are also different, showing polymorphism.

3.2. Spore-Copy Data Augmentation

In the data collection process for the two main pathogenic spores of Wheat Scab, the spores obtained by laboratory culture and actual farmland capture will have different impurities, light, noise, dust, and spore integrities. Therefore, an improved data enhancement method (Spore–Copy) based on Copy–Paste is proposed, which can reduce the error caused by the actual situation of laboratory fitting.

Copy–Paste was originally used as an enhancement technique to diversify the sample space in the field of segmentation. It mainly splits the target from the source image to create a target image to obtain new data samples. In this paper, an improved Copy–Paste data augmentation strategy (Spore–Copy) is proposed for the characteristics of small and polymorphic pathogenic spores of Wheat Scab. Based on the microscopic images of pathogenic spores in the laboratory, a large amount of training data was automatically generated to fit the actual field conditions. In addition, Spore–Copy is also an effective method to expand the sample size, which can enhance the generalization ability of the model [4].

The specific steps of Spore–Copy are as follows: (1) Prepare the data set and labels to be segmented; (2) cut the ROI image of the target spore according to the label, and record the spore species category and save the image; (3) randomly disrupt the ROI image and set the spore replication threshold of the data set to be enhanced. (4) Gaussian noise injection (simulating natural conditions), brightness change (simulating field illumination), image scaling (simulating spore polymorphism), random interception (simulating field spore breakage), and other operations were performed on all cut ROI spore images to fit the actual field conditions. (5) Coordinate transformation was performed, and the target ROI image was randomly embedded into the spore image to be enhanced by Poisson fusion.

The calculation formulae for Gaussian blur and the scaling of ROI spore images are shown in (1) (2); the calculation formula for the Poisson fusion of ROI spore images is shown in (3) [10].

The initial data set of the two main pathogenic spores of Wheat Scab included 7000 images, and the image size was 640 ∗ 640. Then, the data set was expanded to 20,000 images by using the above Spore–Copy enhancement algorithm (7000 images consisted of fa single-species spores and fg single-species spores, while the rest consisted of mixed fa and fg spore images obtained by Spore–Copy data enhancement). In addition, the data set was divided into training, test, and validation sets at a ratio of 8:1:1.

The data set of Wheat Scab spores treated by Spore–Copy is shown in Figure 2. It can be seen from the diagram that the two pathogenic spores fused by the Spore–Copy operation are not much different from the pathogenic spores in the original data set, and the newly embedded spores are more diverse and more suitable for real natural conditions than the original spores.

$$G(x,y)={\displaystyle \frac{1}{2\pi {\sigma }^2}}{e}^{-(x^2+y^2)/2{\sigma }^2}$$

(1)

In the formula, G(x, y) is the pixel value of the ROI image, and σ is the standard deviation of normal distribution.

$$\left[\begin{array}{ccc}{x}_0& y_0& 1\end{array}\right]=\left[\begin{array}{ccc}x& y& 1\end{array}\right] \left[\begin{array}{ccc}sx& 0& 0\\ 0& sy& 0\\ 0& 0& 1\end{array}\right]$$

(2)

In the formula, (x, y) is the coordinate before image pixel scaling, (sx, sy) is the coordinate after image pixel scaling, sx and sy are scaling factors.

$$\underset{f}{min}{\iint }_{\Omega }|\mathsf{\nabla }f-\mathsf{\nabla }V{|}^2 with f{|}_{\mathsf{\partial }\Omega }=f^{\mathrm{*}}{|}_{\mathsf{\partial }\Omega }$$

(3)

In the formula, V is the image to be fused, f^* is the target image, and f is the fused image; Ω represents the region to be fused, and ∂Ω represents the edge part of the fusion region.

4. GSD-YOLO

4.1. GSD-YOLO General Framework

The GSD-YOLO network structure is mainly composed of three modules: Backbone, Neck, and Head. The Backbone module follows the feature extraction layer of the Baseline network Yolov7-tiny, which is mainly composed of Conv, E-ELAN, and MPConv. E-ELAN is an efficient network design method, which solves the problem of a low utilization of model parameters [11,12]. The Neck module mainly uses the “FPN+PAN” structure to realize the mutual transmission of high-dimensional and low-dimensional information. In addition, the lightweight GSConv module is embedded into the maximum channel information dimension and the minimum spatial information dimension to reduce the number of model parameters [10]. At the same time, a balance between model accuracy and speed is realized. It is worth mentioning that the original paper that established GSConv found through experiments that, if GSConv is used throughout the network, the network depth will be greatly deepened and the model inference speed will be reduced. It is a better choice to use GSConv only in the Neck where the channel information dimension is the largest and the spatial information dimension is the smallest. Finally, in the Head module, a decoupled head is added to improve the detection performance and accelerate the network convergence speed [13]. The overall structure of the GSD-YOLO network is shown in Figure 3.

4.2. Decoupled Head

The Yolov7-tiny head is a typically coupled head, and its detection module sends the feature map output by the convolution layer directly into several fully connected layers or convolution layers to generate the desired target location and category. However, due to the different regions focused on during localization and classification, if the two tasks use the same feature map, it will lead to a spatial misalignment problem [14,15]. At the same time, this coupled detection concept requires a large number of parameters and computing resources and is prone to overfitting.

Therefore, we adopted a decoupled head structure to separate the information needed for localization and classification tasks, and finally fused them, which can improve the detection accuracy and accelerate the network convergence speed.

The specific structure of the decoupled head is shown in Figure 4. It can be seen from the diagram that after the decoupled head separates the feature map with different branches, the classification task can better compare the extracted features, while the localization task can pay more attention to the bounding box parameters.

4.3. GSConv Lightweight Module

In the field of detecting Wheat Scab spores in actual agricultural fields, the deployment of equipment results in storage space and power consumption limitations, so it is necessary to carry out lightweight processing on the network model.

Although the traditional ordinary lightweight model can improve the speed of the detector, it lacks detection accuracy. Therefore, this paper uses a new lightweight module GSConv to optimize the model, so as to achieve the balance between model accuracy and speed [16,17].

The specific structure of the GSConv module is shown in Figure 5. It can be seen from the diagram that GSConv combines standard convolution, deep separable convolution, and shuffle mixed convolution to construct a new convolution layer to meet the requirements of both accuracy and speed.

In addition, embedding GSConv will deepen the depth of the network and reduce the speed of model reasoning, so only the lightweight module GSConv is used at a key point in the neck layer of the network model.

5. Experiments

5.1. Experimental Environment and Parameter Settings

The experiment was carried out under the PyTorch framework. The equipment used was NVIDIA GeForce RTX 3070 (NVIDIA Corporation, 2788 San Tomas Expressway, Santa Clara, CA 95051, USA). According to the characteristics of pathogenic spores of wheat head blight, the following training parameters were set: the training batch included 16 images; the training iteration period was 300 s; the target confidence threshold was 0.5; the initial learning rate was 0.01; and the weight attenuation coefficient was set to 0.0005.

5.2. Comparisons with Existing Methods and Evaluation Indicators

In order to verify the performance of the GSD-YOLO (CNN core, one-stage) network proposed in this paper, the comparison network was selected from several different options, namely the baseline network Yolov7-tiny, the traditional one-stage detection algorithm SSD [18], the classic two-stage detection network Faster-RCNN [19], and the Detr network (Transformer core) [20].

In addition, mAP50, Precision, Recall, and other parameters were selected as model evaluation indicators. Some of the test evaluation index calculation formulae are as follows:

$$\mathrm{P}\mathrm{recision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(4)

$$\mathrm{R}\mathrm{ecall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(5)

$$\mathrm{mAP}50={\displaystyle \frac{\sum _{\mathrm{i}=1}^{ \mathrm{n}}{\left({\int }_0^{ 1}{\mathrm{Precision}}_{\mathrm{r}}\mathrm{dr}\right)}_{\mathrm{i}}}{\mathrm{n}}}$$

(6)

In the formula, TP represents the number of objects detected in the data set. FP represents the error in the number of objects detected by the detection model. FN represents the number of objects missed by the detection model.

5.3. Experimental Results and Analysis

A comparison of the detection results between the GSD-YOLO model and the above-mentioned network model is shown in

Table 1. Network detection table. All the models are tested at 640 × 640 resolution, with batch = 16 on NVIDIA GeForce RTX 3070.

Models	mAP0.5 (%)	FPS (f/s)	Weights (MB)	P (%)	R (%)
Yolov7-tiny	0.941	153	6.0	0.862	0.887
SSD	0.815	51	23.9	0.79	0.77
FasterRCNN	0.887	40	41.4	0.795	0.797
Detr	0.878	41	41.3	0.701	0.798
Ours	0.980	114	13.1	0.938	0.954

, and the detection effect diagram is shown in Figure 6. It can be seen from Table 1 that GSD-YOLO has a 3.9% improvement in mAP compared with the original baseline network Yolov7-tiny, and the accuracy and regression rate have been greatly improved. Although the network computing inference time and network parameters are affected to some extent after embedding the decoupled head and the lightweight module GSConv, compared with the other network models, GSD-YOLO still has an advantage, which meets the application requirements of hardware deployment and real-time detection.

From Figure 6, compared with the baseline network Yolov7-tiny, GSD-YOLO has a more detailed feature extraction of the two pathogenic spores, which solves the problem of the two spores being difficult to distinguish. At the same time, GSD-YOLO can also correct the problem of missed detection and false detection in the baseline network; that is, the network has stronger localization and classification performances for the two pathogenic spores.

In addition, in order to explore the role of the components of the GSD-YOLO network, we performed ablation experiments. The experimental results are shown in

Table 2. Ablation study on proposed method. All the models are tested at 640 × 640 resolution.

Methods	mAP0.5 (%)	FPS (f/s)	Weights (MB)	FLOPs (G)
(a) Yolov7-tiny	0.941	153	6.0	13.1
(b) +decoupled head	0.982	102	14.1	54.2
(c) +GSConv	0.937	141	5.7	12.8
(d) +decoupled head +GSConv	0.980	114	13.1	53.1

. It can be seen from the results in the table that the role of the decoupled head is to improve the detection accuracy and the number of floating-point operations, but as the network becomes deeper, its network parameters and detection speed are affected. The function of the GSConv module is to reduce the number of network parameters while maintaining detection accuracy and speed [21,22].

5.4. Model Generalization Experiment

In the setting of detecting spores of Wheat Scab in the field, due to the influence of various environmental factors such as wind, rain, and the atmosphere, microscopic images of spores obtained usually have complex backgrounds and many impurities. At the same time, if other wheat diseases break out, other disease spores will be mixed into the image, and these factors will have a certain impact on the detection. Therefore, the generalization performance of the GSD-YOLO proposed in this paper was tested.

Firstly, a batch of new spore data sets was obtained, and pollen, dust, and other disease bacteria were added. Then, Spore–Copy was used to fit the actual data in the data set from the field. Finally, two experiments were set up. In experiment 1, the spore data set was divided into three types: single fa pathogenic spores, single fg pathogenic spores, and mixed fa + fg pathogenic spores. In experiment 2, the spore data set was divided into four gradients according to the number of spores for the density of pathogenic spores (the number of spores in the image is enhanced by the Spore–Copy pre-control). The experimental results are shown in

Table 3. Table of results for test one.

GSD-YOLO	mAP0.5/AP0.5 (%)	Speed (ms)
Fa	0.972	3.0
Fg	0.983	3.0
Fa + fg	0.962	3.2

and

Table 4. Table of results for test two.

GSD-YOLO	mAP0.5 (%)	Speed (ms)
Density degree 1 (1–5)	0.970	3.1
Density degree 2 (5–10)	0.951	3.0
Density degree 3 (10–20)	0.932	3.2
Density degree 4 (over 20)	0.931	4.2

(the number of experimental images in each group is 2000).

As can be seen from the results in Table 3, GSD-YOLO’s detection accuracy decreases in the face of new data that are more fitting with reality. In addition, comparing the two pathogenic spores of Fg and Fa, the characteristics of Fg are clearer than those of Fg, and it can be more easily identified by the network in the polymorphic fungal environment.

From the results in Table 4, it can be seen that the density of F.graminearum spores has a certain influence on the detection accuracy of GSD-YOLO, and the greater the density, the greater the influence. This is because, when the source density increases, the two main pathogenic spores are altered due to polymorphism, and the growth cycle leads to false detections in the model network; at the same time, the increase in the density of the bacterial source will also lead to adhesion and coverage of the target spores, and these factors will make the neural network miss detections. To sum up, the detection accuracy of the network model will decrease to a certain extent.

6. Conclusions

Aimed at the problem that the distinguishing intra-class and inter-class characteristics of the two main pathogenic spores of Wheat Scab are not clear, we proposed a lightweight decoupled Wheat Scab detection network based on Yolov7-tiny (GSD-YOLO).

The network adopts an improved Spore–Copy data augmentation strategy to fit the actual data and improve the detection performance and generalization ability of the algorithm. At the same time, based on the Yolov7-tiny network, a decoupled head and GSConv lightweight module were added to improve the detection accuracy of the algorithm and reduce the number of parameters in the network model. The experimental results show that the detection accuracy of the algorithm in this paper reaches 98.0%, which is 3.9 percentage points higher than that of the original Yolov7-tiny model. In addition, the detection speed of the algorithm is 114 f/s and the memory is 13.1 MB, which meets the actual needs of hardware deployment and real-time detection. In addition, according to the actual farmland situation, the generalization test of the algorithm is carried out to deduce the feasibility of the algorithm in actual farmland disease detection.

In subsequent research, to make the algorithm more realistic, the laboratory simulation strategy will be optimized to train the network strategy with actual field spore data. Furthermore, in order to make the detection performance of the algorithm more powerful, other common wheat disease spores such as stripe rust and powdery mildew spores will be considered and added to the scab spores that can be detected in the future. It is expected that this will allow us to construct a rapid monitoring model for multiple wheat diseases and provide technical support for monitoring wheat diseases in farmland.