Fast Self-Attention Deep Detection Network Based on Weakly Differentiated Plant Nematodess

Abstract

High-precision, high-speed detection and classification of weakly differentiated targets has always been a difficult problem in the field of image vision. In this paper, the detection of phytopathogenic Bursaphelenchus xylophilus with small size and very weak inter-species differences is taken as an example. Our work is aimed at the current problem of weakly differentiated target detection: We propose a lightweight self attention network. Experiments show that the key feature recognition areas of plant nematodes found by our Self Attention network are in good agreement with the experience and knowledge of customs experts, and the feature areas found by this method can obtain higher detection accuracy than expert knowledge; In order to optimize the computing power brought by the whole image input, we use low resolution images to quickly obtain the location coordinates of key features, and then obtain the information of high resolution feature regions based on the coordinates; The adaptive weighted multi feature joint detection method based on heat map brightness is adopted to further improve the detection accuracy; We have constructed a more complete high-resolution training data set, involving 24 species of Equisetum and other common hybrids, with a total data volume of more than 10,000. The algorithm proposed in this paper replaces the tedious extensive manual labelling in the training process, improves the average training time of the model by more than 50%, reduces the testing time of a single sample by about 27%, optimizes the model storage size by 65%, improves the detection accuracy of the ImageNet pre-trained model by 12.6%, and improves the detection accuracy of the no-ImageNet pre-trained model by more than 48%.

1. Introduction

Bursaphelenchus xylophilus (B. xylophilus) is one of the most important harmful organisms to be detected in the entry–exit phytosanitary work. The annual economic losses caused by plant nematodes are more than 80 billion worldwide [1]. B. xylophilus is small, widely distributed, and harmful. The difference between B. xylophilus and other plant nematodes is very small, which is very difficult to distinguish [2]. Currently, the manual detection is still the main method [3]. As shown in Figure 1, the left is B. xylophilus which is harmful to vegetation, the right is Bursaphelenchus mucronatus (B. mucronatus) which is harmless to the vegetation. It can be found that the overall morphology and characteristics of these two nematodes are very similar and difficult to distinguish.

Although in general, plant nematodes have a very high similarity between different species, there are still distinguishable traits present in some fine-grained characteristic parts. According to expert experience, as shown in Figure 2, different types of plant nematodes still have weak differences in their anus position, length of their tail spikes, and other characteristics [4].

From Figure 2, it can be found that the key parts of different species of plant nematodes have some variability, but the degree of differentiation is very small [5]. The accuracy does not meet the requirements if traditional morphological identification methods are used [6]. In contrast, manual identification or molecular biology methods such as PCR, SVM and Naive Bayes classifiers have the problems of low identification efficiency or huge cost [7,8].

Huang J [9,10] and Doshi, R.A [11,12] Using the influence of nematodes on plants and the changes of plant spectra, remote sensing information is used to identify nematodes. However, this method is indirect and cannot be applied to our scene. The study referenced in [13] used a custom computer vision algorithm (CCVA) and a convolutional neural network (CNN) to identify nematode species, it is proven that both methods can accurately identify two types of nematodes, G. pallida and G. rostochiensis; for Globodera mexicana, the accuracy is low, but it still proves the feasibility of computer-based identification. The author of study [14] used his own cultivated nematode dataset to divide the nematode dataset into juveniles dataset and adult dataset. The neural network method is used to verify its effect on the dataset. The author explores the effectiveness of the volume and neural network on their dataset, as well as the best training process and super parameters suitable for their dataset.

In recent years, with the rapid development of deep learning technology, convolutional neural networks have significantly improved their effectiveness compared with traditional image detection methods [15]. By the method based on the attention mechanism, it is easier for the neural network to focus on the fine-grained detail information, improve the accuracy of image recognition classification under weak differences [6].

For weakly differentiated image classification often requires two steps: step 1. Manually label a large amount of data combined with expert empirical knowledge and train a key attention feature model based on Faster R-CNN [16], YOLO [17] and other general-purpose neural networks; step 2. Based on the fine-grained feature information obtained from the key attention feature model, then further training and testing of classification are performed to obtain the final result [18]. There are several problems in this operation: 1. The industry expert experience knowledge is difficult to obtain, and there is also a certain error in the expert experience, which will have an impact on the detection accuracy [19]; 2. The attention feature model trained by manual annotation of the original high-resolution image and based on networks such as Faster R-CNN will lead to a substantial increase in the model storage size, testing time, training time, which seriously affects the training and testing efficiency [20].

To address the above problems, this paper takes the detection of phytopathogenic B. xylophilus with small size and very weak inter-species differences as an example [5]. Our work has been carried out to focus on the weak inter-species differences and the problems in the process of achieving high-precision and high-speed targets:

a. An open-source plant nematodes high-resolution microscopic image dataset taken by a Zeiss professional microscope was constructed. The dataset image uniform image resolution of 1388 × 1040, involving 24 species of conventional plant nematodes, with a total of 11,237 images. To the best of our knowledge, this is the largest and most diverse plant nematodes dataset that can be found.

b. Construction of a Self-Attention feature network based on low-resolution images, which is capable of quickly and adaptively finding the key feature locations of plant nematodes to be detected in low-resolution images without relying on expert empirical knowledge. Experimentally verifying that the key feature identification areas of B. xylophilus found by this network match extremely well with the empirical knowledge provided by the customs experts.

c. The high-resolution feature information obtained by the Self-Attention feature network adopted a multi-feature joint recognition method based on adaptive weights of the heatmap.

The final detection accuracy is improved from 90% to 99% compared with the expert empirical knowledge input method, the detect model storage size is optimized by 65%, the testing time is reduced by 27%. Compared with other mainstream convolutional neural networks, the algorithm model proposed in this paper not only eliminates the complicated expert prior knowledge input, but also achieves the optimal model size, testing speed, accuracy simultaneously.

2. Materials and Methods

2.1. Dataset

2.1.1. Dataset

In this paper, we constructed the most abundant and diverse plant nematodes dataset that could be found. The dataset involves 24 species of nematodes that are easily confused with B. xylophilus. The total number of dataset images reaches 11,237. All images were taken uniformly with a Zeiss professional microscope Axio Imager Z1 with an objective magnification of 40× and a uniform image resolution of 1388 × 1040 [21]. As shown in Figure 3, it is a schematic diagram of the collected common different kinds of plant nematodes.

2.1.2. Traditional Detection Methods

The process of the traditional detection method is shown in Figure 4 below, which has the following problems: 1. There is too much non-critical redundant information when the entire image is input, which affects the extraction and judgment of key features by the neural network, and then affects the detection accuracy [22]; 2. The computation is time-consuming.

To address the above problems, researchers have made improvements. They combine expert empirical knowledge to construct a neural network based on feature areas attention mechanism through extensive labelling as well as training [23], so as to extract key feature areas to improve the detection accuracy of the neural network. The following Figure 5 shows the combined neural network architecture of the joint Faster R-CNN feature extraction network and the VGG19 [24] classification network.

The above method can eliminate redundant information in non-critical areas of the image to a large extent, improve the detection accuracy. However, there are still the following problems: 1. Based on expert empirical knowledge, a large amount of manual annotation is required in the early stage, and the input of the feature network is still high-resolution entire image, which is very time-consuming in labelling and computation; 2. It is not easy to obtain expert experience, and there is a possibility of misjudgement of expert experience, which leads to inaccurate detection accuracy in part.

Therefore, in order to find a better method that can simultaneously solve the problems of manual labelling, long calculation time, large model complexity, and lack of expert knowledge in the network, the following method is proposed in this paper.

2.2. Methods

In this paper, a deep neural network based on the fast Self-Attention inference is proposed, as shown in Figure 6. The overall network mainly contains three improved parts compared with the traditional method: Firstly, we construct a search network based on the Self-Attention features to replace the tedious labelling and training based on expert empirical knowledge, as shown in Part A below; secondly, the computational overhead of finding key features of the whole image is reduced by fast down-sampling of the high-resolution input, as shown in Figure 6 Part B below. Finally, through the key feature areas coordinates obtained by Part B, we obtain the high-resolution information of the feature areas. Based on the multi-feature areas joint input with adaptive weights, the goal of high-precision and high-speed classification is achieved, as shown in Figure 6 Part C below.

2.2.1. Part A: Self-Attention Feature Network

For any given high-resolution plant nematodes image, to better help the neural network to classify, we need to assist it in eliminating the influence of non-critical redundant information as much as possible.

Generally, we train additional attentional feature recognition networks based on general-purpose neural networks such as Faster R-CNN, with the help of expert empirical knowledge and through extensive time-consuming labelling. Figure 7 (green box detect correctly/red box detect incorrectly) shows the result of the attention network trained based on the labelled data to identify key features. It can be found that the detection network has a certain recognition error. As the network model complexity increases, its false detection rate has decreased, but it also brings the disadvantages of a large model, complex networks, and increased computational time-consuming.

In order to better save labelling time and reduce the cost of storage and computation time consuming by using networks such as Faster R-CNN, a Self-Attention feature network is proposed in this paper. The following figure shows the process of the neural network. As shown in Figure 8 above, the training based on the Self-Attention neural network is divided into two main parts. Step 1: Through the joint training of the Self-Attention Net and Classification Net, the high-precision feature heatmap of the Self-Attention Net is obtained. At this time, both parameters of the Self-Attention Net and Classification Net are trainable. Step 2: Fix the Self-attention Netparameters, find the key feature center coordinates (Px, Py) through the output heatmap to obtain the original high-resolution image information, then calculate the feature areas adaptive weights according to the average brightness of the heatmap areas to input to the classification network train again until loss converges.

The following figure shows the key feature regions obtained from the Self-Attention feature network with different iteration times compared with the features based on the expert knowledge network:

According to Figure 9, it can be found that as the iteration times increases, the key feature areas of the output of the Self-Attention feature network become closer and closer to the expert manual output. When the loss is fully converged, the attention areas obtained according to the method in this paper obtains higher classification accuracy than the expert knowledge. In a sense, the method in this paper finds the feature areas that is better than the expert experience, and there is also a significant improvement in the network complexity and training time consumption. The results are shown as follows

Table 1. Detection comparison network and expert knowledge feature network.

Methods	Annotation Time	Model Parameters	Training Time	Detect Time	Accuracy
Expert Network	2 Day	44 M	7.5 h	1.8 s	0.93
Ours	0	11 M	3 h	1.1 s	0.995

.

2.2.2. Small-Scale Fast Focus Method

Based on the Self-Attention feature network in Section 3.1, we can find the attention feature areas of interest, but this method uses high-resolution images as input, which has serious redundancy of image information and leads to high testing time consumption. In this paper, we use multiple down-sampled low-resolution images to find key feature areas, and maps the coordinates of the found key feature areas to high-resolution images to obtain high-resolution key feature areas, which further saves and optimizes the network computational power, as shown in the picture below.

As shown in Figure 10, for the input high-resolution original image, the scaling is first performed as follows:

$$I^{’}=k\cdot I$$

(1)

where $I^{’}$ is the low-resolution image obtained after image scaling, $k$ is the scaling parameter. Then, the features are extracted from the low-resolution images as follows:

$$F_M(I^{’})=f(P*I^{’})$$

(2)

where the extracted features are denoted as $P*I^{’}$, $*$ denotes the set of operations of convolution, pooling and activation, $P$ denotes the overall parameters, $F_M$ denotes the generated attention map, $f(\cdot )$ denotes the fully connected layer that maps the convolution feature to the feature vectors that can be matched with the plant nematodes category. The coordinates of the centers of key feature areas are obtained by finding the key points in the attention map where the feature values are most obvious, thus guiding the deep learning network to learn the key feature part of plant nematodes adaptively.

$$\{p_x,p_y\}=M_{\max }(F_M)$$

(3)

where p_x and p_y represent the x and y axis coordinates of the key points with the most obvious feature values, which are obtained by calculating the feature heatmap Mmax(·). By the above process, we can get the key areas coordinates on the low-resolution image.

Afterwards, for the coordinates obtained in the low-resolution image, the scaling is reversed according to the scaling parameter:

$$\{P_x,P_y\}=\{\frac{1}{k}{p}_x,\frac{1}{k}{p}_y\}$$

(4)

where P_x and P_y denote the x and y axis coordinates of the center point of the key areas obtained by the low-resolution image on the high-resolution image. Expanding outwards with P_x and P_y as the center, we can get the rectangular box coordinates of the key areas on the original image.

$$A\mathrm{re}a=\left[\begin{array}{cc}(P_x-a,P_y+b)& (P_x+a,P_y+b)\\ (P_x-a,P_y-b)& (P_x+a,P_y-b)\end{array}\right]$$

(5)

where $A\mathrm{re}a$ represents the whole rectangular box, $a$ and $b$ are custom variable parameter. The experimental results with different area sizes and down-sampling times are shown in Figure 11A below. It is verified that the best recognition results are obtained when $a=50$ and $b=50$, that is when the feature area of 100 × 100 is intercepted on the original high-resolution image. At the same time, we gradually increase the down-sampling times. When down-sampling once, both accuracy and efficiency can be taken into consideration. Then, the accuracy will decrease significantly as the down-sampling times increases. The following picture shows the comparison results of detection accuracy under different down-sampling times and feature areas sizes.

2.2.3. Multi-Feature Joint Classification Network Based on Adaptive Weights of Heatmap Brightness

Based on Section 2.2.1 and Section 2.2.2, the original plant nematodes image is input, the image block {P₁, P₂,..., P_N} containing the key feature areas of the plant nematodes are obtained by the Self-Attention feature network and the small-scale fast focus module, where N is the number of blocks. For different areas blocks, the matrix weight α_n is related to the average brightness ratio of the corresponding heatmap areas. After multiplying the obtained feature areas block P_N with the corresponding weight α_n, they are fed into the classification convolutional neural network. The specific process is shown as follows Figure 12.

The loss function used for the neural network is the cross-entropy loss function [9], its expression is as follows:

$$Loss=\frac{1}{N}{\displaystyle \sum _i{L}_i}=\frac{1}{N}{\displaystyle \sum _i-{\displaystyle \sum _i^M{y}_{ic}\log (p_{ic})}}$$

(6)

where M is the number of categories, y_ic denotes the variable, which is 1 if the predicted category of the sample i is the same as the true category (equal to c), otherwise it is 0, P_ic is the predicted probability of the observed sample $i$ belonging to category c.

The following

Table 2. Experimental parameter table of different weights ratio and detection accuracy.

Areas Number	1		2		3		4
Ratio	-	-	1:1	0.54:0.46	1:1:1	0.42:0.36:0.22	1:1:1:1	0.37:0.32:0.20:0.11
Accuracy	0.9895	0.9895	0.9950	0.9953	0.9922	0.9934	0.9905	0.992

and Figure 13 shows the comparison result of the comparison of detection accuracy and weights ratio using heatmap adaptive and equal ratio weights. It can be found using the adaptive ratio weights based on the average brightness of the heatmap can further improve the detection accuracy compared to the equal ratio method for different number of joint feature input experiments.

3. Results

3.1. Experiment Environment Match

The experimental environment in this paper is an Intel Core i7-8700k 3.70 GHZ 12-core CPU, NVIDIA RTX2080 Ti GPU with 11 GB display memory, 32 GB RAM, configured with the TensorFlow open-source deep learning framework on Linux16.04 system [10].

In measuring the model performance, the accuracy rated was selected as the main evaluation metric in this paper [25].

$$Accuracy=\frac{TP+TN}{TP+FP+FN+TN}$$

(7)

Among them, TP (True Positive) is actually positive and predicted as positive; FP (False Positive) is actually negative but predicted as positive; TN (True Negative) is actually negative and predicted as negative; FN (False Negative) is actually a positive class but predicted to be a negative class. The performance of different models can be measured by comparing the accuracy.

The experiments test the algorithm proposed in this paper and also compare it with traditional neural network models such as Faster R-CNN and YOLO. The experiments not only compare the accuracy rate of the models, but also the training and testing time of the models.

For the training hyperparameters, we choose the adaptive iterative learning rate, the kinetic energy is set to 0.9, the weight decay is set to 0.0005.

3.2. Dataset

The plant nematodes dataset for training and testing in this paper includes 24 species of plant nematodes, with a total of 11,237 images, the image resolution is unified to 1388 × 1040, of which 9126 are used for training and 2111 are used for testing.

3.3. Performance Comparison

3.3.1. Results of Self-Attention Network

The Self-Attention network used in this paper further improves the detection accuracy while effectively saving the size of the network model, and the key feature locations output by the neural network match well with the expert experience. As shown in the Figure 14 below, the deviations of the coordinate positions calibrated according to the expert experience are compared with the coordinate positions of the key feature centers output by the Self-Attention network. It can be seen that the key feature positions judged by the Self-Attention network are in good agreement with the expert experience, and the average error rate of the calibrated x and y axis coordinates is only 0.552%.

3.3.2. Efficiency and Accuracy Improvement

The method in this paper avoids a lot of manual labelling and reduces the time cost required for data labelling. Meanwhile, the Self-Attention network in this paper is more lightweight, coupled with the fast search of low-resolution images and the multi-feature joint detection with adaptive weights, it further reduces the model size, training, and testing time, while ensuring the final accuracy of the algorithm.

From the experimental results in Figure 15 and

Table 3. Comprehensive comparison of training, detection, model size, and accuracy of specific parameters.

Method	Training Method	Model Parameters	Annotation Time	Training Time	Detect Time	Acctracy
TOLO	ImageNet	20.8 M	2 Day	5 h	1.3 s	0.84
	No-ImageNet					0.62
Faster RCNN	ImageNet	30 M	2 Day	6 h	1.5 s	0.88
	No-ImageNet					0.64
Expert Network	ImageNet	44 M	2 Day	7.5 h	1.8 s	0.93
	No-ImageNet					0.72
Ours	ImageNet	11 M	0	3 h	1.12 s	0.995
	No-ImageNet					0.98

above, it can be concluded that based on the method in this paper, we avoid the tedious processes of extensive manual labelling. We improve the average training time of the model by more than 50%, reduce the testing time of a single sample by about 27%, optimize the model size by 65%. The detection accuracy of the ImageNet pre-trained model is improved by 12.6%, and the detection accuracy of the no-ImageNet pre-trained model is improved by more than 48%.

4. Conclusions

In this paper, the detection of phytopathogenic Bursaphelenchus xylophilus with small size and very weak inter-species differences is taken as an example. Our work has been carried out in response to the current weakly differentiated target detection problems. Its advantages can be summarized as follows:

• The detection accuracy of our proposed method reaches 99%.
• Compared with other methods, our method is the fastest.
• We constructed a more complete high-resolution training dataset involving 24 species of Bursaphelenchus xylophilus and other common hybrid species with a total amount of data exceeded 10,000.