Next Article in Journal
Enhancing Biogas Production of Corn Stover by Biogas Slurry Reflux Based on Microfiltration Membrane Filtration and Biochar Adsorption
Previous Article in Journal
3D Positioning Method for Pineapple Eyes Based on Multiangle Image Stereo-Matching
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Farmland Soil Block Identification and Distribution Statistics Based on Deep Learning

1
College of Engineering, Anhui Agricultural University, Hefei 230036, China
2
Anhui Province Engineering Laboratory of Intelligent Agricultural Machinery and Equipment, Hefei 230036, China
3
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei 230036, China
*
Author to whom correspondence should be addressed.
Agriculture 2022, 12(12), 2038; https://doi.org/10.3390/agriculture12122038
Submission received: 19 October 2022 / Revised: 25 November 2022 / Accepted: 25 November 2022 / Published: 29 November 2022
(This article belongs to the Section Agricultural Technology)

Abstract

:
Soil block distribution is one of the important indexes to evaluate the tillage performance of agricultural machinery. The traditional manual screening methods have the problems of low efficiency and damaging the original surface of the soil. This study proposes a statistical method of farmland soil block distribution based on deep learning. This method combines the adaptive learning rate and squeeze-and-excitation networks channel attention mechanism based on the original Mask-RCNN and uses the improved model to identify, segment and distribute statistics of the farmland soil blocks. Firstly, the influence of different learning rates and an improved Mask-RCNN algorithm model on training results were analyzed. Secondly, the effectiveness of the model in soil block identification and size measurement was analyzed. Finally, the identified soil blocks were classified accordingly, and the scale problem of soil block distribution after removing edge soil blocks was analyzed. The results show that with the decrease of learning rate, the loss value of model training decreases and the prediction accuracy of model is improved. The average precision value of the improved model increased by 25.29 %, and the recall value increased by 8.92%. The correlation coefficient of the maximum diameter measured by manual measurement and the maximum diameter measured by model algorithm was 0.99, which verifies the feasibility of the algorithm model. The prediction error of the model is the smallest when the camera height is 40 cm. Large-scale detection of soil block size in an experimental field in Hefei, Anhui, with an average confidence of over 97%. At the same time, the soil block is effectively classified according to the set classification standard. This study can provide an effective method for the accurate classification of soil block size and can provide a quantitative basis for the control of farmland cultivation intensity.

1. Introduction

Tillage can change the size of farmland soil blocks; appropriate soil block size distribution can provide a more favorable soil environment, thereby promoting higher crop yields. In most cases, in the agricultural production process, the proportion of energy consumption in the tillage process exceeds more than half of the total energy consumption in the whole planting process [1]. Appropriate soil block size distribution can not only improve the soil environment of farmland but also reduce excessive energy consumption during tillage.
Mean weight diameter (MWD) [2] is one of the most common indicators for assessing soil aggregation size distribution. The commonly used method is to mechanically screen soil samples, place them on a series of screens of different sizes for screening, and determine MWD by screen size and the weight of soil blocks left on screen after screen. This method is inefficient and time-consuming in practical applications [3], while in the screening process, the soil block surface may be damaged, thereby reducing the accuracy of the method.
With the development of digital cameras and high computing power computer technology, the boundary segmentation of objects on images has made great progress [4,5,6]. Image processing technology has also been applied in soil size and quantity statistics. However, due to the very low scale of color change between the soil block and soil matrix, it is difficult to distinguish the boundary of the soil block and determine the diameter of the soil block by conventional methods [1]. In recent years, camera measurements and laser scanners have been widely used to measure the digital elevation model (DEM) of soil surfaces. Many scholars’ research can automatically detect and estimate the boundary from DEM and extract the relevant characteristics of soil blocks [7,8]. In the case of DEM images, classical segmentation methods fail to identify the clods on the support surface, due to the intrinsic non-homogeneity of the clod values (altitudes). Thus, classical image processing tools should be adapted or new ones should be developed [9]. O. Taconet identifies blocks by closing contours of high gradient values on the DEM, which is mainly limited to the inability to identify blocks smaller than 7mm in diameter [10]. It is worth noting that the establishment of DEM usually takes a lot of time, especially the DEM obtained by laser scanner; at the same time, the method of establishing DEM cannot be applied to obtain the size distribution of soil blocks in real-time.
In addition to obtaining soil block information from DEM of soil surfaces, Bosilj, P quantitatively analyzed the particle size distribution of soil aggregates from image spectral information [11,12]. They developed an imaging technique for quantitative analysis of the size distribution of soil aggregates, which can handle a larger sample complexity than previous methods, while handling a smaller sample volume more easily. Ajdadi, F.R. developed a method to assess the quality of soil tillage by extracting texture features from soil images and image processing from an artificial neural network (ANN) classifier [1].
In recent years, with the application of deep learning technology, some complex classification and segmentation problems have been solved with high precision [13,14,15,16]. At present, the commonly used depth model is a convolutional neural network (CNN) [17]. Deep learning is widely used in classification and target detection in different fields, such as the identification of corn plants in smart agriculture [18], the identification of eggplant leaf blight and fruit rot [19], and the identification of tea categories [20]. Semantic segmentation requires image classification and boundary location [21,22]. Azizi, A. proposed a method based on deep learning technology to obtain soil blocks from the surface of cultivated soil and complete high-precision classification tasks [23]. Wang achieved fast detection and removal of soil and stone in mixed potatoes after harvest and proposes a method for detecting soil blocks and stones in potatoes based on an improved YOLO V4 model [24]. Alirezadeh, P. proposed a SoilNet CNN to classify nine images of different soil aggregate sizes from three different heights (the higher the height, the lower the image resolution) at 60, 80, and 100 cm [3]. The technical scheme of soil block identification and classification proposed above based on the deep learning method can achieve good results in specific application scenarios. However, due to the high similarity of the color change range between soil block and soil matrix during field tillage, the accurate identification of farmland soil block becomes a major challenge.
In summary, it has been difficult for the soil screening method to meet the efficiency requirements of large-scale farmland soil block size classification. The application of image processing technology realized the boundary estimation and soil block detection of farmland soil blocks in specific scenes. However, in the field cultivation process, it is difficult to accurately separate soil blocks through image processing due to the similar color of soil blocks and soil matrix. In recent years, with the maturity of deep learning technology, we consider whether it is possible to identify large-scale soil blocks in farmland through deep learning technology so as to quickly make distribution statistics and evaluate the tillage quality of agricultural machinery.
The purpose of this study is to propose an effective method for the identification, size determination, and distribution statistics of farmland soil blocks. This method is based on the Mask-RCNN instance segmentation algorithm. In the training process, the cosine annealing learning rate (CALR) is used to optimize the model, and the squeeze-and-excitation networks (SENet) channel attention mechanism is embedded in the backbone network to improve the speed and accuracy of soil block detection. Based on this recognition, through further analysis of the mask of the detected soil block, the relevant size is extracted, and the identified soil block is graded to achieve the purpose of accurate statistics of soil block distribution.

2. Materials and Methods

2.1. Materials

2.1.1. Image Acquisition

The whole process of image data acquisition of farmland soil blocks was carried out in the experimental field of Anhui Agricultural University. Soil blocks were obtained by rototilling experimental fields with a rototiller. The recording equipment was the Meizu17 pro mobile phone, and the camera parameters were Sony IMX686. In a shadowless environment with uniform illumination, a total of 247 images were taken for individual soil blocks (Figure 1a), cohesive soil blocks (Figure 1b, cohesive soil block refers to two or more soil blocks with edges connected or sheltered), and reference objects (Figure 1c, the reference objects used are cubes, cuboids, spheres). The image size was 4288 × 2848 pixels, and the image was adjusted to 512 × 512 pixels to accelerate the model training.

2.1.2. Training Environment

The research was carried out on the high-performance computing platform of the Anhui Intelligent Agricultural Machinery Engineering Equipment Laboratory. The experimental hardware platform was configured as Intel(R) Xeon(R) Silver 4210 CPU @ 2.20 GHz processor, graphics card NVIDIA Quadro RTX 4000, and 8GB of video memory. The software platform was the Windows 10 operating system, TensorFlow deep learning framework, and CUDA10.0 GPU acceleration library; the programming language was Python 3.6; and the software platform was PyCharm.

2.2. Methods

The purpose of this study is to use an improved Mask-RCNN instance segmentation algorithm to identify and predict the size of farmland soil blocks to classify them. The main steps include image acquisition, dataset production, improved models, and related experiments. The method flow is shown in Figure 2. Firstly, camera equipment is used to take pictures of the soil and resize them to speed up training, and then, LabelMe software is used to label the soil blocks. In order to improve the generalization ability of the data set, data enhancement processing was carried out, and training set, validation set and test set were divided. Thirdly, the adaptive learning rate (ALR) and SENet channel attention mechanism are added to the original Mask-RCNN algorithm to improve the segmentation accuracy of the model. Finally, the effectiveness of the model in soil block recognition is verified by relevant experiments.

2.2.1. Dataset Production

(1) Labeling of soil blocks and reference objects
A total of 247 images were obtained in this study, and each image was manually labeled using LabelMe labeling software. When labeling, the contours of the target are used as the real box, and the label file corresponding to the image was acquired one by one. The image was divided into four classes: soil blocks (soilblock) and reference objects (cube, cuboid, and sphere). For different instances of the same class in an image, numbers were added after the category to distinguish them, such as soilblock_1, soilblock_2….
(2) Data enhancement
In order to enrich the dataset and improve the generalization ability of the deep learning model, the SSD Augmentation data enhancement method is used to simultaneously adjust random saturation, mirror inversion, random clipping, and random rotation at the same time for the annotated image and annotated file, and the dataset samples are expanded to 1235.
(3) Labeling data processing
The labeled image was made into a dataset in COCO format, and the data samples were randomly divided into training set, verification set and test set according to the ratio of 16: 4: 5. In the process of dividing less than one, rounded up by method. The distribution of the number of images in the dataset was shown in Table 1.

2.2.2. Instance Segmentation Model Training

In this study, the Mask-RCNN algorithm was used for the detection of soil blocks. Mask-RCNN is an instance segmentation algorithm proposed by HE KM [25]. Based on the Faster-RCNN target detection algorithm, the algorithm replaces the ROI pooling layer with the ROI Align layer and adds a mask branch. The network output is the bounding box of the target, the category of the target, and the mask of the target region of interest, which can realize instance segmentation, while detecting the target. The overall framework of the algorithm is shown in Figure 3.
The bounding boxes and classes in the figure are completed by Faster-RCNN, and the mask of the target region of interest is completed by a fully convolutional network (FCN). Firstly, the target image containing soil blocks is input, and the backbone features are extracted by using a pre-trained residual network (Resnet101) and FPN to obtain the corresponding feature map. The feature map generates multiple soil block target candidate regions ROI through the RPN network. The feature map and ROI are input into the RoIAlign layer at the same time, and mapped into a unified scale feature vector. Finally, the bounding box of the target, the category of the target and the mask of the target area of interest are extracted through the fully connected layer (FC Layers) and FCN.
The total loss function of Mask-RCNN is:
L = L c l s + L b o x + L m a s k
where Lcls is the loss function of classification confidence, Lbox is the bounding box regression loss function, and Lmask is the average binary cross-entropy loss function. The model is trained iteratively to reduce the value of the loss function until an optimal solution is reached.
Aiming at the phenomenon of adhesion of soil blocks, in order to improve the recognition accuracy and efficiency of the model, the Mask-RCNN model is improved:
(1) ALR. The learning rate (LR) directly affects the convergence state of the model. Choosing the appropriate LR can improve the effect of model training. In the early stages of iterative optimization, a larger LR is used for gradient descent at a faster speed. In the middle and later stages of iterative optimization, the CALR is used to reduce the step size to facilitate the convergence of model training. The principle of CALR is as follows:
η t = η m i n i + 1 2 ( η m a x i η m i n i ) ( 1 + cos ( T c u r T i π ) )
where η t is the calculated learning rate; i is the index of the run; η m i n i and η m a x i are ranges for the learning rate; T c u r is how many epochs have been run currently; T i is the total number of epochs for training the model.
(2) Attention mechanism introduced into backbone network. To improve the accuracy and recognition speed of the algorithm, the SENet channel attention mechanism is introduced [26]. The squeezed operation in SENet is to perform global average pooling on the feature map and compress it into a vector of 1 × 1 × C2. The excitation operation is to use a fully connected neural network to perform a linear transformation on the compressed results to obtain the weight of each channel. The channels of each layer are multiplied by the corresponding weights so that the network pays more attention to useful features and enhances the feature extraction ability. After the identity_block module of the backbone network and before the construction of the feature pyramid, we string into SENet, which improves the detection effect of the model and enhances the extraction ability of soil block features. Figure 4 shows the structure of Resnet101 with the introduction of the SENet attention mechanism.

2.2.3. Camera Calibration

The soil block size extraction needs to obtain the image containing the soil block to be measured in advance. Due to the acquisition equipment itself, the farther the light away from the center of the lens, the greater the distortion, which leads to larger errors in the subsequent extraction of the size of the clods. Therefore, the camera needs to be calibrated before using the algorithm to predict the size of the soil block. At present, the commonly used camera calibration method is the calibration plate calibration method [27,28,29,30].
In this paper, Zhang Zhengyou ’s checkerboard method is used to calibrate the camera to obtain the internal parameters, external parameters, and distortion coefficient of the acquisition equipment [30]. According to the internal parameters and distortion coefficient, the image of the soil block to be measured is de-distorted. In this way, the deformation of the image is reduced and the accuracy of predicting the size of the soil is improved. The camera parameters obtained by Zhang Zhengyou ’s checkerboard calibration method are as follows:
Internal parameter matrix (Mtx):
M t x = [ 3564.99 0 2316.36 0 3561.69 1726.17 0 0 1 ]
Distortion coefficient (Dis):
D i s = [ 0.0019 0.21 0.0019 0.0003 0.48 ]

2.2.4. Extraction and Classification of Soil Block Size

The soil block size is obtained by the following methods: (1) The soil block to be tested is mixed with the reference object and photographed using an acquisition device; (2) Masks of soil blocks and reference objects are obtained by Mask-RCNN instance segmentation algorithm; (3) Using the cv2.findcontours function in OpenCV to obtain the contour of the mask map of the soil block and the reference object; (4) Using cv2.contourAre function to obtain the pixel area of the mask; (5) The cv2.minEnclosingCircle function is used to obtain the minimum circumcircle of the mask contour and then the pixel diameter of the circumcircle can be obtained. Through the above method, the pixel area and the maximum pixel diameter of the soil block and the reference object can be obtained, and the actual size of the soil block can be obtained through the proportional coefficient.
The actual maximum diameter of the soil block is obtained by multiplying the pixel diameter by the proportional coefficient KD. The proportional coefficient KD is obtained by the actual diameter and the pixel diameter of the reference. The relational formula is:
K D = D R e f _ a c t D R e f _ p i x
where D R e f _ a c t is the actual diameter of the reference object, and D R e f _ p i x is the pixel diameter of the reference object on the image.
Similarly, the actual area of the soil block is obtained by multiplying the pixel area by the corresponding scale coefficient KS, and the scale coefficient KS is obtained by the actual diameter of the reference object and the pixel diameter. The relationship is as follows:
K S = S R e f _ a c t S R e f _ p i x
where S R e f _ a c t is the actual area size of the reference object, and S R e f _ p i x is the pixel area size of the reference object on the image.
The specific process of soil block size extraction is shown in Figure 5.
In the test field, five groups of test samples were randomly sampled, and each group of soil samples was sieved with sieves of 25 mm, 50 mm, 75 mm, and 100 mm apertures. The mean weight diameter (MWD) of soil samples was calculated as follows:
D = i = 0 n x i ¯ × ω i
where D is the diameter of the soil block, in millimeters; x i ¯ is the average diameter of each size of soil block; ω i is the weight proportion of some soil blocks with corresponding sizes.
Through Formula 5, the MWD of different soil block sizes of the soil sample can be calculated and divided into five levels, as shown in Table 2.

2.2.5. Evaluation Metrics

Intersection over union (IOU) represents the degree of overlap between the prediction box generated by the model and the real box. The larger the IOU, the more accurate the model prediction. In this paper, when evaluating the accuracy of target instance segmentation, the threshold of IOU is selected as 0.5, that is, when IOU is greater than 0.5, the model prediction is correct, and vice versa. The precision (P) and recall (R) predicted by the model are calculated by the confusion matrix, as shown in Equations (6) and (7). Average precision (AP) is obtained by calculating the area enclosed by the P–R curve and the coordinate axis. The higher the AP value, the better the prediction effect of the algorithm.
P r e c i s i o n = T P T P + F P
R e c a l l = T P T P + F N
where T P denotes the number of true positives, F P represents the number of false positives, and F N shows the number of false negatives.
In addition, we also use the F1 score to evaluate the performance of the algorithm. F1 score is the harmonic mean of P and R, calculated as follows:
F 1 = 2 R e c a l l 1 + P r e c i s i o n 1
The F1 score is a combination of precision and recall. The higher the F1 score, the stronger the model.

3. Model Training Result Analysis

LR is an important hyperparameter in deep learning training. This paper sets up five groups of experiments to explore the impact of LR on the accuracy of model detection and the detection effect of the improved model. Four of them set different LR, 1 × 10−3, 1 × 10−4, 1 × 10−5 and ALR, respectively. The fifth group used ALR and the embedded SENet module. Other hyperparameters were set the same, the number of epochs was set to 120, and the image size was 512 × 512 pixels.
Different models have different optimal LR, so we need to find the most suitable LR before training the model. The changes in training loss values from the five groups of experiments are shown in Figure 6. It can be seen from Figure 6 that the loss values of the five groups of experiments tend to be stable, indicating that the model has reached the optimal effect. Different LR has a certain impact on the loss value of training. When the LR is 1 × 10−3, the model training has a bottleneck, the training loss value tends to be constant, and the LR needs to be reduced; when the learning rate is 1 × 10−4, the training loss value is 0.15, which is higher than the other three groups. LR is 1 × 10−5, using ALR alone, embedding SENet channel attention mechanism and ALR at the same time, these three groups of test training loss values are similar, all are approximately 0.07, indicating that the optimal LR of this model is approximately 1 × 10−5. Models embedded with the SENet channel attention mechanism and ALR are the fastest and have a smoother decline in training loss values during training. As can be seen from Figure 6, the ALR and SENet can well meet the needs of model training and reduce the time to select hyperparameters when training the model.
The evaluation of the training results of the five groups of experiments is shown in Figure 6. When the LR of the first group of experiments is 1 × 10−3, the bottleneck of model training occurs, and the index of the training results cannot be evaluated, which is marked as ‘×’ in the Table 3. When the LR is 1 × 10−4, the AP value is 62.03% and the R is 78.91%; The AP value of the remaining three groups increased by 21.49%, 22.97%, and 25.29%, respectively, and the R value increased by 6.73%, 5.52%, and 8.92%, respectively. Similarly, the F1 score increased by 0.16, 0.16, and 0.19, respectively. Combined with the loss value results in Figure 6, it shows that the improved Mask-RCNN instance segmentation algorithm has a certain improvement in the accuracy of soil block segmentation.

4. Experimental Results and Analysis

4.1. Extraction and Verification of Soil Block Particle Size

In order to verify the effectiveness of the proposed farmland soil block recognition and size prediction based on Mask-RCNN, this paper randomly selected 25 soil blocks of different sizes and used the trained network model to detect soil blocks. As can be seen in Figure 7, this model can be used to identify soil blocks very well. At the same time, the actual value of the artificially measured soil block particle size and the model particle size detection value were counted. Taking the real value of manual measurement as the abscissa and the model detection value as the ordinate, linear fitting was carried out, and the result is shown in Figure 8. The regression equation obtained by fitting is y = 0.98 x 1.11 , and the correlation coefficient is   R 2 = 0.99 . The results show that the extraction of soil block particle size based on Mask-RCNN has a high accuracy, and the measured value of soil block particle size is in good consistency with the manual measured value, which can meet the actual needs of soil block particle size detection.

4.2. The Influence of Different Heights on the Prediction of Soil Block Particle Size

The sampling height has a certain influence on the model prediction of soil particle size. Ten soil blocks were randomly selected and photographed at different heights (30 cm, 40 cm, 50 cm, and 60 cm), and their maximum diameters were counted. Input the image into the prediction model, get the diameter of the soil block at different heights, and count the relative error between it and the measured value. The results are shown in Figure 9.
According to the analysis in Figure 9, the height has a certain influence on the extraction accuracy of soil block size. Overall, the average relative error increases with camera height. The standard deviations at the four heights were 1.83, 1.22, 1.47, and 1.25, respectively. Compared with the other three heights, the dispersion of the average relative error is the smallest at 40 cm height. Based on the analysis of the average relative error and standard deviation, when the height of the camera is 40 cm from the ground, the captured image has the best accuracy in subsequent soil block size extraction.

4.3. Large-Scale Soil Block Particle Size Detection in the Field

In order to verify the accuracy of the improved model in the prediction of large-scale field soil block, two groups of comparative experiments were done. In the experimental field of Anhui Agricultural University, a rotary cultivator was used for rotary tillage of the farmland. Under natural light, a camera was used to collect photos of soil blocks after rotary tillage. The photos were input into the trained model for detection. The detection results are shown in Figure 10. Figure 10a was the input image to be detected. Figure 10b was the detection effect of the unimproved algorithm. Figure 10c was the detection effect of the improved algorithm. Comparing Figure 10b,c, it can be clearly seen that the unimproved model algorithm has a large area of missed detection during detection, and some connected soil blocks cannot be well segmented (the red circle position is drawn in the figure). When there are incomplete soil blocks at the edges of the images, a part of the detection effect is poor, and there is a phenomenon of missed detection of soil blocks. The reason for this phenomenon may be that the annotation of incomplete soil blocks is not involved in the production of datasets. At the same time, the recognition effect of soil blocks is also poor in places where the soil block surface contains straw. It may be caused by occlusion at the straw. The average confidence of the unimproved algorithm for large-scale soil block detection in the field is more than 95%, and the minimum confidence is 70%. The improved algorithm has an average confidence of more than 97% and a minimum confidence of 76% for large-scale soil block detection in the field. The improved algorithm has a certain improvement in recognition accuracy. Overall, the improved algorithm has a better detection effect. While identifying and detecting, the particle size of each soil block can be calculated quickly by using the extraction method of soil block particle size proposed in Section 2.2.4.

4.4. Soil Block Classification

According to the five soil block size levels divided in Section 2.2.4, the farmland soil blocks detected in Section 4.3 were graded, and the classification results are shown in Figure 11.
It can be clearly seen from the figure that the soil blocks extracted based on the model can be effectively graded according to the set size. Different classes are represented by different colors, and the accuracy of classing is based on the accuracy of soil block identification and size measurement. In the classification process, if there is a certain error in the recognition accuracy and the actual diameter of the soil block is near the critical value of the classification, there may be a soil block that originally belongs to class 1 and is misclassified into class 2.
The distribution of soil blocks after classification is counted. The proportion of the number of soil blocks and the total number of identified soil blocks at different levels, the proportion of the area of soil blocks and image frame at different levels, and the proportion of the area of soil blocks and the total area of identified soil blocks at different levels are counted. The statistical results are shown in Table 4. It can be seen from the data in the table that the soil blocks are most widely distributed at C2, C3, and C4, accounting for approximately 90 %.

5. Discussion

5.1. Influence of Soil Block Boundary Determination Method on Soil Block Distribution

If incomplete soil blocks appear at the edges of some images, statistical errors will occur when calculating the maximum diameter of soil blocks by extracting the soil block mask from the model. Therefore, some modifications were made to the model code: the algorithm of removing incomplete soil blocks on the edge was added to the original code. The algorithm is implemented based on the target box coordinates obtained by the above model and the pixel size of the predicted picture. The specific implementation method is as follows:
(1) Obtain the co-ordinates of the four vertices of the target box;
(2) If one of the four vertices of the target box is in the shadow area shown in Figure 12 (that is, outside the image box where the threshold is set), it is judged as an edge incomplete soil block;
(3) When the distribution of different levels of soil blocks is counted, when the maximum size diameter of the edge incomplete soil blocks has reached the maximum classification level, the soil blocks are retained and they participate in the subsequent calculation. The remaining edge soil blocks are no longer counted in the later calculation of soil block distribution;
(4) After the soil block is removed, the scale of the soil block distribution in the original image will change. Therefore, a new image box is set up to make the scale of soil block distribution in the image box as consistent as possible with the original scale.
Using the input image of Section 4.3, it was subjected to the incomplete soil block removal operation, and the result is shown in Figure 13. Compared with Figure 11 and Figure 13, the fine-tuned model algorithm can effectively remove the incomplete soil block at the edge when the image is collected. Removal of incomplete soil blocks, in the subsequent calculation of soil particle size distribution is more in line with the actual situation.
After removing the edge soil blocks, the particle size distribution is shown in Table 5. Combined with Table 4 and Table 5, after removing the edge soil blocks, the proportion of C1 increased. This is because the number of soil blocks decreased after removing the large soil blocks at the edge. Before removing the soil blocks, the sum of the area ratio of all classes of soil blocks in site I and site II to the area of the image box is 0.77 and 0.71 respectively. After removing the soil blocks at the edges, the area percentages become 0.63 and 0.59 without setting the new image box, which deviates greatly from the original scale. Under the reset image box, the area ratios are 0.73 and 0.68, respectively, which is similar to the original image scale. Therefore, although the set image box will cause the calculated area to become smaller, it can ensure that the scale of the image calculation remains unchanged to a certain extent.

5.2. Relationship between Mass of Soil Blocks and Maximum Particle Size

The broken rate of soil can also be used as an important indicator to evaluate the quality of agricultural machinery. The calculation of the broken soil rate is the proportion of the soil mass less than 5 cm to the total mass of the soil sample. We assume that there is a certain correlation between the mass and the maximum soil block diameter. If there is indeed a certain correlation, then we can through the above method, by measuring the maximum diameter of the soil to estimate the quality of soil, to estimate the broken soil rate, in order to more quickly evaluate the quality of agricultural machinery farming.
Starting from the existing experimental conditions, according to the grading standards in Section 2.2.4, under each grading standard, fifteen soil blocks were randomly selected, totaling 75. The maximum diameter and mass of the soil blocks were counted, and a quadratic polynomial fitting was performed on them. The fitting results are shown in Figure 14a. The correlation coefficient between the maximum diameter and mass of the soil block is 0.96, which has a strong correlation. From the fitted image, the fitting effect of soil block diameter before 60mm is better, and vice versa. Therefore, a deeper exploration of the correlations at different levels was carried out. The soil blocks under each grading standard were individually fitted, and the fitting results are shown in Figure 14b. At different classes, the correlation coefficients are 0.35, 0.77, 0.92, 0.73, and 0.75, respectively. It can be seen from the data that in class 3, the correlation between the diameter of the soil block and the quality is the best. The larger the level of the soil block, the greater the shape difference of the soil block, and the worse the correlation. Under C1, the effect of using quadratic polynomial fitting is not ideal, the possible reason is that the soil block is small and the quality change is not obvious. Therefore, the predicted quality of this method may be more suitable for medium-sized soil blocks.

6. Conclusions

In this study, we studied the method based on Mask-RCNN instance segmentation algorithm to identify and measure the size of farmland soil blocks so as to quickly count the distribution of soil blocks. Based on this research method, we draw the following conclusions:
(1) The improved model training loss value is 0.07, the AP value is 87.32, and the R value is 87.83. In the actual soil block detection, the correlation coefficient between the manual measurement value and the model measurement value is 0.986, which has a strong correlation, indicating that the algorithm has a high accuracy in size measurement;
(2) Different heights have a certain impact on the accuracy of model prediction. In this paper, four soil samples at different heights were set up to analyze the accuracy error. It can be seen from the experimental data that the prediction accuracy is higher at 40 cm;
(3) In the actual field of large-scale soil blocks particle size detection, the average confidence is more than 97% and has a good detection effect, but in the image edge incomplete soil block identification and straw under cover there are some defects;
(4) Based on identification and size measurement of soil blocks, the measured soil blocks were classified. The average weight diameter of different soil blocks was obtained from the standard mechanical sieve, and the model can classify the identified soil blocks well. For the graded soil blocks, the distribution statistics can be carried out quickly to evaluate the farming quality of agricultural machinery.

Author Contributions

Conceptualization, L.L.; methodology, L.L.; software, Q.B.; validation, J.L.; data analysis, Q.B.; investigation, Z.L.; resources, Q.Z.; data curation, L.L.; writing—original draft preparation, L.L.; writing—review and editing, Q.Z.; supervision, W.W.; project administration, L.L.; funding acquisition, Q.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by Universities Natural Science Research Project of Anhui Province (Grant No. KJ2020A0105) and Collaborative Innovation Project of Colleges and Universities of Anhui Province (Grant No. GXXT-2020-011).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ajdadi, F.R.; Gilandeh, Y.A.; Mollazade, K.; Hasanzadeh, R.P. Application of Machine Vision for Classification of Soil Aggregate Size. Soil Tillage Res. 2016, 162, 8–17. [Google Scholar] [CrossRef]
  2. Nasr, H.M.; Selles, F. Seedling Emergence as Influenced by Aggregate Size, Bulk Density, and Penetration Resistance of the Seedbed. Soil Tillage Res. 1995, 34, 61–76. [Google Scholar] [CrossRef]
  3. Alirezazadeh, P.; Rahimi-Ajdadi, F.; Abbaspour-Gilandeh, Y.; Landwehr, N.; Tavakoli, H. Improved Digital Image-Based Assessment of Soil Aggregate Size by Applying Convolutional Neural Networks. Comput. Electron. Agric. 2021, 191, 106499. [Google Scholar] [CrossRef]
  4. Liao, J.; Wang, Y.; Zhu, D.; Zou, Y.; Zhang, S.; Zhou, H. Automatic Segmentation of Crop/Background Based on Luminance Partition Correction and Adaptive Threshold. IEEE Access 2020, 8, 202611–202622. [Google Scholar] [CrossRef]
  5. Liu, L.; Mei, T.; Niu, R.; Wang, J.; Liu, Y.; Chu, S. RBF-Based Monocular Vision Navigation for Small Vehicles in Narrow Space below Maize Canopy. Appl. Sci. 2016, 6, 182. [Google Scholar] [CrossRef] [Green Version]
  6. Bogrekci, I.; Godwin, R.J. Development of an Image-Processing Technique for Soil Tilth Sensing. Biosyst. Eng. 2007, 97, 323–331. [Google Scholar] [CrossRef]
  7. Vannier, E.; Taconet, O.; Dusséaux, R.; Darboux, F. A Study of Clod Evolution in Simulated Rain on the Basis of Digital Elevation Models. Catena 2018, 160, 212–221. [Google Scholar] [CrossRef]
  8. Vannier, E.; Dusséaux, R.; Taconet, O.; Darboux, F. Wavelet-Based Clod Segmentation on Digital Elevation Models of a Soil Surface with or without Furrows. Geoderma 2019, 356, 113933. [Google Scholar] [CrossRef]
  9. Chimi-Chiadjeu, O.; Le Hégarat-Mascle, S.; Vannier, E.; Taconet, O.; Dusséaux, R. Automatic Clod Detection and Boundary Estimation from Digital Elevation Model Images Using Different Approaches. Catena 2014, 118, 73–83. [Google Scholar] [CrossRef]
  10. Taconet, O.; Vannier, E.; Le Hégarat-Mascle, S. A Contour-Based Approach for Clods Identification and Characterization on a Soil Surface. Soil Tillage Res. 2010, 109, 123–132. [Google Scholar] [CrossRef]
  11. Bosilj, P.; Gould, I.; Duckett, T.; Cielniak, G. Pattern Spectra from Different Component Trees for Estimating Soil Size Distribution. In Proceedings of the Mathematical Morphology and Its Applications to Signal and Image Processing, Saarbrücken, Germany, 8–10 July 2019; Burgeth, B., Kleefeld, A., Naegel, B., Passat, N., Perret, B., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 415–427. [Google Scholar]
  12. Bosilj, P.; Gould, I.; Duckett, T.; Cielniak, G. Estimating Soil Aggregate Size Distribution from Images Using Pattern Spectra. Biosyst. Eng. 2020, 198, 63–77. [Google Scholar] [CrossRef]
  13. Bai, Z.; Bi, D.; Wu, J.; Wang, M.; Zheng, Q.; Chen, L. Intelligent Driving Vehicle Object Detection Based on Improved AVOD Algorithm for the Fusion of LiDAR and Visual Information. Actuators 2022, 11, 272. [Google Scholar] [CrossRef]
  14. Xie, D.; Chen, L.; Liu, L.; Chen, L.; Wang, H. Actuators and Sensors for Application in Agricultural Robots: A Review. Machines 2022, 10, 913. [Google Scholar] [CrossRef]
  15. He, Z.; Wu, H.; Wu, G. Spectral-Spatial Classification of Hyperspectral Images Using Label Dependence. IEEE Access 2021, 9, 119219–119231. [Google Scholar] [CrossRef]
  16. Hou, W.; Zhang, D.; Wei, Y.; Guo, J.; Zhang, X. Review on Computer Aided Weld Defect Detection from Radiography Images. Appl. Sci. 2020, 10, 1878. [Google Scholar] [CrossRef] [Green Version]
  17. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef] [Green Version]
  18. Gu, Y.; Li, Z.; Zhang, Z.; Li, J.; Chen, L. Path Tracking Control of Field Information-Collecting Robot Based on Improved Convolutional Neural Network Algorithm. Sensors 2020, 20, 797. [Google Scholar] [CrossRef] [Green Version]
  19. Haque, M.R.; Sohel, F. Deep Network with Score Level Fusion and Inference-Based Transfer Learning to Recognize Leaf Blight and Fruit Rot Diseases of Eggplant. Agriculture 2022, 12, 1160. [Google Scholar] [CrossRef]
  20. Cui, Q.; Yang, B.; Liu, B.; Li, Y.; Ning, J. Tea Category Identification Using Wavelet Signal Reconstruction of Hyperspectral Imagery and Machine Learning. Agriculture 2022, 12, 1085. [Google Scholar] [CrossRef]
  21. Lateef, F.; Ruichek, Y. Survey on Semantic Segmentation Using Deep Learning Techniques. Neurocomputing 2019, 338, 321–348. [Google Scholar] [CrossRef]
  22. Yuan, X.; Shi, J.; Gu, L. A Review of Deep Learning Methods for Semantic Segmentation of Remote Sensing Imagery. Expert Syst. Appl. 2021, 169, 114417. [Google Scholar] [CrossRef]
  23. Azizi, A.; Gilandeh, Y.A.; Mesri-Gundoshmian, T.; Saleh-Bigdeli, A.A.; Moghaddam, H.A. Classification of Soil Aggregates: A Novel Approach Based on Deep Learning. Soil Tillage Res. 2020, 199, 104586. [Google Scholar] [CrossRef]
  24. Wang, X.; Li, Y.; Yang, Z.; Zhang, M.; Wang, R.; Cui, L. Detection Method of Clods and Stones from Impurified Potatoes Based on Improved YOLO v4 Algorithm. Trans. Chin. Soc. Agric. Eng. 2021, 52, 241–247. [Google Scholar]
  25. He, K.; Gkioxari, G.; Dollar, P.; Girshick, R. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 386–397. [Google Scholar] [CrossRef] [PubMed]
  26. Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–22 June 2018; pp. 7132–7141. [Google Scholar]
  27. Chen, X.; Song, X.; Cheng, L.; Ran, Z.; Wang, Y.; Liang, W.; Wang, Y. Flexible Calibration Method of Electronically Focus-Tunable Lenses. IEEE Trans. Instrum. Meas. 2021, 70, 1–10. [Google Scholar] [CrossRef]
  28. Chen, X.; Song, X.; Wu, J.; Xiao, Y.; Wang, Y.; Wang, Y. Camera Calibration with Global LBP-Coded Phase-Shifting Wedge Grating Arrays. Opt. Lasers Eng. 2021, 136, 106314. [Google Scholar] [CrossRef]
  29. Chen, X.; Chen, Y.; Song, X.; Liang, W.; Wang, Y. Calibration of Stereo Cameras with a Marked-Crossed Fringe Pattern. Opt. Lasers Eng. 2021, 147, 106733. [Google Scholar] [CrossRef]
  30. Zhang, Z. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef]
Figure 1. Acquire Image. (a) Individual soil blocks. (b) Cohesive soil blocks. (c) Reference objects.
Figure 1. Acquire Image. (a) Individual soil blocks. (b) Cohesive soil blocks. (c) Reference objects.
Agriculture 12 02038 g001
Figure 2. Flow chart of the method.
Figure 2. Flow chart of the method.
Agriculture 12 02038 g002
Figure 3. Mask-RCNN overall framework.
Figure 3. Mask-RCNN overall framework.
Agriculture 12 02038 g003
Figure 4. The structure of Resnet101 with the introduction of the SENet attention mechanism.
Figure 4. The structure of Resnet101 with the introduction of the SENet attention mechanism.
Agriculture 12 02038 g004
Figure 5. Extraction process of soil block size.
Figure 5. Extraction process of soil block size.
Agriculture 12 02038 g005
Figure 6. Training loss value curve.
Figure 6. Training loss value curve.
Agriculture 12 02038 g006
Figure 7. Recognition result.
Figure 7. Recognition result.
Agriculture 12 02038 g007
Figure 8. Correlation analysis between manual measurement value and model detection value of soil block particle size.
Figure 8. Correlation analysis between manual measurement value and model detection value of soil block particle size.
Agriculture 12 02038 g008
Figure 9. Relative error analysis of prediction at different altitudes.
Figure 9. Relative error analysis of prediction at different altitudes.
Agriculture 12 02038 g009
Figure 10. Detection results of large-scale soil blocks in the field. (a) the input image to be detected. (b) the detection effect of the unimproved algorithm. (c) the detection effect of the improved algorithm.
Figure 10. Detection results of large-scale soil blocks in the field. (a) the input image to be detected. (b) the detection effect of the unimproved algorithm. (c) the detection effect of the improved algorithm.
Agriculture 12 02038 g010
Figure 11. Results of the soil blocks grading.
Figure 11. Results of the soil blocks grading.
Agriculture 12 02038 g011
Figure 12. Simple schematic for edge-removal incomplete blocks.
Figure 12. Simple schematic for edge-removal incomplete blocks.
Agriculture 12 02038 g012
Figure 13. The effect of removing the incomplete clod from the edge.
Figure 13. The effect of removing the incomplete clod from the edge.
Agriculture 12 02038 g013
Figure 14. Fitted results. (a) All blocks are fitted with a curve. (b) Soil blocks of different classes are fitted by a single curve.
Figure 14. Fitted results. (a) All blocks are fitted with a curve. (b) Soil blocks of different classes are fitted by a single curve.
Agriculture 12 02038 g014
Table 1. Dataset distribution.
Table 1. Dataset distribution.
ClassDivision RatioTraining SetValidation SetTest SetTotal
Individual soil blocks16: 4: 5506126158790
Cohesive soil blocks16: 4: 52466277385
Reference objects16: 4: 538101260
Table 2. MWD of different soil block sizes obtained by standard mechanical sieve.
Table 2. MWD of different soil block sizes obtained by standard mechanical sieve.
Class 1Class 2Class 3Class 4Class 5
0 < D 21.2021.20 < D 39.5039.50 < D 65.0065.00 < D 93.50D > 93.50
Table 3. Evaluation of training results.
Table 3. Evaluation of training results.
Training ParametersAP(%)R(%)F1
LR = 1 × 10−3×××
LR = 1 × 10−462.0378.910.69
LR = 1 × 10−583.5285.640.85
ALR85.0084.430.85
ALR+SENet87.3287.830.88
Table 4. Distribution statistics of soil blocks.
Table 4. Distribution statistics of soil blocks.
ClassSite ISite II
The proportion of the number of soil blocks and the identified total number of soil blocks at different levelsC10.080.08
C20.360.39
C30.320.37
C40.220.13
C50.020.03
The proportion of the area of soil blocks and image frame at different levelsC10.010.01
C20.090.12
C30.230.26
C40.370.22
C50.070.09
The proportion of the area of soil blocks and the total area of identified soil blocks at different levelsC10.010.01
C20.120.17
C30.310.37
C40.480.32
C50.090.13
Table 5. The distribution of soil blocks after removing the edge incomplete soil blocks.
Table 5. The distribution of soil blocks after removing the edge incomplete soil blocks.
ClassSite ISite II
The proportion of the number of soil blocks and the identified total number of soil blocks at different levelsC10.010.10
C20.380.40
C30.300.35
C40.200.12
C50.020.04
The proportion of the area of soil blocks and image frame at different levelsC10.010.01
C20.090.12
C30.210.25
C40.340.20
C50.080.10
The proportion of the area of soil blocks and the total area of identified soil blocks at different levelsC10.010.02
C20.130.17
C30.290.36
C40.470.30
C50.100.15
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Liu, L.; Bi, Q.; Liang, J.; Li, Z.; Wang, W.; Zheng, Q. Farmland Soil Block Identification and Distribution Statistics Based on Deep Learning. Agriculture 2022, 12, 2038. https://doi.org/10.3390/agriculture12122038

AMA Style

Liu L, Bi Q, Liang J, Li Z, Wang W, Zheng Q. Farmland Soil Block Identification and Distribution Statistics Based on Deep Learning. Agriculture. 2022; 12(12):2038. https://doi.org/10.3390/agriculture12122038

Chicago/Turabian Style

Liu, Lichao, Quanpeng Bi, Jing Liang, Zhaodong Li, Weiwei Wang, and Quan Zheng. 2022. "Farmland Soil Block Identification and Distribution Statistics Based on Deep Learning" Agriculture 12, no. 12: 2038. https://doi.org/10.3390/agriculture12122038

APA Style

Liu, L., Bi, Q., Liang, J., Li, Z., Wang, W., & Zheng, Q. (2022). Farmland Soil Block Identification and Distribution Statistics Based on Deep Learning. Agriculture, 12(12), 2038. https://doi.org/10.3390/agriculture12122038

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop