Fast Detection of Plants in Soybean Fields Using UAVs, YOLOv8x Framework, and Image Segmentation

Abstract

The accuracy of classification and localization of plants on images obtained from the board of an unmanned aerial vehicle (UAV) is of great importance when implementing precision farming technologies. It allows for the effective application of variable rate technologies, which not only saves chemicals but also reduces the environmental load on cultivated fields. Machine learning algorithms are widely used for plant classification. Research on the application of the YOLO algorithm is conducted for simultaneous identification, localization, and classification of plants. However, the quality of the algorithm significantly depends on the training set. The aim of this study is not only the detection of a cultivated plant (soybean) but also weeds growing in the field. The dataset developed in the course of the research allows for solving this issue by detecting not only soybean but also seven weed species common in the fields of Kazakhstan. The article describes an approach to the preparation of a training set of images for soybean fields using preliminary thresholding and bound box (Bbox) segmentation of marked images, which allows for improving the quality of plant classification and localization. The conducted research and computational experiments determined that Bbox segmentation shows the best results. The quality of classification and localization with the application of Bbox segmentation significantly increased (f1 score increased from 0.64 to 0.959, mAP50 from 0.72 to 0.979); for a cultivated plant (soybean), the best classification results known to date were achieved with the application of YOLOv8x on images obtained from the UAV, with an f1 score = 0.984. At the same time, the plant detection rate increased by 13 times compared to the model proposed earlier in the literature.

1. Introduction

The introduction of precision farming technologies is one of the most important ways to overcome the food crisis associated with the rapid growth of the world’s population [1]. The processes of precision farming assume precise impact on crops and cultivated fields in order to increase yields and reduce costs. An important goal is to diminish the influence of interfering factors that limit the growth of useful plants. A number of factors hinder the successful growth of plants: pests, lack or excess of fertilizers, soil conditions, moisture lack or excess, soil salinity, weeds, etc. [2]. Reliable identification of these factors is, of course, possible by visual inspection of the field by an agronomist; however, the large areas of fields prevent the widespread use of manual labor in the monitoring process. The obvious way to improve the situation is the use of automatic means. Nowadays, UAVs represent such means, and they are widely used in various areas of aerial monitoring [3], including monitoring of precision agriculture. To carry out such monitoring, it is necessary to select an appropriate field and flight altitude and take into account the limitations related to weather, battery charge, etc. For example, if you need to obtain an image of individual plants, you need to set the flight altitude to 3–10 m; taking into account the flight time on one battery charge, the monitoring area will be about 1 hectare. To calculate the spectral indices, the flight altitude can be much higher (50 m or more), and accordingly, the area of the field that can be flown around will be up to a hundred hectares. Nevertheless, the processing of the acquired data and images is quite complex, multi-purpose, and one of the most promising tasks attracting great attention from the scientific community [4]. The collected data bear the attributes of so-called big data [3] and practically cannot be processed manually in the operational mode. Although, when using the images obtained from low-flying UAVs equipped with high-resolution cameras, such as those listed in [5], there is a possibility of automatic recognition and classification of objects in the fields, but the specific ways of realization of this process are not so simple. One of the similar tasks is the task of classification and localization of images of cultivated plants and weeds on the received images (detection task). The solution to this problem allows for more accurate forecasting of future yields and application of variable rate technology (VRT) [6], which not only reduces costs but also increases yields. To solve this problem, there are widely used machine learning methods and, above all, frameworks, including convolutional neural networks (CNNs); for example, AlexNet [7], ResNet [8,9], VGG [10], GoogLeNet [11], U-Net, MobileNets, and DenseNet [12]. Such studies demonstrate good results when the quality of classification reaches 90 percent or more. For example, in [7], the task of classifying plant images in the fields sown with soybean was considered. The original RGB images acquired by a UAV from a height of 4 m (resolution 4000 × 3000 pixels) were first segmented using the SLIC super-pixel method and then manually segmented into four classes: soil, soybean, grass, and broadleaf. For each segment, the following features were computed: Gray-Level Co-occurrence Matrix (GLCM), Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBPs), and color distributions in RGB, HSV, and CIELab color spaces. The obtained features were used to train CNNs based on the CaffeNet architecture. The quality of the classification claimed by the authors was 99%. The average processing time of one image was 1.14 s. However, in practice, it is required to solve detection problems, in other words, to localize a plant (to determine its location) and to classify it at the same time. The most advanced framework of this type is YOLO (You Only Look Once) [13,14,15], which is able to solve the problem of object detection in one pass of the signal through the network. In other words, a trained and optimized network can process several dozen high-resolution frames per second. For successful operation of such a model, it is only necessary to train it on an appropriate set of annotated images. Apparently, such extremely attractive properties of the model have catalyzed the growth of its applications for plant detection in fields [16,17]. One of such applications was the task of processing images acquired in soybean fields [15]. Soybean is a plant that contains a large amount of protein, has many useful properties, and is a leader among legumes, significantly surpassing others (beans, peas, chickpeas, etc.) in terms of production. For these reasons, much attention is paid to the optimization of its production on the basis of precision farming technologies. A number of works consider the application of the YOLO algorithm for the detection of cultivated plants and weeds in soybean fields. For example, the task of weed segmentation in a soybean field with an average accuracy of 98% is solved mainly with the use of ground machines [18]. In the work of Sunil et al. [17], the results of applying YOLO to images collected by a ground robot for several crops, including soybeans, are presented. The accuracy values vary from 80.8% to 98% depending on the crop, field, and environmental conditions. However, when using UAV-acquired images, the classification accuracy achieved so far is much lower. Tetila et al. [19] consider the problem of binary identification (soybeans or weeds) using the YOLOv5s6 model and UAVs. The achieved accuracy is 0.93 using a much lighter version of the software. Approximately the same result of soybean classification was obtained in the study [20].

In this paper, we consider the possibility of using a dataset in which only part of the plant images are labeled. Despite this limitation, due to the pre-segmentation, we obtained significantly better soybean classification results with YOLO than in the above-mentioned papers. The goal of this article is to describe a general approach that can be used to build plant detection models using partially segmented image sets and the YOLOv8x framework.

The main contributions of this paper are as follows:

• A new dataset for soybean fields was developed; it includes annotated images of not only soybeans but also seven common weed species found in agricultural fields in Central Asia.
• A method for preprocessing the training data using Bbox segmentation is proposed; it significantly improves the accuracy of classification and localization. The application of the proposed approach led to a significant improvement in the model performance: f1 score increased from 0.64 to 0.959 and mAP@0.5 increased from 0.72 to 0.979.
• The highest classification accuracy to date was achieved using the YOLOv8x algorithm and UAV images (f1 score = 0.984) for the soybean recognition task.
• The proposed model significantly outperformed previously described approaches in terms of detection speed, providing a 13-fold increase in speed compared to the results presented in the literature previously. The trained model processes one image in 20.9 milliseconds and can provide detection in a video stream at a rate of 47 frames per second. That is, plant detection is possible in real time.

The article is arranged as follows:

• The second section describes the materials and methods in detail, including the procedures for data collection, image labeling, and object detection.
• The third section presents the results of the experiments conducted using the proposed approach.
• The fourth section analyzes and discusses the results, focusing on their comparative evaluation and interpretation.
• The conclusion summarizes the results of the study and identifies possible directions for future research.

2. Materials and Methods

The irrigated agricultural field where the survey was conducted is located near the city of Almaty (southeastern Kazakhstan) in the village of Almalybak and belongs to the Kazakh Research Institute of Agriculture and Crop Production (KazRIACP). Figure 1 shows the location of the field. The coordinates of the field are as follows: 43.226108, 76.700016. The area is 1 hectare. The soil cover of the field is represented by foothill light chestnut soils. They are formed on loess-like loams and have a clearly expressed fertile profile. A characteristic feature of light chestnut soils is their high carbonate content. According to the mechanical composition, it belongs to medium loams. The content of coarse dust is 40–45%, physical clay is about 40%, and silt particles decrease along the profile from 13.82 to 8.62%. Almost all mechanical elements are in the aggregate state. The sum of macroaggregates reaches 80–90%, which is typical for loess-like rocks. The soils of the site are characterized by deep groundwater, as well as weak mineralization.

The research process consisted of collecting images of the field during the first phase of soybean development (21 days after planting) (see Appendix A for a list of soybean development phases in practice). The field was photographed using a UAV equipped with a 4K resolution RGB camera. UAV specifications are presented in Appendix B. A total of 376 high-resolution images of the field were acquired. Then, the images of soybean and weed plants were manually marked up in YOLO format [21]. At the preliminary stage, a list and appearance of weeds that may occur in soybean fields of southern Kazakhstan were compiled (see Figure 2).

Images were marked using the publicly available data annotation platform named CVAT service (Computer vision annotation tool) [22]. The obtained labeled images were supplied to the input of the pre-trained YOLOv8x model for its additional training. The main stages of the study are shown in Figure 3.

There is a description of the stages listed in Figure 3 in more detail.

• Field images were obtained during the routine experimental overflights of soybean fields. The overflights were conducted using a commercially available DJI Mavic Mini 2 UAV (SZ DJI Technology Co., Ltd., Shenzhen, China) equipped with an RGB camera with 4K pixel resolution (CMOS 1/2.3″, Effective Pixels: 12 MP, FOV: 83°, resolution 4000 × 2250) at an altitude of 4 to 10 m (see Figure 4a).
• Image labeling was performed using CVAT; therefore, most plant images were boxed and annotated with one of 10 weed and soybean classes (see Figure 4b).
  The set of acquired marked images was divided in an 80/20 proportion into training (300 images) and test images (76 images).
• Segmentation. The essence of the segmentation process is to remove the background along with unlabeled objects in each image. Two approaches were used for this purpose.

  3.1.

  The first approach implemented in the experiment 1 provided threshold segmentation of plant images using the plantcv library [23]. In the process of threshold segmentation, each pixel of the image is compared with a given brightness threshold selected during the preliminary experiments in the range from 0 to 255. The best result is obtained at the threshold equal to 144. The brightness of pixels with a value lower than the threshold is set to 0. Pixels with a brightness higher than the threshold are not changed (see Figure 5a).

  3.2.

  The second segmentation option (experiment 2 or Bbox segmentation) involved removing all unlabeled plant and background images outside the bounding box (Bbox) (see Figure 5b).

• YOLOv8x retraining and evaluation of the results. The YOLOv8x framework, which has 70–80 million trainable parameters [24], was used in the experiments. The YOLOv8x model was pre-trained on the COCO (Common Objects in Context) dataset containing objects of 80 classes ranging from persons and cars to toothbrushes [25,26]. For each dataset, the same set of images was used to test and evaluate the results obtained, but the retraining was performed independently on three datasets: dataset for experiment 1 (threshold segmentation), dataset for experiment 2 (Bbox segmentation), and original dataset without segmentation. To improve the generalization ability of the model and its qualitative indicators, the training dataset was extended by applying the image augmentation library [27]. A total of 1000 augmented images were added and used for training in experiments 1 and 2. The augmentation parameters are described in Appendix C.

Since the applied YOLOv8x framework performs not only plant classification but also plant localization, the metrics of classification quality and localization quality were used to estimate the obtained detection results (see

Table 1. Metrics for assessing the quality of plant detection.

Metric	Formula	Explanation
Classification Indicators [28]
Precision	$P = {\displaystyle \frac{TP}{\left(TP + FP\right)}}$ where true positive (TP) and true negative (TN) are cases of correct operation of the classifier. Accordingly, false negatives (FNs) and false positives (FPs) are cases of misclassification.	Proportion of true positive predictions among all predicted positive cases.
Recall	$R={\displaystyle \frac{TP}{\left(TP+FN\right)}}$	Proportion of true positive predictions among all actual positive cases.
F1 score (a harmonic mean)	$F1score={\displaystyle \frac{2 \ast P \ast R}{\left(P+R\right)}}$	A harmonic mean between accuracy and recall.
Localization Indicators
IoU	$IoU={\displaystyle \frac{Intersection area}{Joint area}}$	A measure that quantifies the overlap between the predicted bounding box (usually a rectangle—Bbox) and the true bounding box. It plays an important role in assessing the accuracy of object localization [29].
$AP$	$AP={\int }_0^1 P\left(R\right)dR$ where P(R) is accuracy versus recall function.	Average accuracy for one class, calculated as the area under the precision–recall curve.
mAP50	$mAP0.5={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N} {AP}_i^{0.5}$ where N is the total number of classes and ${AP}_i^{0.5}$ is the mean value of precision (AP) for class i at IoU = 0.5.	The mean value of accuracy across all classes, used to estimate the overall performance of the model.
mAP50–95	$mAP0.50:0.95={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N} \left({\displaystyle \frac{1}{10}}{\displaystyle \sum _{j=0}^9} {AP}_i^{\left(0.5+0.05j\right)}\right)$ where N is the total number of classes and ${AP}_i^{(0.5+0.05j)}$$\mathrm{is} \mathrm{the} \mathrm{mean} \mathrm{value} \mathrm{of} \mathrm{precision} \left(\mathrm{AP}\right) \mathrm{for} \mathrm{class} \mathrm{i} \mathrm{at} \mathrm{IoU} = 0.5+0.05j$. j is an index taking values from 0 to 9, corresponding to IoU thresholds of 0.5, 0.55, 0.6, …, 0.95.	Mean accuracy averaged over multiple IoU thresholds from 0.5 to 0.95 in steps of 0.05.

).

Computational experiments were performed on a computer equipped with a 12th Gen Intel (R) Core (TM) i9–12900K processor and 125 GB RAM in the environment of a Ubuntu 20.04.6 LTS operating system (kernel 5.15.0–134-generic). The program system is developed in Python 3.9 using the following libraries: pandas v2.3.0, numpy v2.3.1, scikit-learn v1.7.0, torch v2.7.1, ultralytics v8.3.161, opencv-python v4.11.0.86, plantcv v4.8, imgaug v0.4.0, loguru v0.7.3, pyyaml v6.0.2, and wandb v0.21.0.

3. Results

The preliminary steps associated with data preparation resulted in the creation of class-imbalanced datasets 0, 1, and 2 containing images of several tens of thousands of plants and beds (see

Table 2. Characteristics of the original dataset and the datasets prepared in the course of experiments 1 (threshold segmentation) and 2 (Bbox segmentation).

Class Name	Decoding of the claSS Name	0	1	2
0: Bed	Bed	2458	0	0
1: Glycine max	Soybean	42,282	41,368	42,282
2: Amaranthus retroflexus	Common wheatgrass	6155	3840	6155
3: Convolvulus arvensis	Field creeper	385	304	385
4: Setaria glauca	Bristle broom	1897	149	1897
5: Xanthium strumarium	Common dunnitch	513	439	513
6: Cirsium arvense	Pink thistle	532	516	532
7: Echinochloa crusgalli	Chicken millet	0	0	0
8: Hibiscus trionum	Hibiscus trifoliate	15	15	15
9: Abutilon theophrasti	Theophrastus canatum	27	27	27
10: Chenopodium album	White marmoset	0	0	0
11: Apera spica-venti	Common broom, field broom	0	0	0

Note. 0 is an original dataset without segmentation; 1 is a dataset for experiment 1 (threshold segmentation); 2 is a dataset for experiment 2 (Bbox segmentation).

). The images of useful plants (soybeans) are most fully represented, while the images of individual weeds contain from tens to hundreds of images. The original dataset (0) contains a small number (about 10%) of unlabeled plant images. Some weed species (Echinochloa crusgalli, Chenopodium album, Apera spica-venti), which are found in the fields of southern Kazakhstan, are absent in the created datasets.

The mentioned weed classes (7,10,11), which are not represented in the test set, as well as class 0, were not used in the model validation. Therefore, the calculation of detection quality indicators was performed without taking them into account. Since the datasets are not balanced, in other words, the number of plants of different species differs significantly, confusion matrices were visualized to compare the results obtained during the computational experiments. Figure 6 shows the confusion matrix obtained during the experiments with the original (unsegmented) set of images.

Numbers on the diagonal mean the number of correctly classified plants of the corresponding species. Numbers outside the main diagonal mean the number of errors of the first and second kind (FP is above the diagonal, FN is below the main diagonal). It is not difficult to calculate the f1 score for soybeans. It amounts to 0.933.

Figure 7 shows a similar confusion matrix obtained in experiment 1 (threshold segmentation).

Figure 8 shows the confusion matrix obtained in experiment 2 (Bbox segmentation).

The classification quality of cultivated plants became significantly higher (f1 score = 0.979). To verify these assumptions and to assess better the quality of the applied model, a cross-validation (k = 10) of detection models was performed. The f1 score values for all significant classes are given in Appendix D. The averaged results of detection quality for all plant species and separately for soybeans are given in

Table 3. Results of computational experiments for soybean and weed detection.

Training Dataset	mAP50	mAP50–95	Recall	Precision	F1 Score Micro	F1 Score Macro	F1 Score Glycine Max
Original dataset	0.72	0.3	0.599	0.677	0.6356	0.341	0.933
Experiment 1 (threshold segmentation)	0.65	0.348	0.639	0.749	0.6896	0.439	0.939
Experiment 2 (Bbox segmentation and k-fold cross-validation, k = 10)	0.979	0.936	0.941	0.963	0.959	0.873	0.984

Note. The best results are highlighted in bold.

.

The speed of image processing using the YOLOv8x framework in the process of classification and localization of plants was 0.085 sec.

To assess the impact of weather-related distortions, additional experiments were performed using the model under the conditions of fog. White fog of varying intensities was superimposed on the field image. The opacity of the superimposed white layer varied from 0 (original image, completely transparent layer) to 10 (completely opaque white layer) with a step of 0.1. Figure 9 shows the field images with different opacity levels.

Depending on the layer opacity, the classification model showed the following results (

Table 4. Results of computational experiments for different opacity levels.

Opacity Level	F1 Score Micro
0	0.959
0.1	0.85
0.2	0.852
0.3	0.853
0.4	0.828
0.5	0.841
0.6	0.805
0.7	0.78
0.8	0.635
0.9	0.272

).

Image set and programming code can be downloaded at https://drive.google.com/drive/folders/1ObZjrP5eW3zO0bZLeIrNd347nKUlgowU (accessed on 22 July 2025) (see Data Availability Statement).

4. Discussion

Plant detection in fields is an important application of machine learning methods. In particular, it is used to detect plant diseases, crops and weeds [16,30]. The accuracy of crop classification varies from 90% to 98% depending on the model and shooting conditions. The used technologies include the following: RGB, multispectral and hyperspectral cameras and deep learning models: CNN, Vision Transformers, GAN, Vision-Language Models [30]. One of the deep learning models used is YOLOv5; the use of it allowed us to obtain the classification quality in soybean fields from 0.91 [31,32] to 0.93 (f1-score) and the localization accuracy mAP50-95 = 0.48 [33]. In this paper, the experiments were performed with the use of the YOLOv8x model and some dataset preprocessing techniques.

The results obtained in experiment 1 (original dataset) show a quality that roughly corresponds to that reported in the literature (f1 score = 0.93 [20,33]). The result is achieved by careful labeling of the image sets. Nevertheless, it can be improved by applying segmentation techniques. The application of threshold segmentation improved the result for a cultivated plant (f1 score = 0.939), which is already slightly higher than the results achieved previously using YOLO frameworks on UAV images. However, the application of Bbox segmentation allowed us to significantly improve this result as well. The detection quality of Bbox segmentation (experiment 2) is significantly higher compared to both the original dataset and threshold segmentation (experiment 1). The localization quality (mAP50–95 = 0.936) also increased significantly, which allows for more confident identification of the plant location in the field [33]. For soybeans, the classification quality (f1 score = 0.984) is actually close to the maximum known in the literature (f1 score = 0.99) [7].

However, in contrast to the above-mentioned work [7], the classification speed is about 13 times higher (measurements were performed on a computer of the same configuration); in addition, at the same time and with the same speed, the localization of plants is performed, which is important for the subsequent planning of agrotechnical measures.

The experiments show that plant detection by the previously trained network is performed in 20.9 milliseconds (0.1 ms preprocess, 8.7 ms inference, 12.1 ms postprocess). The model can perform detection at a speed of 47.85 frames per second. Therefore, semi-automatic detection of soybeans is possible on large arable fields in real time.

Note that the classification quality of weeds, such as Abutilon theophrasti and Xanthium strumarium, decreased. Part of the results obtained can be explained by the fact that, firstly, the number of weed plants of all species is small, and secondly, Abutilon theophrasti is a very small weed at this growth phase.

It should also be noted that some images are recognized worse than others. Ten photos sorted in order of improving F1 score micro-quality are given in Appendix E. The photos containing a large background area and a small number of plants are recognized to be significantly worse. Fog imitation also has a significant effect on the classification quality (Table 4). A sharp deterioration in the quality of the model is observed, first, in the presence of even a relatively small haze (0.1) and then with an increase in opacity above 0.7. These patterns can serve as a subject for further research aimed at increasing the robustness of the method. Perhaps some image processing techniques that improve image quality will be useful [34,35].

The developed dataset allowed training a model to detect soybean plants from a UAV at early stages of development. In practical application, this allows for making predictions of possible yields and assessing the weediness of field plots. Despite the good results achieved during the research process, there are some limitations that will need to be addressed during the next phases of the study as follows:

• The datasets are not balanced and contain many images of cultivated plants and much fewer weeds. As a result, the quality of weed classification is significantly inferior to the quality of useful plant detection.
  All experiments were performed only during the first stage of soybean growth, when plants are well identified and there is little overlap. Expanding the dataset with field images at later stages of soybean growth will increase the practical applicability of the dataset and the corresponding detection method.
• The photographs of the fields were taken from a low altitude, which on the one hand improves the quality of the images and on the other hand significantly increases the required duration of the UAV flight.

5. Conclusions

The computer vision methods, especially those related to detection, localization, and classification of objects, help in weed control procedures in specific areas of the field. They allow for the implementation of variable rate herbicide application practices within a field. However, the major challenge in developing a weed detection and crop plant counting system is the need for a properly annotated database to distinguish between weeds and crops in the field. This paper utilizes a set of annotated crop images in a soybean field developed by the authors, containing more than 370 highly detailed images annotated for the application of YOLO frameworks. The total number of labeled plants in the images is close to fifty thousand. At the first stage of experiments, we achieved the results generally comparable to those described in the published literature. Further experiments using Bbox segmentation allowed for substantial improvement of these results. A high result of soybean detection on the images obtained from UAVs was achieved.

The developed method of data preparation and training allowed us to surpass the results described earlier [20] (f1 score = 0.93) and to achieve the results obtained using machine vision systems installed on ground robots (f1 score = 0.98). In other words, our proposed and implemented methodology of preprocessing the training set of images allowed us to achieve better results of soybean detection with the help of UAVs and YOLOv8x framework than those known from the literature. The YOLOv8x model used was pre-trained on COCO data images, which do not contain specific images of cultivated plants and weeds. Increasing the set of labeled images can improve the classification and localization of not only useful plants but also all weed classes. The issue of plant detection at subsequent stages of development also remains open. Therefore, in future studies, we plan to

• Improve the balance of classes in the datasets by increasing the number of weed plant images.
• Analyze the possibilities of plant detection at the next phases of soybean development.
• Analyze the dependence of detection quality on the UAV flight altitude and shooting conditions.
• Increase the set of annotated images by surveying fields in different regions of the country.
• Apply the described approach to the detection of other cultivated plants growing in Kazakhstan.