Autonomous Agricultural Robot Using YOLOv8 and ByteTrack for Weed Detection and Destruction

Abstract

Automating agricultural machinery presents a significant opportunity to lower costs and enhance efficiency in both current and future field operations. The detection and destruction of weeds in agricultural areas via robots can be given as an example of this process. Deep learning algorithms can accurately detect weeds in agricultural fields. Additionally, robotic systems can effectively eliminate these weeds. However, the high computational demands of deep learning-based weed detection algorithms pose challenges for their use in real-time applications. This study proposes a vision-based autonomous agricultural robot that leverages the YOLOv8 model in combination with ByteTrack to achieve effective real-time weed detection. A dataset of 4126 images was used to create YOLO models, with 80% of the images designated for training, 10% for validation, and 10% for testing. Six different YOLO object detectors were trained and tested for weed detection. Among these models, YOLOv8 stands out, achieving a precision of 93.8%, a recall of 86.5%, and a mAP@0.5 detection accuracy of 92.1%. With an object detection speed of 18 FPS and the advantages of the ByteTrack integrated object tracking algorithm, YOLOv8 was selected as the most suitable model. Additionally, the YOLOv8-ByteTrack model, developed for weed detection, was deployed on an agricultural robot with autonomous driving capabilities integrated with ROS. This system facilitates real-time weed detection and destruction, enhancing the efficiency of weed management in agricultural practices.

1. Introduction

The world’s population is growing very fast, and with population growth comes food demand. Therefore, weed control in production areas is becoming increasingly important to increase crop yields and food production [1]. In the past years, the most effective method of weed control was known to be the use of herbicides. However, the use of chemical herbicides has negative effects on the environment and human health [2]. Therefore, the goal of reaching healthy and safe agricultural products has led researchers to develop different weed control methods.

Many researchers are using machine vision-based sensors to collect images and applying artificial intelligence algorithms to perform the necessary image processing and modeling to achieve weed identification with minimal human intervention. Traditional methods use characteristics such as color, shape, texture, and distribution to distinguish between crops and weeds. Hamuda et al. proposed a new algorithm based on color features, morphological erosion, and expansion. The proposed algorithm uses HSV color space to distinguish between crops, weeds, and soil. As a result of their study, a sensitivity of 98.91% and an accuracy of 99.04% were obtained [3]. Cho et al. (2002) used an artificial neural network (ANN) model with eight different shape features for weed detection. This model achieved a detection accuracy of 93.3% for crops and 93.8% for weeds [4]. Bakhshipour et al. identified 14 out of 52 tissue features for weed detection in sugar beet using a feature selection method. These were used as input to the ANN algorithm. The average accuracy of the weed detection algorithm was calculated for 40 test images and was 95.3%. Sugar beet detection accuracy was found to be 91.2% [5]. Traditional methods that use the distributive property are effective when crops are planted at regular intervals along the row. Lottes et al. employed a Bayesian approach, a probabilistic model, to differentiate between sugar beets and weeds. They reported achieving classification success with over 95% accuracy in beet fields [6].

The development of machine learning and deep learning methods in the last decade has increased the tendency toward these methods in solving the crop/weed discrimination problem [7]. Review studies conducted in this field also support this finding [8,9]. Support Vector Machine (SVM) [10], K-Nearest Neighbor (KNN) [11], and Random Forest (RF) classifier [12] are the most prominent machine learning models in solving this problem. Islam et al. compared the performance of machine learning algorithms such as Random Forest (RF), Support Vector Machine (SVM), and K-Nearest Neighbors (KNN) to detect weeds using UAV images obtained from a pepper field. According to their results, the accuracy of RF in weed detection was 96%, SVM 94%, and KNN 63% [11]. Despite the high accuracy of machine learning (ML) methods, significant experimental efforts are needed to extract meaningful features [13]. In contrast, deep learning (DL) eliminates the need for hand-crafted features. Instead, it generates deep representations of input images [14]. Supported by large-scale image data, deep learning algorithms can effectively solve the challenge of precise weed detection. Rahman et al. developed DL-based weed detection models in their study. These models include YOLOv5, Retina Net, Efficient Det, Fast RCNN, and Faster RCNN. Retina Net achieved 79.98% detection accuracy despite having a longer inference time, while YOLOv5, which has the fastest inference time, achieved 76.58% detection accuracy [13]. Hu et al. proposed an innovative graph-based deep learning architecture called Graph Weeds Net (GWN), which aims to recognize multiple weed species using RGB images collected from different areas. This architecture achieved an accuracy rate of 98.1% [15]. Yu et al. compared the performance of deep convolutional neural network (DCNN) models—VGGNet, GoogLeNet, and DetectNet—in detecting emerging weeds in Bermuda grass. The study found that while the VGGNet model achieved impressive results across different mowing heights and surface conditions, DetectNet emerged as the most effective DCNN architecture for detecting a variety of broadleaf weeds alongside dormant Bermuda grass [16]. CNN models employ a two-stage process, making them slightly slower compared to single-stage YOLO models. Hussain et al. collected approximately 24,000 images from potato fields under varying weather conditions, including sunny, cloudy, and partly cloudy environments. These images were tested using YOLOv3 and Tiny-YOLOv3 models for the detection of lamb’s quarters weed and potato plants infected with early blight, as well as healthy potato plants. For the weed dataset, the mAP values of the Tiny-YOLOv3 and YOLOv3 models were determined to be 78.2% and 93.2%, respectively [17]. Ruigrok et al. reported that they effectively controlled 96% of weeds and identified potato plants with an accuracy of 84% using the YOLOv3 algorithm. Their study focused on recognizing crops rather than identifying weeds [18]. Junior and Ulson proposed a real-time weed detection system based on the YOLOv5 architecture. Their model was evaluated on a custom dataset consisting of five weed species, both with and without transfer learning. The results demonstrated that the system is functional, achieving a 77% accuracy rate while detecting weeds at 62 FPS [19]. Rehman et al. developed a novel detection model based on the YOLOv5 architecture, providing a method for distinguishing between soybean plants and weeds. They compared the model’s performance against different YOLO versions and the transformer-based RT-DETR, reporting superior results with a mAP of 73.9 [20]. Li et al. developed a weed detection algorithm called YOLOv10n-FCDS, which identified Sagittaria trifolia, a common weed in rice fields, with an accuracy of 87.4%. Based on the obtained results, the rice fields were divided into sections, and customized spray prescriptions were formulated for each section at varying application rates [21].

Improving both detection accuracy and speed simultaneously in the models used by the researchers will positively affect the success of robotic systems. In addition, the development of weeding robot prototypes has progressed significantly, particularly with the use of machine vision for weed detection. Astrand and Baerveldt developed an autonomous agricultural mobile robot that discriminates between sugar beet and weeds with K-Nearest Neighbor classifiers using three features. Although the accuracy rate of the system seems to be satisfactory at 96%, the process of extracting individual plants out of a scene has been conducted manually [22]. Bawden et al. developed a modular agricultural robot platform with a mechanical and chemical weeding system based on machine vision. The performance of different color spaces in weed detection was investigated, and plant species were classified using both local binary patterns and covariance features. They preferred to use covariance features on the robot platform. During the training and testing phases, which were conducted using only 40 images, it was reported that the weed classification system was able to classify broad-leaved and grass plants with 96% accuracy and distinguish individual weed species with 92.3% accuracy [23].

In addition to academic research, company R&D departments are developing robotic agricultural platforms for weed control. One such example is BoniRob (Bosch, Baden-Württemberg, Germany) (Figure 1a) created by Deepfield Robotics (Germany), a Bosch-owned startup [24]. BoniRob can effectively eliminate weeds by burying them deep in the soil, achieving an accuracy rate of 80%. Ecorobotix has developed two different models in this field. The ARA model (Figure 1b) is designed as a high-precision sprayer used behind a tractor [25]. The AVO model (Figure 1c) is an autonomous system powered by solar energy and has the capacity to spray up to 10 hectares per day [26]. Carbon Robotic (Figure 1d) has developed an autonomous laser-powered weed management solution with its agricultural robot platform, arguing that conventional chemicals negatively affect soil health [27].

Agricultural research has shown that the use of autonomous and semi-autonomous robots can enhance productivity while also promoting healthy production and environmental sustainability. In this context, the integration of image processing-based deep learning methods into agricultural robots is expected to provide significant contributions.

State-of-the-art weed detection and removal robots are optimized for industrial use with advanced multispectral and hyperspectral cameras, LiDAR systems, high-performance processors, and durable mechanical components. Due to these features, they tend to be highly expensive. Academic research plays a crucial role in the development of commercial robots and paves the way for more efficient, cost-effective, and environmentally friendly solutions in the future.

Despite numerous academic studies on weed detection and destruction robots, ensuring that these systems operate with high accuracy and speed remains a challenging research area that requires further development. This study aims to contribute to this gap by developing a prototype of a low-cost weed detection and removal robot with an accuracy/speed balance by integrating an object-tracking algorithm into a new generation deep learning method.

The main objective of this study is to develop an autonomous agricultural robot capable of real-time differentiation between crops and weeds using the YOLOv8-ByteTrack deep learning (DL) algorithm without feature extraction. The low-cost prototype system successfully distinguished weeds in real time. Additionally, various YOLO deep learning models were compared, and the results were presented. One of the objectives of this study is to minimize the negative impact of chemical herbicides on soil and water resources by replacing chemical spraying with laser-based weed control. This approach contributes significantly to sustainable agricultural practices by targeting only the necessary areas for intervention.

2. Materials and Methods

The proposed autonomous agricultural robot is composed of two main parts: an object detection/tracking part and an autonomous navigation part. An overview of the autonomous agricultural robot for weed detection and its block diagram are presented in Figure 2 and Figure 3, respectively.

2.1. Autonomous Navigation

2.1.1. Differential Drive

The agricultural robot utilizes a differential drive system, which enables movement and rotation by varying the speed of its two motors. It is equipped with 36 V DC motors and a gearbox, along with an additional 1:2 ratio gearbox connected by a chain. The differential drive can be characterized by the robot’s X and Y positions on a plane and angular position, θ, as illustrated in Figure 4.

The position of a mobile robot can be described by two systems, inertial and robot coordinate. The inertial coordinate system is a global frame fixed in the environment in which the robot moves. The robot coordinate system is a local frame that is attached to the autonomous agricultural robot and moves with it. {XI, YI} Inertial fixed reference frame and {XR, YR} the relationship between the Robot Frame is represented by the following matrix. The expression P [x y θ] shows the position of the robot in the cartesian coordinate system [28]:

$${{\epsilon }^{.}}_R=R(\theta ){{\in }^{.}}_1$$

(1)

$$=R(\theta ).{\left[\begin{array}{ccc}·& ·& ·\\ x& y& \theta \\ \end{array}\right]}^T$$

(2)

$$R(\theta )=\left[\begin{array}{ccc}\cos \theta & \sin \theta & 0\\ -\sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right]$$

(3)

where (x, y) denotes the coordinates of the center point of the two wheels. Q denotes the orientation of the rear two movable wheels. v represents the linear velocity of the robot; ω represents the angular velocity of the robot and represents the traction wheel radius [29].

$$\left\{\begin{array}{l}v={\displaystyle \frac{r_R{\omega }_R+r_L{\omega }_L}{2}}\\ \omega ={\displaystyle \frac{r_R{\omega }_R-r_L{\omega }_L}{b}}\end{array}\right\}$$

(4)

where ${\omega }_R$ and ${\omega }_L$ are the speeds of the right and left wheel, $r_R$ and $r_L$ are the radii of the right and left wheel, respectively, and b is the distance between both drive wheels and the axis of symmetry. Equation (5) can be rewritten as follows [29]:

$$\left[\begin{array}{c}v\\ w\end{array}\right]=\left[\begin{array}{l}{\displaystyle \frac{r_{\mathrm{R}}}{2}} {\displaystyle \frac{r_R}{2}}\\ {\displaystyle \frac{r_R}{b}} {\displaystyle \frac{r_L}{b}}\end{array}\right]\left[\begin{array}{l}{\omega }_R\\ {\omega }_L\end{array}\right]$$

(5)

2.1.2. Localization and Mapping

Localization and mapping of the agricultural robot are based on the distributed framework of the ROS (Robot Operating System). As an open-source software library and framework, ROS supports the development of robot applications and can operate across multiple machines. In ROS, files compatible with the system are referred to as packages, and a collection of these packages is known as a stack; stacks can be shared and distributed among users. The ROS framework can be programmed using popular programming languages such as C++ 14 and Python 3.9. Firstly, the ROS compatible with the Linux operating system was installed on the system computer. Then, localization and mapping operations of the autonomous agricultural robot using 2D lidar were performed with the SLAM (Simultaneous Localization and Mapping) application via ROS. In this robot for autonomous navigation, we used packages such as Ros Serial (https://github.com/ros-drivers/rosserial), RF2O Package (https://github.com/MAPIRlab/rf2o_laser_odometry), Hector SLAM (https://github.com/tu-darmstadt-ros-pkg/hector_slam), Gmapping (https://github.com/ros-perception/slam_gmapping), and Navigation Stack (https://github.com/ros-planning/navigation, all accessed on 17 January 2025).

Lidar plays a critical role in the process of acquiring depth information by providing high measurement accuracy and fine angular resolution. Various studies have been conducted to enhance the 3D imaging performance of Lidar technology. For example, since obtaining a depth map requires multiple reconstruction calculations, CS-based dual-frequency laser 3D imaging methods, which work with only two calculations, have been preferred by researchers [30]. Additionally, Lidar and camera fusion-based systems have demonstrated effective performance in extracting 3D contours [31]. In this study, depth information between the agricultural robot and obstacles was obtained using Lidar, and this information proved to be sufficient for the robot to avoid obstacles and create a map.

Many SLAM algorithms can be utilized on the ROS platform. In the experiments conducted by Santos et al. [32], the Gmapping algorithm demonstrated a particularly low error rate and CPU overhead. This algorithm employs the Particle Filtering (PF) probabilistic approach, which is grounded in the Sequential Monte Carlo Technique for sampling particle distributions. The Sequential Monte Carlo Technique estimates the robot’s position using data gathered from sensors [33]. Additionally, the Gmapping algorithm integrates both scan matching and odometry information to reduce the number of particles. Figure 5 presents the flowchart of autonomous navigation.

2.2. Weed Detection and Tracking

2.2.1. YOLO (You Only Look Once)

YOLO (You Only Look Once) emerged in the field of computer vision with a paper published in 2015 by Joseph Redmon et al. [34]. YOLO is a one-stage deep learning algorithm using a convolutional neural network with a high success rate in accurately detecting objects in image and video data [35,36]. Its ability to perform detection with a single propagation makes it ideal for real-time applications requiring high detection speeds [37]. The YOLO algorithm decomposes the image and passes it through a convolutional neural network in a single pass, generating a result vector about the image. Initially, the entire image is divided into cells of size m × n. Each cell is analyzed by the convolutional neural network to detect objects within it and draw bounding boxes around them. During this process, the algorithm checks whether the object’s center point falls within the cell. If the midpoint of the object is inside the cell, the center, height, width, and confidence score (x, y, w, h, and a confidence score) of the detected object are calculated. To determine the best bounding box, the Intersection Over Union (IOU) metric is applied between the cells.

Following the release of the first YOLO version, YOLOv2 [38], YOLOv3 [39], and YOLOv4 [40], nine models were released in 2016, 2018, and 2020, respectively. In 2020, YOLOv5 [41] introduced by a company called Ultralytics, was released on a GitHub (https://github.com/) repository by the company’s founder and CEO, Glenn Jocher. In 2022, YOLOv6 [42] was developed by Meituan researchers, followed by YOLOv7 [43]. Finally, in 2023, YOLOv8 [44] was released by Ultralytics. After the end of the experimental tests of this study, YOLOv9 [45], YOLOv10 [46], and YOLOv11 [47] were published, respectively.

The loss function in the YOLO algorithm can be analyzed under three main categories: location loss, confidence loss, and classification loss. The sum of these losses constitutes the overall loss function. The formula is as follows [48]:

$$\begin{array}{l}{\lambda }_{coord}{\displaystyle \sum _{i=0}^{s^2}{\displaystyle \sum _{j=0}^B{\parallel }_{ij}^{obj}}}\left[{\left(x_i-\widehat{x_i}\right)}^2+{\left(y_i-\widehat{y_i}\right)}^2\right]\\ +{\lambda }_{coord}{\displaystyle \sum _{i=0}^{s^2}{\displaystyle \sum _{j=0}^B{\parallel }_{ij}^{obj}}}\left[{\left(\sqrt{{\omega }_i}-\sqrt{\widehat{{\omega }_i}}\right)}^2+{\left(\sqrt{h_i}-\sqrt{\widehat{h_i}}\right)}^2\right]\\ +{\displaystyle \sum _{i=0}^{s^2}{\displaystyle \sum _{j=0}^B{\parallel }_{ij}^{obj}}}{\left(C_i-\widehat{C_i}\right)}^2\\ +{\lambda }_{noobj}{\displaystyle \sum _{i=0}^{s^2}{\displaystyle \sum _{j=0}^B{\parallel }_{ij}^{nobj}}}{\left(C_i-\widehat{C_i}\right)}^2\\ +{\displaystyle \sum _{i=0}^{s^2}{\parallel }_i^{obj}{\displaystyle \sum _{c\in classes}{\left(p_i(c)-\widehat{p_i(c)}\right)}^2}}\end{array}$$

(6)

$${\lambda }_{coord}{\displaystyle \sum _{i=0}^{S^2}{\displaystyle \sum _{J=0}^B}}$$

(7)

The symbols x, y, w, h, c, and p define the center coordinates, width, height, confidence level, and category probability of the bounding box, respectively. The hat (ˆ) symbol in the formulae refers to the estimated values. λcoord is the highest in order to have more importance in the first term.

2.2.2. YOLO Models

This study is based on YOLOv5 and YOLOv8 algorithms for weed detection. YOLOv5 was inspired by YOLOv3 and YOLOv4 and the arithmetic set feature was added to provide a more effective detection. YOLOv5 uses the Auto Anchor algorithm instead of manually determined anchor boxes to increase the speed of the R-CNN algorithm. In addition, K-means are applied to the size of the bounding boxes to obtain a better priority value. The YOLOv5 architecture consists of three main components: Backbone, Neck, and Head. The architecture of YOLOv5 is shown in Figure 6.

YOLOv8 features a backbone akin to the C2f module found in YOLOv5, with some modifications to the CSPLayer. This model employs an anchor-free detector that processes box prediction, classification, and regression tasks independently. The output layer of YOLOv8 utilizes a sigmoid activation function to generate the objectivity score, indicating the probability of an object being present within the bounding box [29,41]. Additionally, it adopts CioU for bounding box loss and binary cross-entropy for classification loss, enhancing the detection of small objects [49]. Figure 7 illustrates the architecture of YOLOv8.

2.2.3. Pruning and Quantization

DeepSparse is a CPU inference tool that reduces the computation time required to increase the inference speed of the neural network. Due to this advantage, DeepSparse has been stated to offer a 5.8 times speedup for YOLOv5s running on the same system [47]. Pruning reduces the number of parameters in the model by removing less significant weights, while quantization compresses the model by reducing the bit precision of weights and activations (e.g., from 32-bit floating-point to 8-bit integer). These modifications balance model size, processing speed, and detection accuracy. While a slight reduction in accuracy is observed, the model continues to maintain a high detection rate. This indicates that the effectiveness of pruning and quantization is not significantly compromised. One of the greatest advantages of pruning and quantization is the notable increase in inference speed. This improvement in processing speed enhances the system’s real-time capabilities, allowing the robot to process more frames per second and make faster decisions an essential factor for dynamic agricultural environments. Pruning and quantization techniques can be applied in YOLOv5 versions to increase the extraction speed without loss of accuracy. In this study, quantization and pooling operations are performed on the lean YOLOv5 model trained on our dataset using the SparseML library (https://github.com/neuralmagic/sparseml, accessed on 17 January 2025). In addition, the performance of the different sparse models (quantization and pooling) obtained through transfer learning is also compared.

2.2.4. Weed Tracking with ByteTrack

Multi-object tracking algorithms are essential for detecting and effectively tracking the movements of multiple objects. They play a crucial role in machine vision, particularly in monitoring several objects in recorded images or live streams. Commonly used methods in object tracking algorithms include ByteTrack, SORT, DeepSORT, and StrongSORT [50].

The ByteTrack object tracking algorithm has low hardware requirements due to its simple structure, which enables fast processing. By incorporating objects with low confidence scores into its evaluation, it enhances overall tracking accuracy. This capability allows ByteTrack to achieve an effective balance between tracking speed and accuracy [51]. For example, ByteTrack consistently outperformed SORT and DeepSORT in detecting vehicles and people in highway timelapse videos [52]. Moreover, object tracking algorithms such as DeepSORT, StrongSORT, DeepOCSORT, and BOT-SORT employ the Re-Identification (ReID) method, which significantly reduces the network’s processing speed [51].

As a result of the literature review, the ByteTrack algorithm, proposed by Zhang et al. [53]. In 2021, was selected as the object-tracking algorithm for this study. ByteTrack differentiates itself in the field of object tracking, treating each detection as a byte. This algorithm allows objects with low detection scores to be recovered through a secondary association pass, without being limited to high-scoring bins, thus providing a more robust association process. ByteTrack’s simple structure is a key advantage; it relies on precise detections and byte-level association, avoiding complex appearance and motion models [54]. ByteTrack workflow is shown in Figure 8.

2.3. Metrics

In order to evaluate the accuracy of the YOLO models used in this study, several important metrics were determined: recall, precision, and mean average precision (mAP). In addition, in real-time systems, the frame rate (FPS—Frames Per Second) is a factor that directly affects system performance. While increasing the model performance, it is also important that the FPS value ensures the sustainability of the real-time system.

Sensitivity measures the model’s ability to accurately predict true positive values, while precision refers to the rate at which all classes are correctly predicted. The mAP indicates the accuracy score between the true bounding box and the detected box. A high score indicates that the model provides high accuracy in its detections.

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(8)

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(9)

$$F1score=2\times {\displaystyle \frac{P\times R}{P+R}}$$

(10)

$$AP={\displaystyle \underset{0}{\overset{1}{\int }}P(R)dR}$$

(11)

$$mAP={\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}AP(i)}}{n}}$$

(12)

True positives (TP) refer to instances where both the predicted results and the ground truth values are positive samples. False positives (FP) occur when the predicted result is positive, but the ground truth value is negative. False negatives (FN) indicate cases where the predictions are negative, while the ground truth is positive. AP(i) refers to the average accuracy for the current species, and n signifies the number of categories [56].

3. Results and Discussion

The dataset used in this study was obtained from the Roboflow Weeds Dataset, a publicly available dataset that includes images of different weed species captured in diverse field conditions. The dataset contains 3926 images of weeds commonly found in agricultural settings [57]. An additional 200 images of weeds were incorporated into the dataset to enhance the model’s performance in its intended environment. In all datasets, Dandelion, Heliotropium indicum, Young Field Thistle (Cirsium arvense), Plantago lanceolata, Eclipta, and Urtica dioica were identified as significant weed species, and examples of images are depicted in Figure 9.

To ensure the robustness of the deep learning model, images were collected under various conditions. The dataset includes images taken in bright sunlight, partial shade, and low-light conditions, simulating real-world agricultural environments. Some images were captured under dry, wet, and cloudy conditions, which affect the contrast and appearance of weeds. Images include different angles, soil textures, and crop backgrounds to improve model generalization.

Augmentation techniques were used to further enhance the dataset’s variability and improve the model’s ability to generalize, including brightness and contrast adjustment, rotation and flipping, Gaussian noise injection, scaling and cropping, and color jittering. These augmentations were performed using Roboflow’s built-in augmentation pipeline and additional processing in Python using the albumentations library.

The implementation of these techniques will positively impact the three key performance aspects of deep learning algorithms. The use of diverse lighting conditions and augmentations helps the model recognize weeds in different scenarios, reducing overfitting. By simulating real-world field conditions, the model can effectively differentiate weeds from crops, even under challenging circumstances. Data augmentation improves mAP scores by exposing the model to multiple variations, making it more resilient to unseen images.

In this experimental study, six different YOLO models (the YOLOv5, the YOLOv5 Pruned and Quantized, the YOLOv5 Pruned and Quantized with Transfer Learning, the YOLOv8, the YOLOv9, and the YOLOv11) were trained and tested for the detection of seven different weed species (Dandelion weeds (Figure 9a), Heliotropium indicum (Figure 9b), Young field thistle cirsium arvense (Figure 9c), Cirsium arvense (Figure 9d), Plantago lanceolata (Figure 9e), Eclipta (Figure 9f), and Urtica diocia (Figure 9g)). The training of the models was performed over 100 epochs.

Results of YOLO Models

When the results of the YOLOv5 model are analyzed, the mAP@0.5 value is 0.930, the recall value is 0.884, and the precision value is 0.938. When the model was run on the CPU, it reached approximately 15 fps., which can be seen in Figure 10.

The model can be optimized and pre-diluted (pruned and quantified) by adapting it to the dataset via SparseML. This process is called Sparse Transfer Learning. The YOLOv5 Pruned and Quantized with Deepsparse achieved an mAP@0.5 value of 0.909, a recall value of 0.834, and a precision value of 0.921. Figure 11a shows that the fps reached 47.1 on the CPU. The YOLOv5 model was pruned and quantized to improve fps. As a result, the fps increased to 116 (Figure 11b) on the CPU, while mAP, precision, and recall values remained largely unchanged. The mAP@0.5 value for this model is 0.909, the recall value is 0.835, and the precision value is 0.924.

Figure 12a shows the precision curve for YOLO models: the YOLOv5, the YOLOv5 Pruned and Quantized, and the YOLOv5 Pruned and Quantized with Transfer Learning models. When pruning and quantification were applied to the YOLOv5 model, a decrease in the positive predictive value was observed. The recall value, defined as the probability of detection, is higher in the YOLOv5 model compared to the other two models. Change in recall values in epochs can be seen in Figure 12b.

Mean Average Precision (mAP) is essential for model evaluation because it offers a comprehensive metric that encompasses both precision and recall across various weed classes. The mAP@0.5 (Figure 13a) evaluates the ability of models to accurately detect weeds by measuring sensitivity at a threshold of 0.5 IoU. On the other hand, mAP@0.5:0.95 (Figure 13b) comprehensively evaluates the detection of weeds by averaging the sensitivities obtained at various IoU thresholds. The mAP@0.5:0.95 value for the YOLOv5 without pruning and quantization is 0.6473. In contrast, the pruned and quantized model with transfer learning has an mAP@0.5:0.95 value of 0.587. Finally, the pruned and quantized model has an mAP@0.5:0.95 value of 0.571.

Despite parameter reduction, it was observed that pruned and quantized models did not have a significant negative impact on robustness. Overall, the models maintained high detection accuracy despite variations in field conditions, lighting, and weed species.

The developed agricultural autonomous robot uses a model trained with YOLOv8 for object tracking. The purpose of the tracking algorithms is not to reevaluate the weed image that has completed the detection process. It is seen as an important process to burn the areas identified as weeds with laser and to avoid wasting time by burning this area again. SORT, DeepSORT, ByteTrack, and BotSort applications were tested for object tracking in this model. As a result of the comparisons made by the trial and error method, the ByteTrack application showed the best performance. Sort and BotSort tracking algorithms were not preferred because they are outdated and increase the processing time. For this reason, YOLOv8 ByteTrack, the tracking algorithm of the YOLO model, was used.

The performance of YOLOv8 is shown in Figure 14. The mAP@0.5 value of this model is 0.921, the recall rate is 0.865, and the precision value is 0.938. This model, running on a GPU, performed at an average speed of 18 fps.

YOLOv9 introduces modifications to the network architecture, particularly in feature extraction and optimization strategies, aiming to improve detection speed while maintaining high accuracy. The mAP@0.5 value of this model is 0.893, the recall rate is 0.803, and the precision value is 0.905. The model performed well in detecting weeds but exhibited a slightly lower recall, indicating a higher rate of false negatives compared to other models. Despite this, it maintained a strong balance between precision and detection speed.

YOLOv11 builds on advancements from previous YOLO versions, incorporating improved object localization techniques and an enhanced detection backbone. The mAP@0.5 value of this model is 0.926, the recall rate is 0.870, and the precision value is 0.915. The model demonstrated strong performance in weed detection in terms of both precision and detection speed. However, compared to other models, except for YOLOv9, it exhibited slightly lower precision due to a higher false positive rate. The YOLO models were tested with the generated data set. The performance comparisons of these models are given in

Table 1. Results of the comparative performance analysis of YOLO models.

Model	mAP	Precision	Recall	Fps
YOLOv5	0.930	0.938	0.884	15
YOLOv5 Pruned and Quan. with Tf.	0.909	0.921	0.834	47
YOLOv5 Pruned and Quan.	0.909	0.924	0.835	116
YOLOv8 ByteTrack	0.921	0.938	0.865	18
YOLOv9	0.893	0.905	0.803	21
YOLOv11	0.926	0.915	0.870	24

.

The models presented in Table 1 were tested on a laptop equipped with an NVIDIA GTX 1650 (4GB VRAM) (NVIDIA Corporation, Location: Santa Clara, CA, USA) and an Intel i5 processor with 16GB RAM (Intel Corporation, Location: Santa Clara, CA, USA). Comparing these models based only on performance metrics can lead to errors in real-time systems. Therefore, the detection speed (fps) of the model is also important for system performance. To enhance the detection accuracy, pruned and quantized processes were implemented, resulting in a substantial increase in fps from 15 to 116. However, this improvement came at the cost of a reduction in the model’s detection accuracy. As a result, considering both accuracy and speed performance, YOLOv8 was preferred for use in the autonomous agricultural robot system. Additionally, the integration of YOLOv8 with the ByteTrack object tracking feature further supports this decision. YOLOv8-ByteTrack achieved a frame rate of 18 FPS, meaning that processing a new frame took an average of 55 ms. When head tracking was performed using ByteTrack, an additional latency of approximately 5 ms per frame was introduced, resulting in a total delay of around 60 ms. In tests conducted solely on the CPU (Intel i5 without GPU acceleration), the FPS dropped to a range of 4–6, significantly limiting the practicality of real-time deployment on a CPU. In terms of computational costs, running YOLOv8-ByteTrack on the GTX 1650 utilized approximately 75–85% of the GPU resources. Memory consumption ranged between 3 GB VRAM and 5–7 GB RAM. Under load, the system’s power consumption was estimated to be around 40 W–50 W.

Some recent studies have been carried out on the detection weed performance of models such as Support Vector Machine (SVM) [10], YOLO [35], Faster R-CNN [58], and RetinaNet [59]. While considerable efforts have been made on weed detection, very few studies were focused on the detection and speed performance together. This study sets itself apart from previous research by not only comparing model performances based on existing or acquired images but also by implementing the proposed model within a machine vision hardware prototype.

The deep learning model based on ByteTrack and YOLOv8 is designed for generalization across multiple environments. However, its adaptation to new agricultural settings can be further improved through several approaches. First, the model can be retrained using novel datasets of weeds and crops specific to a particular region or agricultural application. Alternatively, transfer learning can be utilized to leverage the pre-trained model without the need for full retraining, enabling the identification of new weed species. Transfer learning allows the model to efficiently adapt to new weed types or different crop species without requiring training from scratch. For instance, if the model was initially trained to detect seven different weed species, it can be fine-tuned using new images to accommodate fields with different weed compositions while preserving learned representations of plant structures. If a new weed species emerges in the field, a small, labeled dataset containing images of this weed can be used for incremental learning to enhance recognition. Instead of retraining the entire network, transfer learning enables fine-tuning of only the last few layers, making adaptation more efficient.

4. Conclusions

Weeds can negatively impact crop development, so it is important to keep them under control. However, the presence of various weed species and the resemblance of some species to crops complicate the identification process. This study integrates ByteTrack, an effective object-tracking system for the detection and eradication of weeds in agricultural areas by considering seven different weed species under a single category with the deep learning algorithm YOLOv8 in a hardware prototype. The experimental results indicate that the ByteTrack-YOLOv8 model achieved a detection speed of 18 FPS and a mAP@0.5 value of 92.1%. This model was compared with six different YOLO models using the same dataset. The proposed model achieved a detection accuracy slightly lower than YOLOv5. However, for this study, the YOLOv8 model was favored due to detection speed and object tracking capabilities. Although modified YOLOv5 models did improve detection speed, this enhancement came at the expense of detection accuracy, making them less desirable. As a result, the ByteTrack-YOLOv8 model provides satisfactory results in weed detection and tracking, along with strong hardware compatibility.

One of the primary limitations of this study is the high computational power required by the YOLO model due to its operation with large-scale datasets. This challenge has made it difficult to run the model on the embedded system (Jetson Nano). Additionally, the model used in the agricultural robot was developed to recognize seven common weed species. However, the dominant weed species may vary across different geographical regions. Therefore, to ensure the model adapts to local weed species, it must be retrained using an appropriate dataset that accounts for variations in weather and lighting conditions.

Agricultural robots can detect not only weeds but also harmful insects, diseases, and the ripeness level of crops. This enables the development of a more efficient and sustainable agricultural system. Furthermore, the integration of artificial intelligence and agricultural robot technologies, as demonstrated in this study, facilitates the development of environmentally friendly applications.

In future studies, it is recommended to use BLDC motors instead of DC motors to enhance the autonomous driving performance of agricultural robots. Additionally, integrating the IMU with wheel encoders and the Extended Kalman Filter will allow for more accurate odometry readings. Along with these hardware upgrades, several approaches can be applied to improve the performance of YOLO, such as enhancing the model architecture and optimizing the training processes. These improvements could make YOLO more effective, especially in real-time and large-scale applications.