Research Status and Development Trends of Deep Reinforcement Learning in the Intelligent Transformation of Agricultural Machinery

Abstract

With the acceleration of agricultural intelligent transformation, deep reinforcement learning (DRL), leveraging its adaptive perception and decision-making capabilities in complex environments, has emerged as a pivotal technology in advancing the intelligent upgrade of agricultural machinery and equipment. For example, in UAV path optimization, DRL can help UAVs plan more efficient flight paths to cover more areas in less time. To enhance the systematicity and credibility of this review, this paper systematically examines the application status, key issues, and development trends of DRL in agricultural scenarios, based on the research literature from mainstream Chinese and English databases spanning from 2018 to 2024. From the perspective of algorithm–hardware synergy, the article provides an in-depth analysis of DRL’s specific applications in agricultural ground platform navigation, path planning for intelligent agricultural end-effectors, and autonomous operations of low-altitude unmanned aerial vehicles. It highlights the technical advantages of DRL by integrating typical experimental outcomes, such as improved path-tracking accuracy and optimized spraying coverage. Meanwhile, this paper identifies three major challenges facing DRL in agricultural contexts: the difficulty of dynamic path planning in unstructured environments, constraints imposed by edge computing resources on algorithmic real-time performance, and risks to policy reliability and safety under human–machine collaboration conditions. Looking forward, the DRL-driven smart transformation of agricultural machinery will focus on three key aspects: (1) The first aspect is developing a hybrid decision-making architecture based on model predictive control (MPC). This aims to enhance the strategic stability and decision-making interpretability of agricultural machinery (like unmanned tractors, harvesters, and drones) in complex and dynamic field environments. This is essential for ensuring the safe and reliable autonomous operation of machinery. (2) The second aspect is designing lightweight models that support edge-cloud collaborative deployment. This can meet the requirements of low-latency responses and low-power operation in edge computing scenarios during field operations, providing computational power for the real-time intelligent decision-making of machinery. (3) The third aspect is integrating meta-learning with self-supervised mechanisms. This helps improve the algorithm’s fast generalization ability across different crop types, climates, and geographical regions, ensuring the smart agricultural machinery system has broad adaptability and robustness and accelerating its application in various agricultural settings. This paper proposes research directions from three key dimensions-“algorithm capability enhancement, deployment architecture optimization, and generalization ability improvement”-offering theoretical references and practical pathways for the continuous evolution of intelligent agricultural equipment.

1. Introduction

With advancements in cloud positioning and collaborative precision positioning (collaborative precision positioning refers to the technology permitting multiple positioning devices or systems to work together to achieve more accurate positioning information) technologies, agricultural automation is transitioning from single-dimensional navigation tasks to multidimensional intelligent operations, significantly reducing labor intensity and enhancing production efficiency [1,2,3]. Modern agricultural automation has evolved under the impetus of precision agriculture [4], which is closely tied to efficient and intelligent navigation systems for agricultural machinery. High-precision positioning and navigation are prerequisites for agricultural robots to perform autonomous field operations [5]. During navigation, robots must first determine their absolute or relative positions to facilitate subsequent path planning and trajectory tasks [6,7]. Unlike navigation scenarios for pedestrians, mobile robots, vehicles, or UAVs [8,9,10,11,12], agricultural environments feature unstructured terrain, including uneven soil, dynamic crop distributions (e.g., varying plant heights during growth cycles), and transient obstacles (e.g., fallen branches or irrigation tools); unmanned agricultural machinery imposes higher demands on navigation due to operational speed and equipment action requirements [13]. Consequently, there is a pressing need to develop instruction-based navigation methods for agricultural machinery to achieve efficient full-field coverage [14]. Current autonomous driving technologies for agricultural machinery, leveraging high-precision positioning and path planning, have markedly improved efficiency in tasks such as plowing and sowing. Furthermore, meeting the diverse demands of delicate operations, such as fruit picking and field weeding, relies on the flexible control of intelligent agricultural end-effectors. Recent breakthroughs in sensor technology, materials science, and computer science have driven innovations in decision-making, perception and localization, structural optimization, intelligent control, and operational management for these end-effectors, which are critical to enhancing harvesting efficiency [15,16,17,18]. Additionally, the application of low-altitude UAV technology in agriculture has emerged as a transformative practice, injecting new momentum into agricultural production through data-driven models [19]. In scenarios such as precision pesticide spraying and crop growth monitoring, UAVs have demonstrated remarkable advantages in autonomous decision-making [20]. However, the unique complexity of agricultural production environments poses multidimensional challenges to traditional automation technologies:

(1)

Dynamic environmental factors (e.g., changing crop distributions, sudden obstacles) demand real-time perception and adaptive responses.

(2)

Hybrid task requirements (e.g., coupling continuous control with discrete decision-making) necessitate the development of hybrid intelligent decision systems.

(3)

Resource constraints (e.g., limited computational power and energy consumption) impose stricter demands on system efficiency.

In agricultural environments, traditional methods struggle to handle real-time environmental changes and are highly sensitive to sensor noise and uncertainties. To address these limitations, researchers are actively exploring the adaptability of intelligent algorithms in agricultural scenarios. While deep learning (DL), a representative perception intelligence technology, excels in feature extraction and complex data processing, its reliance on extensive training data and high computational costs in dynamic environments (e.g., orchards and crop fields) limit its practicality. In contrast, reinforcement learning (RL), which relies on environmental interaction and trial-and-error learning, demonstrates unique advantages in addressing dynamic and stochastic challenges [21,22,23]. Nevertheless, traditional RL algorithms face dimensional catastrophe in real-world control tasks due to constrained action and sample spaces in complex agricultural environments (e.g., narrow passages, dense crop obstructions, and dynamic obstacles). To bridge these gaps, researchers proposed deep reinforcement learning (DRL) [24], which integrates the perceptual capabilities of DL with the decision-making strengths of RL in unknown environments. In summary, compared to traditional rule-based automation, DRL overcomes the rigidity limitations of rule systems in real-time environmental changes by integrating dynamic perception with autonomous decision-making. It can adaptively address the complexity, uncertainty, and resource constraints of agricultural scenarios, achieving more flexible and efficient intelligent decision-making. DRL, through the deep coupling of perception and decision-making, offers innovative strategies to resolve the dilemma of “environmental dynamism, task complexity, and resource constraints” in agricultural scenarios.

This paper reviews the recent literature to provide a comprehensive understanding of DRL advancements in agricultural production environments. We first introduce DRL fundamentals and classifications, then analyze its applications, challenges, and future directions, aiming to guide further research and deployment of DRL in agriculture.

2. Deep Reinforcement Learning Technology Framework and Algorithm Architecture

2.1. Deep Reinforcement Learning Algorithm Framework

DRL, the integration of deep learning (DL) and reinforcement learning (RL), aims to address intelligent decision-making in complex environments. Its core lies in end-to-end learning, which eliminates the need for labeled data. Instead, agents learn action policies by interacting with the environment through raw input information, refining strategies via trial-and-error to develop highly adaptive intelligence [25].

DRL encompasses a wide range of algorithms [26], including classical frameworks such as Deep Q-Networks (DQNs) [27], Deep Deterministic Policy Gradient (DDPG) [28], and Twin Delayed Deep Deterministic Policy Gradient (TD3) [29]. DQNs use deep networks to handle high-dimensional states (limited to discrete actions); DDPG outputs continuous actions via Actor–Critic but suffers from overestimation; TD3 introduces dual Critics, delayed updates, and policy smoothing to reduce variance and error, resulting in greater stability. Depending on whether environmental models are explicitly constructed during training, DRL algorithms can be categorized into model-based and model-free approaches [30,31,32,33] (Figure 1) (

Table 1. Comparison of model-based vs. model-free deep reinforcement learning algorithms.

Attribute	Model-Based DRL	Model-Free DRL
Core Concept	Builds a model of environment dynamics for policy optimization	Learns optimal policy directly through interaction with environment
Sample Efficiency	High—can generate abundant data via the learned model	Low—requires large amounts of real-world interaction
Generalization Ability	Strong—suitable for transfer and multi-task learning	Moderate—relies on experience replay or exploration mechanisms
Typical Application Scenarios	Structured environments (e.g., greenhouse climate control)	Complex, unstructured environments (e.g., field navigation)
Representative Algorithms	PILCO [37], MBPO [38], Dreamer [39]	DQN, DDPG, TD3
Major Challenges	Model error accumulation, increased training complexity	High data demand, unstable convergence
Agricultural Use Cases	Crop growth modeling, irrigation/fertilization control	Tractor path tracking, UAV spraying missions

). Model-based DRL algorithms seek optimal policies based on learned environmental dynamics, encompassing methods like fine-tuning algorithms [34] and augmented intelligence frameworks [35]. In contrast, model-free DRL algorithms acquire optimal policies through direct agent–environment interactions, broadly divided into value function-based and policy gradient-based methods [36]. Compared to model-based approaches, model-free DRL avoids constructing explicit environmental dynamics, learning directly from interactions, making it more suitable for high-dimensional, dynamic, or unknown agricultural scenarios. This section focuses on model-free DRL algorithms, specifically value function and policy gradient variants.

2.1.1. Value Function-Based DRL Algorithms

The DQN algorithm, a widely adopted RL variant, has been successfully applied to robot path planning and similar fields [40,41,42]. In DQNs, agents utilize neural networks to approximate the optimal action value function, mapping states to the expected long-term rewards of actions. The network is trained through a variant of Q-learning, where agents learn from state transitions and rewards [43,44,45]. When applying a DQN to robot path planning, agents are first trained on a set of sample environments. During training, agents explore the environment and learn to select optimal actions for each state. Performance is evaluated based on their ability to navigate to target locations while avoiding obstacles [46,47,48]. However, a traditional DQN’s single-network architecture is prone to Q-value overestimation, potentially destabilizing policy convergence. This limitation has driven researchers to propose improved frameworks like Double Q-learning.

To mitigate overfitting caused by a single Q-network, Hasselt et al. [49] introduced a Double Q-learning structure combined with deep learning, outperforming standard DQNs. Building on Double-DQNs, Zhang et al. [50] developed a path smoothing and tracking control method capable of tracking linear and polygonal paths. Compared to traditional pure pursuit control (PPC), their approach significantly reduced stabilization time and corner overshoot in high-speed scenarios. Yue et al. [51] employed a Double-DQN architecture to design a reinforcement learning-based obstacle avoidance controller for agricultural robots, achieving efficient and reliable autonomous navigation in complex farm environments. Zhigang Ren et al. [52] implemented a Double-DQN-based path-tracking control algorithm for orchard traction spray robots, demonstrating superior path-tracking accuracy and stability compared to conventional control algorithms. These studies confirm that value function-based DRL algorithms effectively reduce errors caused by inaccurate environmental modeling in agriculture, thereby enhancing operational efficiency.

2.1.2. Policy Gradient-Based DRL Algorithms

Policy gradient-based DRL algorithms primarily include DDPG, Trust Region Policy Optimization (TRPO) [53], and Asynchronous Advantage Actor–Critic (A3C) [54] (Figure 2). The core idea of these algorithms is to adjust policy parameters via gradient ascent to maximize the expected long-term cumulative reward. Owing to their flexibility and adaptability, policy gradient-based DRL has become a critical branch of DRL, particularly excelling in complex control tasks.

Designing effective reward mechanisms and strategies for multi-agent collaboration and competition remains challenging due to intricate relationships between agents. For instance, Hu et al. [55] enhanced the DDPG algorithm by designing reward functions based on COLREGs (Collision Avoidance Regulations) and introducing potential-based reward shaping to guide agents toward optimal policies. Takahashi et al. [56] proposed a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) method to learn near-optimal solutions from prior knowledge. Ye et al. [57] integrated an ε-greedy exploration strategy into MADDPG to balance exploration and exploitation during training, accelerating convergence and improving performance. Dynamic environments pose another significant challenge, as traditional deep learning models trained in static settings struggle to adapt to real-time changes. To address this, Chen et al. [58] improved the reward–penalty mechanism of DDPG by incorporating adaptive artificial potential fields (APF-DDPG). This enhancement boosted learning speed, obstacle avoidance success rates, and continuous information utilization during training. For efficient computational resource utilization, Yoko Sasaki et al. [59] developed an A3C-based motion learning algorithm for autonomous mobile robots, reducing training costs and simplifying collision avoidance design. In summary, researchers have made notable progress in reward design for multi-agent systems, policy stability, and dynamic environment adaptability. These advancements not only enhance learning efficiency and decision-making precision in complex scenarios but also provide foundational insights for future explorations in multi-objective collaborative optimization and robust policy design under non-stationary conditions.

Compared to value function-based DRL algorithms, policy gradient-based approaches directly search for optimal policies in the policy space through end-to-end learning. Their simpler architecture makes them more suitable for handling continuous and high-dimensional action spaces.

3. Applications of DRL in Agricultural Production Environments

With the rapid advancement of artificial intelligence, DRL has been widely adopted in agricultural production environments due to its robust learning and decision-making capabilities [60]. For instance, DRL-powered agricultural transplanting robots have reduced mechanical seedling damage rates to 2.82% through multi-agent collaborative path optimization. Meanwhile, DRL-enhanced plant protection drones achieved a 41.68% improvement in pesticide deposition coverage rate while reducing operational path overlap to 5.56%, enabled by dynamic environment modeling and action space adaptation. This section explores its applications in three key areas: agricultural ground platform navigation, motion planning for intelligent agricultural end-effectors, and low-altitude UAV operations, highlighting the technical strengths and operational advantages of smart agricultural systems to support the development of an intelligent agricultural ecosystem.

3.1. Agricultural Ground Platform Navigation

3.1.1. Application Background

Agricultural ground platforms, including autonomous tractors and combine harvesters, face significant navigation challenges in unstructured farmland environments characterized by uneven crop distributions, soil undulations, and transient obstacles. These dynamic, unstructured environments demand robust navigation systems to handle variable conditions while improving operational efficiency and reducing energy consumption. Although traditional navigation technologies have made progress through sensor fusion and algorithm enhancements, their inherent limitations become evident in agricultural scenarios. For example, Freitas et al. [61] developed an orchard obstacle detection system that identifies obstacles over 15 cm when unobscured by grass and adjusts vehicle speed based on proximity yet fails to detect smaller obscured obstacles. Blok et al. [62] achieved lateral deviations below 5 cm at 0.5 m/s speeds using a particle filter with laser beam modeling but showed limited adaptability in high-speed or sudden obstacle scenarios. Wang et al. [63] improved the A*-SVR method to reduce lateral deviations by 29.57% to 6.90 cm between tree rows, though performance degrades significantly under model mismatches due to reliance on precise environmental modeling. These methods generally suffer from static environmental perception and rigid algorithmic adjustments, struggling to adapt to real-time agricultural dynamics. In contrast, DRL overcomes dependence on preset models through autonomous environmental interaction and policy optimization, offering a novel pathway to break through traditional navigation limitations in unstructured agricultural settings.

3.1.2. DRL-Based Solution and Key Advantages

In the field of precision agriculture navigation, traditional end-to-end decision-making frameworks based on multi-sensor fusion (e.g., vision/LiDAR/GNSS integration) can achieve basic path tracking but face challenges such as complex parameter tuning and insufficient environmental adaptability, particularly under dynamic scenarios like uneven crop distributions, undulating terrain, or sudden obstacles [64]. To address these complex control issues, DRL is emerging as a core technology for intelligent navigation in agricultural machinery, leveraging its autonomous decision-making advantages through environmental interactions. In agricultural ground vehicle control, innovations in DRL algorithms primarily focus on dynamic performance optimization and system lightweighting. For instance, Zhang et al. [50] pioneered the application of Double-DQNs to agricultural vehicle navigation. By integrating nonlinear normalization and reward function design, they optimized dynamic path-tracking performance, overcoming the limitations of traditional methods in sharp-turning and high-speed scenarios. Yan et al. [65] proposed a lightweight, portable DRL control algorithm. By incorporating path curvature states, the algorithm enhanced control accuracy in curved and high-curvature paths, demonstrating centimeter-level precision and robust performance in both simulations and field trials. This method has been successfully applied to unmanned agricultural machinery for straight-line operations and headland turning. To meet the multidimensional demands of smart agriculture, DRL technology is expanding into multi-agent collaboration and specialized environmental adaptation. Examples of this include the following: Fan et al. [66] developed an SMO-Rainbow strategy for UAV path planning in smart tourism agriculture. By combining hierarchical reinforcement learning (HRL) with techniques like Double-DQN and Dueling DQN, they reduced model complexity and improved performance, addressing inefficiencies in data collection, training complexity, and dynamic adaptability. Hu [67] introduced a DRL-based navigation method for orchard inspection robots using the Soft Actor–Critic (SAC) algorithm. Ordered stochastic curriculum learning was employed to resolve GPS inaccuracy, sparse rewards, and environmental adaptability challenges. Furthermore, DRL is demonstrating significant potential beyond physical navigation, extending into the core decision-making and resource optimization layers of smart agriculture systems. Devarajan et al. [68] proposed a two-stage DRL framework (DONSA$/MACO-DQN + RL-DQN) for intelligent agricultural management. The first stage employs an Ant Colony Optimization-enhanced Deep Q-Network (MACO-DQN) to optimally offload diverse monitoring tasks (e.g., fire/pest detection, irrigation scheduling, soil/climate monitoring) to edge, fog, or cloud computing nodes based on latency, energy consumption, and computing power. The second stage utilizes a reinforcement learning-enabled DQN (RL-DQN) model for the actual prediction and monitoring of these agricultural activities. This approach demonstrated superior performance (98.5% precision, 99.1% recall, 98.1% F-measure, and 98.5% accuracy) and faster convergence compared to traditional methods, highlighting DRL’s capability in optimizing complex system-level tasks and resource allocation within the smart agriculture paradigm.

Table 2. Comparison of performance of agricultural machinery path planning methods.

Method	Path Accuracy	Adaptability	Computational Load	Suitable Terr
PPC	Medium	Low	Low	Flat Fields
MPC	High	Medium	High	Greenhouses
DRL(DQN,PPO)	High	High	Medium	Unstructured fields

compares agricultural machinery navigation methods. Next, we introduce several typical agricultural ground platform algorithms:

• Double-DQN algorithm (Figure 3a): The Double-DQN is an enhanced algorithm developed from the foundation of DQNs, with the primary objective of addressing the issue of Q-value overestimation inherent in DQNs [49]. The central principle involves decoupling the evaluation of Q-values from the selection of actions, employing two distinct neural networks to execute these tasks independently. This separation enhances the precision of Q-value estimation, thereby enabling the agent to learn an optimal policy with greater stability throughout the decision-making process. As depicted in the leftmost input section of the accompanying figure, the input comprises speed and two error terms. These elements collectively characterize the agent’s current environmental state, serving as the foundational basis for subsequent decision-making. The central component of the figure, the deep network, constitutes the critical processing unit of the Double-DQN. This network accepts the input state and processes it through a multi-layer neural network architecture, performing feature extraction and transformation to map the relationship between states and Q-values. The network’s parameters are iteratively updated to progressively approximate the true Q-value function. In the output phase, the action yielding the highest Q-value is designated as the final decision action. This selection is executed via a “max” operation, identifying the optimal action that drives the agent to perform the corresponding operation within the environment, subsequently receiving a reward, and transitioning to the next state. The figure provides a clear illustration of this workflow: from the input state, through deep network processing to derive Q-values, to the ultimate selection of actions based on these Q-values. This visualization encapsulates the fundamental operational framework and decision-making paradigm of Double-DQNs.
• Soft Actor–Critic (SAC) algorithm (Figure 3b): The SAC algorithm is a policy gradient method rooted in maximum entropy reinforcement learning [69]. Its core principle involves not only pursuing high cumulative rewards during policy optimization but also maximizing the entropy (uncertainty) of the policy. This encourages the agent to thoroughly explore the environment while maintaining effective control. During training, the agent executes actions determined by the current policy (output by the Actor network), interacts with the environment, and collects experiential data (such as states, actions, and rewards), which is stored in a replay buffer. Data sampled from this replay buffer is then used to update the networks.
• SMO-Rainbow strategy (Figure 3c): The rainbow algorithm integrates multiple improved algorithms into the DQN algorithm [70]. The network architecture uses a fully connected neural network with two hidden layers and a dueling layer, as shown in Figure 2. The network input is the environment state S, with 1024 and 512 neurons in the hidden layers, respectively. The dueling layer includes both a value function network and an advantage function network, which together output the predicted Q-values for each option. The hidden layers and the dueling layer use ReLU (rectified linear unit) activation functions, and the loss function is Huber loss.

These studies collectively reveal the technological breakthrough trajectory of DRL in agricultural navigation: through adaptive state representation construction (such as the introduction of curvature parameters), hierarchical policy optimization (such as the SMO-Rainbow framework), and innovative training mechanisms (such as curriculum learning), they systematically address inherent deficiencies of traditional methods in dynamic environment perception, multi-objective optimization, and hardware adaptation. With the integration and evolution of lightweight deployment and edge computing technologies, DRL-driven high-precision navigation is reshaping the operational paradigms of precision agriculture, providing core technological support for low-consumption and efficient unmanned farm operations.

3.2. Motion Planning for Intelligent Agricultural End-Effectors

3.2.1. Application Background

In recent years, DRL has demonstrated significant advantages in motion planning for intelligent agricultural end-effectors. However, fruit harvesting and weeding require the precision control of manipulators in dynamic environments with occlusions and soft targets—challenges where rule-based control falls short. By leveraging autonomous agent–environment interaction mechanisms, DRL effectively addresses the limitations of traditional methods in dynamic environmental adaptability and complex constraint handling. From fruit picking to field weeding, and from multi-end-effector collaboration to dynamic path planning, DRL is rapidly advancing agricultural robots toward intelligent operation.

3.2.2. DRL-Based Solution and Key Advantages

As a core driver for the intelligent upgrade of agricultural robotic arms, DRL’s capabilities in autonomous environmental interaction and dynamic policy optimization are progressively overcoming the technical bottlenecks of traditional motion planning in complex farmland scenarios. In perception modeling, researchers construct high-dimensional state spaces through multi-source heterogeneous data fusion. For example, Zhang et al. [71] employed the SAC algorithm to develop a 6-degree-of-freedom (6-DOF) robotic arm assembly system. By integrating multi-source data such as end-effector pose, joint angles, and contact forces into the state space, they achieved a 93% success rate in flexible assembly tasks for 3C products. To address the morphological diversity of crops, Sheng et al. [72] innovated a joint framework combining point cloud feature extraction and grasp evaluation networks, enabling the stable grasping of irregular targets like tomatoes and peppers. This advances DRL’s adaptation from general-purpose robotic manipulation to specialized agricultural scenarios. As perception modeling matures, real-time responsiveness in dynamic environments has become a focal point. Xuan et al. [73] modeled dynamic target grasping using Markov decision processes, incorporating Kalman filtering for trajectory prediction and Proximal Policy Optimization (PPO) for 6-DOF robotic arm control. Their approach improved success rates by 21% compared to traditional methods. Simultaneously, Liu et al. [74] proposed a safety verification mechanism, building a closed-loop system (trajectory prediction-collision detection-dynamic replanning) on the TD3 framework. This enabled robotic arms to generate dynamic obstacle avoidance paths within 12.1 milliseconds, achieving a 3.7-fold speed improvement over Rapidly Exploring Random Trees (RRTs). For multi-robot collaborative tasks with complex constraints, DRL exhibits robust optimization capabilities. With advancements in single-robot intelligence, distributed decision-making frameworks are emerging to address multi-agent coordination challenges. Xie et al. [75] constructed a multi-agent Markov game model, coordinating a four-robot harvesting system via self-attention mechanisms. This reduced task completion time by 10.7% in scenarios involving 50 targets. Bu et al. [76] combined artificial potential fields with DDPG, designing a multi-level reward function incorporating obstacle avoidance penalties and contact force constraints. Their method reduced seedling damage rates to 2.82% in inter-row weeding tasks. These studies demonstrate that explicit constraint integration into reward functions allows DRL to balance operational efficiency and safety metrics effectively. To tackle the inherent uncertainties of agricultural environments, successful deployment relies on breakthroughs in simulation-to-reality transfer. Current research utilizes high-fidelity digital twin systems built on physics engines like MuJoCo and PyBullet. By introducing over 20 domain randomization parameters—such as wind disturbances and soil adhesion—the generalization capabilities of robotic arms trained in virtual environments are significantly enhanced. From multimodal perception to swarm collaboration, and from dynamic obstacle avoidance to virtual–real migration, DRL not only elevates robotic arm precision by orders of magnitude but also redefines the technological paradigm of agricultural automation. It shifts mechanical systems from rigid program execution to autonomous environmental cognition, injecting core momentum into fully autonomous operations for fruit picking, precision spraying, and beyond. A representative framework diagram of motion planning algorithms for intelligent agricultural robotic end-effectors is illustrated in Figure 4:

• Twin Delayed Deep Deterministic Policy Gradient (TD3) (Figure 4a): TD3 is an improved deterministic policy gradient algorithm. Its core idea involves enhancing the stability of reinforcement learning by integrating dual critic networks, smooth target network updates, and delayed policy updates. During execution, the agent executes actions (“a”) according to the policy output by the “Actor Network π₀”, interacts with the environment, and observes the next state (“s′”). These experiences—including the current state, action, and next state—are then stored (typically in a replay buffer for practical applications).
• A deep reinforcement learning network with self-attention (Figure 4b): This deep reinforcement learning network incorporates a self-attention mechanism. Its core idea is to leverage the self-attention mechanism to capture global dependencies among different features within the input observations, thereby extracting more effective features for value estimation and action selection in reinforcement learning. The agent acquires observational data from the environment and inputs it into a feature extractor. The data first undergo FNC and ReLU processing, performing preliminary feature extraction and nonlinear transformation on the raw observations. The self-attention mechanism then processes these initially extracted features, computing correlation weights between features. By comparing each feature against others, the self-attention mechanism determines the importance of each feature, ultimately generating a new feature representation that highlights the most critical features for the task.
• Deep Deterministic Policy Gradient (DDPG) algorithm (Figure 4c): The DDPG algorithm is a reinforcement learning method that combines the Q-learning and policy gradient techniques [28]. It employs two neural networks—a policy network (Actor) and a value network (Critic)—to represent the policy and value function, respectively, utilizing experience replay and target networks to stabilize the training process. At its core, DDPG optimizes the value function via a deterministic policy to identify optimal policies in continuous action spaces. The agent executes actions generated by the online policy network πθ (st) during environmental interaction, observing the next state (s_t+1) and receiving an immediate reward (r_t). These experiences (s_t, a_t, r_t, and s_t+1) are stored in an experience replay buffer. Through continuous updates to the online networks and soft updates to the target networks, the agent learns an effective policy to maximize cumulative rewards.

DRL overcomes the limitations of traditional agricultural robotic arms by enabling autonomous environmental interaction and dynamic policy optimization, which is crucial for complex farmland scenarios. Its multidimensional perception decision loop integrates robot dynamics with environmental physics, achieving real-time responses such as millisecond obstacle avoidance.

Table 3. DRL Gains in Agricultural Manipulator Tasks.

Metric	Traditional	DRL Solution
Obstacle reaction time	42 ms	12.1 ms
Multi-target success rate	65%	87%
Multi-arm task time	120 s	107 s

summarizes the performance comparison between DRL schemes and traditional methods in robot task scenarios.

3.3. Agricultural Low-Altitude Drone Operations

3.3.1. Application Background

As aerial platforms for agricultural intelligence, unmanned aerial vehicles (UAVs) have witnessed rapid adoption in modern farming systems, driven by their low-cost deployment and operational flexibility. Currently, UAVs are undergoing a paradigm shift—transitioning from mechanized task execution to autonomous decision-making—through the integration of DRL. This evolution enables UAVs to dynamically optimize mission parameters (e.g., flight altitude, spraying intensity, and obstacle avoidance) in response to real-time field conditions, thereby enhancing both task adaptability and resource-use efficiency [77,78] (Figure 5). By leveraging the state representation capabilities of deep neural networks and the dynamic policy optimization mechanisms of reinforcement learning, DRL has successfully addressed core challenges faced by traditional UAVs in complex farmland scenarios, such as delayed environmental responses and difficulties in multi-objective coordination. Significant breakthroughs have been achieved in critical areas, including pesticide spraying, path planning, and dynamic perception.

3.3.2. DRL-Based Solution and Key Advantages

In pesticide spraying scenarios, DRL has enabled a transformative leap from static path execution to dynamic trajectory optimization. For instance, Hu et al. [80] designed a wireless sensor network (WSN)-assisted UAV trajectory correction system that integrates real-time wind speed and direction data from ground sensors. By employing a dual-algorithm framework combining a DQN and Particle Swarm Optimization (PSO), they generated optimal spraying paths. Experimental results demonstrated a 41.68% increase in pesticide settlement coverage and a reduction in overlapping coverage to 5.56%. Fu et al. [81] further enhanced traditional DQNs by introducing a Bidirectional Long Short-Term Memory (Bi-LSTM) network to construct a BL-DQN model. This approach achieved 19.9% fewer steps for efficient coverage in a 10 × 10 grid map, validating DRL’s robustness in pesticide spraying across complex terrains. Path planning technologies have also evolved significantly, showcasing DRL’s advantages in multi-physics coupled environments. Kang et al. [82] proposed a control frequency adaptive (CFA) scheduling method, where a reinforcement learning agent dynamically adjusts PID-based attitude and position control frequencies. Under sudden wind disturbances, this method, combined with quadrotor dynamics and Q-learning, reduced waypoint tracking time by 12.8%. Huang et al. [83] addressed obstacle avoidance in mountainous orchards by developing a 3D path planning system using deep Q-learning. A multi-tiered reward mechanism—incorporating terrain slope, crop density, and pest distribution weights—reduced path redundancy by 31.4% and saved battery consumption by 18.2%. Agricultural UAVs not only address the limitations of satellite-based remote sensing (e.g., temporal resolution constraints and cloud interference) but also demonstrate superior operational advantages, including real-time responsiveness, cost-effectiveness, and operator-friendly simplicity [84]. Furthermore, DRL integration is driving the evolution of agricultural UAVs toward embodied intelligence—a paradigm characterized by self-adaptive decision loops and enhanced environment–body–task integration through sensorimotor learning. Almalki et al. [79] developed a dual-cognition UAV system that integrates DRL for autonomous navigation and Faster R-CNN for vegetation detection, achieving 98% detection accuracy in Saudi Arabia’s arid valleys. Chen et al. [85] proposed the MTPI-MTSA hybrid algorithm, which optimizes network initialization parameters through transfer learning and automatically enhances datasets using multi-Thompson sampling reinforcement learning. This method reduced vegetation segmentation network training time by 93.7% and improved the Intersection over Union (IoU) metric to 90.9%. Additionally, Gözen and Özer [86] innovated action-sequence reward functions and introduced diagonal movement commands for UAV visual tracking, boosting tracking accuracy by 3.87% on the VisDrone dataset and effectively resolving challenges such as small-target occlusion and scale variations. Collectively, these advancements outline a clear trajectory for DRL-powered agricultural UAVs: transitioning from single-task optimization (e.g., spray coverage) to multi-objective synergy (e.g., balancing energy efficiency and precision) and evolving from discrete functional modules (perception, planning, control) to intelligent closed-loop systems. This progression establishes a comprehensive technical framework that integrates environmental perception, dynamic decision-making, and precise execution into a unified continuum. With ongoing breakthroughs in 3D path planning and multimodal perception fusion, DRL-driven UAVs are redefining operational paradigms in precision plant protection and ecological monitoring, offering scalable, intelligent solutions to support global agricultural transition toward low-carbon practices. A representative algorithmic framework for low-altitude agricultural UAV operations is depicted in Figure 6:

• Neural network-based value function approximator of reinforcement learning (Figure 6a): Hu et al.’s neural network architecture, as shown in Figure 3, consists of two fully connected layers instead of the convolutional layers found in typical Deep Q-Networks. Additionally, the rectified linear unit (ReLU) is used as the activation function, with each layer having 32 neurons. The input is the state S = (U, X_w, w), and the output is the corresponding Q-value.
• QFP and CFA models (Figure 6b): QFP is trained to predict the next position of a quadcopter controlled by PID in an environment free from external influences. After training, the estimated next state by QFP is used as input for CFA. Then, CFA is trained to adjust the position and attitude control frequencies against external influences by maximizing a given reward function. The approximate external influence is calculated as the difference between the current state (enclosed in a red dashed box) and the estimated state (enclosed in a green dashed box). CFA is trained to balance the position and attitude control frequencies to achieve time-optimal waypoint tracking control.
• Multi-Task State Aggregation (MTSA) algorithm (Figure 6c): The MTSA algorithm aims to enhance the efficiency and performance of reinforcement learning by combining multi-task learning with state aggregation techniques. Its core principle leverages shared information across tasks to accelerate learning while reducing state space complexity through state aggregation, enabling more effective handling of complex environments. The agent interacts with the environment, selecting actions based on the current state. These actions are generated by a Q-network, which guides the agent’s behavior by estimating the value of state–action pairs. During training, state-aggregated training is accelerated using MPI (Message Passing Interface). This approach enables efficient processing of large-scale state spaces.

4. Key Challenges of DRL in Agricultural Applications

DRL has shown significant advantages in agricultural ground platform navigation, robotic arm motion planning, and low-altitude drone operations, it still faces numerous severe challenges during practical implementation. Overall, these challenges mainly manifest in three dimensions: environmental complexity, limitations of algorithms and computational resources, and issues of safety and reliability. The following analysis will delve into each dimension in detail.

4.1. Environmental Complexity Issues

Agricultural production sites are typically unstructured environments characterized by high dynamism and uncertainty. Regarded as semi-natural systems, agricultural settings inherently involve numerous stochastic and seasonal factors [15], posing significant challenges for autonomous agricultural systems [87]. For instance, variations in crop growth cycles, undulating terrain, climatic conditions, and sudden weather disturbances impose heightened adaptability requirements on automated equipment for navigation and task execution. Particularly in highly dynamic and unstructured agricultural environments—which may include obstacles of varying sizes (e.g., branches, leaves), gusts of wind, multi-obstacle occlusions, and fluctuating light conditions [88]—the unpredictable distribution of crops, diverse topography, and erratic obstacles render traditional experience- or rule-based methods inadequate in addressing these environmental complexities.

The technical bottlenecks of path planning in dynamic environments are further exacerbated under such conditions. In dynamic scenarios, if sensors detect unexpected obstacles, path planning systems must rapidly recalculate trajectories to avoid collisions while progressing toward the target location [89]. Conventional path planning approaches often struggle to deliver satisfactory performance under these constraints. Although DRL technology offers adaptive learning capabilities, continuously evolving dynamic factors in real-world environments (e.g., crop growth stages, sudden weather changes) can cause trained policies to become obsolete quickly, compromising long-term stability.

Table 4. Advantages and disadvantages of path planning methods.

Method	Advantages	Disadvantages	Reference
RRT	Simple, low computation	Static-only; non-optimal paths	Yang et al. [90]
A*	Fast search	Static-only; high computational burden; non-smooth	Hart et al. [91]
APF	Fast convergence	Local minima problem	Chen & Zhang [92]
Dijkstra’s Alg	Easy implementation	High complexity; static-only	Dijkstra et al. [93]
PRM	Handles complex spaces	Static-only; non-optimal; time-consuming	Yan et al. [94]
Genetic Alg	Solves NP-hard and multi-objective	High complexity; premature convergence	Tsai et al. [95]
CNN	Handles dynamic, unknown	High complexity; many hyperparameters; non-optimal	Wu et al. [96]
DQN	Adaptive	Non-optimal; hard to implement	Tong et al. [97]
DDPG	Supports continuous action spaces; suitable for high-dimensional control	Unstable training; high computational resource demands	Tian et al. [98]
TD3	Improved stability over DDPG; reduces overestimation	Complex parameter tuning; insufficient real-time performance	Liu et al. [74]
SAC	Balances exploration-exploitation; adapts to complex dynamic environments	High algorithmic complexity; long training time	Haarnoja et al. [99]
PPO	Stable training; suitable for multi-task scenarios	Requires fine hyperparameter tuning; slow convergence	Baumgartner et al. [100]

summarizes the strengths and weaknesses of contemporary path planning methods.

The above content categorizes and summarizes various path planning methods, but their application advantages, limitations, and suitability in specific agricultural scenarios still require further exploration. Below, we analyze the applicability and challenges of different methods by combining three typical scenarios proposed in this paper: navigation for agricultural ground platforms, motion planning for intelligent agricultural end-effectors, and low-altitude drone operations. For agricultural ground platform navigation, the core requirements include dynamic path planning, real-time obstacle avoidance, high-precision path tracking, and adaptability to unstructured farmland environments (such as crop growth interference and terrain undulations), as shown in

Table 5. Advantages and limitations of agricultural ground platform navigation.

Method	Advantages	Limitations	Applicability Case
RRT	Simple, low computation	Only suitable for static environments, the path is not optimal, and it cannot handle dynamic obstacles	For initial path exploration of static obstacles in farmland (such as avoiding fixed gullies); it should be combined with other methods to optimize the path.
A*	Fast search	Training is unstable, requires high computational resources, and has limited real-time performance	For initial global path planning of agricultural machinery; it needs to be combined with local dynamic adjustments.
DDPG	Supports continuous action spaces; suitable for high-dimensional control	Parameter tuning is complex, and hardware compatibility is poor	Zhang et al. used DDPG to optimize tractor dynamic path tracking, improving response speed by 30%.
TD3	Improved stability over DDPG; reduces overestimation	High algorithm complexity and long training cycles	Liu et al. implemented dynamic obstacle avoidance path generation for agricultural machinery based on TD3, reducing the response time to 12.1 milliseconds.
SAC	Balances exploration–exploitation; adapts to complex dynamic environments	Only applicable for static maps; path is not smooth; high computational burden	Hu adopted an SAC-designed orchard inspection robot navigation strategy to address the issue of sparse GPS signals.

. However, its limitations lie in the fact that dynamic farmland environments (such as crop growth interference and sudden obstacles) cause traditional methods (like RRT and A*) to fail; while DRL methods (such as TD3 and SAC) are more adaptable, they rely on high-performance computing devices, making it difficult to deploy them in edge environments in real time. For intelligent agricultural end-effector motion planning, the core requirements include high-precision trajectory control, dynamic obstacle avoidance (such as moving crops), multi-arm collaboration, and real-time response (<50 ms), as shown in

Table 6. Advantages and limitations of motion planning for intelligent agricultural end-effectors.

Method	Advantages	Limitations	Applicability Case
PPO	Training is stable and suitable for multi-task scenarios	Requires fine-tuning of parameters, with slow convergence speed	Xu et al. achieved dynamic grasping with a 6-degree-of-freedom robotic arm using PPO, improving the success rate by 21%
DQN	Strong adaptive learning ability; suitable for discrete action spaces (such as decision-making)	Difficult to handle continuous actions	For discrete task selection of end-effectors (such as spray switch control); it needs to be combined with policy gradient methods
MADDPG	Supports multi-agent collaboration	High risk of strategy drift requires complex reward function design	Xie et al. developed a multi-robotic arm Markov game model, reducing task completion time by 10.7%
APF	Converges quickly; suitable for local obstacle avoidance	Prone to becoming stuck in local minima; unable to handle complex constraints	Chen et al. combined APF with DDPG to design an obstacle avoidance reward mechanism, reducing the seedling damage rate to 2.82%

. However, its limitations are that agricultural end-effectors need to balance efficiency and safety at millimeter-level precision, which traditional methods (like APF) cannot meet under dynamic constraints; although DRL methods (such as MADDPG) support collaborative optimization, they require high-precision simulation environment migration support. For low-altitude drone operations, the core requirements include multi-objective coordination (such as balancing coverage and energy consumption), 3D path planning, wind disturbance resistance, and real-time decision-making (<100 ms), as shown in

Table 7. Advantages and limitations of agricultural low-altitude drone operations.

Method	Advantages	Limitations	Applicability Case
Genetic Alg	Suitable for multi-objective optimization	High computational complexity; prone to premature convergence	For offline global path planning (such as large field area division); but it needs to be combined with online DRL dynamic adjustment
DQN + PSO	Combining the adaptability of DQN with the global search capability of PSO to enhance path optimality	Implementation is complex and requires data fusion from multiple sensors	Hu et al. designed a drone trajectory correction system assisted by WSN, which increased pesticide deposition coverage by 41.68%
TD3	Adapt to continuous action space	Insufficient real-time performance; difficulties in deploying at the edge	For drone attitude control in simulation environments, a lightweight model such as TinyDRL is needed to adapt to actual hardware
SAC	Supports complex dynamic environments (such as sudden changes in wind speed), with high efficiency	The need for extensive training data makes it difficult to quickly transfer to new scenarios	Almalki et al. combined SAC with Faster R-CNN to achieve vegetation detection in arid river valleys with an accuracy of 98%

. However, its limitations are that drones need to achieve multi-objective optimization under resource constraints, which traditional methods (like genetic algorithms) cannot respond to online; hybrid architectures (such as DQN + PSO) improve performance but depend on high-precision environmental modeling and edge-cloud collaborative computing.

In summary, in terms of agricultural ground platforms, DRL methods such as TD3 and SAC are gradually replacing traditional rule-based algorithms but need to address edge computing bottlenecks. For agricultural end-effectors, multi-agent DRL like MADDPG is driving collaborative automation but relies on high-fidelity simulation-to-reality transfer. In the case of low-altitude agricultural drones, hybrid architectures combining DRL with optimization algorithms have become mainstream but require breakthroughs in lightweight deployment and cross-scenario generalization. Notably, even if path planning algorithms can handle dynamic environments, agricultural automation systems face another fundamental challenge: technical limitations in data acquisition and processing. Agricultural operations rely heavily on data from vision systems, LiDAR, GNSS, and environmental sensors, which are often plagued by noise and latency. Moreover, significant disparities in data formats, accuracy, and update frequencies across different sensors complicate multi-sensor data fusion. While DRL algorithms approximate Q-value functions via deep neural networks during training, enabling agents to learn and optimize path planning strategies [101], they must process high-dimensional and imperfect state information. This compromises model convergence and decision robustness, particularly in scenarios involving transient obstacles or abrupt local environmental changes. Future research must therefore prioritize breakthroughs in sensor fusion and noise-resistant algorithm design to enhance model adaptability.

4.2. Algorithm and Computational Resource Limitations

Despite demonstrating theoretical potential in agricultural automation, the practical deployment of DRL faces systematic challenges arising from computational constraints, data acquisition, and real-time responsiveness.

First, computational resource bottlenecks directly limit algorithmic performance. DRL’s training and inference heavily rely on GPU parallel computing capabilities [102]. However, field operation platforms must simultaneously process multimodal data streams, including multispectral imaging (crop vitality assessment [103]), thermodynamic analysis (water stress detection [104]), computer vision (pest/disease detection [105]), and LiDAR point clouds (vegetation density reconstruction [106]). These demands impose stringent requirements on edge computing devices, with existing agricultural machinery often suffering from delayed policy updates due to insufficient computational power. For instance, during fruit harvesting operations using agricultural robotic end-effectors, oscillations of target fruits during the grasping process may significantly increase the computational load of motion planning algorithms. This elevated demand for real-time processing could induce mechanical blockages (e.g., delayed actuator responses or trajectory conflicts), thereby compromising the practical utility of automated harvesting systems in field applications [107].

Second, high data acquisition costs and simulation-to-reality transfer hurdles exacerbate deployment difficulties. DRL’s reliance on extensive trial-and-error interactions compels researchers to use virtual simulation platforms like MuJoCo and PyBullet as substitutes for costly field trials. However, discrepancies persist between simulated and real-world environments in terms of lighting randomness, soil physical properties, and other factors. Even with domain randomization strategies to inject perturbed parameters, path planning errors in simulation-to-reality transfer remain unresolved. Furthermore, the massive computational resources required for simulation further elevate technical barriers to application.

More critically, real-time decision-making poses a pressing contradiction. Traditional DRL frameworks, designed to balance multi-objective optimization (e.g., waypoint tracking and sudden obstacle avoidance), often employ complex neural networks and multi-level reward functions. This leads to latency in online computation and decision-making during real-world operations. In high-speed scenarios, such as hilly terrain operations, such delays can trigger cumulative trajectory deviations or even cascading control failures. Thus, reducing computational complexity and energy consumption while maintaining decision accuracy is a critical breakthrough point for future research.

4.3. Safety and Reliability Issues

Agricultural automation equipment must not only perform tasks efficiently but also ensure the safety of on-site personnel and crops. Consequently, safety and reliability issues have emerged as a core challenge in technology promotion. In recent years, agricultural machinery accidents have occurred frequently in China. Statistical data indicate that 220 agricultural machinery incidents were reported nationwide in 2021, resulting in 46 fatalities and significant economic losses [108]. These recurring accidents highlight that the safety and reliability of automated agricultural equipment have become critical factors constraining the development of agricultural modernization.

In practical applications, agricultural robots often operate in human–robot collaborative environments. For example, when deploying spraying drones in complex agricultural settings, it is critical to account for obstacles such as heavy machinery, moving vehicles, and coexisting robotic agents to prevent collisions while simultaneously fulfilling the dual requirements of “precision spraying” and “safety” [109]. In tasks such as autonomous navigation and robotic arm grasping, inaccurate risk quantification in decision-making mechanisms may lead to erroneous device actuation, potentially causing crop damage or operator injuries. Therefore, safety verification and confidence estimation for DRL strategies, combined with real-time fault detection modules, are imperative for reliable operations. Furthermore, long-term deployed agricultural equipment often exhibits policy drift—a phenomenon where initially trained strategies gradually deteriorate due to dynamic environmental changes and evolving task requirements. This necessitates continuous online learning or periodic policy updates. Achieving adaptive adjustments while maintaining system stability (i.e., avoiding cascading erroneous decisions) has emerged as a key research frontier. Potential solutions include structured stochastic curriculum learning, transfer learning, and online adaptation algorithms; however, their practical efficacy remains to be empirically validated under real-world agricultural scenarios.

5. Future Research Directions for DRL in Agricultural Applications

With AI’s evolution, DRL applications in agriculture will rapidly advance toward higher efficiency, intelligence, and integration. Although DRL has demonstrated significant potential in the intelligentization of agricultural machinery, its practical deployment still faces three major technical barriers: First, the contradiction between complex neural network structures and the edge computing resources of agricultural machinery results in existing systems struggling to meet real-time response requirements in actual operations. Second, the black-box nature of strategies fundamentally conflicts with the safety requirements of human–machine coexistence scenarios in farmland, and the lack of quantitative verification mechanisms may lead to catastrophic failures. Third, the differences in physical properties between simulation environments and real farmland make virtual training strategies frequently ineffective in field applications, as evidenced by repeated cases. To overcome these bottlenecks, coordinated innovation across the three dimensions of algorithm–architecture–verification must be pursued:

(1)

Complex Hybrid Architectures and Algorithmic Innovations: Develop hybrid decision frameworks combining model predictive control with DRL to enhance stability and interpretability; innovate algorithms merging value and policy methods (e.g., SAC + DQN) to boost convergence and generalization; introduce attention mechanisms and multi-scale state representations to improve strategy expressiveness in complex scenarios.

(2)

Edge-Cloud Collaborative Computing and Lightweight Deployment: Establish edge-cloud collaborative architectures, training complex models in the cloud while deploying lightweight, quantized models (e.g., TinyDRL, ONNX-quantized) on edge devices for low latency and energy efficiency. Leveraging 5G/6G networks enables real-time model updates and strategy synchronization, reconstructing intelligent farm IT infrastructure.

(3)

Meta-Learning and Self-Supervised DRL for Rapid Adaptation: Integrate meta-learning and self-supervised tasks to enable few-shot policy generalization across crop varieties, climates, and regions. Construct self-supervised tasks from historical trajectories and visual representations to achieve plug-and-play agricultural agents.

In summary, DRL’s future in agriculture lies in simultaneous breakthroughs in algorithmic depth and application breadth. Through algorithm innovation, system integration, and computing architecture collaboration, DRL will become the core engine driving smart agriculture toward higher efficiency, lower carbon emissions, and increased precision, providing key technology support for sustainable global agriculture.

6. Conclusions

DRL, as a cutting-edge artificial intelligence technology, has emerged as a core driver of the intelligent transformation of agricultural machinery, demonstrating significant potential for decision optimization in complex and dynamic farmland environments. This paper systematically reviews the research progress of DRL in the autonomous navigation of agricultural machinery, motion planning for intelligent robotic arms, and optimization of low-altitude unmanned aerial vehicle (UAV) operations, highlighting its technical advantages in enhancing agricultural automation efficiency and adaptability. Key challenges and future directions for practical applications are also identified. Key findings include the following:

• Autonomous Navigation and Path Planning: Hybrid DRL algorithms integrating value function and policy gradient approaches (e.g., Double-DQN and DDPG) have achieved centimeter-level precision in agricultural vehicle path tracking (e.g., reducing errors to ±3 cm in [44]). Dynamic state representations (e.g., path curvature parameters) significantly improved robustness in complex farmland scenarios. However, the long-term stability of high-precision path tracking requires further validation, particularly under dynamic conditions such as crop growth interference and sudden obstacles.
• UAV Operation Optimization: Multi-objective reward mechanisms (e.g., balancing coverage efficiency and energy consumption) demonstrated preliminary success in pesticide spraying tasks (e.g., a 41.68% increase in coverage in [67]). However, their generalization across heterogeneous farmland scenarios necessitates further validation through cross-regional and multi-crop experiments.
• Multi-Agent Collaboration: Distributed DRL frameworks with attention mechanisms (e.g., MADDPG) reduced task completion time by 10.7% in collaborative harvesting scenarios ([63]). Yet, real-time responsiveness and safety of multi-agent systems require systematic optimization under hardware constraints.

Nevertheless, the practical deployment of DRL in agriculture faces persistent challenges, including unstructured environmental dynamics, real-time algorithmic constraints, and safety risks in human–machine collaboration. Future research should prioritize the following:

• Hybrid DRL architectures coupled with model predictive control to enhance decision interpretability and robustness;
• Edge-cloud collaborative computing frameworks for deploying lightweight models with low-latency field response and high energy efficiency;
• Cross-scene adaptive agents developed through meta-learning and self-supervised mechanisms to advance agricultural machinery from task-specific to generalized intelligence.

In conclusion, by synergizing algorithmic depth and systemic breadth, DRL is poised to resolve the trilemma of “complex environmental conditions, multitask coordination, and computing resources” in agriculture, serving as a core engine to propel smart agriculture toward sustainable, low-carbon, and fully autonomous practices. This evolution will provide critical technological support for global food security and agricultural modernization.