Multi-Level Attention Dueling Double Deep Q-Network for Local Path Planning

Abstract

Deep reinforcement learning (DRL) has shown considerable potential in local path planning for autonomous robots. However, existing DRL methods still suffer from limited training efficiency, poor generalization, and weak sim-to-real transferability in complex environments. To address these issues, this paper proposes a Multi-Level Attention Dueling Double Deep Q-Network (MLA-D3QN) framework, which progressively enhances feature extraction, spatial perception, and modality fusion through three attention levels: rule-based attention for obstacle contour extraction, implicit neural multi-scale spatial attention for environment perception, and bidirectional cross-attention for multi-modal feature alignment. Simulation results show that MLA-D3QN outperforms baseline and comparison methods in terms of convergence speed and average reward. Real-world experiments are conducted on a Scout mini platform with 50 trials in simple task scenarios (sparse obstacles, short distance) and 50 trials in complex task scenarios (dense obstacles, long distance). The proposed method achieves success rates of 98% in simple tasks and 94% in complex tasks. Compared to CNN-D3QN and D3QN, MLA-D3QN improves success rates by 10 percentage points (vs. CNN-D3QN) and 38 percentage points (vs. D3QN) in simple tasks, and by 34 percentage points (vs. CNN-D3QN) and 84 percentage points (vs. D3QN) in complex tasks. Path costs are reduced by 24.0% (vs. CNN-D3QN) and 59.9% (vs. D3QN). These results validate the effectiveness of MLA-D3QN in improving generalization and sim-to-real transferability for local path planning in complex environments.

1. Introduction

Path planning is a core technology that enables robots to operate autonomously in unknown and complex environments, aiming to generate a collision-free and efficient path from the start point to the goal. Despite significant progress in recent years, path planning remains a critical challenge in robotics. Current research on autonomous navigation mainly focuses on two categories of methods: improved traditional algorithms and artificial intelligence-based approaches. Enhanced methods based on algorithms such as A* [1,2,3] and RRT* [4,5] can generate accurate paths in known global environments, while local planners such as DWA [6,7,8] and APF [9,10,11,12] have been increasingly refined for dynamic environments. However, traditional methods generally lack autonomous decision-making capabilities and often struggle to adapt to complex and unknown environments. Specifically, traditional algorithms rely on pre-defined rules (e.g., cost functions, potential fields, or fixed sampling strategies) and cannot intelligently fuse multi-modal data such as vision, LiDAR, and map information. Moreover, they are unable to adaptively adjust parameters according to environmental changes, which limits their performance in dynamic or unstructured scenarios. With the rapid development of reinforcement learning, intelligent path planning methods based on reinforcement learning have attracted increasing attention. As a trial-and-error machine learning paradigm, reinforcement learning enables an agent to learn optimal decision-making strategies through continuous interaction with the environment, making it increasingly applicable to robotic path planning.

According to the functional roles of networks in reinforcement learning algorithms, existing RL-based path planning methods can generally be classified into three categories: value-based methods, policy-based methods, and Actor–Critic methods, which combine both value estimation and policy optimization. In addition, with the development of multi-sensor fusion technology, multimodal fusion reinforcement learning has gradually become an important research direction.

Value-based reinforcement learning guides agents to select optimal actions by learning state–action value functions, namely Q-values. In path planning tasks, such methods optimize navigation decisions by estimating the long-term return of different actions. Lv et al. improved the Deep Q-Network (DQN) algorithm by introducing an experience-value evaluation network and a dense connection structure, thereby enhancing convergence speed, planning success rate, and path accuracy [13]. Lin et al. combined reinforcement learning with sampling-based planning and proposed a value-estimation-based Rapidly-exploring Random Tree (RRT), which improves spatial exploration efficiency and node utilization in environments with limited information [14]. Yin et al. addressed local path planning in dynamic and unknown environments by using an n-step dueling double DQN framework with LiDAR data as input, which enhanced exploration capability and obstacle avoidance performance in sparse-reward environments and was successfully deployed on a real robot [15]. Jaramillo-Martínez et al. designed a reward function that encourages shorter paths and fewer turns, improving path quality and navigation stability, and demonstrated that reinforcement learning outperforms the traditional A* algorithm in planning performance [16]. Xie et al. proposed an improved DQN algorithm based on dynamic decision-making to minimize total travel time. Moreover, studies incorporating auxiliary tasks, reflective reward mechanisms, and traditional algorithms such as A* have further enhanced the expressiveness of value functions and the effectiveness of policy optimization, making path planning for unmanned vehicles in complex dynamic environments more efficient and reliable [17,18,19,20].

Policy-based reinforcement learning methods directly optimize the decision policy and have been widely investigated for improving planning efficiency, enhancing adaptability to complex environments, and enabling multi-robot collaboration in path planning. Chen et al. introduced a Transformer-based imitation reinforcement learning method that significantly improves the collaborative capability of multiple robots without explicit communication, achieving a high planning success rate in dense environments [21]. Jiang et al. combined hybrid-strategy path planning with reinforcement learning and traditional curve planning, thereby improving planning success rates and generalization ability in complex scenarios such as automated parking [22]. Zhao et al. addressed training efficiency and multi-objective optimization by proposing a progressive policy enhancement framework, which combines sparse rewards with process rewards to improve DRL performance in both static and dynamic obstacle environments [23]. Nie et al. focused on Automated Guided Vehicles (AGVs) in dynamic environments and employed a Proximal Policy Optimization (PPO) algorithm with sample regularization and an adaptive learning rate, improving exploration capability, convergence speed, and path planning stability [24]. Wahdan et al. proposed an adaptive reinforcement learning model based on Twin Delayed Deep Deterministic Policy Gradient (ARL-TD3), which can respond to complex and changing road environments in real time to improve decision-making and trajectory control [25]. Related studies have also integrated traffic flow and bandwidth variation data into reinforcement learning frameworks to optimize travel time and data transmission capability for unmanned vehicles in urban environments [26].

Actor-Critic reinforcement learning methods for unmanned vehicle path planning combine a policy network (Actor) and a value network (Critic) to achieve efficient path generation and tracking in complex dynamic environments. Liu et al. proposed an Adaptive Soft Actor-Critic (ASAC) framework integrated with Model Predictive Control (MPC), which not only improved path tracking accuracy but also significantly reduced energy consumption, outperforming traditional control methods and human driving performance [27]. Ibrahim addressed collision avoidance by generating safe and smooth paths using model-free Actor-Critic reinforcement learning. By sorting the inputs of obstacle distances and relative directions, the algorithm’s adaptability in multi-obstacle environments was improved [28]. The Soft Actor-Critic (SAC) algorithm has been widely applied to highway decision-making and motion planning, achieving safe, efficient, and comfortable automatic lane changing and cruise control [29]. For indoor mobile robots, Yang et al. introduced action entropy to optimize paths [30]. Additionally, Yu et al. proposed a global path planning method combining artificial potential fields with SAC. By incorporating a risk assessment module, they significantly accelerated convergence speed and reduced computation time, improving planning efficiency [31]. Research shows that the Actor-Critic architecture, through stable training and continuous action space handling, provides strong technical support for unmanned vehicle path planning, promoting autonomous driving systems towards greater intelligence and safety [32,33,34].

Multimodal reinforcement learning methods integrate multiple types of perceptual information, such as images, LiDAR data, and environmental features, to enable efficient autonomous navigation for robots and unmanned vehicles in complex environments. Tan et al. proposed an architecture that combines deep reinforcement learning with multi-sensor information fusion, where semantic segmentation and a lightweight multimodal data fusion network are employed to improve planning adaptability and decision-making performance [35]. Yan et al. developed a Q_EDQ algorithm for multimodal traffic scenarios by incorporating dual Q-learning mechanisms and dynamic exploration strategies. Their method achieves fast convergence and global optimal path search, outperforming traditional genetic algorithms, and is suitable for integrated planning across multiple transportation modes, including subways, buses, and shared cars [36]. Furthermore, Zhang et al. addressed cooperative path planning for unmanned aerial vehicle (UAV) swarms by combining multimodal cooperative particle swarm optimization with reinforcement learning, effectively handling multiple constraints while improving algorithmic performance and stability [37].

In recent years, attention mechanisms have been increasingly incorporated into multimodal reinforcement learning for path planning, enabling models to dynamically focus on task-relevant features from multi-source perceptual inputs. Wu et al. adopted an end-to-end architecture in which multimodal data are fed into an attention module, allowing the system to automatically learn and emphasize important environmental information. Combined with an LSTM model, this method enables real-time path adjustment and decision optimization in dynamic navigation tasks [38]. In multi-agent systems, Ma et al. employed attention mechanisms to facilitate information exchange among neighboring agents and exploit historical states, thereby improving the success rate and efficiency of cooperative path planning, particularly in large-scale complex environments [39]. For multi-UAV cooperative tasks, Wang et al. proposed an attention-based reinforcement learning method that jointly considers survival probability, path length, and load balancing to achieve real-time cooperative path planning [40]. In global path planning, attention mechanisms have also been used to dynamically capture both local and global information, effectively filtering irrelevant features and improving the convergence and generalization capabilities of deep reinforcement learning models [41]. These studies indicate that combining multimodal information with attention mechanisms can significantly enhance the adaptability and decision-making capability of reinforcement learning-based path planning in complex dynamic environments. As shown in Table 1, different reinforcement learning methods for path planning are summarized in terms of their categories, representative algorithms, advantages, and limitations.

Table 1. Summary of reinforcement learning methods for path planning.

Category	Representative Methods	Ref.	Advantages	Limitations
Value-based	DQN, D3QN	[13,14,15,16,17,18,19,20]	Simple architecture, stable training, suitable for discrete action spaces	Poor generalization, limited multimodal fusion capability
Policy-based	PPO, TRPO	[21,22,23,24,25,26]	Suitable for continuous action spaces, stable policy updates	High sample complexity, slow convergence
Actor-Critic	SAC, TD3, A3C	[27,28,29,30,31,32,33,34]	Combines value estimation and policy optimization, handles continuous actions well	Complex architecture, limited perception capability
Multimodal Fusion RL	Tan et al., Yan et al.	[35,36]	Integrates multiple sensor inputs, richer environment understanding	Static fusion strategies, poor cross-modal alignment
Attention-based RL	Wu et al., Ma et al.	[38,39,40,41]	Dynamic feature selection, focuses on task-relevant information	Single-level attention, limited multi-scale perception, weak sim-to-real transfer

Despite these advances, current DRL-based local path planners still face several challenges when processing multimodal inputs such as LiDAR data, cost maps, and kinematic states. First, different modalities exhibit substantial differences in scale, structure, and semantic representation, making it difficult to establish effective correspondences through simple feature concatenation. Second, existing fusion strategies are often based on static combinations and cannot dynamically adjust the contribution of each modality according to changes in the environment. Third, the fixed receptive fields of conventional CNNs limit their ability to simultaneously capture multi-scale information, such as global obstacle layouts and local boundary details. These limitations restrict policy generalization and increase the difficulty of transferring learned policies from simulation to real-world platforms.

The recent work by Valadbeigi et al., which addresses robust output consensus in heterogeneous multi-agent systems using a model-free Q-learning framework, exemplifies the growing trend of integrating learning-based methods into intelligent optimization and control system design. This line of research motivates our work on deep reinforcement learning for autonomous robot path planning [42].

To address these challenges, this paper proposes a novel Multi-Level Attention Dueling Double Deep Q-Network (MLA-D3QN) framework. The main contributions are summarized as follows:

• LoG Rule-based Attention: A fixed Laplacian of Gaussian (LoG) convolutional layer is employed to preprocess the local cost map and explicitly extract obstacle contours. This design introduces strong geometric priors into the network and reduces the burden of learning low-level edge features from scratch.
• Implicit Neural Multi-scale Spatial Attention (IN-MSA): A continuous implicit neural representation with learnable Fourier frequency encoding is introduced to sample the cost map at adaptive sub-pixel coordinates. This enables the network to capture multi-scale spatial structures, ranging from global obstacle layouts to narrow passage entrances, without relying on an explicit feature pyramid. Consequently, the reinforcement learning agent can better balance short-range obstacle avoidance and long-range navigation.
• Multi-head Cross-attention Fusion: A bidirectional cross-attention mechanism is designed to enable dynamic interaction between kinematic states and map features. Specifically, kinematic states query map features to evaluate traversability, while map features query kinematic states to focus on navigation-relevant self-motion cues. This context-aware fusion strategy improves semantic alignment between heterogeneous modalities.

Compared with traditional DQN, DDQN, and D3QN architectures that typically rely on low-dimensional state vectors (e.g., LiDAR scans or goal distances) or simple raw sensor inputs, the proposed MLA-D3QN introduces high-dimensional cost map observations as part of the state space. To effectively process these high-dimensional inputs, the multi-level attention mechanism progressively extracts task-relevant features: the LoG-based attention captures geometric contours, the IN-MSA module perceives multi-scale spatial structures, and the cross-attention fusion dynamically aligns heterogeneous modalities. This hierarchical design enables efficient feature fusion and significantly improves training performance. Furthermore, real-world experiments on a Scout mini platform validate the framework’s sim-to-real transferability, demonstrating that policies learned in simulation can be successfully deployed on physical robots with minimal performance degradation.

2. Experimental Methods

This paper proposes an autonomous navigation framework that integrates real-time environmental perception, multimodal feature fusion, and trajectory tracking. As shown in Figure 1, the framework consists of three main modules: a perception and mapping module, a Multi-Level Attention Dueling Double Deep Q-Network (MLA-D3QN) decision-making module, and a trajectory tracking module.

Figure 1. Overall framework of the proposed system. (a) System-level architecture: the main modules (perception and mapping, decision-making, trajectory tracking) and their information flow. (b) Perception and mapping module: LiDAR point clouds are processed by FAST-LIO and projected into a dynamic 2D cost map, which is then concatenated with kinematic and navigation state observations to form the reinforcement learning input. (c) Decision-making module: the MLA-D3QN network integrates multi-level attention mechanisms and outputs optimal actions guided by the loss function. (d) Trajectory tracking module: Model Predictive Control (MPC) follows the planned trajectory.

The three modules are detailed as follows:

• Perception and Mapping Module: This module uses LiDAR as the primary sensor and employs the FAST-LIO algorithm to construct a dynamic 2D cost map in real time. After aligning and concatenating the cost map with the kinematic and navigation states, the observation state input for reinforcement learning is formed.
• Decision-Making Module: This module adopts a D3QN architecture integrated with a multi-level hybrid attention mechanism. The dual-network structure improves the stability of Q-value estimation, while the multi-level attention network adaptively extracts multi-scale spatial features. In addition, multi-head cross-attention is employed to achieve deep fusion of heterogeneous modalities. Guided by the reward function, the module is iteratively optimized to output optimal obstacle avoidance decisions.
• Trajectory Tracking Module: This module employs Model Predictive Control (MPC) to accurately track the decision trajectory.

Through closed-loop iteration among perception, decision-making, and control, the framework realizes an automated process from sensor input to execution output, thereby enhancing navigation stability and robustness.

2.1. Dynamic 2D Cost Map Construction

The 2D grid map serves as the core environmental representation for mobile robot path planning. In this paper, 3D LiDAR data are transformed into a dynamic 2D grid map in real time through point cloud projection. The FAST-LIO framework [43] is employed to obtain registered point clouds by fusing LiDAR and IMU data through a tightly coupled iterated Kalman filter. FAST-LIO (Fast LiDAR-Inertial Odometry) is a computationally efficient and robust framework for real-time state estimation. It tightly couples LiDAR point clouds and IMU measurements using an iterated Kalman filter (IKF), which linearizes the measurement model at the estimated state to achieve higher accuracy than standard extended Kalman filters. Unlike conventional methods that process LiDAR scans as a whole, FAST-LIO updates the state incrementally as each point arrives, significantly reducing computational latency. The framework also compensates for motion distortion caused by the sequential sampling of LiDAR scans through a backward propagation process. As a result, FAST-LIO can reliably reconstruct 3D point cloud maps even during aggressive motion or in degenerate environments (e.g., feature-less corridors), which is essential for real-time 2D cost map generation in this paper. The environment is then divided into grids at a predefined resolution, and traversability is determined based on point cloud height: grids containing high-elevation points are marked as obstacles, those containing low-elevation points are marked as traversable, and those without point cloud observations are marked as unknown. To enhance robustness, voxel-grid downsampling, obstacle inflation, and dynamic region clearing are applied. Ultimately, a 2D grid map is generated and updated in real time as the unmanned vehicle moves, as shown in Figure 2.

Figure 2. Dynamic 2D Cost Map Construction. (a) 3D point cloud map constructed by FAST-LIO from LiDAR scans. The point cloud is color-coded by elevation or intensity, typically ranging from blue/purple (low elevation/low intensity) to red (high elevation/high intensity), resembling a rainbow spectrum. This color mapping helps distinguish ground points from obstacle points. (b) Corresponding dynamic 2D grid map after point cloud projection, where black areas represent obstacles, dark green areas denote unknown space, and white areas indicate traversable regions. The map is dynamically updated as the vehicle moves.

During the mapping process, the system dynamically refreshes the local map around the unmanned vehicle’s pose, thereby incorporating the instantaneous positions of moving obstacles into the environmental representation in real time.

2.2. Partially Observable Markov Decision Process Formulation

Due to the limited field of view and sensing range of LiDAR, the unmanned vehicle cannot obtain the true global state of the environment. Therefore, this paper formulates the problem as a Partially Observable Markov Decision Process (POMDP). To alleviate the challenges posed by partial observability, this paper adopts an approximate reconstruction strategy based on multimodal feature stacking, defining the system input as a heterogeneous joint observation vector that encompasses both spatial topology and the vehicle’s own physical characteristics, consisting of the following two parts:

1.

Local Cost Map Observation ($M_{cost}^{\left(t\right)}$): The grayscale map of the dynamic 2D grid map is used to characterize the environmental topological structure.

2.

Kinematic and Navigation State Observation ($X_{state}^{\left(t\right)}$): To compensate for the dynamic information lost due to the limited field of view in environmental perception, a low-dimensional state observation vector is defined as follows:

$$X_{state}^{\left(t\right)}={\left[v_t,{\omega }_t,{\displaystyle \frac{d_t}{d_{max}}},\mathrm{s}\mathrm{i}\mathrm{n}(\Delta {\theta }_t),\mathrm{c}\mathrm{o}\mathrm{s}(\Delta {\theta }_t),lidar\right]}^T$$

(1)

where $v_t$ and ${\omega }_t$ denote the linear velocity and angular velocity at the current time step, respectively; $d_t$ is the current distance to the target point, and $d_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is its maximum possible value, used to normalize the distance information; $\Delta {\theta }_t$ represents the yaw angle between the current heading and the target direction; and lidar denotes the LiDAR-based perceptual features. Specifically, the LiDAR scans the area within a 270° field of view in front of the vehicle. This scanning range is evenly divided into 27 sectors at 10° intervals. For each sector, the distance to the nearest obstacle is measured, yielding 27 distance values. These values are then normalized and assembled into a 27-dimensional feature vector, which provides a compact representation of the surrounding obstacle distribution. This vector encodes the vehicle’s self-motion state and navigation-related information into a compact representation serving as input for reinforcement learning.

The actual input to the system is no longer a single absolute state, but a heterogeneous joint observation vector $o_t$ that includes both spatial topology and its own physical characteristics.

$$o_t=\{M_{cost}^{\left(t\right)},X_{state}^{\left(t\right)}\}$$

(2)

By combining the local cost map ($M_{cost}^{\left(t\right)}$) and the kinematic-navigation state vector ($X_{state}^{\left(t\right)}$) into such a heterogeneous joint observation ($o_t$), the agent receives a more comprehensive representation of both environmental topology and self-motion dynamics. This formulation lays a solid foundation for the subsequent design of the attention-based deep reinforcement learning framework.

2.3. Reward Function Design

In robot path planning based on deep reinforcement learning, the reward function serves as the value guidance for policy optimization and directly determines the convergence direction of the agent’s behavior. To enable the robot to balance obstacle avoidance safety with path efficiency in grid-based environments, this section constructs a reward function that integrates sparse primary rewards with dense auxiliary penalties. At each time step $t$, the agent receives an immediate reward $r_t$ based on its current state, defined as follows:

$$r_t=\{\begin{array}{cc}{R}_{\mathit{step}}+R_{\mathit{reach}},& reach\\ R_{\mathit{step}}+R_{\mathit{collision}},& collision\\ R_{\mathit{step}},& otherwise\end{array}$$

(3)

The definitions are as follows: $R_{\mathit{step}}$ is a constant step penalty of −1 that imposes time and energy costs and encourages continuous motion to minimize arrival time. $R_{\mathit{reach}}$ is a reward of +80 triggered when the target threshold is reached, providing sparse terminal guidance. $R_{\mathit{collision}}$ is a penalty of −100 activated when obstacle proximity violates the safety radius, enforcing strict collision avoidance. Together, the step penalty encourages efficient exploration, the collision penalty establishes safety boundaries, and the reaching reward defines the optimization objective, balancing sparse and dense signals to achieve robust and efficient path planning.

3. Multi-Level Attention Deep Network Topology Architecture

To achieve efficient and safe autonomous navigation in complex environments, an unmanned ground vehicle requires not only robust feature extraction capabilities but also the selective attention ability akin to that of a human driver. When driving, humans do not process all information within the field of view equally; rather, cognition follows a hierarchical progression from perceiving object contours to visually screening salient features and ultimately to intention-driven attention. Inspired by this mechanism, this paper proposes the network architecture of the Multi-Level Attention Dueling Double Deep Q-Network (MLA-D3QN), as illustrated in Figure 3.

Figure 3. Multi-Level Attention Deep Network Topology Architecture. (a) Overall Q-network architecture of MLA-D3QN, illustrating the information flow from input to output. (b) Implicit Neural Multi-Scale Spatial Attention (IN-MSA) module for extracting multi-scale spatial features from the local cost map via learnable query points and Fourier encoding. (c) CNN-based dimensionality reduction module for processing map features. (d) Multi-head cross-attention fusion module for dynamic interaction between kinematic states and map features.

The architecture constructs a progressively layered three-tier attention system, aiming to address three key challenges in DRL-based local path planning. The first level adopts LoG-based rule attention, aiming to accelerate training convergence by explicitly extracting obstacle contours. The second level introduces the IN-MSA module, aiming to enable the agent to balance long-range navigation and short-range obstacle avoidance through multi-scale spatial feature extraction. The third level employs a multi-head cross-attention fusion module, aiming to enhance model generalization and sim-to-real transferability by aligning heterogeneous modalities.

3.1. LoG Rule-Based Attention

In deep reinforcement learning, an agent typically needs to learn environmental features through a large number of trial-and-error interactions. However, in unmanned vehicle navigation tasks, environmental boundaries, such as walls, curbs, and obstacle edges, exhibit well-defined geometric properties. Rather than requiring the neural network to learn these low-level geometric features from scratch at a high computational cost, it is more efficient to pre-extract key features using a prior operator and construct a rule-guided attention mechanism.

To evaluate the sensitivity of different edge detection operators to obstacles, we computed the mean pixel values in obstacle regions and free space regions on a representative local cost map. The ratio is defined as the mean value in obstacle regions divided by that in free space. The results are shown in Table 2.

Table 2. Comparison of different operators.

Operator	Mean Value in Obstacle Region	Mean Value in Free Space	Ratio
Roberts	0.3025	0.1495	2.02
Prewitt	0.8987	0.2889	3.11
Sobel	1.2119	0.4060	2.98
LoG	0.1478	0.0309	4.78

The values in Table 2 were calculated as follows: for each operator, we applied its convolution kernel to a representative local cost map, then computed the average pixel intensity in obstacle regions (black areas) and free space regions (white areas). The ratio is the obstacle region mean divided by the free space mean. A higher ratio indicates stronger sensitivity to obstacles. The results show that the LoG operator achieves a ratio of 4.78, outperforming the other operators, which indicates that it is more sensitive to obstacles and can extract more effective obstacle features.

For a continuous two-dimensional function $f(x,y)$, which in this paper represents the 2D cost map (grayscale image of the local environment), the Laplacian of Gaussian is defined as the convolution of the Gaussian function with the Laplacian operator:

$$\mathrm{L}\mathrm{o}\mathrm{G}\left(x,y\right)={\nabla }^2\left[G_{\sigma }\left(x,y\right)\times f\left(x,y\right)\right]$$

(4)

where the Gaussian function is given by $G_{\sigma }(x,y)={\displaystyle \frac{1}{2\pi {\sigma }^2}}{e}^{-{\displaystyle \frac{x^2+y^2}{2{\sigma }^2}}}$, and $\sigma$ is the scale parameter. Due to the commutativity of linear systems, the above computation is equivalent to first convolving the image with the Laplacian of Gaussian kernel:

$$\mathrm{L}\mathrm{o}\mathrm{G}\left(x,y\right)=\left[{\nabla }^2{G}_{\sigma }\left(x,y\right)\right]\times f\left(x,y\right)$$

(5)

By directly applying the Laplacian operator to the Gaussian function, the explicit expression of the LoG kernel is obtained:

$${{\nabla }^{2}G}_{\sigma }\left(x,y\right)={\displaystyle \frac{1}{\pi {\sigma }^4}}\left({\displaystyle \frac{x^2+y^2}{2{\sigma }^2}}-1\right)e^{-{\displaystyle \frac{x^2+y^2}{2{\sigma }^2}}}$$

(6)

where $(x,y)$ are the spatial coordinates in the image plane; ${\nabla }^2$ denotes the Laplacian operator ${\nabla }^2={\displaystyle \frac{{\partial }^2}{\partial x^2}}+{\displaystyle \frac{{\partial }^2}{\partial y^2}}$ and ${\nabla }^2{G}_{\sigma }(x,y)$ is the Laplacian of Gaussian kernel, which combines Gaussian smoothing with Laplacian edge detection.

In a discrete grid map, this continuous function is typically discretized into a $k\times k$ convolution kernel (e.g., $5\times 5$ or $9\times 9$). The coefficients of the kernel sum to zero, and the kernel exhibits a “Mexican hat” shape, with a positive peak at the center and a negative annular ring around it. A typical discrete approximation is:

$${\mathrm{K}}_{\mathrm{L}\mathrm{o}\mathrm{G}}\approx \left[\begin{array}{ccccc}0& 0& -1& 0& 0\\ 0& -1& -2& -1& 0\\ -1& -2& 16& -2& -1\\ 0& -1& -2& -1& 0\\ 0& 0& -1& 0& 0\end{array}\right]$$

(7)

In this kernel, the Gaussian component serves to smooth noise and suppress spurious edges, while the Laplacian component detects intensity discontinuities and generates zero-crossing points.

3.2. Implicit Neural Multi-Scale Spatial Attention

In unmanned vehicle path planning tasks in complex environments, the local map contains spatial structure information at multiple scales. Large-scale obstacle layouts influence long-range navigation intent, mid-scale corridor widths determine traversability, and small-scale obstacle edges affect short-range obstacle avoidance accuracy. Traditional methods typically employ multi-resolution feature pyramids or multi-branch convolutional networks to capture multi-scale information. However, these methods often suffer from computational redundancy and limited scale adaptability. To address this issue, this paper proposes an implicit neural network-based multi-scale spatial attention mechanism. By leveraging the continuity and differentiability of implicit neural representation (INR) and combining learnable sampling points with multi-frequency Fourier encoding, the proposed mechanism achieves efficient multi-scale spatial feature extraction from local maps. The specific structure is shown in Figure 4.

Figure 4. Implicit Neural Multi-Scale Spatial Attention (IN-MSA) Network Framework.

Unlike traditional methods that represent the map as discrete grid features, IN-MSA employs an implicit neural field to parameterize the mapping from coordinates to feature values as a continuous function $f_{\theta }:{\mathbb{R}}^2\to {\mathbb{R}}^d$, offering three key advantages: sub-resolution precision, differentiability and optimizability, and implicit encoding of structural priors.

Traditional spatial attention mechanisms (e.g., CBAM (Convolutional Block Attention Module, it is a lightweight attention module for convolutional neural networks that sequentially applies channel and spatial attention to adaptively refine intermediate features.)) dynamically compute attention weights based on input features, with computational cost growing linearly with the size of the feature map. In contrast, IN-MSA adopts an implicit static learnable sampling strategy: a set of optimizable query point coordinates serves as the input to the implicit neural network, with these parameters shared across all samples. The network automatically discovers the spatial regions that are most informative for decision-making through end-to-end training. A learnable set of query points $Q=\{q_i{\}}_{i=1}^K$ is defined, where $K=G\times G$ (with $G=7$, i.e., 49 sampling points). The query points are parameterized in Logit form as follows:

$$q_i=\sigma \left({\mathcal{l}}_i\right)=\mathrm{Sigmoid}\left({\mathcal{l}}_i\right)$$

(8)

where $l_i\in {\mathbb{R}}^2$ is a learnable parameter, where $\sigma (\cdot )$ denotes the Sigmoid function, i.e., $\sigma \left(x\right)=\mathrm{Sigmoid}(x)=1/(1+e^{-x})$, ensuring that $q_i\in (\mathrm{0,1}{)}^2$. The query points can adaptively shift from an initial uniform distribution to information-dense regions (e.g., obstacle edges, narrow passage entrances).

To endow the implicit neural network with the ability to perceive multi-scale spatial structures, IN-MSA applies Fourier feature encoding to each query point coordinate $q_i=(x_i,y_i)$. Inspired by the positional encoding used in NeRF (Neural Radiance Fields, it is a neural representation for 3D scenes that uses positional encoding to capture high-frequency details.), a multi-frequency sine-cosine encoding is adopted to map low-dimensional coordinates into a high-dimensional feature space, enabling the MLP to fit high-frequency variations. To accommodate map details at different scales, a learnable frequency scaling factor $s$ is introduced:

$$\gamma (q_i)=[q_i,\stackrel{N_f-1}{\underset{k=0}{⨁}}\left\{sin\left(2^k\pi s{q}_i\right),cos\left(2^k\pi s{q}_i\right)\right\}]$$

(9)

where $N_f=6$ is the number of frequencies. $s=0.5+1.0\cdot \mathrm{Sigmoid}\left(\beta \right)$ is a learnable scaling factor used to automatically adjust the range of perceived frequencies. The symbol $⨁$ denotes the concatenation operation.

Implicit Fourier encoding covers all scales through a continuous frequency domain, and with the frequency scaling factor $s$, the network can adaptively adjust the range of scales it perceives.

When the local map $M\in {\mathbb{R}}^{1\times H\times W}$ (with $H=W=100$) is input as a state feature component, mean filtering downsampling is first applied to the map. This step reduces the influence of noise on the gradient and lowers the map resolution to facilitate subsequent sampling:

$$\tilde{M}=\mathrm{AvgPool}2\mathrm{d}(M,\mathrm{kernel}\_\mathrm{size}=2,\mathrm{stride}=2,\mathrm{padding}=0)$$

(10)

where $M$ is the input local cost map; ${\mathbb{R}}^{1\times H\times W}$ denotes a real-valued tensor with 1 channel, height $H$, and width $W$ (with $H=W=100$); $\mathrm{AvgPool}2\mathrm{d}(\cdot )$ is the 2D average pooling operation; $\mathrm{kernel}\_\mathrm{size}=2$ means a $2\times 2$ pooling window; $\mathrm{stride}=2$ means the window moves by 2 pixels each step, reducing the spatial dimensions by half; $\mathrm{padding}=0$ means no zero-padding is applied; and $\tilde{M}$ is the downsampled map after pooling, with dimensions ${\mathbb{R}}^{1\times 50\times 50}$.

Subsequently, bilinear interpolation is used to sample the occupancy value at each query point on the smoothed map $\tilde{M}$. Given the query point coordinate $q_i$, it is necessary to convert it to the normalized device coordinates required for bilinear interpolation:

$$g_i=2\times q_i-1$$

(11)

$$o_i=\mathrm{GridSample}(\tilde{M},g_i)\in [0,\mathrm{ }1]$$

(12)

where $g_i$ is the normalized device coordinate obtained by mapping $q_i$ from $[0, 1]$ to $\left[-1, 1\right]$, and $o_i$ is the occupancy value at the sampling point. The differentiability of the bilinear interpolation operation “GridSample” ensures that the query point positions can be optimized via the gradient of the loss function.

The final feature of each query point is obtained by concatenating its positional encoding $\gamma \left(q_i\right)$ with the occupancy value $o_i$, which is then decoded by an implicit neural network (MLP):

$$f_i={\mathrm{MLP}}_{\mathsf{\theta }}\left(\left[\gamma \right(q_i),o_i]\right)\in R^{d_p}$$

(13)

where $\gamma \left(q_i\right)\in {\mathbb{R}}^{2+4N_f}$ (with $N_f=6$, hence $\gamma \left(q_i\right)\in {\mathbb{R}}^{26}$) is the Fourier feature encoding of the query point coordinate $q_i$; $o_i\in [0, 1]$ is the occupancy value sampled at $q_i$ from the smoothed map; $\left[\gamma \left(q_i\right),o_i\right]$ denotes the concatenation of the positional encoding and the occupancy value; ${\mathrm{MLP}}_{\theta }(\cdot )$ is a multi-layer perceptron with parameters $\theta$; $f_i\in {\mathbb{R}}^{d_p}$ is the output feature vector of the $i$-th query point; $d_p=10$ is the feature dimension per point; and the MLP consists of two hidden layers, each with 96 dimensions. The MLP continuously maps the concatenated features into the feature space.

The features of all query points are flattened and then aggregated through a linear layer to form the multi-scale map representation:

$$z_{\mathrm{map}}=\mathrm{Flatten}\left(\right[f_1,\dots ,f_K\left]\right)\in R^{d_{\mathrm{out}}}$$

(14)

where $f_i\in {\mathbb{R}}^{d_p}$ is the feature vector of the $i$-th query point (with $d_p=10$); $K=49$ is the total number of query points; $\left[f_1,\dots ,f_K\right]$ denotes the concatenation of all query point features; $\mathrm{Flatten}(\cdot )$ flattens the concatenated features into a one-dimensional vector; ${\mathbb{R}}^{d_{\mathrm{out}}}$ indicates the output space with dimension $d_{\mathrm{out}}=32$; and $z_{\mathrm{map}}$ is the resulting multi-scale map representation. The introduction of IN-MSA in this section enables the network to learn primarily which positions to query in the continuous space and how to decode the queried results into task-relevant feature representations.

In summary, the proposed IN-MSA module operates in three stages: (1) adaptive sampling: a set of learnable query points is optimized to focus on information-dense regions; (2) Fourier encoding: each query point is mapped to a multi-scale feature vector that captures both coarse and fine spatial structures; (3) implicit decoding: an MLP transforms the encoded features into a compact map representation. This design allows the reinforcement learning agent to perceive both global obstacle layouts and local boundary details without relying on a fixed multi-scale pyramid, thereby improving navigation performance in complex environments.

3.3. Multi-Head Cross-Attention Fusion

Traditional fusion methods employ simple vector concatenation or element-wise addition. Such static linear fusion ignores the differences in data distribution across modalities and struggles to capture the deep correlations between the state and the environment. To address this issue, this section introduces a multi-head cross-attention mechanism and constructs a two-level modality interaction architecture.

The core idea of cross-attention is to use information from one modality as the query (Q) to dynamically retrieve and aggregate the key (K) and value (V) from another modality. The multi-head mechanism captures cross-modal semantic correlations from different representation subspaces through multiple parallel attention heads. Its mathematical formulation is as follows:

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(Q,K,V\right)=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(15)

$$\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{H}\mathrm{e}\mathrm{a}\mathrm{d}\left(Q,K,V\right)=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left({\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_1,\dots ,{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_h\right)W^O$$

(16)

where $Q$ (query), $K$ (key), and $V$ (value) are matrices derived from the input features of different modalities; $d_k$ is the dimension of the key vector, and the scaling factor $\sqrt{d_k}$ prevents the dot product from becoming too large, which could lead to vanishing gradients; $\mathrm{softmax}(\cdot )$ normalizes the attention weights to a probability distribution; $h$ is the number of attention heads; ${\mathrm{head}}_i=\mathrm{Attention}(Q{W}_i^Q,K{W}_i^K,V{W}_i^V)$ is the output of the $i$-th attention head; $\mathrm{Concat}(\cdot )$ denotes the concatenation operation; and $W^O$ is the output projection matrix that linearly transforms the concatenated multi-head outputs.

This paper designs a bidirectional interaction mechanism: (1) Kinematic state-guided map validation: using the kinematic state as Q and the map features as K/V, the real-time state corrects map latency and captures dynamic obstacles; (2) Map prior-guided state focusing: using the map features as Q and the kinematic state as K/V, the global continuity of the map compensates for the sparsity of long-range information.

Through this mechanism, the network no longer passively receives all environmental information, but actively retrieves highly relevant regions from multiple semantic perspectives based on its own state, fundamentally addressing the spatial and semantic misalignment problem of multimodal data.

4. Experimental Validation

4.1. Simulation Training

To simulate real-world scenarios, this paper establishes a simulation environment based on Gazebo-ROS. By incorporating various maps collected during the actual mapping process, obstacle regions are cropped and randomly spliced to construct diverse training scenarios. The reinforcement learning model is trained in a Python 3.8 environment, and the main parameter settings are shown in Table 3.

Table 3. Training hyperparameters.

Parameter	Value	Description
gamma	0.99	Discount factor
total_iterations	2 × 106	Training iterations
train_start_iterations	2 × 104	Training warmup steps
batch_size	256	Mini-batch size
epsilon_noise	0.15	Noise coefficient for exploration
capacity	1 × 105	Capacity of experience replay buffer
optimizer.name	Adam	Optimizer type
optimizer.learning_rate	1 × 10−4	Learning rate

To validate the effectiveness of the proposed algorithm, it is trained and compared against four baseline methods: the original D3QN without map input, the CNN-D3QN without attention mechanism, the MA-D3QN with only multi-head attention, and the SAC algorithm. The reward function convergence curves of each method are shown in Figure 5.

Figure 5. Training reward curves of different algorithms.

From the training curves shown in Figure 5, it can be observed that as the number of training iterations increases, the cumulative rewards of all algorithms exhibit a gradually rising trend, indicating that each algorithm is able to learn effective obstacle avoidance strategies through interaction with the environment. In terms of convergence speed, the proposed MLA-D3QN algorithm demonstrates a significant advantage, entering the convergence region after approximately 50 k iterations, while MA-D3QN and CNN-D3QN stabilize after around 75 k and 100 k iterations, respectively. Regarding model performance, MLA-D3QN achieves significantly higher reward values than the other compared algorithms, demonstrating that the multi-level attention mechanism introduced in this paper can effectively improve sample efficiency and model performance. The detailed training data are presented in Table 4.

Table 4. Performance comparison of different algorithms.

	MLA-D3QN	MA-D3QN	CNN-D3QN	D3QN	SAC
Average Reward	−0.175	−7.54	−14.42	−26.39	−42.70
Maximum Reward	43.47	42.90	39.93	28.10	28.21
Test Success Rate (Optimal Model)	100%	100%	100%	100%	100%
Average Task Time (s) (Optimal Model)	3.66	3.71	4.07	5.19	5.23

From Table 4, it can be observed that the proposed MLA-D3QN outperforms the comparative algorithms across all metrics. In terms of average reward, MLA-D3QN achieves −0.175, significantly higher than MA-D3QN (−7.54), CNN-D3QN (−14.42), D3QN (−26.39), and SAC (−42.70). Regarding the maximum reward, MLA-D3QN also attains the optimal value (43.47), slightly outperforming MA-D3QN (42.90). In terms of simulation test success rate, all five algorithms achieve 100% under their optimal models. However, in terms of average task time, MLA-D3QN requires only 3.66 s, outperforming MA-D3QN (3.71 s), CNN-D3QN (4.07 s), D3QN (5.19 s), and SAC (5.23 s), representing a reduction of approximately 29.5% compared to D3QN.

To comprehensively validate the effectiveness of the proposed MLA-D3QN algorithm, we also conducted comparative experiments against two recently proposed advanced methods: the lightweight dual-path Kolmogorov-Arnold convolutional network (KAU-Net [44]) and the triple-branch hybrid dynamic-static alignment network (TriHDSA [45]). The results are shown in Figure 6. KAU-Net enhances nonlinear feature extraction through a dual-path architecture with Kolmogorov-Arnold convolution kernels, and employs a multi-scale information fusion (MSIF) module to achieve collaborative capture of local and global multi-scale features. It is primarily designed for medical optical image segmentation tasks, and in this comparative experiment, it replaces the traditional convolutional neural network for map feature extraction. TriHDSA adopts a triple-branch parallel structure, including a hybrid alignment branch (HAB), an elastic adjustment branch (EAB), and an adaptive balancing branch (ABB), and improves cross-modal matching accuracy in vision-language tasks through a joint dynamic-static feature alignment strategy. In this comparative experiment, TriHDSA is simply reproduced at the feature fusion stage. Although both methods perform well in their original tasks, their performance in the simulation training of this task, as shown in Figure 6, is relatively limited. They are inferior to the proposed MLA-D3QN algorithm in key metrics such as convergence speed, average reward, and training stability.

Figure 6. Training reward curves of different algorithms (MLA-D3QN, KAU-Net, and TriHDSA).

As shown in the figure, the proposed MLA-D3QN converges significantly faster than the two comparison algorithms, reaching a stable state within fewer training episodes. Moreover, in terms of training stability, the reward curve of MLA-D3QN exhibits the smallest fluctuation, while both KAU-Net and TriHDSA show varying degrees of oscillation. This demonstrates that the proposed method possesses stronger robustness and training consistency in reinforcement learning-based path planning tasks. The detailed results are presented in Table 5.

Table 5. Performance comparison of MLA-D3QN, KAU-Net, and TriHDSA.

	MLA-D3QN	KAU-Net	TriHDSA
Average Reward	−0.175	−78.32	−29.36
Maximum Reward	43.47	41.40	44.66
Test Success Rate (Optimal Model)	100%	100%	100%
Convergence speed	50 k	--	200 k
Computational complexity (MACs, i.e., Multiply-Accumulate Operations)	4.28 M	2.68 M	4.26 M

Based on the simulation training data, MLA-D3QN demonstrates superior overall performance compared to these algorithms. First, in 30 repeated tests with randomly generated environments, the maximum reward values of the three algorithms are close (43.47 vs. 41.40 vs. 44.66), likely due to the occasional sparseness of the randomly generated test maps. Therefore, the average reward serves as a more reliable indicator of overall robustness. MLA-D3QN achieves an average reward of −0.175, outperforming KAU-Net (−78.32) by 78.15 and TriHDSA (−29.36) by 29.19. Second, although KAU-Net has the lowest computational complexity (2.68 M MACs), its reward curve oscillates drastically, and its convergence time cannot be determined (marked as “—”), revealing poor training stability. Third, MLA-D3QN achieves the fastest convergence (50 k steps), which is 4 times faster than TriHDSA (200 k steps), while its computational complexity (4.28 M MACs) is comparable to that of TriHDSA (4.26 M MACs). In summary, MLA-D3QN outperforms the comparison algorithms in the three core dimensions: overall performance, training efficiency, and stability.

4.2. Real-World Transfer Experiment

In this paper, the Scout mini is used as the experimental platform. The vehicle is equipped with a 16-beam LiDAR, a Jetson Nano central processing unit, and an inertial measurement unit (IMU). In complex obstacle environments, the experiments are divided into two categories: simple tasks and complex tasks, to validate the transfer capability of the proposed algorithm across scenarios with varying difficulty levels. The real-world experimental environment is shown in Figure 7.

Figure 7. Real-world experimental environment and schematic diagram. (a) Scout mini unmanned ground vehicle platform equipped with a 16-beam LiDAR, an IMU, and a Jetson Nano onboard computer. (b) Photograph of the real-world experimental environment. (c) Schematic plan view of the experimental scenarios, where the green dashed box indicates the simple task scenario and the red dashed box indicates the complex task scenario.

The physical parameters of the unmanned ground vehicle and the planning parameters are shown in Table 6.

Table 6. Physical and planning parameters of the unmanned ground vehicle.

Parameter Category	Parameter Name	Value
UGV Physical Parameters	Vehicle size (length × width)	0.6 m × 0.5 m
	Maximum linear velocity	1.0 m/s
	Maximum angular velocity	2.0 rad/s
Planning Parameters	Goal tolerance radius	0.05 m
	Obstacle tolerance radius	0.3 m
	Sampling period ΔT	0.1 s
Performance Parameters	Average replanning time	0.3 s

Based on the trained MLA-D3QN model, the unmanned ground vehicle makes real-time decisions at a fixed sampling period of $\Delta T=0.1\hspace{0.17em}\mathrm{s}$. At each sampling instant, the model outputs optimal actions (linear acceleration and angular acceleration) according to the current local cost map and kinematic states (linear velocity, angular velocity, distance to the goal, heading deviation, etc.), thereby updating the vehicle’s linear and angular velocities. Within the velocity constraints (maximum linear velocity of $1.0\hspace{0.17em}\mathrm{m}/\mathrm{s}$, maximum angular velocity of $2.0\hspace{0.17em}\mathrm{rad}/\mathrm{s}$), the system simulates and samples a smooth trajectory from the current position to the goal. Subsequently, the trajectory tracking module employs Model Predictive Control (MPC) to accurately follow the planned trajectory. The task is considered successful when the vehicle enters the goal tolerance radius ($0.05\hspace{0.17em}\mathrm{m}$). A collision penalty is triggered when the distance to an obstacle is less than the obstacle tolerance radius ($0.3\hspace{0.17em}\mathrm{m}$). In dynamic environments, the system updates the trajectory in real time with an average replanning time of $0.3\hspace{0.17em}\mathrm{s}$ to handle moving obstacles.

The performance of MLA-D3QN in simple and complex tasks is shown in Figure 8.

Figure 8. Performance of MLA-D3QN in simple and complex tasks.

Figure 8 presents the planning results of the unmanned vehicle in the physical environment visualized in Rviz. The white unmanned vehicle model represents the initial position of the vehicle, and the green curve denotes the obstacle avoidance path planned by the MLA-D3QN algorithm. Black represents the inflated obstacles in the 2D grid map, while in the 3D point cloud map, different colors (e.g., red, blue, etc.) represent obstacles at different heights. Figure 8a–c show examples of planning in simple task scenarios, while Figure 8d–f show examples in complex task scenarios, illustrating typical successful trials in which the algorithm reaches the target. To validate the robustness of the algorithm, 50 repeated experiments with randomly selected target points were conducted in both task scenarios. The experimental results demonstrate that the algorithm achieves a success rate of 98% in simple task scenarios and 94% in complex task scenarios. To evaluate the performance of different algorithms, this section conducts comparative experiments on the MLA-D3QN, CNN-D3QN, and D3QN algorithms in both simple and complex task scenarios. The experimental results are shown in Figure 9.

Figure 9. Comparison of planning results of different algorithms in simple and complex tasks.

Figure 9a,b present the experimental results in complex task scenarios, while (c) shows the comparative experimental results in the simple task scenario. The three curves in different colors represent the paths planned by the three algorithms, respectively. In the complex task scenarios shown in (a) and (b), the D3QN algorithm fails completely, as its planned path collides with obstacles or fails to reach the goal, while CNN-D3QN also struggles with low success rates. In contrast, only the proposed MLA-D3QN algorithm achieves satisfactory planning performance in complex tasks. In the simple task scenario (c), all algorithms are generally able to complete the task. The detailed experimental results of 50 repeated experiments with randomly selected target points are presented in Table 7. The path cost in Table 7 is defined as the ratio of the actual path length to the straight-line distance between the start and goal points.

Table 7. Experimental results of different algorithms in simple and complex tasks.

	Simple Task Success Rate	Complex Task Success Rate	Simple Task Path Cost	Complex Task Path Cost
MLA-D3QN	98%	94%	1.16	1.27
CNN-D3QN	88%	60%	1.38	1.67
D3QN	60%	10%	1.98	3.17

As shown in Table 7, the proposed MLA-D3QN algorithm achieves the best performance in both simple and complex tasks. In terms of success rate, MLA-D3QN attains 98% in simple tasks and 94% in complex tasks, significantly outperforming CNN-D3QN (88%/60%) and D3QN (60%/10%). Notably, in complex task scenarios, MLA-D3QN improves the success rate by 34 and 84 percentage points compared with CNN-D3QN and D3QN, respectively, demonstrating that the proposed multi-level attention mechanism effectively enhances the algorithm’s generalization capability and sim-to-real transfer ability in complex environments.

In terms of path cost, MLA-D3QN achieves path costs of 1.16 in simple tasks and 1.27 in complex tasks, both lower than those of CNN-D3QN (1.38/1.67) and D3QN (1.98/3.17). In complex tasks, MLA-D3QN reduces the path cost by 24.0% compared to CNN-D3QN and by 59.9% compared to D3QN, indicating that the proposed algorithm plans paths that are closer to the optimal straight-line route, thus achieving higher path quality.

In summary, MLA-D3QN outperforms the compared algorithms in both success rate and path cost, validating the effectiveness of the proposed multi-level attention mechanism in improving the path planning performance of unmanned ground vehicles.

During the dynamic map update process, the system triggers path replanning. To validate the responsiveness of the proposed algorithm in dynamic environments, experiments were conducted in scenarios with dynamic obstacles. The replanning effect is shown in Figure 10.

Figure 10. Path replanning results in a dynamic obstacle scenario.

As shown in Figure 10, the area within the small red circle represents a pedestrian obstacle. As the pedestrian moves, the system updates the dynamic cost map in real time and continuously performs path replanning accordingly to generate the optimal trajectory. These results indicate that the proposed algorithm can respond sensitively to changes in dynamic obstacles.

5. Conclusions

This paper proposed MLA-D3QN, a multi-level attention deep reinforcement learning framework for unmanned ground vehicle local path planning. By integrating a LoG rule-based attention for geometric prior extraction, an implicit neural multi-scale spatial attention for adaptive scale perception, and a multi-head cross-attention fusion mechanism for dynamic modality alignment, the framework effectively addresses cross-modal semantic misalignment and multi-scale feature extraction challenges. Simulation results show that MLA-D3QN achieves an average reward of −0.175 and an average task time of 3.66 s, outperforming D3QN, CNN-D3QN, MA-D3QN, and SAC. Real-world experiments on a Scout mini platform further demonstrate a 98% success rate in simple tasks and 94% in complex tasks, with path costs reduced by 24.0% and 59.9% compared to CNN-D3QN and D3QN, respectively, validating the algorithm’s superior generalization and sim-to-real transfer capability. Despite the promising results, the proposed MLA-D3QN framework has several limitations. First, the training speed is slow, requiring 2 million steps (approximately 366 min). Second, the framework relies on a 2D cost map and does not directly use raw point cloud or image data, potentially losing fine-grained environmental details. Third, it has only been validated on a Scout mini platform in indoor environments; its generalization to other platforms or outdoor terrains remains unverified. Fourth, due to the possible retention of point clouds from moving obstacles, the framework may struggle with highly dynamic environments. These limitations will be addressed in future work. In addition, future work will focus on incorporating multi-timescale temporal fusion to handle highly dynamic scenarios, as well as extending the framework to more complex environments, including multi-agent coordination and extreme terrain navigation.