EMS-SLAM: Dynamic RGB-D SLAM with Semantic-Geometric Constraints for GNSS-Denied Environments

Abstract

Global navigation satellite systems (GNSSs) exhibit significant performance limitations in signal-deprived environments such as indoor spaces and underground spaces. Although visual SLAM has emerged as a viable solution for ego-motion estimation in GNSS-denied areas, conventional approaches remain constrained by static environment assumptions, resulting in a substantial degradation in accuracy when handling dynamic scenarios. The EMS-SLAM framework combines the geometric constraints and semantics of SLAM to provide a real-time solution for addressing the challenges of robustness and accuracy in dynamic environments. To improve the accuracy of the initial pose, EMS-SLAM employs a feature-matching algorithm based on a graph-cut RANSAC. In addition, a degeneracy-resistant geometric constraint method is proposed, which effectively addresses the degeneracy issues of purely epipolar approaches. Finally, EMS-SLAM combines semantic information with geometric constraints to maintain high accuracy while quickly eliminating dynamic feature points. Experiments were conducted on the public datasets and our collected datasets. The results demonstrate that our method outperformed the current algorithms of SLAM in highly dynamic environments.

1. Introduction

SLAM is a key technology enabling self-awareness in intelligent mobile devices, with applications in fields like mobile robots, drones, and so on [1,2]. While GNSS-based navigation systems have been widely adopted in open environments, their positioning accuracy severely degrades in denied environments such as indoor spaces, urban canyons, and underground tunnels due to signal occlusion or interference [3,4]. This limitation has driven significant research interest in alternative localization methods, particularly visual SLAM, which leverages the ubiquity and low cost of cameras to achieve environment-aware positioning without relying on external signals [5]. Due to the increasing number of visual SLAM systems being developed, the challenge of performing well in GNSS-denied environments has been addressed, some of which include PTAM [6], ORB-SLAM2 [7], VINS-Mono [8], and ORB-SLAM3 [9].

However, visual SLAM’s applicability is limited by its reliance on a static environment assumption, which restricts its use in real-world scenarios. Dynamic environments in real applications introduce numerous erroneous correspondences, resulting in an inaccurate state estimation in visual SLAM systems [10,11]. Although the algorithms for generalizing SLAM systems perform well in certain scenarios, they still encounter issues when dealing with dynamic objects. In recent years, various methods for dealing with dynamic objects have been presented. These include deep learning-based methods and geometry-based approaches as well as hybrid methods.

Most existing geometry-based methods that are used for dealing with dynamic objects rely on the information collected from rigid objects and motion consistency. Kim et al. [12] proposed a dense visual odometry method that used an RGB-D camera to estimate the sensor’s ego-motion. Wang et al. [13] utilized an RGB-D algorithm for detecting moving objects in an indoor environment that was able to achieve an improved performance by taking into account geometric constraints and mathematical models. Geometry-based methods usually depend on predefined thresholds to eliminate dynamic feature points, often leading to over-detection or under-detection.

Methods for deep learning that are used for removing dynamic features rely on semantic data collected from images [14]. Li et al. [15] proposed a system that used semantic segmentation and mixed information fusion to improve its localization accuracy and remove dynamic feature points. Xiao et al. [16] proposed a dynamic object detection method using an SSD object detector with prior knowledge and a compensation algorithm to enhance the recall rate. They also developed a feature-based visual SLAM system that minimized pose estimation errors by selectively tracking dynamic objects. Liu et al. [17] presented a dynamic SLAM algorithm based on the ORB-SLAM3 framework. It could perform better in real-time tracking accuracy when dealing with complex environments. The reliance on prior knowledge can lead to errors in deep learning such as when it mistakenly recognizes a static object as a dynamic one [18]. Simple networks may not recognize complex dynamic objects effectively, while complex networks may affect the system’s real-time performance.

Combining geometric methods with deep learning can fully leverage the strengths of both approaches Dynamic environments are first extracted from deep learning, and these methods then refine the extracted semantic information using geometric constraints. Bescos et al. [19] presented the DynaSLAM framework, which is an extension of ORB-SLAM2. It features multi-view geometry and masks-RCNN for removing dynamic points. In a previous paper, Zhao et al. [20] proposed a workflow that enabled the accurate segmenting of objects in dynamic environments. Yang et al. [21] first identified predefined dynamic targets using object detection models and then verified the depth information of dynamic pixels through multi-view constraints. Wu et al. [22] proposed YOLO-SLAM, which filtered dynamic feature points with Darknet19-YOLOv3 and geometric constraints. Wang et al. [23] proposed an algorithm combining semantic segmentation and geometric constraints to improve SLAM accuracy in dynamic indoor environments.

Although the methods above-mentioned have achieved significant improvements in accuracy, they still have the following limitations:

(1)

Systems that rely on semantic information often face difficulties in detecting unknown dynamic features. For example, chairs are typically considered static in semantic labels, but the system may fail to identify this dynamic behavior if they move.

(2)

The misclassification of stationary dynamic objects as dynamic can lead to excessive feature point removal and affect the data integrity. For example, a person sitting on a chair may be mistakenly identified as a dynamic object.

(3)

The complexity of semantic segmentation networks leads to inadequate real-time performance and reduced processing efficiency.

The EMS-SLAM is an improved SLAM algorithm that adopts an RGB-D structure and addresses the limitations and disadvantages of the current methods. This method incorporates geometric constraints and semantic information to suppress dynamic points. In addition to being able to adapt to complex environments, it also meets the requirements of real-time applications. The EMS-SLAM system framework is shown in Figure 1. The main contributions it makes are summarized below.

(1)

A feature matching algorithm leveraging graph-cut RANSAC for accurately estimating the pose of cameras;

(2)

A degeneracy-resistant geometric constraint method that combines epipolar constraints with multi-view geometric constraints, effectively resolving the degeneracy issues inherent in purely epipolar approaches;

(3)

EMS-SLAM, a real-time RGB-D SLAM system driven by an RGB-D sensor, is constructed. A new parallel thread is integrated into EMS-SLAM to extract semantic information;

(4)

A dynamic feature suppression algorithm is proposed that combines the advantages of geometric and semantic constraints to efficiently suppress dynamic points. The proposed method can greatly enhance the system’s capability to adapt to complex environments and ensure a stable and accurate performance.

2. Method

2.1. System Framework

The widely used ORB-SLAM2 system is a high-accuracy visual SLAM that includes three main components: loop-closing, tracking, and local mapping. Building on this foundation, EMS-SLAM has been enhanced for dynamic environments by incorporating additional functional modules. Figure 1 illustrates the EMS-SLAM architecture, which consists of four synchronized threads: tracking, object detection, local mapping, and loop closing.

In EMS-SLAM, RGB-D camera frames are simultaneously processed by the tracking and object detection threads during operation. Within the tracking thread, the RGB image is used to extract ORB feature points. These features are then processed using a GC-RANSAC matching method to ensure the accurate identification of inliers and outliers. The fundamental matrix between frames is then computed using the inlier points, and the fundamental matrix is then used as prior for the geometric constraints. The object detection thread employs a MobileNetV3-SSD model to detect objects in the scene. The object detection thread detects objects in the image and filters out dynamic ones. Since object detection is computationally intensive, the tracking thread has to wait for detection results after calculating the geometric information. To reduce latency, a lightweight object detection model is instead of the semantic segmentation model, which significantly reduces the processing delay.

Subsequently, the tracking thread accurately filters out dynamic points using the dynamic point suppression strategy and utilizes the remaining static points to estimate the camera’s pose. The system then transitions to the local mapping and loop closure threads. Finally, bundle adjustment optimization is carried out to improve the trajectory’s accuracy and system performance.

2.2. Object Detection

In the EMS-SLAM, we used MobileNetV3-SSD to extract semantic information from the input RGB image. The SSD algorithm, proposed by Liu et al. [24] in 2016, effectively balances the detection speed and accuracy. MobileNetV3 [25], a lightweight model introduced by Google in 2019, builds upon its predecessors to achieve high detection accuracy with minimal memory usage. Because SSD’s base network, VGG16, has numerous parameters and is unsuitable for embedded platforms, we replaced it with MobileNetV3 to construct the MobileNetV3-SSD object detection model.

We selected NCNN as the inference framework for efficient inference on mobile devices and embedded platforms. NCNN’s optimizations allow the MobileNetV3-SSD model to perform object detection with low latency and high computational efficiency. The training of the model was carried out on the PASCAL VOC 2007 dataset, which ensured its generalizability across different scenarios.

2.3. Feature Matching Method Based on GC-RANSAC

In SLAM systems, the accuracy of the initial pose estimation is important [26,27]. However, in dynamic environments, moving objects can lead to feature point mismatches, compromising the pose estimation accuracy. In order to improve the accuracy of the initial pose estimation, we proposed a method that uses the GC-RANSAC algorithm [28]. Our method addresses the problem of separating inliers and outliers through geometric and spatial consistency to improve the feature point identification accuracy.

We employed the epipolar constraint to determine whether a pair of matching points was an inlier or an outlier. Between two consecutive frames, for a given pair of matching points $(P_i,P_i^{\prime })$, they must satisfy the epipolar constraint $l_i=F{P}_i$, where $l_i$ is the epipolar line corresponding to $P_i$ in the other view, computed from the fundamental matrix $F$. The RANSAC algorithm’s iterative iteration process classifies the matching point pairs into outliers or inliers according to their constraint. The classification is achieved by optimizing the following energy function:

$$F(L)={\displaystyle \sum _i}C(L_i)+\lambda {\displaystyle \sum _{(i,j)\in G}}Q(L_i,L_j)$$

(1)

where $L=\{L_i\in \{0,1\left\}\right|i=1,\mathrm{\dots },n\}$ denotes the label assignment for the matching point $(P_i,P_i^{\prime })$, and $G$ represents the neighborhood graph.

The unary term $C(L_i)$ represents the geometric relationship, defined as:

$$C(L_i)=\left\{\begin{array}{c}K(\varphi (p_i,p_i^{\prime },\theta ),\epsilon )\\ 1-K(\varphi (p_i,p_i^{\prime },\theta ),\epsilon )\end{array}\right.\begin{array}{c}\mathrm{if}{L}_i=0\\ \mathrm{if}{L}_i=1\end{array}$$

(2)

and

$$K(\delta ,\epsilon )=e^{-{\displaystyle \frac{{\delta }^2}{2{\epsilon }^2}}}$$

(3)

where $\theta$ is the angle parameter of the fundamental matrix $F$, $\varphi (p_i,p_i^{\prime },\theta )$ represents the distance between the fundamental matrix and the matching point, $\epsilon$ is the inlier–outlier threshold, and the function is a Gaussian kernel function. $L_i=0$ indicates that the matching point $(P_i,P_i^{\prime })$ is an inlier; otherwise, it is an outlier.

The pairwise term $Q(L_i,L_j)$ is used to describe the spatial consistency between neighboring feature points, defined as:

$$Q(L_i,L_j)=\left\{\begin{array}{c}1\\ ((C(L_i)+C(L_j))/2\\ (1-(C(L_i)+C(L_j))/2\end{array}\right.\begin{array}{c}\mathrm{if}{L}_i\ne L_j\\ \mathrm{if}{L}_i=L_j=0\\ \mathrm{if}{L}_i=L_j=1\end{array}$$

(4)

By minimizing this energy function, the GC-RANSAC algorithm effectively separates inliers and outliers [29]. We then used the inliers to estimate the fundamental matrix $F$, providing accurate prior information for the epipolar constraints and enhancing the accuracy in filtering out dynamic feature points. Figure 2 illustrates an example of inlier selection using our proposed method, with the procedure summarized in Algorithm 1.

Algorithm 1 Feature Matching Method Based on GC-RANSAC
Input:	$F_1$: Previous frame; $F_2$: Current frame; $F_r$: Reference frame.
Output:	$F$: The fundamental matrix
1:  Match features between frames $F_1$ and $F_2$
2:  Get static feature points by GC-RANSAC
3:  for each static feature point $p_i$ in $F_2$ do
4:    $P_i$← FindCorresponding3DPoint($p_i$,$F_r$)
5:  end for
6:  Estimate fundamental matrix $F$ by GC-RANSAC
7:  return $F$

2.4. Epipolar Constraints

In dynamic environments, current networks can recognize object classes but cannot identify their motion status. Therefore, to eliminate dynamic points originating from static objects, we make full use of epipolar constraints to determine the motion state of feature points. The distance between the feature points and their epipolar lines is calculated, and the points that exceed a certain threshold are regarded as dynamic.

Figure 3 illustrates the optical centers of the cameras, denoted as $C_1$ and $C_2$. The lines $l_1$ and $l_2$ represent the epipolar lines. The spatial point $P$ corresponds to the feature points $p_1$ and $p_2$ observed in the previous and current frames, respectively:

$$p_1={[a_1,b_1]}^{\mathrm{T}},p_2={[a_2,b_2]}^{\mathrm{T}}$$

(5)

The coordinate $(a_1,b_1)$ and $(a_2,b_2)$ of the feature points of the two frames are shown in the above equation. Their corresponding homogeneous coordinates are:

$$P_1={[a_1,b_1,1]}^{\mathrm{T}},P_2={[a_2,b_2,1]}^{\mathrm{T}}$$

(6)

The epipolar line $l_2$ can be determined as follows in the current frame:

$$l_2=\left[\begin{array}{c}A\\ B\\ C\end{array}\right]=F{P}_1=F\left[\begin{array}{c}{a}_1\\ b_1\\ 1\end{array}\right]$$

(7)

where $A$, $B$, $C$ represent the line vector components, and $F$ represents the fundamental matrix. The epipolar constraints can be expressed as:

$$P_2^{\mathrm{T}}F{P}_1=P_2^{\mathrm{T}}{l}_2=0$$

(8)

The deviation distance $D_i$ of feature point $p_i$ to epipolar line $l_i$ is defined as follows:

$$D_i={\displaystyle \frac{\left|P_i^{T}F{P}_1\right|}{\sqrt{A^2+B^2}}}$$

(9)

The spatial point $P$ can be static or dynamic. If it is static, the corresponding feature point in the current frame is $p_2$, and its deviation distance is $D_2=0$. On the other hand, if it is dynamic, the corresponding feature point is $p_4$, and its deviation distance is $D_4>0$.

Figure 3 shows that if a point $P$ is static, its feature point $p_2$ should be on the epipolar line $l_2$. However, due to the noise in the environment, its deviation is not always zero but within the empirical threshold $D_{thresh}$. When the point moves, its distance $D$ between the epipolar line and the feature point can be calculated. This is used to determine the dynamic state of the feature.

When point $P$ moves along the epipolar direction to $P^{″}$, its corresponding feature point $p_5$ lies on epipolar line $l_2$. Therefore, epipolar constraints cannot effectively identify these dynamic objects in such cases. To address this issue, we introduced multi-view geometric constraints, building on epipolar constraints, to detect further missed dynamic feature points.

2.5. Muti-View Geometric Constraints

Dynamic scenes create significant changes in the angle and depth of the feature points in the reference frames and current scenes. Dynamic feature points can be eliminated using angular and depth information. This principle is shown in Figure 4.

Figure 4 shows that the keyframe feature point $x_{kf}$ in the reference frame is projected onto the current frame to obtain the feature point $x_{cf}$ and its projected depth $Z_{cf}$, generating the corresponding 3D point $X$. The depth information is obtained by calculating the depth value $Z_{cf}$ and $Z_{kf}$ in the current frame. The depth difference $\Delta Z$ is defined as $\Delta Z=\left|Z_{kf}-Z_{cf}\right|$. The angular deviation $\Delta \theta$ is the angle between the reprojections of $x_{kf}$ and $x_{cf}$, which is determined using the cosine rule.

By combining the depth information and angular deviation, a feature point $x_{cf}$ is identified as dynamic under multi-view geometric constraints when the depth difference $\Delta Z$ exceeds the threshold $Z_{thresh}$ or the angular deviation $\Delta \theta$ exceeds the threshold ${\theta }_{thresh}$. Here, $Z_{thresh}$ and ${\theta }_{thresh}$ are the depth and angular thresholds, respectively.

2.6. Dynamic Feature Suppression Method

This paper proposed a geometric information-based dynamic feature suppression method aimed at reducing the over-reliance on depth learning. First, the target category in formation obtained through the detection module was used as prior knowledge for the system to distinguish between moving and non-moving objects. Since different object categories exhibit distinct motion characteristics, the system assigns each feature point a weight based on its object category, with values ranging from 0 to 1. Indoor objects were classified into three categories based on their motion status: active motion, passive motion, and absolutely static, with different weights assigned to each category. The classification details are shown in

Table 1. Classification of indoor objects and movement status.

Object Type	Category	Status
Human, animals, etc.	Moving	Active movement
Chairs, books, water cups, etc.	Relative	Passive movement
Air conditioning, frames, lamps, etc.	Static	Absolutely static

.

Subsequently, epipolar constraints were used to calculate the distance between feature points and their associated epipolar lines, which was then integrated with feature weights for a more refined comparison. If the distance was below the threshold, it underwent multi-view geometric analysis to examine the angle and depth changes, further determining whether it was dynamic or static. If the distance was below the empirical threshold, the point underwent multi-view geometric analysis to examine the changes in depth and angle, further determining whether it was dynamic or static. By applying these rules, the system can accurately determine the status of all feature points in the current frame. Algorithm 2 provides a detailed explanation of the suppression strategy for dynamic feature points.

Algorithm 2 Dynamic Feature Rejection Strategy
Input:	$F_1$: Previous frame; $F_2$: Current frame; $P_1$: Previous frame’s feature points; $P_2$: Current frame’s feature points; Depth image; Standard empirical thresholds;$F$: The fundamental matrix.
Output:	$S$: Static feature points in the current frame.
1:  ReferenceFrames ← ComputeReferenceFrames($F_2$)
2:  for each matched pair ($p_1$,$p_2$) in $P_1$, $P_2$do:
3:    if dynamic object exists and $P_2$ is in a dynamic area then
if epipolar line distance and movement probability is within threshold then
Compute3DPointInReferenceFrames($p_2$)
if depth and angle are within thresholds then
$S$ ← $p_2$
end if
end if
else
if epipolar line distance is within threshold then
Compute3DPointInReferenceFrames($p_2$)
if depth and angle are within thresholds then
$S$← $p_2$
end if
8:      end if
9:     end if
18:    end for
18:  return $S$

3. Results

This section presents an evaluation of the performance of the system using publicly available TUM RGB-D [30] and Bonn RGB-D [31] datasets. Subsequently, ablation and comparative experiments were performed to validate the effectiveness of each module in the dynamic scenes. Additionally, we tested the algorithm on self-collected data. Finally, an in-depth analysis of the system’s real-time performance was carried out. We used the relative pose error (RPE) and absolute trajectory error (ATE) to evaluate the accuracy of SLAM. The optimal results for each sequence have been highlighted in bold. To address uncertainties in object detection and feature extraction, each algorithm was tested ten times per sequence, and the median value was taken as the final result. All experiments were conducted on a computer running Ubuntu 18.04, equipped with an Intel i7-9750H (2.60 GHz) CPU, an NVIDIA GeForce GTX 1660 GPU, and 16 GB of RAM.

3.1. Evaluation on the TUM RGB-D Dataset

In 2012, the TUM group released the RGB-D dataset, which has since become a standard in the SLAM sector. The dataset features various dynamic scenes, each with its own accurate ground truth trajectory. It was obtained through the use of a high-accuracy capture system. We tested EMS-SLAM by performing five dynamic scenes. Among these, the sitting sequence featured a person sitting in a chair with slight limb movement, representing a low-dynamic environment, while the walking sequence featured a person walking continuously, representing a high-dynamic environment.

EMS-SLAM was compared with ORB-SLAM2 to assess performance improvements, as shown in Figure 5 and Figure 6. As indicated by the ATE results in

Table 2. Result of ATE on the TUM dataset (m).

Sequences	ORB-SLAM2				EMS-SLAM				Improvements
	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D
fr3_s_static	0.0087	0.0078	0.0057	0.0072	0.0039	0.0050	0.0043	0.0032	55.17%	35.90%	24.56%	55.56%
fr3_w_half	0.4462	0.4096	0.3860	0.1770	0.0236	0.0182	0.0155	0.0151	94.71%	95.56%	95.98%	91.47%
fr3_w_rpy	0.5396	0.5012	0.4974	0.1999	0.0304	0.0224	0.0172	0.0205	94.37%	95.53%	96.55%	89.74%
fr3_w_static	0.4032	0.3696	0.3164	0.1627	0.0068	0.0061	0.0056	0.0030	98.32%	98.35%	98.23%	98.16%
fr3_w_xyz	0.6826	0.6086	0.6661	0.3091	0.0135	0.0113	0.0097	0.0074	98.02%	98.14%	98.54%	97.60%

, in high-dynamic sequences, EMS-SLAM achieved a 96.36% improvement in average RMSE accuracy compared with ORB-SLAM2, while the improvement in low-dynamic sequences was 55.17%. This was due to the smaller size and limited movement of dynamic objects in low-dynamic sequences. The RPE results in

Table 3. Result of RPE on the TUM dataset (m).

Sequences	ORB-SLAM2				EMS-SLAM				Improvements
	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D
fr3_s_static	0.0093	0.0082	0.0074	0.0044	0.0049	0.0049	0.0049	0.0005	47.31%	40.24%	33.78%	88.64%
fr3_w_half	0.3685	0.2072	0.0491	0.3047	0.0354	0.0337	0.0358	0.0108	90.39%	83.74%	27.06%	96.45%
fr3_w_rpy	0.3374	0.2344	0.1137	0.2426	0.0739	0.0572	0.0452	0.0467	78.10%	75.60%	60.26%	80.74%
fr3_w_static	0.2182	0.0950	0.0169	0.1965	0.0131	0.0104	0.0064	0.0079	93.99%	89.05%	62.13%	95.98%
fr3_w_xyz	0.3752	0.2944	0.2394	0.2326	0.0554	0.0459	0.0379	0.0310	85.23%	84.41%	84.17%	86.68%

and

Table 4. Result of RPE on the TUM dataset (°).

Sequences	ORB-SLAM2				EMS-SLAM				Improvements
	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D
fr3_s_static	0.2899	0.2606	0.2484	0.1271	0.1536	0.1296	0.1079	0.0824	46.99%	50.29%	56.57%	35.17%
fr3_w_half	7.9219	4.4695	1.2568	6.5406	0.4842	0.3952	0.3287	0.2797	93.89%	91.16%	73.85%	95.72%
fr3_w_rpy	6.4220	4.5134	2.2990	4.5685	0.6977	0.4584	0.3259	0.5259	89.14%	89.85%	85.82%	88.49%
fr3_w_static	3.8068	1.6993	0.3888	3.4065	0.1912	0.1668	0.1474	0.0934	94.98%	90.18%	62.08%	97.26%
fr3_w_xyz	7.1415	5.6403	4.6159	4.3804	0.4100	0.2890	0.2391	0.2908	94.26%	94.87%	94.82%	93.36%

indicate that the accuracy of the RMSE was significantly improved by EMS-SLAM compared with ORB-SLAM2. In high-dynamic sequences, the RMSE accuracy improvement was 93.07% and 86.92%, while in low-dynamic instances, it was 47.31% and 46.99%, respectively.

These findings highlight the system’s effectiveness in high-dynamic sequences, with limited improvement in low-dynamic scenarios due to the minimal impact of dynamic objects on SLAM. The ATE results presented in Figure 5 highlight the differences between the ground-truth and estimated trajectories. EMS-SLAM significantly improved the accuracy compared with ORB-SLAM2. Figure 6 presents the RPE results, where EMS-SLAM showed reduced errors.

The proposed algorithm was further evaluated by comparing its performance with other mainstream SLAM systems including DSLAM [32], RS-SLAM [33], DS-SLAM [34], RDS-SLAM [32], RDMO-SLAM [35], Blitz-SLAM [36], YOLO-SLAM [22], and SG-SLAM [37].

Table 5. Results of the RMSE of ATE on the TUM RGB-D dataset (m).

Sequences	DSLAM	RS-SLAM	DS-SLAM	RDS-SLAM	RDMO-SLAM	Blitz-SLAM	YOLO-SLAM	SG-SLAM	EMS-SLAM
fr3_s_static	0.0096	0.0066	0.0065	0.0084	0.0066	-	0.0066	0.0060	0.0039
fr3_w_half	0.0354	0.0425	0.0303	0.0807	0.0304	0.0256	0.0283	0.0268	0.0236
fr3_w_rpy	0.1608	0.1869	0.4442	0.1604	0.1283	0.0356	0.2164	0.0324	0.0304
fr3_w_static	0.0108	0.0067	0.0081	0.0206	0.0066	0.0102	0.0073	0.0073	0.0068
fr3_w_xyz	0.0874	0.0146	0.0247	0.0571	0.0226	0.0153	0.0146	0.0152	0.0135

presents the experimental results for five dynamic scene sequences. Aside from the fr3_w_static sequence, where EMS-SLAM performed slightly worse than RDMO-SLAM, it achieved a higher average accuracy than other algorithms across the remaining sequences.

3.2. Evaluation on the Bonn RGB-D Dataset

The University of Bonn’s Robotics and Photogrammetry Laboratory released the Bonn RGB-D Dynamic Dataset in 2019, which features 24 dynamic sequences that can be used for various scenarios such as walking people and moving objects. These sequences were recorded using ground-truth camera trajectories from OptiTrack Prime 13. Nine representative sequences were selected to evaluate the system performance. In the crowd sequence, three people were simulated walking randomly indoors, while the moving_no_box sequence depicted the process of moving a box from the ground to a table without obstructing the line of sight. In the person_tracking sequence, the camera followed a slow-moving individual, and in the synchronous sequence, two people were shown moving at the same speed and in the same direction. These highly dynamic scenes present significant challenges for traditional SLAM systems.

The EMS-SLAM system was compared with three standard SLAM systems to verify its robustness and accuracy: ORB-SLAM2, SG-SLAM, and YOLO-SLAM. The evaluation results of the nine scene sequences are presented in

Table 6. Results of the RMSE of ATE on the Boon RGB-D dataset (m).

Sequences	ORB-SLAM2	YOLO-SLAM	SG-SLAM	EMS-SLAM
crowd	0.8632	0.033	0.0234	0.0165
crowd2	1.3573	0.423	0.0584	0.0576
crowd3	1.0772	0.069	0.0319	0.0426
moving_no_box	0.1174	0.027	0.0192	0.0137
moving_no_box2	0.1142	0.035	0.0299	0.0216
person_tracking	0.7959	0.157	0.0400	0.0228
person_tracking2	1.0679	0.037	0.0376	0.0331
synchronous	1.1411	0.014	0.3229	0.0069
synchronous2	1.4069	0.007	0.0164	0.0231

. In the crowd3 sequence, frequent mutual occlusions, overlapping actions, and changing viewpoints among people in the scene led to false positives and missed detections, affecting the overall performance of the algorithm. In the synchronous2 sequence, the dynamic objects moved synchronously at similar speeds and in the same direction, causing the motion differences relied upon by the epipolar constraints to become indistinct, which in turn affected the accuracy of the motion estimation. Although EMS-SLAM’s performance was slightly inferior to other algorithms on the crowd3 and synchronous2 sequences, EMS-SLAM significantly outperformed the other algorithms in the remaining sequences. These results not only further highlight the EMS-SLAM system’s exceptional accuracy and robustness in dynamic environments, but also showcase its generalization ability across various complex dynamic scenarios.

3.3. Effectiveness Validation of Each Module

3.3.1. Ablation Experiments

The experiments were performed to evaluate the performance of the multi-view geometric and feature matching modules in the TUM RGB-D dataset. Trajectory estimation results were compared across three configurations: EMS-SLAM without the improved feature matching module, EMS-SLAM without the multi-view geometric module, and the complete EMS-SLAM system.

The results of the ablation procedures presented in

Table 7. Results of ATE from the ablation experiments on the TUM RGB-D dataset.

Sequences	Without Improved Feature Matching Module				Without Multi-View Geometric Module				Complete System
	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D
fr3_s_static	0.0086	0.0070	0.0056	0.0050	0.0067	0.0057	0.0048	0.0036	0.0060	0.0050	0.0043	0.0032
fr3_w_half	0.0283	0.0215	0.0175	0.0185	0.0284	0.0215	0.0176	0.0185	0.0236	0.0182	0.0155	0.0151
fr3_w_rpy	0.0755	0.0608	0.0491	0.0448	0.0797	0.0615	0.0457	0.0505	0.0304	0.0224	0.0172	0.0205
fr3_w_static	0.0090	0.0074	0.0064	0.0052	0.0081	0.0070	0.0064	0.0040	0.0068	0.0061	0.0056	0.0030
fr3_w_xyz	0.0244	0.0216	0.0192	0.0113	0.0161	0.0132	0.0113	0.0092	0.0135	0.0113	0.0097	0.0074

are shown in detail. The system’s performance in highly dynamic sequences was 36.37% higher than that of the configuration without GC-RANSAC. In contrast, in low-dynamic sequences, the performance was 30.23% lower. Compared with the system without the multi-view geometric algorithm, the complete system achieved an average improvement of 24.68% in highly dynamic sequences and 10.23% in low-dynamic sequences. The main reason is that the smaller dynamic areas and magnitudes reduced the impact of geometric constraints in low-dynamic sequences.

3.3.2. Comparative Experiments

The experiments were conducted to evaluate the effectiveness of the dynamic feature suppression strategy. Three different approaches were used: a geometric algorithm, a semantic algorithm, and a combined algorithm. The results of the experiments are shown in Figure 7 and show the effectiveness of the different approaches in detecting dynamic points.

(1)

EMS-SLAM (S): Removes dynamic feature points only using semantic information.

(2)

EMS-SLAM (G): Removes dynamic feature points only using geometric information.

(3)

EMS-SLAM (S+G): Removes dynamic feature points by combining semantic and geometric information.

The results, shown in Figure 7 and

Table 8. Results of ATE from the comparative experiments on the TUM RGB-D dataset.

Sequences	EMS-SLAM (S)				EMS-SLAM (G)				EMS-SLAM (S+G)
	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D	RMSE	Mean	Median	S.D
fr3_s_static	0.0059	0.0055	0.0053	0.0018	0.0070	0.0061	0.0053	0.0035	0.0060	0.0050	0.0043	0.0032
fr3_w_half	0.1347	0.0905	0.0708	0.0998	0.0448	0.0342	0.0281	0.0290	0.0236	0.0182	0.0155	0.0151
fr3_w_rpy	0.1436	0.1184	0.0843	0.0813	0.0222	0.0162	0.0109	0.0152	0.0304	0.0224	0.0172	0.0205
fr3_w_static	0.0095	0.0079	0.0066	0.0053	0.0082	0.0071	0.0061	0.0041	0.0068	0.0061	0.0056	0.0030
fr3_w_xyz	0.0145	0.0124	0.0108	0.0075	0.0159	0.0115	0.0079	0.0109	0.0135	0.0113	0.0097	0.0074

, demonstrate the comparative performance of each method. ORB-SLAM2, which does not handle dynamic features, indiscriminately extracted feature points in dynamic areas. EMS-SLAM (S), using only semantic information, misclassified many static feature points as dynamic such as those on a computer monitor. EMS-SLAM (G), relying on geometric information, still misidentified some dynamic features as static such as those on a person’s knee while seated. In contrast, EMS-SLAM (S+G), combining semantic and geometric information, effectively removed all dynamic features associated with moving objects while retaining the static object features, outperforming both of the other algorithms.

The two algorithms based on individual information had different strengths and weaknesses across sequences. In contrast, the algorithm delivered the best results in all test sequences by combining the geometric and semantic information. However, in the fr3_w_rpy sequence, complex camera movements caused dynamic objects to blur, resulting in the geometric algorithm having significantly lower errors than the semantic and combined algorithms. Based on the experimental results in Table 8, in high-dynamic scenes, the combination of semantic and geometric methods improved the RMSE accuracy by 79.17% and 8.70%, respectively, when compared with the semantic or geometric methods individually. In low-dynamic scenes, the combined method yielded results that were nearly identical to those obtained using semantic or geometric methods alone. This suggests that the integration of semantic and geometric information effectively compensates for the limitations of each method, thereby enhancing the overall performance of the system.

3.4. Evaluation Using Our Collected Dataset

EMS-SLAM was tested on the dynamic scene data collected with a Kinect v2 camera to validate its performance in complex dynamic scenarios. The data, collected in a laboratory setting with people walking, included two dynamic environments named sdjz_w_static_01 and sdjz_w_static_02. The datasets contained color and depth images at a resolution of 640 × 480. The trajectories of the EMS-SLAM and ORB-SLAM2 were evaluated by keeping the Kinect v2 stationary while collecting data. As a result, the Kinect v2 sensor’s actual trajectory was a fixed point in space, and the error was computed as the distance between its estimated and actual trajectory.

The extraction of feature points for the various sequences presented in Figure 8 was performed for the two different systems: ORB-SLAM2 and EMS-SLAM. Compared with ORB-SLAM2, EMS-SLAM effectively removed all feature points on dynamic objects. As shown in

Table 9. Results of the RMSE on our collected dataset.

Sequences	Error Type	ORB-SLAM2	EMS-SLAM	Improvement
Sdjn_w_static_01	ATE (m)	0.1568	0.0087	94.45%
	Translational RPE (m)	0.2393	0.0078	96.74%
	Rotational RPE (°)	0.3534	0.0270	92.36%
Sdjn_w_static_02	ATE (m)	0.3627	0.0136	96.25%
	Translational RPE (m)	0.0811	0.0051	93.71%
	Rotational RPE (°)	0.1453	0.0130	91.05%

, the RMSE accuracy of the ATE improved by 94.45% and 96.25% for the sdjz_w_static_01 and sdjz_w_static_02 sequences, respectively.

3.5. Timing Analysis

The performance of EMS-SLAM was compared with that of other systems by measuring the average time it took for each frame on the TUM dataset, as shown in

Table 10. Time analysis (ms).

Algorithm	Average Tracking Time	Hardware Platform
ORB-SLAM2 (OURS)	28.25	Intel i7 CPU, NVIDIA GeForce GTX 1660 GPU
EMS-SLAM (OURS)	34.47	Intel i7 CPU, NVIDIA GeForce GTX 1660 GPU
YOLO-SLAM	696.09	Intel Core i5-4228U CPU
DS-SLAM	59.40	Intel i7 CPU.P4000 GPU
RDS-SLAM	57.50	Nvidia RTX 2080TiGPU
YS-SLAM	57.57	Imel i7 CPU, Nvidia GTX 3060 GPU. 6 GB RAM

. Systems such as DS-SLAM, YS-SLA [38], and RDS-SLAM utilize pixel-level semantic segmentation networks, significantly increasing their average frame processing time. While YOLO-SLAM employs an object detection algorithm, its real-time performance is limited by hardware constraints. In contrast, EMS-SLAM uses a lightweight object detection network, greatly enhancing the inference efficiency and processing speed on resource-constrained devices. The average frame processing time for EMS-SLAM increased by only 6.22 milliseconds compared with ORB-SLAM2, fully satisfying the real-time performance requirements of SLAM.

4. Discussion

The EMS-SLAM algorithm proposed in this paper has made significant progress in the field of real-time SLAM, particularly in dynamic environments. By integrating a feature matching method based on GC-RANSAC, we enhanced the initial camera pose estimation, which is crucial for achieving accurate localization in complex scenarios. Additionally, our degeneracy-resistant geometric constraint method, which combines epipolar constraints with multi-view geometric constraints, further strengthens the robustness of the algorithm, addressing common limitations faced by traditional SLAM methods. The inclusion of a parallel object detection thread ensures that the system can handle dynamic objects in real-time, making it suitable for a wide range of practical applications.

However, during the experiments, we observed that the performance of EMS-SLAM may be compromised when images are blurred, particularly in the extraction of semantic information. This issue is particularly pronounced when the camera experiences significant motion or operates in low-light conditions, both of which are common challenges in real-world applications. In these scenarios, feature extraction and matching become unreliable, potentially leading to reduced accuracy in the final pose estimation.

Future improvements will focus on addressing this limitation by enhancing the semantic information extraction process. This could involve integrating more advanced image processing techniques, such as deep learning-based feature extraction methods, which are better equipped to handle blurry or low-quality images.

5. Conclusions

The proposed EMS-SLAM is a real-time SLAM algorithm designed to enhance the accuracy and robustness in dynamic environments. To more accurately estimate the initial camera pose, we proposed a feature matching method based on GC-RANSAC that utilized geometric and spatial consistency. In addition, for the degeneracy problem of epipolar constraints, we presented a degeneracy-resistant geometric constraint method that combines epipolar constraints with multi-view geometric constraints. A parallel object detection thread was added to EMS-SLAM. Finally, we proposed a dynamic point suppression method that integrated the semantic information and geometric constraints. Experimental results demonstrated that EMS-SLAM outperformed other state-of-the-art SLAM algorithms on the TUM RGB-D, Bonn RGB-D, and our collected datasets. Ablation and comparative experiments further validated the contributions of each module.