A Semantic Segmentation-Based GNSS Signal Occlusion Detection and Optimization Method

Abstract

Existing research fails to effectively address the problem of increased GNSS positioning errors caused by non-line-of-sight (NLOS) and line-of-sight (LOS) signal attenuation due to obstructions such as buildings and trees in complex urban environments. To address this issue, we dig into the environmental perception perspective to propose a semantic segmentation-based GNSS signal occlusion detection and optimization method. The approach distinguishes between building and tree occlusions and adjusts signal weights accordingly to enhance positioning accuracy. First, a fisheye camera captures environmental imagery above the vehicle, which is then processed using deep learning to segment sky, tree, and building regions. Subsequently, satellite projections are mapped onto the segmented sky image to classify signal occlusions. Then, based on the type of obstruction, a dynamic weight optimization model is constructed to adjust the contribution of each satellite in the positioning solution, thereby enhancing the positioning accuracy of vehicle-navigation in urban environments. Finally, we construct a vehicle-mounted navigation system for experimentation. The experimental results demonstrate that the proposed method enhances accuracy by 16% and 10% compared to the existing GNSS/INS/Canny and GNSS/INS/Flood Fill methods, respectively, confirming its effectiveness in complex urban environments.

1. Introduction

In modern navigation systems, the Global Navigation Satellite System (GNSS) represents a widely adopted high-precision positioning technology. Capable of providing all-weather and high-precision positioning services, this system has evolved into a critical technology underpinning various domains, including transportation infrastructure and smart city development [1,2]. GNSS can deliver reliable positioning accuracy in open environments [3]. However, in complex environments such as urban canyons and tree cover, it is susceptible to non-line-of-sight (NLOS) errors due to obstructions and other disturbances, leading to a sharp drop in positioning accuracy or even loss of positioning capability [4,5].

Scholars worldwide have conducted extensive research on mitigating the impact of NLOS signals in positioning and navigation systems. In this paper, we primarily categorize the research into three approaches: receiver design, NLOS identification based on signal feature parameters, and NLOS detection assisted by external sensors or information sources. In the domain of receiver design, Suzuki [6] proposed a rotating GNSS antenna configuration to mitigate NLOS-induced errors. Jiang [7] employed a delay lock loop that estimates multipath and combined it with a vector tracking loop to mitigate NLOS induced errors. Xu [8] developed a vector tracking loop (VTL)-based algorithm that uses metrics like equivalent noise bandwidth and multi-correlator peak time delays to detect and correct NLOS errors. However, in urban environments with dynamic positioning where obstacles may change their positions and shapes, these approaches show limited applicability. In the context of signal characteristic parameter-based approaches, Han [9] developed a weighted model incorporating azimuth and elevation angles to enhance positioning accuracy, primarily addressing positioning errors induced by multipath effects. Zidan [10] utilized pseudo-range and Doppler measurements in conjunction with a decision tree classifier to categorize satellite signals as either line-of-sight (LOS) or NLOS. These classified signals are then processed based on their respective categories. Ng [11] developed a signal weighting model aimed at reducing positioning errors, utilizing pseudo-range measurements, Signal-to-Noise Ratio (SNR), and satellite elevation angle. Nonetheless, these weight-based models largely rely on empirical tuning and statistical assumptions. When positioning conditions degrade and measurement errors become more complex, these models require recalibration and refinement [12]. Hsu [13] leveraged key features including SNR, SNR fluctuation amplitude, pseudo-range residuals, and pseudo-range fluctuation rates, achieving a signal classification accuracy of 75% through a Support Vector Machine (SVM) model. Luo [14] enhanced signal recognition accuracy to 81.98% by utilizing a gradient-enhanced decision tree with SNR, pseudo-distance residual and elevation angle as input light gradient boosting machine (LGBM) model. Jang [15] alleviates NLOS errors by integrating a multivariate long short-term memory fully convolutional network with a binary tree. Liu [16] developed a convolutional neural network (CNN)-based model for NLOS signal detection, utilizing six double-difference features extracted from GNSS observations. Li [17] employed machine learning techniques that leveraged pseudo-range, SNR, elevation angle and azimuth angle to detect NLOS signals. These machine learning-based approaches for classifying NLOS/LOS signals through signal feature analysis have significantly reduced reliance on human intervention and empirical knowledge. However, in complex environments, signals are prone to noise-induced distortions, which can compromise model accuracy. Particularly in complex urban settings, such interference may lead to classification errors in GNSS signals categorization and result in erroneous conclusions.

In terms of incorporating external sensors or information sources. Bétaille [18] distinguished between NLOS and LOS signals by integrating a 3D city model and applying pseudo-range measurement corrections to the localization solution, which requires an initial high-accuracy localization result. Luo [19] proposed an enhanced 3DMA RT algorithm to detect NLOS/LOS signals. However, in areas with incomplete 3D building models, the detection performance decreases as it relies on the accuracy and timeliness of the 3D models. Zhang and Wang [20,21] leveraged LiDAR-generated 3D point clouds to rapidly capture the surrounding environment, enabling effective detection of NLOS signals. However, LiDAR systems remain relatively costly and the rapid processing of 3D point cloud data imposes specific computational resource requirements. With the continuous advancement of vision sensor technology and the gradual reduction in cost, fisheye cameras have become an increasingly attractive option due to their high cost-effectiveness. Marais [22] employed the fisher clustering algorithm in combination with a fisheye camera to classify the sky region into occluded and open sky areas for NLOS/LOS signals identification. Kato [23] employed the K-Means clustering algorithm to detect and extract the sky view range. Sánchez [24] utilized the Canny edge detection method to extract sky visibility ranges. Wen [25] determined NLOS/LOS signals by analyzing the average pixel values in the neighborhood of the satellite’s projected position on the camera image. Our team extracted the sky region using image expansion, erosion, and edge detection and then integrated it with the Flood Fill algorithm to determine NLOS/LOS signals [26]. Nevertheless, when capturing a sky image using a fisheye camera, the color of buildings or its edges may resemble the sky, or the sky area may be fragmented into multiple regions. These factors can lead to deviations in sky area extraction from its actual extent. For example, if the Flood Fill algorithm’s initial position is set in an occluded region, there is a high likelihood of extraction errors. Additionally, most existing fisheye-based approaches classify regions only as sky or obstruction without further distinguishing between different types of occluders such as buildings or trees.

As reviewed above, most existing studies, after classifying GNSS signals as LOS or NLOS, apply a binary weighting strategy in the positioning solution. However, these approaches often overlook the distinct propagation effects introduced by different types of obstructions. In practice, satellite signals passing through tree canopies are mainly affected by diffraction and scattering due to vegetation, and often still retain a direct LOS component. In contrast, building-induced obstructions completely block direct signals, allowing only refracted or reflected components to reach the receiver—an inherently different propagation mechanism [27,28,29]. These two types of obstructions result in distinct GNSS error characteristics and positioning biases. Therefore, distinct optimization strategies should be applied to NLOS signals caused by buildings and LOS signals with attenuation due to tree canopies. In this paper, signals obstructed by tree canopies are not categorized as NLOS but are treated as attenuated LOS signals.

To address these issues, building on the team’s previous research, this paper proposes a semantic segmentation-based GNSS signal occlusion detection and optimization method. Leveraging the capability of deep learning to adaptively extract semantic features such as edges and textures, the proposed method effectively distinguishes between sky, vegetation-obstructed, and building-obstructed regions. A fisheye camera and vehicle-mounted sensors are used to capture sky images and heading information. The sky images are then processed using a deep learning model to segment them into sky, tree, and building regions. The received GNSS signals are projected onto the segmented sky image based on the satellite’s altitude angle, azimuth angle, and the carrier’s heading angle, which are derived from the navigation message. Subsequently, based on the pixel classification of the projection location, GNSS signals are identified as NLOS, attenuated LOS, or LOS. Building on this, a satellite signal weight updating strategy that accounts for the impact of different occlusions is further proposed. Finally, different weight optimization strategies are designed for satellite signals affected by different types of obstructions, enabling efficient utilization of various GNSS signals and enhancing positioning accuracy in urban environments. This paper is organized as described below.

Section 1 primarily reviews research conducted by scholars on NLOS signal identification and processing. Section 2 presents the methods for detecting and identifying NLOS signals, along with the weight optimization techniques used in both NLOS signal detection and localization. Section 3 provides the experimental results, analysis, and a comparison of our method with existing approaches. Finally, Section 4 offers conclusions and outlines future work.

The contributions of this paper are as follows:

• To more accurately extract the sky region and types of occlusions from sky images, this study leverages the semantic feature learning capabilities of deep learning models, particularly their adaptability to edges and textures, for image segmentation. The extraction accuracy is compared with that of existing methods.
• Based on satellite attitude information, the satellite projection is mapped onto the images processed by the deep learning model, enabling the classification and detection of GNSS signal types. These signals are categorized into LOS, attenuated LOS, and NLOS.
• The degree of occlusion is quantified by calculating the shortest pixel distance from the satellite projection point to the nearest sky region. Additionally, a weight optimization scheme is developed based on different types of obstacles.
• Different optimization strategies are applied depending on the type of GNSS signal occlusion. Comparative results demonstrate that the proposed method achieves more accurate positioning and navigation performance than existing approaches.

2. Materials and Methods

In urban environments, complex factors such as buildings, trees, and other occluding objects hinder the LOS propagation of GNSS signals, thereby posing significant challenges to vehicle positioning accuracy. Currently, although environment-aware methods have been applied, they struggle to accurately identify specific occlusions in overhead images and frequently produce NLOS signal recognition errors. In recent years, deep learning technology has gained widespread adoption across various fields particularly in image recognition due to its outstanding performance, where it demonstrates significant potential. In light of this, we propose an innovative approach that leverages deep learning technology for semantic segmentation of sky images. This method enables precise identification of various obstructions, thereby effectively avoiding misclassification of NLOS signals and enhancing the accuracy and reliability of vehicle positioning systems. The primary steps of the methodology proposed in this paper are illustrated in Figure 1.

As illustrated in the overall methodological flowchart (Figure 1), first, sky image data and satellite navigation messages are collected using a fisheye camera and MEMS based integrated navigation device. Training and validation datasets for the model are then created based on the acquired sky environment data. Subsequently, the acquired sky environment data are processed to generate the training and validation datasets required for model training. Next, using the generated weighting model, semantic segmentation is applied to the skyward environmental data collected along the vehicle’s driving path. Satellite projection is then conducted by incorporating the satellite altitude and azimuth information decoded from the navigation message. Based on the pixel classification at the projected locations, the GNSS signal class is identified, and its corresponding weight is optimized using different strategies. Finally, the combination with the IMU is used to obtain the localization and navigation results.

2.1. Semantic Segmentation of Sky-Directed Images

The fisheye camera can capture sky images synchronized with GNSS signals, effectively capturing the receiver’s surroundings and reflecting the geometric distribution of obstructions such as surrounding buildings and trees. To detect and identify the GNSS signal type, it is necessary to extract and classify both the direct sky view and the sheltered area.

In reference [26], edge detection and the Flood Fill algorithm are used for sky region extraction. However, in complex urban environments, buildings such as flyovers and street signs may obscure the center of the image. When the seed points selected by the Flood Fill algorithm fall on occlusions, incorrect or invalid sky region extraction may occur. Other commonly used segmentation algorithms include K-Means clustering and Canny edge detection; however, parameter adjustment during preprocessing relies on experience, which leads to discrepancies between the extracted and actual sky regions. Moreover, these methods typically segment images into only two categories: sky and obstruction. As discussed in Section 1, tree and building occlusions have fundamentally different impacts on GNSS signal propagation. Therefore, it is necessary to further refine obstruction classification during image segmentation to identify specific types of occluders. This enables the development of targeted weighting strategies in the optimization stage based on the identified obstruction type.

To address the aforementioned issues and leverage the advantages of deep learning in image segmentation, this study employs the DeepLabv3+ algorithm (as detailed in references [30,31]) to process fisheye sky images and reliably extract sky regions, vegetation-obstructed areas, and building-obstructed areas. The process is primarily divided into the following steps.

2.1.1. Image Acquisition

In this study, the image data were collected using a camera interfaced through the Robot Operating System (ROS). The overall acquisition process is illustrated in Figure 2.

As shown in Figure 2, the collected data are stored in the ROS bag format. During the acquisition process, the status of the data stream and the integrity of the data are checked to ensure the completeness and reliability of the final dataset. Once verified, the ROS bag files are parsed. During parsing, the image data undergo necessary format conversion and resizing to ensure compatibility with subsequent processing and analysis requirements.

2.1.2. Image Preprocessing and Model Training

Urban sky images include objects beyond buildings and trees such as street lights and power lines, but these elements alone are insufficient to produce a NLOS signal. Therefore, when using the Labelme tool to produce the labels required for deep learning training, objects such as street lights and antennas present in the sky region that do not produce NLOS signals are not extracted separately. Corresponding json files are generated during the labeling process.

After creating the labels, the corresponding 8-bit color mask images are generated for training based on these label files, as shown in Figure 3.

The preprocessed images, along with the original sky-view images captured by the vehicle, are fed into the DeepLabV3+ model for training and validation to obtain the semantic label of each pixel. The overall process is illustrated in Figure 4.

2.1.3. Semantic Segmentation of Images

Based on the parameter settings during training, multiple weight models are generated. The format of the collected data (video or image) and the data are processed using either video streaming or batch processing of image folders. To facilitate subsequent satellite projection onto the image, this paper employs the batch processing method for image folders.

As shown in Figure 5, this paper classifies the pixels within the segmented sky image into three categories: sky, tree shaded regions, and building shaded regions. The sky is represented in red, the tree shaded region in green, and the building shaded region in black.

2.2. Satellite Signal Projection

The method proposed in this paper projects the tracked satellites onto the semantically segmented image and distinguishes the GNSS signal categories. The GNSS signals are mapped onto the segmented image based on the satellite’s elevation angle and azimuth angle, which are derived from the heading angle obtained by the onboard instrument and the navigation message received by the receiver. The projection modeling framework is illustrated in Figure 6.

The fisheye camera used in this study follows the modeling approach described in reference [26], where the pixel distance from the satellite projection point to the image center is defined in (1).

$$R_l=2·F_c·\tan \left({\displaystyle \frac{\pi }{4}}-{\displaystyle \frac{E}{2}}\right)$$

(1)

where $F_c$ represents the focal length of the camera.

Thus, based on the given image center coordinates, the pixel coordinates of the satellite projection on the image are determined as shown in (2).

$$\left\{\begin{array}{c}{x}_s=x_c+R_l\times \sin (H+A)\\ y_s=y_c+R_l\times \cos (H+A)\end{array}\right.$$

(2)

where $\left(x_c,y_c\right)$ represent the pixel coordinates of the image center, $\left(x_s,y_s\right)$ denote the pixel coordinates of the satellite projection on the image, $H$ is the heading angle, and $A$ is the satellite azimuth angle.

An example of satellite projection for a calendar element is shown in Figure 7, where the green color represents an unobstructed LOS signal, with C denoting BDS, G denoting GPS, and E denoting Galileo. The satellite signal types depicted in Figure 7 are listed in

Table 1. Epoch Signal Recognition and Detection Results.

	LOS	Attenuation	NLOS
PRN	E12 E17 C9 G5	G11 G13 E23 G20	C4 E22 G29

, including C4, C5, C9, G4, G5, G20, G29, G11, G29, E23, E22 and E17. The occluded satellites in this calendar element are G11 and G29, as indicated by red X marks.

2.3. GNSS Signal Weight Optimization

Following the signal classification process described in Section 2.2, and as discussed in Section 1, building and tree occlusions affect satellite signals in different ways. Building obstructions typically block the direct LOS, resulting in refracted NLOS signals. In contrast, tree canopy interference leads to diffracted and scattered signals, which are attenuated but still maintain a direct LOS component [32,33]. In light of these differences, this study does not apply a unified weighting strategy during the optimization stage. Instead, distinct weighting schemes are designed, respectively, for NLOS and attenuated LOS signals.

To characterize the degree of signal occlusion, it is based on the pixel distance from the occluded signal to the nearest sky range, which is determined by counting the pixel coordinates of the satellite projected onto the image, as given in (3).

$$dis{t}_{\min }=\underset{j=1\cdot \cdot \cdot \cdot \cdot \cdot n}{\min }\sqrt{{\left(x_{con}^j-x_s\right)}^2+{\left(y_{con}^j-y_s\right)}^2}$$

(3)

where $n$ represents the number of pixel points at the boundary of the sky region and $dis{t}_{\min }$ denotes the shortest pixel distance to the main sky region based on the pixel coordinates of the satellite projection points. $x_{con}^j$ and $y_{con}^j$ correspond to the horizontal and vertical coordinates of the boundary points, while $x_s$ and $y_s$ represent the horizontal and vertical coordinates of the satellite projection points.

In GNSS positioning, satellite signals are typically weighted based on parameters such as elevation angle and Signal-to-Noise Ratio (SNR). A common approach involves classifying signals into LOS and NLOS categories using a fixed SNR threshold, followed by applying a uniform weighting strategy to NLOS signals. However, the selection of such a threshold is highly sensitive: a threshold that is too low may misclassify LOS signals as NLOS, while a threshold that is too high may result in NLOS signals being incorrectly treated as LOS [34]. In this paper, we optimize the weights of GNSS signals affected by various occlusions based on their types and the degree of occlusion, as characterized. The formula is presented in (4).

$$w_n=\left\{\begin{array}{ll}0,\hfill & dis{t}_{\min }>R_{\max }\hfill \\ {\displaystyle \frac{dis{t}_{\min }}{\gamma }}\times {\displaystyle \frac{1}{{\sin }^{2}E}}\times {10}^{-{\displaystyle \frac{SN{R}_n-a}{b}}}\times {\displaystyle \frac{b}{{10}^{\frac{-\left(c-a\right)/b-1}{c-a}\left(SN{R}_i-a\right)+1}}}\hfill & dis{t}_{\min }\le R_{\max }\&\&I=I_{tree}\hfill \\ {\displaystyle \frac{dis{t}_{\min }}{\beta }}\times {\displaystyle \frac{1}{{\sin }^{2}E}}\times {10}^{-{\displaystyle \frac{SN{R}_n-a}{b}}}\times {\displaystyle \frac{b}{{10}^{\frac{-\left(c-a\right)/b-1}{c-a}\left(SN{R}_i-a\right)+1}}}\hfill & dis{t}_{\min }\le R_{\max }\&\&I=I_{other}\hfill \end{array}\right.$$

(4)

where $SN{R}_n$ denotes the SNR ratio of the nth satellite. $a$,$b$,$c$ are empirical values for SNR weighting [34,35]. $\gamma$ and $\beta$ are empirically determined parameters for different types of obstructions, obtained through multiple sets of experiments. $dis{t}_{\min }$ is the shortest pixel distance to the main sky region calculated from the pixel coordinates of the satellite projection point. $SN{R}_n$ indicates the SNR of the satellite corresponding to NLOS signals. $R_{\max }$ is the threshold value for satellite image coordinates based on the distance from the sky boundaries. $I$ denotes the picture type. $I_{tree}$ represents the occlusion object type, which is tree. And $\&\&$ denote concatenation.

Finally, the optimization method is applied to GNSS positioning calculations. The weight matrix obtained based on (4) is shown in (5). For the GNSS position calculation process, consult reference [36].

$$W=\left[\begin{array}{cccc}{w}_1& & & \\ & w_2& & \\ & & \cdot \cdot \cdot & \\ & & & w_n\end{array}\right]$$

(5)

As shown above, the flowchart of the proposed method in this paper is shown in Algorithm 1.

Algorithm 1. GNSS Signal Occlusion detection and Correction

Input: GNSS information, heading information, and sky environment information.

Output: Weights $\left(w\right)$ GNSS signal categories, weights, positioning results.

Steps:

for i = 1, 2, … N

Segment the Nth sky environment image following the steps in Section 2-B.

for j = 1, 2, … M

Obtain the array of sky edge pixel coordinates.

Based on (1) and (2), construct a satellite signal projection model to project the                Mth satellite from the image’s corresponding epoch onto the segmented image.

Based on (3), construct an occlusion-degree representation model to calculate                the distance from the projection point to the sky edge $\left(dis{t}_{\min }\right)$

Obtain the GNSS signal category.

if $dis{t}_{\min }>R_{\max }$

Satellite discarding.

Obtain 0.

else if $dis{t}_{\min }\le R_{\max }\&\&I=I_{tree}$

Use the optimization method for tree occluders from (4).

Obtain w.

else

Use the optimization method for buildings or other occluders from (4).

Obtain w.

end

end

end

Perform positioning calculation, obtain the positioning results.

3. Vehicle-Mounted Experiment

To verify the effectiveness of the proposed method, we constructed a vehicle-mounted experimental platform and conducted validation and analysis based on measured data.

3.1. Experimental Platform

The experimental instrument used in this study primarily comprises a USB fisheye camera, a high-precision MEMS-based integrated navigation system, and a high-precision fiber-optic integrated navigation system. The camera captures images at a resolution of 640 × 480 pixels and a rate of 20 frames per second. The high-precision MEMS-based integrated navigation system used to collect satellite navigation messages and IMU information is the CGI-410 (Huace, Hangzhou, China). The HY-P1750 high-precision fiber-optic integrated navigation system comprising a KVH1750 IMU (KVH Industries, Middletown, RI, USA) and a NovAtel GNSS receiver (NovAtel, Calgary, AL, Canada) is used as the reference system. The reference positioning results are derived from post-processing calculations performed with the Inertial Explorer (version 9.00, NovAtel, Canada). Figure 8 displays an enlarged view of the experimental platform and instrument used in this study, while

Table 2. Data Acquisition Hardware Specifications.

GNSS and Camera
SPAN-SE	dual-frequency L1/L2GPS + GLONASS + B1-2/B2bBDS
Combination	CGI-410
Focal length	2.2 mm
VFOV	90%
Image resolution	640 × 9480
Image frame rate	20 fps

lists the parameters of the associated hardware. In the container shown in Figure 8, the NovAtel GNSS receiver and KVH1750 IMU are integrated alongside the display, with the CGI-410 positioned above the HY-P1750. The image data captured by the camera are stored on the NVIDIA Jetson Xavier NX motherboard (NVIDIA Corporation, Santa Clara, CA, USA) housed within the platform.

The hardware specifications of the primary experimental instrument shown in Figure 8 are presented in Table 2.

Figure 9 illustrates the driving path of the experimental vehicle, with the GNSS receiving antenna positioned at approximately 100 cm in height. The primary sources of occlusion along the vehicle’s route are trees and buildings. The presence of tall buildings captured by the camera indicates that the signal path between the satellites and the GNSS antenna is influenced by building height, which may lead to degraded GNSS performance.

3.2. Selection of $\gamma$ and $\beta$

In the weight optimization model, $\gamma$ and $\beta$ are empirical parameters used to represent the influence of different obstructions (such as trees and buildings) on GNSS signal. To determine appropriate values for these parameters, multiple sets of experiments were conducted. Figure 9a,b show the two trajectories from replicate experiments.

For the two trajectories depicted in Figure 9, multiple combinations of parameters $\gamma$ and $\beta$ were tested to determine the most appropriate range. Through multiple experiments, we found that when parameter $\gamma$ ranges between 4 and 23 and parameter $\beta$ ranges between 10 and 70, the model demonstrates better performance with lower positioning errors. Therefore, we selected three combinations from our experiments for further analysis. Figure 10 and Figure 11 illustrate the X, Y, and Z direction errors for three different combinations of $\gamma$ and $\beta$ applied to Path 1 and Path 2, respectively.

From Figure 10 and Figure 11, it can be seen that in the X and Y directions, the combination of $\gamma =10$, $\beta =50$ performs more stably, while the other two combinations show more sudden changes and larger errors. In most cases, the error is smaller than the other two combinations.

Table 3. RMSEs of Path 1 and Path 2.

	Path 1 RMSE(m)			Path 2 RMSE(m)
	X	Y	Z	X	Y	Z
$\begin{array}{l}\gamma =20\\ \beta =60\end{array}$	2.37	5.83	7.09	1.53	2.82	5.40
$\begin{array}{l}\gamma =10\\ \beta =50\end{array}$	1.77	5.20	4.57	0.99	1.65	4.14
$\begin{array}{l}\gamma =5\\ \beta =30\end{array}$	2.73	6.36	5.26	1.17	2.58	3.83

corresponds to the root-mean-square errors (RMSEs) of Path 1 and Path 2.

As shown in Table 3, the combination of $\gamma =10$ and $\beta =50$ yields lower RMSEs in the X and Y directions on both paths compared to the other two combinations. Specifically, compared to the $\gamma =20$, $\beta =60$ combination, the RMSE in the X and Y directions on Path 1 is reduced by 25% and 36%, respectively, and by 35% and 41% on Path 2. Compared to the $\gamma =5$, $\beta =30$ combination, the RMSE is reduced by 35% and 18% on Path 1 and by 15% and 36% on Path 2. The

Table 4. RMSE of Path 1 and Path 2 with $\gamma =10$.

	Path 1 RMSE(m)			Path 2 RMSE(m)
	X	Y	Z	X	Y	Z
$\beta =10$	1.83	5.20	5.84	1.55	1.84	4.43
$\beta =20$	1.84	5.18	5.08	1.26	1.80	4.31
$\beta =30$	1.79	5.34	4.79	1.00	1.83	4.24
$\beta =40$	1.77	5.70	4.69	1.02	1.88	4.19
$\beta =50$	1.77	5.20	4.57	0.99	1.65	4.14
$\beta =60$	1.81	5.43	4.62	1.01	1.70	4.12
$\beta =70$	1.85	5.58	4.63	1.01	1.74	4.10

presents the RMSE in the X, Y, and Z directions on both paths for experiments where parameter $\gamma$ is fixed at 10 and parameter $\beta$ is varied.

Table 5. RMSE of Path 1 and Path 2 with $\beta =50$.

	Path 1 RMSE(m)			Path 2 RMSE(m)
	X	Y	Z	X	Y	Z
$\gamma =5$	1.80	6.07	4.76	1.06	1.68	4.30
$\gamma =10$	1.77	5.20	4.57	0.99	1.65	4.14
$\gamma =15$	1.77	5.20	4.63	0.98	1.94	4.08
$\gamma =20$	1.78	5.24	4.65	0.97	1.95	4.06
$\gamma =25$	1.79	5.11	4.66	0.97	1.96	4.03

presents the RMSE values in the X, Y, and Z directions on both paths when parameter $\gamma$ is varied while parameter $\beta$ is fixed at 50.

Although the lowest individual RMSE values do not occur precisely at $\gamma =10$ and $\beta =50$, this combination provides relatively low and balanced errors across all directions and both paths, with fewer outliers. Therefore, after comprehensive evaluation, this combination is adopted in the subsequent comparative experiments conducted on Figure 12.

3.3. Experimental Results and Analysis

To validate the effectiveness of our proposed approach, comparative experiments with multiple methods were conducted along the trajectory shown in Figure 12. The figure demonstrates that the selected scene satisfies the complexity required for our experiments, featuring a mixture of canopy and building occlusions.

In this paper, sky image segmentation results are evaluated using three different schemes and compared with the reference segmentation results. The reference segmentation results are obtained from result files generated through manual sky image segmentation using Labelme. The area of the region is calculated based on the total number of pixels it contains. This comparison aims to assess the effectiveness of semantic segmentation in extracting sky regions and occlusion types, relative to traditional image processing methods.

• Canny Edge Detection Method: The original sky image is first converted to a grayscale image, followed by dilation and erosion operations. Then, Canny edge detection is applied. In this paper, this method is referred to as Canny.
• Flood Fill Method: The center pixel of the image is first selected as the seed point. The algorithm then propagates to neighboring pixels, identifying all points with the same or similar color and filling them with a new color to form a connected region. In this paper, this method is referred to as Flood Fill.
• The proposed method in this paper involves training and segmenting images using DeepLabV3+, with obstructions classified as buildings and trees. This method is referred to as Deep-Air.

The Canny and Flood Fill methods are compared and evaluated against the proposed Deep-Air method. The original images used for segmentation are shown in Figure 13a, while the segmentation results of the three methods are presented as follows: (b) shows the sky region segmented using the Flood Fill method, (c) displays the result obtained with the Canny method, and (d) presents the segmentation outcome using the proposed Deep-Air method. In the segmentation results, the sky region is marked in red, tree occlusions in green, and building occlusions in black.

Table 6. Performance Comparison of Three Sky Region Segmentation Methods.

Methods	Sky Area Accuracy	Number of Pixels
Canny	79.1%	44,789
Flood Fill	89.0%	62,790
Deep-Air	98.9%	57,196

presents a comparative analysis of the segmentation performance of the three methods.

Table 6 presents the sky region segmentation results for the three methods. As shown in the table, the similarity between the color of building edges and the sky region in the collected data causes the Canny edge detection method to deviate significantly from the reference value, resulting in an accuracy of only 79.1%. In contrast, the Flood Fill method achieves an accuracy of 89.0%, while the Deep-Air method attains 98.9%.

Figure 14 illustrates the sky region segmentation results for the three methods in a scenario where the obstacle occludes the sky, dividing it into multiple regions. In this case, the seed point selected by the Flood Fill method falls on the obstacle.

Since the Flood Fill method determines whether surrounding pixels belong to the sky region based on the attributes of the seed point, the seed point itself must also be located within the sky region. The Flood Fill method relies on the assumption that the area directly above the vehicle is generally unobstructed, with the center of the image chosen as the seed point. As shown in Figure 14b, when processing the original sky image in Figure 14a using the Flood Fill method, the selected seed point is the image center, which belongs to the occlusion category. This causes the sky region segmentation to fail, resulting in an empty sky region. When using the Canny method for sky region extraction, the original image must undergo edge detection preprocessing. Since the threshold in this process is set empirically, there is a high probability of under-segmentation. As shown in Figure 14c, the extracted sky region is smaller than the actual area, with a segmented area of 13,799 pixels. In contrast, the proposed method effectively extracts the initial sky region and the tree occlusion region, with areas of 16,844 and 111,154 pixels, respectively, as shown in Figure 14d. It can be seen that when the image center is occluded, the Flood Fill method fails to extract the sky region due to its algorithmic limitations. In contrast, both the Canny method and the proposed method effectively extract the sky region, with the latter additionally identifying the tree occlusion region.

Due to the movement of the experimental vehicle, a scenario arises as illustrated in Figure 15a where the sky region is primarily located in the middle and right portions of the image. Figure 15b–d, respectively, present the sky region extraction results using the three methods.

Figure 15a displays the original sky-direction image, in which the center region is classified as sky. For sky region extraction, all three methods successfully identify a portion of the sky. However, when using the Flood Fill method, only the sky region containing the seed point is extracted, resulting in an area of 4085 pixels. When using the Canny method, the original image is processed by dilation and erosion before extracting the sky regions, resulting in multiple sky regions with a total area of 9252 pixels. In contrast, when using the Deep-Air method, the sky region in the center and right portions of the image, along with the tree occlusion region, are successfully extracted. The sky region covers an area of 15,987 pixels, and the tree occlusion area spans 107,537 pixels. It can be observed that the proposed method achieves higher accuracy in extracting occlusion regions and differentiating obstructions types in multi-sky region scenarios where the middle region is identified as sky.

In complex urban environments, the carrier’s skyward image is often divided into two parts by columnar objects such as power lines. Figure 16a presents the results of different image segmentation methods.

Figure 16b–d, respectively, show the results of the three methods for sky region extraction. Figure 16b shows that when the Flood Fill method is applied, only the sky region in the upper part of the image is extracted due to its algorithmic characteristics. In contrast, Figure 16c,d show that the Canny algorithm and the Deep-Air method successfully extract the sky region below the traffic signal pole. However, the Canny method applies a dilation and erosion operation so that the extracted beam region differs from its actual size, leading to misjudgment of the NLOS signal. In practice, factors such as the distance between the carrier and the beam and the beam’s thickness reduce the likelihood that the beam affects the NLOS signal. Therefore, it is generally considered an unobstructed region. The Deep-Air method, as shown in Figure 16d, correctly judges the beam portion as a sky region.

Figure 13, Figure 14, Figure 15 and Figure 16 demonstrate that the method proposed in this paper outperforms the other two existing methods in accurately extracting the sky region. This is particularly evident in cases involving multiple sky regions or sky regions separated by obstacles such as lines or poles, where the proposed method excels at extracting the sky regions from each section. These results confirm the practical feasibility and effectiveness of using semantic segmentation to extract sky regions and categorize obstructions.

To further verify the performance of the proposed algorithm, the positioning and navigation results are evaluated based on the outputs of three different methods along Figure 12 and compared with the reference trajectory.

(1) GNSS/INS/Canny Solution: The Canny method is used for sky extraction and is re-assigned based on LOS and NLOS satellites. It is then loosely integrated with motion estimation from INS and is referred to as GNSS/INS/Canny in the text.

(2) GNSS/INS/Flood Fill Solution: The Flood Fill method is used for sky extraction; LOS and NLOS satellites are re-assigned accordingly. It is then loosely integrated with motion estimation from INS and is referred to as GNSS/INS/Flood Fill in the text.

(3) GNSS/INS/Deep-Air Solution: The proposed method in this paper is used for sky extraction and is re-assigned based on GNSS satellite signal categories. It is then integrated with motion estimation from INS and is referred to as GNSS/INS/Deep-Air in the text.

In this study, GNSS observations are selected from the L1 and L2 bands for GPS, the B1-2 and B2b bands for BDS, and the G1 and G2 bands for GLONASS. The road segment used for data collection and method evaluation is shown in Figure 12. This segment is selected due to its mixed shading environment, which includes trees, high-rise buildings, and other structures. A comparison of the trajectories of the different methods is shown in Figure 17.

As shown in the trajectory diagram in Figure 17, the method proposed in this paper closely aligns with the reference trajectory. However, in some cases, the number of LOS satellites received by certain receiver elements is insufficient to support accurate localization, as illustrated in Figure 18, which leads to significant deviations from the reference trajectory. Additionally, a low hardware installation position may cause signal interference from pedestrians or nearby vehicles. As shown in Figure 8, the GNSS receiving antenna on the experimental platform is positioned at a height of only 100 cm, lower than the fisheye camera. The selected experimental area is densely built-up, with high pedestrian and vehicle activity. This complex environment significantly degrades satellite signal quality, leading to increased positioning errors and reduced localization accuracy.

As shown in Figure 18, the dataset used for experimental acquisition contains a significant number of NLOS signals, fulfilling the data requirements for validating the performance of the proposed methods. Figure 19 illustrates the error trends along the X, Y, and Z directions for the GNSS/INS/Flood Fill, GNSS/INS/Canny, and GNSS/INS/Deep-Air methods. As shown in Figure 19, the proposed method produces less error and smoother results compared to other methods. When the number of LOS signals drops below four, as seen in the time intervals of 2300–2600 and 3000–3500, errors are more likely to occur. The method proposed in this paper employs distinct weighting optimization techniques for NLOS signals caused by building occlusions and signal attenuation due to trees. Additionally, the method is more effective at identifying signal types in road sections with special conditions. As a result, the probability of a sudden increase in error in the other two methods is significantly higher than the proposed method, in terms of both likelihood and magnitude.

As shown in Figure 19, the error trend of the GNSS/INS/Deep-Air method is smoother and closer to zero compared to the other two methods. Additionally, its error deviation is smaller at most time points. However, in the Z-axis error during epochs 4500–5000, the proposed method shows a larger deviation than the others. This may be due to misclassification of NLOS signals as attenuated LOS signals, as the projection point lies within a tree-covered area that is actually a mixed region of trees and buildings, where buildings behind the trees contribute to NLOS errors. Another possible cause is dense tree coverage with large inter-leaf gaps, which appear small in the camera image due to distance, leading to the area being extracted as tree occlusion and LOS signals being misclassified as attenuated NLOS. The RMSE and mean errors (MEs) in all three directions are presented in

Table 7. RMSE and ME in the X, Y, and Z directions.

	RMSE (m)			ME (m)
	X	Y	Z	X	Y	Z
GNSS/INS/Flood Fill	0.80	2.91	3.09	−0.35	2.21	1.65
GNSS/INS/Canny	0.88	2.99	3.25	0.37	2.40	1.79
GNSS/INS/Deep-Air	0.64	2.75	2.52	−0.05	2.25	1.14

.

Table 7 shows that the RMSE and the ME for the proposed method are smaller than the other three methods. In the X direction, the improvement is 20% better compared to GNSS/INS/Flood Fill and 27% better compared to GNSS/INS/Canny. In the Y direction, the improvement is 5% over GNSS/INS/Flood Fill and 8% over GNSS/INS/Canny. In the Z direction, the RMSE is improved by 18% compared to GNSS/INS/Flood Fill and by 22% compared to GNSS/INS/Canny. The reason why the GNSS/INS/Canny and GNSS/INS/Flood Fill methods’ stability and accuracy are lower than those of the proposed method is likely the presence of occluded objects, multi-sky regions, and sky regions divided by bars at the center of the image, as previously mentioned. These factors result in errors in signal type classification during the vehicle’s movement, which in turn reduces localization accuracy.

In summary, the detection and identification of NLOS/LOS signals are influenced by factors such as building color and sky region distribution, resulting in fluctuations in the stability and accuracy of the positioning process. The method proposed in this paper outperforms the others in improving both the accuracy and stability, as measured by the average error and root-mean-square error. Figure 20 presents the cumulative distribution function (CDF) graphs for the three methods across three directions.

Figure 20 illustrates the CDF of the error in the X, Y, and Z directions for each method, showing the probability distribution of error values in each direction. This representation provides insights into the likelihood of data points occurring within a specific interval, aiding in a comprehensive understanding of the error distribution characteristics. The leftward shift of the curve intuitively indicates the improvement in accuracy. The figure shows that, among the three methods, GNSS/INS/Flood Fill, GNSS/INS/Canny, and GNSS/INS/Deep-Air, the CDF curves for the GNSS/INS/Deep-Air method exhibit a pronounced leftward shift in all three dimensions, particularly along the Y-axis, where the shift is most notable. For example, when the probability is 80%, the error thresholds for the three methods are as follows. In the X-direction, they are 1.04, 1.25 and 0.74 m. In the Y-direction, they are 3.79, 3.89 and 2.73 m. In the Z-direction, they are 3.86, 3.98 and 3.28 m. This systematic advantage further confirms the accuracy and reliability of the method proposed in this paper.

4. Conclusions

To address the impact of NLOS signals induced by complex scenes on positioning and navigation, this paper proposes a GNSS signal occlusion detection and optimization method by integrating sky-semantic segmentation, and different optimization methods are chosen for signals affected by different obstructions. The method is implemented using a fisheye camera mounted on top of the test vehicle. To enhance the detection and classification of GNSS signal categories, the DeepLabv3+ model is employed to segment the skyward area into three classes: sky, trees, and buildings. Finally, optimization methods are chosen based on the obstruction’s category where the GNSS signal is located and integrated with IMU data to obtain positioning and navigation results. Compared to GNSS/INS/Canny and GNSS/INS/Flood Fill, the GNSS/INS/Deep-Air method improves X-axis accuracy by 27% and 20%, Y-axis accuracy by 5% and 8%, and Z-axis accuracy by 22% and 18%. Overall positioning accuracy is enhanced by 16% and 9% compared to GNSS/INS/Canny and GNSS/INS/Flood Fill, respectively.

Although the method proposed in this paper improves positioning accuracy, our vision system may encounter challenges such as lighting issues in scenarios with extensive obstructions like bridges. In such cases, a supplementary system that provides relative positioning can be integrated for assistance. In future work, we will incorporate additional approaches to enhance the proposed methodology and conduct a comparative analysis of the performance and results achieved by different deep learning models within the scope of our research.