Optical Coastal GNSS-Denied Navigation for Reduction in AUV Underwater Navigation Error

Abstract

The paper addresses the problem of maritime navigation in coastal areas without access to satellite positioning data. The proposed approach relies on visual information from a monocular camera supported by map-derived coastal information. Although the operational concept assumes the use of publicly available cartographic data, the method’s preparation and evaluation require additional processing steps, including high-resolution aerial imagery, GIS-based extraction of coastal features, semantic segmentation, and trained convolutional neural network models. The problem discussed in the paper concerns, for example, Autonomous Underwater Vehicles that seek to reduce underwater dead-reckoning navigation error by surfacing and using information about what is visible around them, in a way similar to how a human would. To solve the above problem, a system was proposed that compares the camera’s representation of the observed coastline with the map representation of the area where the vehicle is most likely located. The system was validated using real-world data. The tests revealed that the information contained in a flat map is insufficient for accurate position estimation. Accuracy is also significantly affected by errors in the semantic segmentation used to extract land features from camera images, as well as by potential errors in the camera viewing angle. The achieved accuracies are sufficient for navigation away from land, but when operating close to land, the proposed system appears significantly insufficient. The paper specifies the system and reports the results.

1. Introduction

Autonomous Underwater Vehicles (AUVs) are a tool increasingly used for various purposes [1], e.g., to support deep-sea divers [2], monitor critical underwater infrastructure, research the marine environment, and detect and neutralise sea mines. However, operating such vehicles in an underwater environment [3] is not as easy as with flying, surface, or land drones. The first problem AUVs face compared to their surface counterparts is significant communication difficulties [4]. Radio communication is impossible underwater, and the only available option is acoustic communication [5]. However, in this case, the problem lies in the acoustic channel’s very low bandwidth.

Another challenge for AUVs in the underwater environment is navigation [6]. Drones operating over the surface of the water have easy access to information from satellite systems [7] such as the US GPS and the European GALILEO, enabling highly accurate and frequent positioning. Even if this access is difficult or impossible, continuous observation of the surrounding world [8], which is rich in characteristic objects that serve as reference points for navigation systems, is still possible. In the case of the underwater environment, where visibility is very low [9], sometimes limited to a few meters, as in the North Sea or the Baltic Sea, for example, there is no such comfort as in the case of operating in a clear surface environment [10].

Underwater navigation currently relies on three main approaches. The first approach, represented by systems such as LBL and USBL, uses external infrastructure for navigation [11]. For example, suppose that we have a system of three or four underwater “satellites” with known, unchanging positions that transmit an acoustic signal allowing us to determine the distance to the signal source. In that case, similarly to satellite systems, we can easily determine the receiver’s position. Unfortunately, such solutions can only be applied when operating in a location where our underwater “satellite” system can be placed, for example, around a critical infrastructure facility. Away from such a system, we must use another approach.

One option is to use underwater observation systems, such as sonar [12] (i.e., underwater acoustic radars). If the AUV operates near a seabed rich in recognisable landmarks, we can use them as navigational cues. For example, we can navigate along an underwater pipeline or use the SLAM method [13], which, thanks to continuous environmental observation, can enable safe navigation. But what if the seabed is flat and devoid of anything distinguishable from the background? Then we have to resort to dead reckoning.

Dead reckoning [14] is an onboard system independent of external sources of information. It is based on the measurement of speed, direction, and depth (In reality, the measurement most often does not involve speed, direction, or depth. Other parameters are measured, e.g., pressure, acceleration, and angular velocity) using onboard sensors. The problem in this case, however, is that the position error increases with the duration of the underwater mission, due to sensor imperfections. To reduce dead-reckoning navigation error, AUVs surface to utilise satellite systems. Unfortunately, in a GNSS-denied environment, satellite system support is unavailable. The only accessible information is what is visible on the surface. Far from land, this is the sun and stars. Closer to land, we can use landmarks such as urban areas, forests, beaches, isolated buildings, and harbour infrastructure.

In the simplest case, if we have image data from the area where the AUV will be operating, for example, photos of characteristic objects in the area (usually buildings) along with their precise locations, the vehicle can search for these objects and use them as reference points [15]. If at least two such objects can be found, then two bearings to these objects, determined using the onboard compass and the object-oriented camera, will be sufficient to determine the AUV’s position.

If only one such object can be detected, then, in addition to the bearing, the distance to the object is also required. For this purpose, we can use the solution proposed in [16]. It involves estimating distance by comparing the size of a standard object with the number of pixels it occupies in the image.

However, if we do not have imagery from the area where the AUV will operate, we can use the algorithm proposed in [17]. It is based on extracting the land outline from images captured by a camera around the AUV’s current position and comparing the size of the visible land, expressed in pixels, which, under the algorithm’s assumptions, corresponds to the distance to the land, with data from a digital terrain model.

It seems that one of the most challenging localisation scenarios for an AUV operating in a GNSS-denied coastal environment occurs when neither prior image data nor a terrain elevation model is available. Under such conditions, the vehicle observes the coastline from a considerable distance, making it impossible to extract distinctive landmarks or individual objects from camera images. Instead, only coarse characteristics of the visible shoreline can be observed, such as the general coastline outline and the presence of broad terrain classes, including urban areas, forests, or farmland.

This paper investigates the feasibility of using such limited visual information to estimate position in coastal environments. Rather than proposing a complete navigation solution, the study aims to evaluate which types of information contained in omnidirectional coastline observations can support localisation and which are insufficient for reliable position estimation. To this end, an Optical Coastal Navigation System (OCNS) was developed as an experimental framework for comparing omnidirectional image representations of the observed coastline with flat-map representations generated from publicly available geographic data sources, such as Google Maps.

The contribution of the paper is as follows:

• An experimental Optical Coastal Navigation System (OCNS) for coastline-based localisation in GNSS-denied coastal regions has been developed and specified.
• Three localisation strategies based on different combinations of angular and distance-related coastline information have been formulated and evaluated.
• The limitations and practical applicability of flat-map-based coastline matching have been systematically investigated using real-world data.
• The OCNS was validated using data collected in the Gulf of Gdańsk region.

The rest of the paper is as follows: Section 2 specifies the OCNS, Section 3 reports the experiments, and the final Section 4 summarises the paper.

2. Optical Coastal Navigation System

The OCNS consists of three modules. The first module, the Camera Representation Module (CRM), prepares a representation of the AUV’s surroundings, SR, as seen by the camera. The second module, the Map Representation Module (MRM), generates the set $S^{MR}$ of map representations $MR$. The OCNS assumes that the AUV knows its approximate position and the maximum position error $E_{max}^{P^{AUV}}$. The MRM generates map representations $MR$ around the AUV’s position, estimated via dead-reckoning underwater navigation, and produces $SR$. All $MR$s are generated at positions whose distance from the estimated position does not exceed $E_{max}^{P^{AUV}}$. The final module, the Estimation Module (EM), estimates the AUV’s position by comparing the $SR$ with all $MR$s generated around it. The $MR$, which is the most similar to $SR$, indicates the position of the AUV. All OCNS modules are detailed in the following sections.

2.1. Camera Representation Module

The purpose of this subsystem is to generate a structured representation of the environment surrounding the vehicle, denoted as $SR$, at the AUV’s position. The camera orientation enables the assignment of an absolute azimuth angle to individual pixel columns. This mapping allows the projection of image-based observations into a global angular reference frame. The processed image data are used to construct a compact representation of the surrounding scene, defined as follows:

$$SR=\{O^{SR},H\}$$

(1)

where $O^{SR}$ denotes a set of detected areal objects grouped by semantic classes, and H represents the angular profile of the observed scene height.

The object component $O^{SR}$ is defined as follows:

$$O^{SR}=\left\{O_i^{SR}\right\},i\in \{\mathrm{building},\mathrm{forest},\mathrm{other}\}$$

(2)

where each subset $O_i^{SR}$ contains areal objects belonging to a specific semantic category obtained from the segmentation process.

Each object is represented as a continuous angular segment

$$o_{ij}^{SR}={(s_{i,j},e_{i,j})}^{SR},s_{i,j},e_{i,j}\in \langle 0, 360),e_{i,j}>s_{i,j}$$

(3)

where $s_{i,j}$ and $e_{i,j}$ denote the starting and ending azimuth angles of the j-th object of class i. This formulation assumes that each object occupies a contiguous region in the angular domain.

The set of objects of a given class is defined as follows:

$$O_i^{SR}=\left\{o_{ij}^{SR}:o_{ij}^{SR}\cap o_{ik}^{SR}=\varnothing ,k\ne j,k,j=1,\dots ,N_i^{SR}\right\}$$

(4)

which ensures that angular segments assigned to objects of the same class do not overlap. In addition to the object-based representation, a discrete height profile of the surroundings is constructed as follows:

$$H=<h_1,h_2,\dots ,h_{360}>$$

(5)

where $h_{\theta }$ denotes the estimated height of the observed scene (terrain or above-ground objects) in the direction corresponding to azimuth angle $\theta \in \{1,\dots ,360\}$.

The height values $h_{\theta }$ are derived from the segmentation results obtained from all processed images. For each angular direction, the corresponding pixel column is selected based on the closest azimuth match.

The resulting representation $SR$ combines two descriptions of the environment:

• $O^{SR}$ represents detected objects as angular intervals grouped by semantic class;
• H provides a dense angular profile of scene height.

2.2. Map Representation Module

The purpose of the Map Representation Module (MRM) is to generate a structured description of the environment surrounding a given observation point, based on geospatial data sources such as a digital surface model. The representation is defined in a global angular reference frame and captures both the semantic structure of the scene and the spatial distribution of distances.

The MRM generates a set of map representations:

$$S^{MR}=\left\{M{R}_k\right\}$$

(6)

where each element $M{R}_k$ corresponds to a representation constructed for a specific observation point k with given spatial coordinates. The structure of a single map representation $MR$ is defined as follows:

$$MR=\{O^{MR},D\}$$

(7)

where $O^{MR}$ denotes the object-based description of the surroundings, and D represents the angular distance profile.

The object component is expressed as follows:

$$O^{MR}=\left\{O_i^{MR}\right\},i\in \{\mathrm{building},\mathrm{forest},\mathrm{other}\}$$

(8)

where each subset $O_i^{MR}$ contains objects belonging to a specific semantic class. The classification is derived from map data, where terrain and above-ground structures are identified and assigned to predefined categories.

Each object is represented as an angular interval:

$$o_{ij}^{MR}={(s_{i,j},e_{i,j})}^{MR},s_{i,j},e_{i,j}\in \langle 0,360),e_{i,j}>s_{i,j}$$

(9)

where $s_{i,j}$ and $e_{i,j}$ denote the starting and ending azimuth angles of the object, respectively. The angles satisfy constraints ensuring that each object occupies a continuous region in the angular domain. If an object spans across the $0^{\circ }/{360}^{\circ }$ boundary, it is represented as two separate angular intervals.

The sets of objects of a given class are defined as follows:

$$O_i^{MR}=\left\{o_{ij}^{MR}:o_{ij}^{MR}\cap o_{ik}^{MR}=\varnothing ,k\ne j,k,j=1,\dots ,N_i^{MR}\right\}$$

(10)

which guarantees that angular intervals corresponding to objects of the same class do not overlap. As a result, each direction is associated with at most one object of a given semantic category.

In addition to the object-based description, a distance profile is defined as follows:

$$D=\langle d_1,d_2,\dots ,d_{360}\rangle$$

(11)

where $d_{\theta }$ denotes the distance to the nearest visible object in the direction corresponding to azimuth angle $\theta$. The profile is defined with a resolution of one degree, providing a dense representation of spatial relationships in the environment. If no object is detected along a given direction within the map extent, the corresponding value is set to $d_{\theta }=0$.

The resulting representation $MR$ combines two descriptors:

• $O^{MR}$, representing objects as non-overlapping angular intervals grouped by semantic class;
• D, providing a dense angular profile of distances.

2.3. Estimation Module

The EM estimates the AUV position by comparing $SR$ with $MR$s distributed on the map around the rough vehicle position, seeking the best match between the representations. It has three variants: EM1, EM2, and EM3. The difference between the variants lies in the comparison they make. EM1 compares $O^{SR}$ with $O^{MR}$, EM2 compares H with D, and EM3 is a combination of EM1 and EM2.

2.3.1. EM1

To calculate the matching $M^{EM1}$ between $O^{SR}$ and $O^{MR}$, EM1 determines the common part of both representations $O^C=\left\{O_i^C\right\}$ and the differences between them $O^{D1}=\left\{O_i^{D1}\right\}$, $O^{D2}=\left\{O_i^{D2}\right\}$, where $i\in \{\mathrm{building},\mathrm{forest},\mathrm{other}\}$. The subsets of $O_i^C,O_i^{D1},O_i^{D2}$ are defined as follows:

$$O_i^C=\bigcup _{\begin{array}{c}1\le j\le N_i^{SR}\\ 1\le k\le N_i^{MR}\\ o_{ij}^{SR}\cap o_{ik}^{MR}\ne ⌀\end{array}}\left(o_{ij}^{SR}\cap o_{ik}^{MR}\right).$$

(12)

$$O_i^{D1}=O_i^{SR}\setminus O_i^C$$

(13)

$$O_i^{D2}=O_i^{MR}\setminus O_i^C$$

(14)

The final matching is defined as follows:

$$M^{EM1}(O^{SR},O^{MR})={\displaystyle \frac{{\sum }_{i}L\left(O_i^C\right)}{1+{\sum }_i\left(L\left(O_i^{D1}\right)+L\left(O_i^{D2}\right)\right)}}$$

(15)

where $L\left(O\right)$ is the total length of all objects in O: $L\left(O\right)={\sum }_j(e_j-s_j)$.

To account for possible observation angle errors during image data acquisition, as well as the range of view from a given observation point, the matching $M^{EM1}(O^{SR},O^{MR})$ is calculated multiple times for different $O^{SR}$s and $O^{MR}$s. The final result is the best match between all tested $O^{SR}$s and $O^{MR}$s.

In the case of $O^{SR}$, various variants of this representation are tested, each rotated relative to the original by an angle in the range $<-E_{max}^A,E_{max}^A>$, where $E_{max}^A$ is the maximum assumed observation angle error. The $O^{MR}$ modification filters out data whose distance from the map observation point at which $MR$ is generated exceeds the assumed threshold $R^O$. The EM1 operation is specified by Algorithm 1.

Algorithm 1 EM1.

Require:  $O^{SR}$, $O^{MR}$, $E_{max}^A$, $\Delta E^A$, $R_{max}^O$, $R_{min}^O$, $\Delta R^O$, D Ensure:  $M^{EM1}(O^{SR},O^{MR})$   1: $R^O\leftarrow R_{min}^O$   2: $M_{best}\leftarrow 0$   3: while $R^O\le R_{max}^O$ do   4:       $O_R^{MR}\leftarrow filter(O^{MR},R^O,D)$▹ Remove $(s^{MR},e^{MR})$ whose average distance to the coast exceeds $R^O$   5:       $shift\leftarrow -E_{max}^A$   6:       while $shift\le E_{max}^A$ do   7:             $O_{shift}^{SR}\leftarrow rotate(O^{SR},shift)$   8:             Generate $M^{EM1}(O_{shift}^{SR},O_R^{MR})$ according to Equation (15)   9:             if $M_{best}<M^{EM1}(O_{shift}^{SR},O_R^{MR})$ then 10:                   $M_{best}\leftarrow M^{EM1}(O_{shift}^{SR},O_R^{MR})$ 11:             end if 12:             $shift\leftarrow shift+\Delta E^A$ 13:       end while 14:       $R^O\leftarrow R^O+\Delta R^O$ 15: end while 16: return $M_{best}$

2.3.2. EM2

EM2 uses the vector H, containing the pixel heights of the coastline extracted from the semantically segmented image, to derive an approximate distance estimate $\widehat{D}\left(H\right)$ to the visible land. This estimate is not based on a direct geometric relationship between pixel height and range, since the true terrain elevation is generally unknown and the observed coastline profile depends on multiple factors, including the shape and local topography. Instead, the entire vector H is treated as an image-derived feature describing the appearance of the visible coastline from the current observation point. Although the resulting distance estimate may be inaccurate, it still provides useful information about the coastline’s relative proximity. It can therefore support navigation in GNSS-denied environments, where dead-reckoning errors may accumulate over time.

The estimate $\widehat{D}\left(H\right)$ is subsequently compared with vectors D, each of which defines the distance to the coastline from a given map point. To estimate D, three regression methods are used, i.e., (i) Linear Ordinary Least Squares (LOLS), (ii) Linear Total Least Squares (LTLS) [18], and (iii) Kernel Regression (KR). The first two methods are well-known, while the KR is defined in Algorithm 2.

To build each of the regressors, training pairs $\left\{(h_i^L,d_i^L)\right\}$ are used, defining the pixel height of the land $h_i^L$ and the corresponding distance to the land $d_i^L$ for viewing angle i degrees. These pairs are generated from sample images of the coast recorded at points with known coordinates, which allows the distance $d^L$ to be determined.

Obtaining an effective regressor, however, requires filtering the entire available training dataset and removing pairs that could degrade the accuracy of the estimate. The pairs that should be removed primarily correspond to semantic segmentation errors, which can generate land where it is very distant and consequently invisible, or can “lose” land where it should be visible. Another example of faulty pairs involves cases where land is erroneously elevated or lowered.

Algorithm 2 KR.

Require:  ${\left\{(h_i^L,d_i^L)\right\}}_{i=1}^{N^L}$ – learning set, $\sigma$, h Ensure:  $\widehat{D}\left(h\right)$   1: $S\left(h\right)\leftarrow 0$   2: for $i=1$ to $N^L$ do   3:       $w_i\left(h\right)\leftarrow K(h-h_i^L)=exp\left(-{\displaystyle \frac{{(h-h_i^L)}^2}{2{\sigma }^2}}\right)$   4:       $S\left(h\right)\leftarrow S\left(h\right)+w_i\left(h\right)$   5: end for   6: if $S\left(h\right)>0$ then   7:       $\widehat{D}\left(h\right)={\displaystyle \frac{{\sum }_{i=1}^{N^L}{w}_i\left(h\right)d_i^L}{{\sum }_{i=1}^{N^L}{w}_i\left(h\right)}}$   8: else   9:       $i^{\ast }=arg{min}_i|h-h_i^L|$ 10:       $\widehat{D}\left(h\right)=d_{i^{\ast }}^L$ 11: end if 12: return $\widehat{D}\left(h\right)$

To filter the training data, EM2 employs the DBSCAN clustering algorithm [19]. It clusters the data and identifies outliers, which are then removed from the training dataset.

After filtering the training data and building a regressor, EM2 compares the vector H included in $SR$ with the vectors D, each representing a specific position on the map and corresponding $MR$ representation. Matching H to a single D is performed according to Algorithm 3. It compares H and D at each position i (viewing angle) in both vectors and returns the sum of all comparisons. There are three cases to consider when comparing vectors. The first case is when $h_i>0$ and $d_i>0$, i.e., when land is visible in the image, and the distance to the coastline is sufficiently small for the land to be visible from the observation point. In this case, the result is the absolute difference between the exact distance $d_i$ and the estimated distance $\widehat{D}\left(h_i\right)$. The second case occurs when $d_i=0$ and $h_i>0$, i.e., when it is impossible to see the land because of the distance to it ($d_i=0$—see description of MRM, Section 2.2); however, it is visible in the image ($h_i>0$). The algorithm strongly punishes this case. The last case considered occurs when $d_i$ and $h_i$ agree on land invisibility, which improves the comparison result. The case in which the map indicates that the land may be visible ($d_i>0$), whereas the $SR$ does not confirm this ($h_i=0$), is not considered by the algorithm. The reason is that $d_i>0$ only indicates the potential for land visibility; it does not guarantee that it is visible. In consequence, $h_i$ and $d_i$ disagree; however, in reality, they can agree.

2.3.3. EM3

The EM3 is executed in two phases. First, EM2 is run, producing matchings between H and the set of Ds generated at different map points scattered around the approximated AUV position. Then, the best K matchings are compared, and if they are very similar, making the correct decision very difficult, so EM1 is run for the K best $MR$s. The final result is the EM1 best matching.

Algorithm 3 EM2.

Require:  H, D, regressor, $E_{max}^{EM2}$ Ensure:  $M^{EM2}(H,D)$   1: $E1\leftarrow 0$   2: $E2\leftarrow 0$   3: for $i=1$ to 360 do   4:       if $h_i>0$ and $d_i>0$ then   5:            $E1\leftarrow E1+|\widehat{D}\left(h_i\right)-d_i|$   6:       else if $d_i=0$ and $h_i>0$ then           ▷ error in the image   7:            $E1\leftarrow E1+E_{max}^{EM2}$   8:       else if $d_i=0$ and $h_i=0$ then   9:            $E2\leftarrow E2+1$ 10:       end if 11: end for 12: return $E2/(1+E1)$

In Algorithm 4, $I{n}_{EM2}$ denotes all additional input parameters required by EM2, such as the regressor and the maximum error value, whereas $I{n}_{EM1}$ denotes all additional input parameters required by EM1, such as angular-error limits, rotation step, and filtering parameters. The sets $\mathcal{D}$ and ${\mathcal{O}}^{MR}$ contain map representations generated for N candidate map points. Thus, each candidate point is represented by a pair $(D_j,O_j^{MR})$. EM2 is first used to rank all candidate points based on distance-related coastline information. If the best EM2 score is sufficiently better than the K-th score, the best EM2 candidate is accepted as the final result. Otherwise, the K best candidates are considered ambiguous and are re-evaluated using EM1, which exploits angular information about areal objects.

Algorithm 4 EM3.

Require:  H, $\mathcal{D}={\left\{D_j\right\}}_{j=1\dots N}$, $O^{SR}$, ${\mathcal{O}}^{MR}={\left\{O_j^{MR}\right\}}_{j=1\dots N}$, K, $T^{EM3}$, $I{n}_{EM2}$, $I{n}_{EM1}$ Ensure:  $j_{best}$   1: ${\mathcal{M}}^{EM2}\leftarrow \varnothing$   2: for $j=1$ to N do   3:       $m_j^{EM2}\leftarrow EM2(H,D_j,I{n}_{EM2})$   4:       Add $(j,m_j^{EM2})$ to ${\mathcal{M}}^{EM2}$   5: end for   6: Sort ${\mathcal{M}}^{EM2}$ in descending order according to $m^{EM2}$   7: $\mathcal{K}\leftarrow$ indices of the K best elements of ${\mathcal{M}}^{EM2}$   8: $(j_1,m_1^{EM2})\leftarrow$ the best element of ${\mathcal{M}}^{EM2}$   9: $m_K^{EM2}\leftarrow$ the EM2 score of the K-th element of ${\mathcal{M}}^{EM2}$ 10: if $m_1^{EM2}-m_K^{EM2}>T^{EM3}$ then 11:       return $j_1$ 12: end if 13: $M_{best}\leftarrow 0$ 14: $j_{best}\leftarrow j_1$ 15: for each $j\in \mathcal{K}$ do 16:       $m_j^{EM1}\leftarrow EM1(O^{SR},O_j^{MR},D_j,I{n}_{EM1})$ 17:       if $m_j^{EM1}>M_{best}$ then 18:             $M_{best}\leftarrow m_j^{EM1}$ 19:             $j_{best}\leftarrow j$ 20:       end if 21: end for 22: return $j_{best}$

3. Experiments

To evaluate the OCNS, its performance was verified using real-world data from the Gulf of Gdańsk region in Poland. The objective of the evaluation was not to determine a single average localisation error but to assess the spatial resolution at which coastline observations remain sufficiently distinctive to support reliable position estimation. Preliminary experiments using randomly distributed candidate positions yielded highly variable errors that depended on the local coastline geometry, limiting the usefulness of aggregate statistical measures. Therefore, candidate map representations were generated at predefined distances from the reference position. This methodology enables evaluation of the localisation capability of the proposed approach as a function of spatial separation. It provides practical information regarding the scale at which optical observations can be used to constrain position uncertainty in GNSS-denied environments. From an operational perspective, the obtained results can be used to determine the minimum spacing between candidate positions generated around a dead-reckoning estimate. If the spacing is significantly smaller than the method’s effective spatial resolution, the resulting map representations become too similar to be reliably distinguished. Conversely, using a spacing comparable to or larger than the estimated resolution increases the likelihood that the map representation corresponding to the actual vehicle position will be correctly identified as the best match to the observed coastline.

The verification tests compared all three position estimation methods: EM1, EM2, and EM3. The results of all tests are presented at the end of this section.

3.1. SR

3.1.1. Dataset

To validate the proposed method under real-world conditions, an experimental dataset was acquired consisting of 873 images collected at 70 measurement locations in the Gulf of Gdańsk—see Figure 1. For each recorded image, the camera’s geographic position and the azimuth of its optical axis were recorded, enabling precise spatial referencing of the observations.

A key parameter influencing the characteristics of the recorded images is the height of the imaging sensor above the water surface. To ensure consistency and to reflect the operational conditions of a surfaced unmanned platform, all images were acquired at a fixed height of 1 m above sea level.

The image acquisition process was carried out using a camera equipped with an integrated GNSS receiver and a digital compass. The focal length was set to a value corresponding to 25 mm for a full-frame sensor, resulting in a horizontal field of view equal to ${\alpha }_h=71.{52}^{\circ }$. Under these conditions, full ${360}^{\circ }$ coverage could theoretically be achieved using a minimum of 6 images. In practice, between 8 and 16 images were recorded at each measurement point to reduce the influence of distortions and ensure sufficient overlap between consecutive observations.

3.1.2. Semantic Segmentation

To prepare $SR$s, semantic segmentation [20,21] was performed on the collected image dataset using the YOLOv8 Convolutional Neural Networks (CNN) [22,23]. The training of CNNs was based on another image dataset comprising approximately 10,000 high-resolution images acquired under diverse environmental conditions, supplemented with publicly available coastal imagery. To ensure reliable segmentation performance, all images were manually annotated.

Before CNN training, the images were resized and partitioned into overlapping sub-images to match the network input requirements while preserving local spatial details. To improve robustness and generalisation, the training images were augmented using random rotations, photometric transformations, and noise-based perturbations. In particular, the applied augmentation operations included image rotation, saturation modification, exposure/brightness variation, and additive noise. These transformations were selected to simulate typical changes in camera orientation, illumination conditions, atmospheric visibility, and image sensor noise that may occur during maritime image acquisition. The dataset was divided into training and validation subsets in an 8:2 ratio.

Two CNN models were designed: a multi-class model (MC-CNN) and a simplified model (SM-CNN). The MC-CNN included an extended set of object classes (e.g., vessels, offshore structures such as wind turbines, and other maritime objects), which reduced misclassification errors (e.g., classifying a ship as a building). However, these additional classes were not directly used in the final detection stage.

The SM-CNN was designed with three primary classes (sea, sky, land), where all terrain-related objects were aggregated into the land class. Additionally, during training, an auxiliary class representing objects located on the sea surface was included to reduce false detections and improve segmentation robustness, particularly in distant regions.

To semantically segment the collected images, the outputs of both CNNs were combined. When the MC-CNN provided reliable classification results, the corresponding semantic labels (building or forest) were used directly; objects belonging to the remaining classes were classified as other. In cases where these classes were not detected but the SM-CNN indicated the presence of land, the corresponding region was assigned to the class other, consistent with the definition $O^{SR}=\left\{O_i^{SR}\right\},i\in \{\mathrm{building},\mathrm{forest},\mathrm{other}\}$.

The quality of segmentation was evaluated in terms of its efficacy in facilitating the preparation of $SR$s. Illustrative results are presented in Figure 2. The SM-CNN demonstrated an effective ability to distinguish among the predominant maritime scene regions, namely the sea, sky, and land. This capability enabled the reliable localisation of the shoreline and the delineation of the sea–sky boundary. The MC-CNN provided supplementary data on the coastal zone, primarily by identifying forest and building regions that serve as reference objects in the proposed method.

The primary challenges encountered during segmentation predominantly arose in instances where the land region constituted a minuscule proportion of the scene, particularly when the shoreline was distant and manifested as a narrow strip proximate to the horizon. In such cases, the network occasionally had difficulty reliably detecting land, which could result in partial omission, fragmentation, or reduced-confidence classification. This limitation was primarily attributable to the relatively small apparent size of coastal objects in the image, rather than to errors in the segmentation of substantial sea and sky regions.

3.1.3. SR Design

Based on segmented images, a spatial representation $SR$ was constructed for each measurement point. The representation was defined with an angular resolution of $R=1^{\circ }$, resulting in a discretized form $SR=<h_1,h_2,\dots ,h_{360}>$, consistent with the definition of the observation vector H.

3.2. MR

The generation of the $MR$s requires the use of geospatial data describing the area in which the system operates. In contrast to the previous approach based on elevation models [17], this study employs high-resolution aerial imagery as the primary source of spatial information.

The MRs were constructed by integrating multiple high-resolution images using dedicated geographic information system (GIS) software ArcGIS Pro 3.6 into a single, spatially consistent mosaic. The source GIS data were georeferenced and processed in the WGS84 geodetic reference system, ensuring a common spatial reference for all input layers and reference objects. For map visualisation and generation of the mosaic presented in Figure 3, the data were projected into the WGS84/UTM zone 34N coordinate system. This projection was selected because it is appropriate for the Gulf of Gdańsk region and, as a conformal projection, preserves local angular relationships while limiting distortions over the analysed area. The resulting image had a resolution of approximately $(40,000\times 30,000)$ pixels, covering an area of about (2500 km²), corresponding to the region of interest considered in the experimental evaluation.

To ensure consistency with the $SR$, the assembled dataset was subjected to a segmentation process within the GIS environment. The segmentation distinguished three classes: buildings (urbanised areas), forest, and other land regions. This classification scheme directly reflects the structure of the set $O^{MR}=\left\{O_i^{MR}\right\}$, where $i\in \{\mathrm{building},\mathrm{forest},\mathrm{other}\}$, thus maintaining semantic compatibility between the map-based and image-based representations.

Due to the large size of the generated dataset, an additional preprocessing step was applied to reduce storage requirements and improve computational efficiency. The segmented image was converted to grayscale, preserving class information while significantly reducing data volume. As a result (Figure 3), the final high-resolution 8-bit image occupied approximately 0.9 GB of storage, which was sufficiently compact to enable efficient processing and the generation of map representations $MR$ for subsequent analysis.

The $SR$s were constructed at a set of predefined observation points, which served as reference locations for the analysis. At each reference location, a corresponding map representation, denoted as $M{R}_{i=0}$, was also generated. Furthermore, five additional $MR$ sets were created: $MR$-1.6, $MR$-2.4, $MR$-3.2, $MR$-4, and $MR$-8. Each set differed in the distance $D^{MR}$ = 1.6, 2.4, 3.2, 4, and 8 km between the reference position and the positions of the generated $MR$s. All $MR$s belonging to a given set were located at the same distance $D^{MR}$ from the reference position.

The purpose of introducing multiple $D^{MR}$ values was to analyse how the spatial separation between candidate positions affects the distinguishability of their corresponding map representations. When candidate positions are located close to one another, their associated (MR)s may be highly similar, making reliable identification of the correct position difficult even when the matching algorithm performs correctly. By gradually increasing $D^{MR}$, the generated map representations become more distinct, allowing the practical localisation capability of the method to be evaluated as a function of the spatial separation between candidate positions rather than through a single aggregate error metric.

Each set of map representations contained a maximum of 37 $MR$s, i.e., one $MR$ generated at the reference position ($MR\text{-}{D}_{i=0}^{MR}$) and up to 36 additional $MR$s ($MR\text{-}{D}_{i=1\dots 36}^{MR}$) evenly distributed every 10 degrees on a circle of radius $D^{MR}$. Candidate positions located on land were omitted.

Example spatial distributions of $MR\text{-}{D}_{i=0\dots 36}^{MR}$ are illustrated in Figure 4, where points located on land (and thus discarded) are marked in red, while correctly identified sea points are shown in green.

For each candidate point, the map representation $MR$ was generated by analysing the map in all azimuth directions with a resolution of $1^{\circ }$, resulting in a distance profile $D=\langle d_1,d_2,\dots ,d_{360}\rangle$. If land was not visible in a given direction, then $d_i=0$; otherwise $d_i>0$. Distances were calculated using the map resolution of approximately 1 m per pixel.

3.3. Results

The accuracy of the EMs was estimated separately for each $MR\text{-}{D}^{MR}$. The accuracy for one $MR\text{-}{D}^{MR}$ was calculated as the number of $SR$s out of all 70 $SR$s that were correctly associated with the correct $MR$, i.e., with $M{R}_{i=0}$ generated in the $SR$ position. Formally, the EM evaluation can be written as follows:

$$F(EM,MR\text{-}{D}^{MR},SR)={\displaystyle \sum _{i=1}^{70}}{\displaystyle \frac{\varphi (S{R}_i,MR\text{-}{D}_i^{MR})}{70}}$$

(16)

$$\varphi (SR,MR\text{-}{D}^{MR})=\left\{\begin{array}{cc}1\hfill & \mathrm{if}arg\underset{j=0,\dots ,36}{max}{M}^{EM}(SR,MR\text{-}{D}_j^{MR})=0\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(17)

3.3.1. EM1

The parameter setting for EM1 was as follows: $E_{max}^A$ = 5, and 10 deg, $\Delta E^A$ = 1 deg, $R_{max}^O$ = 8, 16 and 24 km, $R_{min}^O$ = 6 km, $\Delta R^O$ = 2 km, $O^{MR}=\left\{O_{building}^{MR}\right\}$, $\{O_{building}^{MR},O_{forest}^{MR}\}$, and $\{O_{building}^{MR},O_{forest}^{MR},O_{other}^{MR}\}$.

EM1 testing started with $MR$-8 and ended with $MR$-4 due to highly unsatisfactory results. In the case of $MR$-8, the best result was F(EM1,$MR$-8) = 0.45, i.e., only 32 correct decisions, identifying the SR position as the measurement position. This result was obtained for $O^{MR}=\left\{O_{building}^{MR}\right\}$, $E_{max}^A$ = 5 deg, and $R_{max}^O$ = 16 km. In other cases, i.e., when the map also included forests or other areas, the results were even worse. In the case of $MR$-4, and once again the best option, $O^{MR}=\left\{O_{building}^{MR}\right\}$, only 20 correct decisions were obtained, yielding the result F(EM1,$MR$-4) = 0.28, and demonstrating the very low accuracy of EM1, which decreases with decreasing distance $D^{MR}$.

There are several reasons for this. One is, of course, the erroneous operation of semantic segmentation, which either fails to “see” certain areas or artificially creates them in the image. Analysis of the segmented images, however, indicates that the impact of segmentation errors is not decisive.

Another reason for the poor EM1 results is potential compass errors in the camera. During testing, the maximum error was first assumed to be $E_{max}^A$ = 5 deg, then increased to 10 deg. Increasing $R_{max}^A$ at first glance seems to be the best way to eliminate compass error. The problem, however, was that increasing $E_{max}^A$ to 10 deg resulted in even more frequent association of $SR$ and $MR$ objects (usually urbanised areas), even though these did not correspond. In most cases, EM1, “looking” further and further by gradually increasing $R^O$ (see Algorithm 1), could match $SR$ with many good $MR$s even though these $MR$s were located far from each other. Often, the difference between the best match $M_{best}^{EM1}$ and the other matches was negligible, as shown in Figure 5.

The first two lines in this figure show a situation in which EM1 made an incorrect decision, associating $SR$ with the incorrect winning $MR$s. Figure 5a,c show the matchings of $SR$ with winning $MR$s, while Figure 5b,d show $SR$s together with $M{R}_{i=0}$, i.e., $MR$s generated on the map at the location where $SR$ was recorded. In addition to $SR$s and $MR$s, the figures also show the common parts between the two representations. It can be seen that the differences in the common parts between the winning $MR$s and $M{R}_{i=0}$ are very small.

In some cases, $M{R}_{i=0}$ was a winning $MR$, but subsequent analysis of matches between individual $SR$ and $M{R}_{i=0}$ objects showed that they refer to objects that are actually distant from each other.

It seems that the main factor leading to difficulties in pairing $SR$ with the correct $M{R}_{i=0}$ is the lack of information about the distances to objects observed in $SR$. The lack of this information causes Algorithm 1 to search for increasingly distant $SR$ matches on the map and often associates $SR$ with objects that are either inland or cannot be seen from the sea surface due to the large distance. The solution may be to limit $R_{max}^O$. However, the experiments revealed that this solution causes EM1 to miss many objects, resulting in even worse performance.

3.3.2. EM2

Testing this variant of position estimation required training images of the coast for regressors designed to estimate distance to land from visual data. These training images were recorded in the Gulf of Gdańsk, like the other images used in the experiments. The training set consisted of 86 images of the coast recorded at various positions in the Gulf. These images were then semantically segmented to extract land and determine its pixel height. The pixel values were then associated with distances to land, which was possible given information about the recording position and the camera viewing angle. After creating the training set, it was filtered using DBSCAN (eps = 0.7, min_samples = 6), and the resulting data were used to construct the aforementioned regressors, which are core components of EM2 and Algorithm 3.

This variant of the estimation module proved to be more accurate than EM1, as shown in

Table 1. Results of EM2—% of correct decisions, a correct decision is matching $SR$ with $M{R}_{i=0}$.

	LOLS	LTLS	KR
$MR$-1.6	28	34	41
$MR$-2.4	36	40	55
$MR$-3.2	42	45	64
$MR$-4	48	48	83
$MR$-8	55	58	86

. The best option turned out to be Kernel Regression (KR($\sigma =3$); see Algorithm 2). For higher $D^{MR}$, this variant achieved the following results: F(EM2,$MR$-4) = 0.83 and F(EM2,$MR$-8) = 0.86. Unfortunately, for smaller $D^{MR}$ values, EM2 still makes many mistakes. For $MR$-3.2 and $MR$-2.4, slightly less than half of the decisions are incorrect, and for $MR$-1.6, most decisions are incorrect.

An example of EM2 malfunction for $MR$-4 is shown in Figure 6. This figure shows the four situations in which EM2 made the largest errors, i.e., those in which the matching between the SR and $M{R}_{i=0}$ differed significantly from the best matching. The graphs show the pixel height H representing the $SR$, the distance to land D representing the $M{R}_{i=0}$, and the distance D representing the best-matching $MR$.

Analysis of Figure 6 shows two potential causes of errors. In the first upper graph, where the EM2 error is highest, the most likely cause is incorrect distance estimation by the KR. For the entire angular range for which land is visible on the $SR$, its height is low, suggesting a large distance to land, and this is, in fact, the case. Distance D at the $SR$ position for the land visibility range is approximately 10 km. Meanwhile, the winning $MR$, that is, the $MR$ best matched with $SR$, is approximately 4–5 km closer. This means that KR found the distance to land to be closer. The reason is that the training data used to construct the KR were closer to the decision corresponding to the winning $MR$ than $M{R}_{i=0}$. The situation presented in the upper graph must simply differ from what we most often observe in the Gulf of Gdańsk. A land height of approximately 30 pixels most often corresponds to a distance of about 5 km, not 10 km.

A different situation occurs in the remaining graphs, where the EM2 error is lower. In this case, it turns out that for the area in which KR is applied, the best-matching $MR$s, even though they are 4 km away from $SR$, are very similar to $M{R}_{i=0}$.

3.3.3. EM3

EM3 was run for $K=4$, $T^{EM3}=0.00005$ and $E_{max}^{E}M2=$ 10,000 and the best EM2 variant with KR regressor. It turned out that adding EM1 to EM2 in situations where multiple MRs achieved similar matching scores slightly improved the results of the original EM2 variant, but only for MR-4 and MR-8. The results obtained for these cases are as follows: F(EM3,$MR$-4) = 0.86, F(EM3,$MR$-8) = 0.9. For $MR$-4, improvement occurred for 2 $SR$s, and for $MR$-8, for 3 $SR$s. For $MR$-4, both correctly classified cases were previously incorrectly classified. They are shown in Figure 7 along with the best-matching $MR$s chosen by EM2. For $MR$-8, improvement was observed for previously misclassified $SR$s in 4 cases, and 1 previously correctly classified $SR$ was assigned to an incorrect $MR$.

4. Summary

This paper investigated the feasibility of using optical observations of the coastline to estimate position in GNSS-denied maritime environments. Three localisation approaches were evaluated.

The first approach, EM1, relied solely on the angular arrangement of areal objects visible on both maps and camera images. Experimental results demonstrated that this approach was ineffective because angular information alone was insufficient to uniquely associate observed objects with their map counterparts in areas containing numerous visually similar features.

The second approach, EM2, incorporated an estimate of the distance to the coastline derived from image features describing the visible land. This additional information significantly improved localisation performance compared with EM1, demonstrating that approximate range-related information plays a critical role in coastline-based localisation. However, the achieved localisation resolution remained limited. The experimental results indicate that the method can reliably discriminate between candidate positions separated by approximately 4 km, which is insufficient for precise coastal navigation.

The third approach, EM3, combined the distance-related information used by EM2 with the angular information employed by EM1. In ambiguous situations, angular observations were used as an additional decision criterion. This approach provided a modest improvement over EM2, increasing the probability of correctly identifying the reference position, but without fundamentally changing the achievable localisation resolution.

The obtained results indicate that the proposed Optical Coastal Navigation System should not be considered a replacement for conventional navigation systems in applications requiring precise coastal positioning. Instead, its current capabilities suggest potential use as a coarse localisation or position-correction mechanism in GNSS-denied environments, where even approximate position information may help constrain accumulated navigation errors and reduce the search area for more accurate localisation methods.

The experiments also demonstrated that reliable estimation of the distance to the visible coastline is the key factor limiting localisation performance. Therefore, future research will focus on improving distance estimation using additional image features. In particular, texture characteristics and the roughness of the extracted coastline silhouette will be investigated, as both features are expected to exhibit distance-dependent behaviour that may improve the discriminative capability of the proposed localisation framework.