Water Surface Spherical Buoy Localization Based on Ellipse Fitting Using Monocular Vision

Abstract

Spherical buoys serve as water surface markers, and their location information can help unmanned surface vessels (USVs) identify navigation channel boundaries, avoid dangerous areas, and improve navigation accuracy. However, due to the presence of disturbances such as reflections, water obstruction, and changes in illumination for spherical buoys on the water surface, using binocular vision for positioning encounters difficulties in matching. To address this, this paper proposes a monocular vision-based localization method for spherical buoys using elliptical fitting. First, the edges of the spherical buoy are extracted through image preprocessing. Then, to address the issue of pseudo-edge points introduced by reflections that reduce the accuracy of elliptical fitting, a multi-step method for eliminating pseudo-edge points is proposed. This effectively filters out pseudo-edge points and obtains accurate elliptical parameters. Finally, based on these elliptical parameters, a monocular vision ranging model is established to solve the relative position between the USV and the buoy. The USV’s position from satellite observation is then fused with the relative position calculated using the method proposed in this paper to estimate the coordinates of the buoy in the geodetic coordinate system. Simulation experiments analyzed the impact of pixel noise, camera height, focal length, and rotation angle on localization accuracy. The results show that within a range of 40 m in width and 80 m in length, the coordinates calculated by this method have an average absolute error of less than 1.2 m; field experiments on actual ships show that the average absolute error remains stable within 2.57 m. This method addresses the positioning issues caused by disturbances such as reflections, water obstruction, and changes in illumination, achieving a positioning accuracy comparable to that of general satellite positioning.

1. Introduction

Spherical buoys, serving as water surface markers, are widely used in scenarios such as dragon boat racing course markers, hazard warnings for water bodies, swimming area demarcations, and aquaculture region delineations (Figure 1a–d). Accurate positional information of these buoys can assist USV in precisely identifying navigation channels and hazardous areas, thereby optimizing their navigation routes. Additionally, using spherical buoys as reference points can help them correct their own positions and improve navigation accuracy. Furthermore, spherical buoys can be used to measure water flow velocity [1] and assess the impact of ocean currents on wave buoy measurements [2], with the premise being the accurate determination of the spherical buoys’ positions. In recent years, the “RobotX” international maritime challenge has required USVs to navigate within designated paths while avoiding collisions with gateways formed by paired buoys and black spherical buoys randomly placed, as shown in Figure 1e [3,4]. Similarly, in the “Roboboat” international autonomous boat competition, USVs face similar tasks, as shown in Figure 1f, where they must autonomously traverse courses marked by spherical buoys and navigate through obstacles consisting of spherical buoys with a variety of colors [5,6].

Currently, spherical target localization methods are primarily divided into two categories: fusion of vision and LiDAR and pure vision. Woo et al. [7] used LiDAR to obtain the position information of spherical buoys and combined it with images from a monocular camera. They detected spherical buoys using the Adaboost algorithm and determined their colors based on the Euclidean distance between the HSV values of the detected buoys and reference values. Stanislas et al. [8] built maps using LiDAR data, extracted obstacle lists from these maps, and classified features using a random forest algorithm. They also used HSV color space to determine the color of spherical buoys from images captured by a monocular camera. Although these methods can utilize the high localization accuracy of LiDAR and cameras to obtain position and feature information of spherical buoys, they require precise calibration between LiDAR and cameras. Moreover, feature point matching is challenging in dynamic water environments, which imposes higher demands on system performance. Pure vision-based spherical buoy localization methods are mainly divided into monocular and binocular vision localization. Binocular vision can directly obtain depth information and achieve high localization accuracy [9,10]. However, due to the reflection and water obstruction of spherical buoys on the water surface and the influence of wind and waves causing image tilt during navigation, binocular vision localization faces difficulties in matching and has a large computational load. In contrast, monocular vision localization can effectively avoid these issues. This method requires only one camera and can obtain depth information of targets in images to some extent, featuring a simple structure and fast computation speed [11,12,13].

Regarding the monocular vision-based localization of spherical targets, Penne et al. [14] proposed an analytical method that calculates the center of the sphere using the unique positive real solution of a cubic equation. This method is applicable when the spherical projection forms a complete ellipse. Van et al. [15] utilized monocular camera calibration information and the known actual diameter of the sphere to predict the pixel diameter of the sphere in the image using a small convolutional neural network. They then converted this into three-dimensional coordinates to achieve localization of the sphere. This method can be used to determine the accurate position of a basketball during shooting. Hajder et al. [16] precisely estimated the center position of a sphere in three-dimensional space by detecting ellipses from calibrated camera images and using the known radius of the sphere. This method is suitable for target localization in robotic vision. Zhao et al. [17] improved localization accuracy by combining elliptical projection transformation and experimental site correction techniques. They used data from two identical monocular cameras capturing the trajectory of a sphere (such as a ping-pong ball) and jointly corrected it with experimental site data. This method was designed for automatic hitting robots to quickly identify and locate ping-pong balls during high-speed continuous motion. Budiharto et al. [18] placed the sphere on the camera’s optical axis and estimated the distance between the camera and the sphere using the camera’s tilt angle. This system is primarily applied to soccer robots for ball localization during matches. Guan et al. [19] calculated the three-dimensional coordinates of the sphere’s center in the camera coordinate system based on the projected area and centroid of the sphere extracted from images, combined with the known radius of the sphere and camera internal parameters. They used geometric relationships and similar triangles to achieve this. This method is suitable for three-dimensional localization of spheres in indoor environments.

The above monocular vision localization methods can achieve three-dimensional positioning when the spherical projection contour is complete. However, the positioning of spherical buoys in a water environment faces multiple challenges:

(1)

Water obstruction can cause partial loss of the target’s edge, compromising the integrity of the spherical contour;

(2)

Water surface reflections can lead to false detection, introducing pseudo-edge points into the target contour;

(3)

Changes in illumination affect the grayscale distribution of the image, further weakening the visual distinction between reflections and the actual buoy. Additionally, intense illumination may cause overexposure on the surface of the spherical buoy, weakening local edge features and affecting contour extraction.

Given these challenges, to achieve accurate positioning of spherical buoys using monocular vision in a water environment, two key prerequisites must be met: first, accurately detecting the spherical buoy, and second, reconstructing the contour of the buoy based on the detection.

Current visual-based water surface buoy detection methods are mainly divided into two directions: deep learning-based methods and traditional image processing methods. For water surface buoy detection based on deep learning, Liu et al. [20] proposed an improved YOLOv4 model by integrating reverse depth separable convolution to optimize the network structure. Zhao et al. [21] presented a method combining image enhancement with YOLOv7 for buoy detection. Kang et al. [22] built a detection model based on the YOLOv5 algorithm, using Mosaic data augmentation and feature fusion network technology. Zhang et al. [23] proposed a method based on improved Faster R-CNN for detecting floating objects on the water surface. Zhang et al. [24] presented a method based on improved RefineDet for water surface target detection. However, these methods only generate bounding boxes for detected targets and do not extract their contours. Moreover, they often require the integration of multi-sensor data (such as LiDAR) or the use of stereo cameras to obtain the three-dimensional position information of the buoys. This paper aims to use a more economical monocular vision approach to detect the position of spherical buoys, making the aforementioned deep learning-based detection methods unsuitable for our research scenario.

In deep learning methods, research focused on extracting target contours is mainly concentrated in the field of edge detection. Relevant studies include the following: Cao et al. [25] proposed a new deep refinement network in 2021, combining multiple refinement aspects to obtain substantial edge detection features; Huan et al. [26] proposed a context-aware tracking strategy (CATS) for edge detection; Lin et al. [27] proposed a lateral refinement network algorithm for extracting target edges; Liu et al. [28] designed a fully convolutional neural network based on structural features of each layer for contour extraction; Khan et al. [29] automatically learned to extract image color, geometric features, and edge information to remove pseudo-edge points from contours; and Zheng et al. [30] utilized a convolutional residual neural network, inputted the global image, and combined semantic information for model training, enabling the trained model to detect and locate pseudo-edge points in contours. Although these methods effectively extract target contours and remove pseudo-edge points, they have limitations in our research scenario: they struggle to effectively restore contours lost due to water obstruction and have limited effectiveness in handling local edge feature weakening caused by strong illumination.

Furthermore, whether it is object detection or edge detection based on deep learning, these methods require a large amount of manually annotated training data and depend on multiple rounds of network training. They often require high-performance computing hardware, making it difficult to deploy them effectively on resource-constrained onboard computing platforms.

In contrast, traditional image processing methods are more practical in resource-constrained scenarios due to their simplicity. Aqthobilrobbany et al. [31] proposes a self-navigating system for robotic boats based on HSV color space, achieving visual tracking of spherical buoys through color threshold segmentation. Tran et al. [32] developed a buoy detection system using circular Hough transformation, integrating shape and color features to reduce the interference of illumination changes while maintaining real-time processing capabilities, suitable for automated maritime navigation tasks. However, these traditional methods do not involve contour extraction.

In traditional methods involving contour extraction, relevant research include the following: Shafiabadi et al. [33] used Canny and Sobel filters to detect crack contours in images for analyzing wall cracks; Isar et al. [34] used a denoising system based on the hyperanalytic wavelet transform before edge detection to enhance robustness; Dwivedi et al. [35] used continuous wavelet transformation to detect edge-forming streamlines; Li et al. [36] proposed a method for fitting edge points based on grayscale centroids and moving least squares; and Chen et al. [37] combined wavelet transformation with cubic B-spline interpolation to achieve higher precision in edge detection. However, these traditional contour extraction methods have limitations in our scenario: they cannot effectively remove pseudo-edge points from contours, and they struggle to restore missing contour parts caused by water obstruction. Additionally, they have limited effectiveness in handling local edge feature weakening caused by strong illumination.

To address the above issues, this paper proposes a spherical buoy localization method based on ellipse fitting using monocular vision. The method first preprocesses images of spherical buoys captured by a camera to extract edge points. Then, based on the extracted edge points, it uses grayscale feature analysis for preliminary screening to remove some pseudo-edge points, reducing the number of pseudo-edge points in the input data for Random Sample Consensus (RANSAC). It then uses RANSAC to perform a second screening on the remaining edge points to further eliminate pseudo-edge points. After that, it uses the least squares method to fit an ellipse to the screened edge points, obtaining high-precision ellipse parameters. Finally, based on the ellipse parameters, this paper establishes a monocular vision-based spherical buoy localization model. The model converts the position of the spherical buoy in the world coordinate system into pixel coordinates using a pinhole camera model and calculates the Euclidean distance from the endpoints of the projected ellipse’s major axes to the camera’s optical axes center. It normalizes these distances with the camera’s focal length and applies the arctangent function to calculate the corresponding viewing angles. By determining the angle difference, it uses the known radius of the spherical buoy and geometric relationships to solve the relative distance between the spherical buoy and the camera. Based on relative distances and the USV’s position from satellite observation in the scene, we estimated the precise location of the spherical buoy in space, achieving global positioning of the spherical buoy.

The main contributions of this paper are summarized as follows:

• A monocular vision-based localization method using elliptical fitting is proposed to address the issues of partial edge loss caused by water obstruction and the weakening of local edge features due to strong light, thereby improving the positioning accuracy of spherical buoys in environments with water obstruction and high illumination.
• A multi-step pseudo-edge elimination method for elliptical fitting is proposed to solve the problem of reflection interference. By performing multi-step edge point filtering on images, this method effectively reduces the impact of reflections and other interferences on elliptical fitting.
• Experimental verification was conducted. Through experiments, the effectiveness and accuracy of this method in real-world environments were validated, demonstrating its practical value in the localization of spherical buoys on the water surface.

The remainder of this paper is organized as follows: Section 2 describes the image preprocessing process; Section 3 presents the monocular vision localization method based on ellipse fitting; Section 4 provides simulation verification; Section 5 presents shipboard verification; and Section 6 concludes the paper.

2. Image Preprocessing

Figure 2 illustrates the image preprocessing process for spherical buoys. In real-world environments, images captured by a USV can result in the objects on the water surface and their reflections appearing tilted relative to the pixel coordinate system’s vertical axes due to the roll angle. To correct this tilt, this paper uses an inertial measurement unit (IMU) to measure the roll angle of the USV and applies an image rotation algorithm to correct the image.

Assuming the rotation center is the origin of the pixel coordinate system, the pixel coordinates before correction are denoted as $\left(u^{\prime },v^{\prime }\right)$, the roll angle is denoted as $\phi$, and the corrected pixel coordinates $\left(u,v\right)$ are given by:

$$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\cos \phi & \sin \phi & 0\\ -\sin \phi & \cos \phi & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}^{\prime }\\ v^{\prime }\\ 1\end{array}\right]$$

(1)

Using the inverse mapping method, the rotated pixel points are determined according to the mapping relationship to correct the original pixel points. The corrected image is shown in Figure 2b.

Let the corrected RGB image be denoted as follows:

$$I_c(u,v)=(r(u,v) g(u,v) b(u,v))$$

(2)

Based on the environmental and color information of the spherical buoys, the image is segmented to obtain the binary image of the spherical buoys. The segmentation strategy is expressed as follows:

$$I_{bin}(u,v)=\left\{\begin{array}{l}1 {\lambda }_{r}r(u,v)+{\lambda }_{g}g(u,v)+{\lambda }_{b}b(u,v)\ge 0\\ 0 {\lambda }_{r}r(u,v)+{\lambda }_{g}g(u,v)+{\lambda }_{b}b(u,v)<0\end{array}\right.,$$

(3)

where, ${\lambda }_r$, ${\lambda }_g$, and ${\lambda }_b$ represent the coefficients of the three channels used to adjust the intensity of different color channels. Their values are positive or negative depending on the color of the spherical buoys. For example, for red spherical buoys, ${\lambda }_r$ is positive, while ${\lambda }_g$ and ${\lambda }_b$ are negative.

After target segmentation processing, a binary image is obtained, as shown in Figure 2c. However, the image contains noise and holes, which affect subsequent detection and reduce the detection rate of the spherical buoys. To solve this problem, this paper uses morphological opening processing. A circular structural element $K_r$ with a radius of 4 is selected to first erode and remove noise and then dilate to fill gaps, as follows:

$$I_{open}(u,v)=(I_{bin}(u,v)\Theta K_r)\oplus K_r$$

(4)

The binary Figure 2e obtained from the above processing contains spherical buoys and their reflections. Since there are often multiple spherical buoys in the image, based on 8-connectivity, connected region labeling is performed on the regions of the spherical buoys and their reflections to distinguish between different buoys. Then, the Canny edge operator is used to extract the edges of each spherical buoy, with the extraction results shown in Figure 2f.

3. Monocular Vision-Based Localization Method Using Elliptical Fitting

Based on the edge points generated from the image preprocessing in the previous section, this section further explores the localization method for spherical buoys. The structure of the monocular vision localization system is shown in Figure 3. Due to the obstruction caused by water, which results in incomplete contours of the spherical buoys, a localization method based on ellipse fitting is proposed. This method leverages the characteristic that the projection of a sphere in a camera appears as a circle or an ellipse [14].

3.1. Obtaining Ellipse Parameters

In water environments, the presence of water surface reflections often results in a large number of pseudo-edge points among the obtained edge points. These pseudo-edge points significantly affect the accuracy of ellipse fitting using the least squares method, leading to biases in the estimation of ellipse parameters such as the center and radius, which in turn impact the localization accuracy of spherical buoys. Therefore, accurately estimating ellipse parameters is crucial.

RANSAC is a robust algorithm used to estimate mathematical model parameters from data containing outliers [38]. Its core idea is to find the model that best represents the data through random sampling and inlier detection. However, when the proportion of pseudo-edge points is high, RANSAC’s performance significantly deteriorates because random sampling is more likely to select pseudo-edge points, resulting in inaccurate ellipse model estimation.

In addition to considering the ellipse model, this paper also starts from the actual scenario of water surface spherical buoys. It uses the different grayscale values between the spherical buoy area and the reflection area to perform grayscale characteristic analysis on the reflection boundary line area, thereby determining the position of the reflection boundary line. However, in practical applications, changes in lighting conditions significantly affect the grayscale distribution of images, leading to a reduction in the grayscale value differences in the reflection boundary line area, which in turn affects the accurate positioning of the reflection boundary line.

Since using grayscale analysis or RANSAC alone has limitations, this paper proposes a multi-step pseudo-edge removal ellipse fitting method. This method first uses grayscale analysis to preliminarily screen edge points and remove some pseudo-edge points, thereby reducing the number of pseudo-edge points input into RANSAC. Then, it uses RANSAC to perform a second screening on the remaining edge points to further eliminate pseudo-edge points. Finally, it uses the least squares method to fit an ellipse to the screened edge points, obtaining high-precision ellipse parameters. Figure 4 illustrates the process, and the specific steps are as follows:

(1)

Edge Point Reordering

Let the set of detected edge points be $P={\left\{p_i=\left(x_i,y_i\right)\right\}}_{i=1}^{N_1}$, where $p_i$ represents the coordinates of the $i$th edge point. These points are reordered according to the natural order of the ellipse (clockwise or counterclockwise) to obtain a new ordered set $P^{\prime }={\left\{{p_i}^{\prime }\right\}}_{i=1}^{N_1}$.

(2)

Edge Point Supplementation

Due to the possibility that image preprocessing may not fully extract the target edge contour, resulting in gaps in the contour, a set of points $Q$ is defined for the missing regions. Through B-spline interpolation, a set of supplementary points $Q^{\prime }$ is generated. The mathematical expression for the B-spline is as follows:

$$C(u)={\displaystyle \sum _{i=0}^{n-1}{N}_{i,k}}(u)\cdot p_i^{\prime },$$

(5)

where $C(u)$ is the B-spline curve; $p_i^{\prime }$ represents the $n$ control points selected from the ordered set $P^{\prime }$; ${\displaystyle \sum _{i=0}^n{N}_{i,k}}(u)$ is the $i$th B-spline basis function; $k$ is the order of the B-spline; and $u$ is the parameterization variable. By using B-spline interpolation to generate the supplementary point set $Q^{\prime }$ and by merging it with the original point set $P^{\prime }$, the following closed contour is constructed:

$$P^{″}=P^{\prime }\cup Q^{\prime }.$$

(6)

The constructed closed contour is shown in Figure 5:

(3)

Candidate Point Selection

Let the grayscale value of each row in the image be $I(y,x)$, where $y$ is the row number and $x$ is the image width. For the closed contour $P^{″}$, extract the pixel grayscale value of each row within the contour via the following expression:

$$I_y=\{I(y,x)\mid (x,y)\in P^{″}\}.$$

(7)

Calculate the median grayscale value for each row:

$$M\left(y\right)=\mathrm{Median}(I_y).$$

(8)

Connect the median grayscale values $M\left(y\right)$ of all rows to form a median grayscale curve $M$, and calculate the gradient of the median grayscale curve $M$:

$$G(y)={\displaystyle \frac{\partial M(y)}{\partial y}}.$$

(9)

Find the row number corresponding to the maximum gradient value:

$$y^{*}=\arg \underset{y}{\max } G(y).$$

(10)

The edge points above the split line are identified as candidate points for the true contour:

$$P_{\mathrm{candidate}}=\{p_i\in P^{″}\mid y_i<y^{*}\}.$$

(11)

The median grayscale curve, gradient curve, and candidate points are shown in Figure 6a–c:

(4)

Random Sampling

Randomly select five points from the candidate point set $P_{\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}}$, and use the least squares method to fit the ellipse model through the following minimized objective function:

$$\min {{\displaystyle \sum _{i=1}^5\left(A{x}_i^2+B{x}_i{y}_i+C{y}_i^2+D{x}_i+Ey+F\right)}}^2.$$

(12)

Solve for the ellipse parameters $A$, $B$, $C$, $D$, $E$, and $F$.

(5)

Inlier Detection

For each point $p_i\in P_{\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}}$, calculate its algebraic distance $d_i$ to the ellipse model:

$$d_i=A{x}_i^2+B{x}_i{y}_i+C{y}_i^2+D{x}_i+Ey+F,$$

(13)

if $\left|d_i\right|<\delta$ (preset threshold), it is judged as an inlier; otherwise, it is judged as an outlier. Count the number of inliers of the current ellipse model. If the number of inliers of the current ellipse model is greater than the historical optimal value, update the optimal ellipse model and inlier set.

(6)

Iterative Optimization

Repeat the above two steps (random sampling and inlier detection) $k$ times. Each iteration generates a candidate ellipse model, and its quality is evaluated by inlier detection. Finally, the ellipse model with the largest number of inliers is selected as the optimal model.

(7)

Ellipse Fitting

Use the optimal inlier set, and use the least squares method to fit the final ellipse model, as shown in Figure 7:

3.2. Positioning Calculation

Currently, monocular vision ranging methods are mainly divided into three categories [11,12,13]: The first category is to calculate the distance between the camera and the target based on the geometric relationship established by the imaging model; the second category is to calculate the actual distance by establishing the proportional relationship between the actual length of the target and its physical length in the image and then using the principle of triangle similarity; the third category is to measure distance by establishing a mathematical regression model. This paper is based on the first method, establishing a ranging model based on the parameters of the projected ellipse. Building on this, we utilize the known satellite observation data from the USV in the scene to obtain the coordinates of each spherical buoy in the geodetic coordinate system.

3.2.1. Relative Distance Calculation

When the center of the sphere is located on the camera’s optical axis, its projection on the image plane is circular; in other cases, its projection is elliptical. These projected ellipses have the following characteristics [17]:

• The image plane center $O$ lies on the line containing the major axes $q_1{q}_2$ of the ellipse.
• The image point $S_1$ of the center of the sphere is located on the line segment $q_1{q}_2$ of the major axes of the ellipse.

The pinhole imaging model of the spherical buoy is shown in Figure 8a:

Where $O_c{X}_c{Y}_c{Z}_c$ is the camera coordinate system, $OXY$ is the image coordinate system, and $O_{0}UV$ is the pixel coordinate system. $q_1$ and $q_2$ are the endpoints of the major axes of the ellipse formed by the projection of the spherical buoy on the image plane. $P_1$ and $P_2$ are the tangent points where extensions of $O_c{q}_1$ and $O_c{q}_2$ intersect the sphere. $S_0$ is the center of the spherical buoy; $S_1$ is the projection of the sphere’s center on the image plane; $O_2$ is the center of the projected ellipse; and $\theta$ is the angle formed by $q_1$, $q_2$, and the origin $O_c$ of the camera coordinate system.

Using the pinhole camera model to transform world coordinates $\left(X_w,Y_w,Z_w\right)$ to pixel coordinates $\left(u,v\right)$, the transformation expression is as follows:

$$Z_C\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/dx& 0& u_0\\ 0& 1/dy& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0_{1\times 3}& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right],$$

(14)

where $f$ is the camera focal length; $dx$ and $dy$ are the physical dimensions of a single pixel in the $X$ and $Y$ axes directions, respectively; $\left(u_0,v_0\right)$ are the pixel coordinates of the image plane center; $R$ and $t$ represent the rotation matrix and translation vector from the world coordinate system to the camera coordinate system, respectively; and $Z_c$ is the depth coordinate value of the spherical buoy in the camera coordinate system. Let the camera intrinsic parameter matrix be:

$$M_1=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right],$$

(15)

where $f_x=f/dx$, $f_y=f/dy$ represent the effective pixel focal lengths in the $X$-axis and $Y$-axis directions of the image, respectively. Let the extrinsic parameter matrix be $M_2=\left[\begin{array}{cc}R& t\end{array}\right]$, so we can obtain the following:

$$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M_1{M}_2\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right].$$

(16)

Ignoring camera distortion and other factors, the projection of the spherical buoy is always perpendicular to the line connecting the sphere’s center to the camera’s optical center. The points where rays emitted from $O_c$ are tangent to the sphere form a circle $C_1$. These rays and $O_c$ form a right circular cone with $O_c$ as its vertex. According to the above geometric relationships, we can obtain:

$$|O{q}_1|=\sqrt{{\left[(u_1-u_0)dx\right]}^2+{\left[(v_1-v_0)dy\right]}^2},$$

(17)

$$|O{q}_2|=\sqrt{{\left[(u_2-u_0)dx\right]}^2+{\left[(v_2-v_0)dy\right]}^2},$$

(18)

$$\theta =\arctan {\displaystyle \frac{|O{q}_1|}{f}}-\arctan {\displaystyle \frac{|O{q}_2|}{f}},$$

(19)

where $\left(u_1,v_1\right)$ are the pixel coordinates of $q_1$, and $\left(u_2,v_2\right)$ are the pixel coordinates of $q_2$. Since $P_2$ is a tangent point on the sphere, according to the properties of tangents to a circle, $O_c{P}_2$ is perpendicular to $P_2{S}_0$, and $O_c{S}_0$ is the angle bisector of $\theta$. Let the radius of the spherical buoy be $r$, then the distance from the sphere’s center to the origin of the camera coordinate system is as follows:

$$|O_c{S}_0|={\displaystyle \frac{r}{\sin (\theta /2)}}.$$

(20)

3.2.2. Estimate the Spherical Buoy’s Coordinates in the Geodetic Coordinate System

Since the projection point $S_1$ of the sphere center $S_0$ on the image plane coincides with the center $O_2$ of the ellipse only when the sphere center $S_0$ is on the optical axes of the camera, and $S_1$ and $O_2$ do not coincide in other cases, $O_2$ cannot be directly regarded as the projection point of the sphere center. Based on the property that $O_c{S}_0$ is the angular bisector of $\theta$, Equations (17) and (18) can be combined to obtain:

$$|O{S}_1|=f\tan \left[{\displaystyle \frac{1}{2}}(\arctan {\displaystyle \frac{|O{q}_1|}{f}}+\arctan {\displaystyle \frac{|O{q}_2|}{f}})\right].$$

(21)

Since points $S_1$, $q_1$, and $O$ are collinear, $S_1$ and $q_1$ are projected onto the $X$ axes of the image coordinate system, forming two similar triangles. As shown in Figure 8b, $X_1$ and $X_2$ are the projections of $S_1$ and $q_1$ onto the $X$ axes, respectively. By the theorem of similar triangles, the pixel coordinates of $S_1$ can be obtained:

$$\left[\begin{array}{c}{u}_{s_1}\\ v_{s_1}\end{array}\right]={\displaystyle \frac{|O{S}_1|}{|O{q}_1|}}\left[\begin{array}{c}{u}_1-u_0\\ v_1-v_0\end{array}\right]+\left[\begin{array}{c}{u}_0\\ v_0\end{array}\right].$$

(22)

Estimate the spherical buoy’s coordinates in the geodetic coordinate system using the relative distance between the camera coordinate system’s origin and the sphere’s center, the USV’s position from satellite observation, and the heading information. As shown in Figure 9, the positional relationship between the spherical buoy and the USV is established. $O_w{X}_w{Y}_w$ represents the world coordinate system; $O_s{X}_s{Y}_s$ represents the body coordinate system of the vehicle, where the origin $O_s$ is the center of gravity of the USV; $\beta$ is the angle between the line connecting the center of the sphere and the center of gravity of the USV and the $Y_w$ axes of the world coordinate system; $\epsilon$ is the angle between the $X_s$ axes of the body coordinate system and the $Y_w$ axes of the world coordinate system, which is the heading angle of the USV; and $\alpha$ is the angle between the line connecting the center of the sphere and the center of gravity of the USV and the $X_s$ axes of the body coordinate system:

$$\alpha =\arctan {\displaystyle \frac{|u_{s_1}-u_0|dx}{f}},$$

(23)

$$\beta =\left\{\begin{array}{cc}\epsilon -\alpha & u_{s_1}<u_0\\ \epsilon +\alpha & u_{s_1}\ge u_0\end{array}\right..$$

(24)

Assuming the body frame coincides with the camera frame and the USV’s position from satellite observation is $\left(x_1,y_1\right)$, the estimated coordinates of the spherical buoy in the geodetic coordinate system are obtained as:

$$\left[\begin{array}{l}{x}_2\\ y_2\end{array}\right]=\left[\begin{array}{l}{x}_1\\ y_1\end{array}\right]+\sqrt{|O_c{S}_0{|}^2-h^2}\left[\begin{array}{l}\sin \beta \\ \cos \beta \end{array}\right],$$

(25)

where $h$ is the vertical distance from the camera’s optical center to the horizontal plane containing the center of the sphere.

4. Simulation Verification

4.1. Scene Simulation

To verify the accuracy of the proposed localization method in this paper, simulation experiments were conducted under different conditions and compared with the method in reference [19]. The simulation scenario is shown in Figure 10, where Figure 10a is a three-dimensional scene in which 30 spherical buoys are arranged in five rows and six columns; the red, blue, and green arrows represent the $X_c$, $Y_c$, and $Z_c$ axes of the camera coordinate system, respectively. The projection of the spherical buoys obtained by the camera located in the position shown in Figure 10a is shown in Figure 10b.

To ensure that the camera mounted on the USV can effectively capture the spherical buoys along its path during the defined route, the simulation experiment path is set as shown in Figure 11. In the experiment, the initial coordinates of the USV are set to (5, 18), and the initial heading angle is set to 0 degrees. As the USV travels along the defined path, every 1 m, a coordinate calculation is performed for each spherical buoy appearing in the projection image, and the total number of times each spherical buoy appears in the projection image is recorded as $J$, as well as the result of each coordinate calculation $\left({\widehat{x}}_k^j,{\widehat{y}}_k^j\right)$, where $k$ represents the spherical buoy number, $k=1,2,\dots ,30$, $j=1,2,\dots ,J$. Then, based on the frequency of appearance of each spherical buoy and the results of each coordinate calculation, the mean of its coordinates is computed and used as the estimated coordinate value $\left({\widehat{x}}_k,{\widehat{y}}_k\right)$ of the spherical buoy in the geodetic coordinate system:

$$\left[\begin{array}{l}{\widehat{x}}_k\\ {\widehat{y}}_k\end{array}\right]=\left[\begin{array}{l}{\displaystyle \frac{1}{J}}{\displaystyle \sum _{j=1}^J{\widehat{x}}_k^j}\\ {\displaystyle \frac{1}{J}}{\displaystyle \sum _{j=1}^J{\widehat{y}}_k^j}\end{array}\right].$$

(26)

To evaluate the accuracy of the localization method, the mean absolute error is used as an intuitive metric:

$$MAE={\displaystyle \frac{1}{30}}{\displaystyle \sum _{k=1}^{30}{‖(x_k,y_k)-({\widehat{x}}_k,{\widehat{y}}_k)‖}_2},$$

(27)

where $\left(x_k,y_k\right)$ represents the true coordinates of the spherical buoy in the geodetic coordinate system.

4.2. Impact of Noise Interference

The projection of the spherical buoy is subject to interference from environmental and image processing parameters, and the combined effect of these factors may lead to errors in extracting the feature parameters of the spherical buoy. To simulate the effects of these disturbances, the camera focal length was set to 12mm and the camera height was set to 30cm in the simulation experiment, and comparative experiments were conducted with different pixel noise intensities. A noise level of 0.00 was selected to simulate the case of no pixel noise to verify the correctness of the method in this paper. In addition, noise levels of 0.25 and 0.50 were selected to simulate the interference of small and moderate pixel noise, respectively, to test the anti-interference ability of the method in this paper. The results of the simulation experiments are shown in Figure 12a–c.

Figure 12a–c show the simulation experiment results using the method from reference [19] and the method presented in this paper under different noise intensities. As can be seen from the figures, under the condition of 0.00 noise, the mean absolute error calculated by both methods is 0, indicating that both methods can correctly calculate the absolute coordinates without noise interference, which verifies the correctness of the method in this paper. However, as the noise increases, the mean absolute error calculated by both methods increases. This is because as the pixel noise increases, the proportion of pixel noise in the projection area of the spherical buoy increases. Therefore, the influence of pixel noise on the absolute coordinate solution gradually increases, and the mean absolute error obtained by solving the absolute coordinates also gradually increases. Under the conditions of 0.25 and 0.50 noise, the mean absolute errors obtained by the method in reference [19] are 0.86 m and 0.93 m, respectively, while the mean absolute errors obtained by the method in this paper are 0.42 m and 0.56 m, respectively. The mean absolute error is reduced by 51.1% and 39.7%, respectively, which verifies the anti-interference effect of the method in this paper.

4.3. Impact of Camera Height

To verify the influence of camera height on the algorithm in this paper, the camera focal length was set to 12 mm and the noise was set to 0.25 in the simulation. Comparative experiments were conducted with camera heights of 30 cm, 60 cm, and 90 cm. The results of the simulation experiments are shown in Figure 13a–f.

Figure 13a–c show the simulation experiment results obtained using the two methods at different camera heights. As can be seen from the figures, as the camera height increases, the mean absolute error obtained using the method in this paper stabilizes within 0.62 m, while the mean absolute error obtained using the method in reference [19] increases. Furthermore, at the same height, the mean absolute error obtained using the method in this paper is reduced by more than 30.6% compared to the method in reference [19]. It can be seen that the method in this paper is more stable and accurate in locating spherical buoys at different camera heights.

Figure 13d–f represent the projection situations when the camera height is 30 cm, 60 cm, and 90 cm, respectively. As can be seen from the figures, as the camera height increases, the projection area of the spherical buoy in the camera gradually decreases; at the same time, the occlusion between the spherical buoys in the image decreases, which improves the visibility of the spherical buoys in the image so that a more complete projection shape of the spherical buoys can be obtained. The mean absolute error when using the method in this paper at a camera height of 30 cm is relatively the smallest, which indicates that although increasing the camera height can improve visibility, the reduction in the projection area has a more significant impact on the method in this paper. This also provides a basis for the selection of camera height in real-world experiments.

4.4. Impact of Camera Focal Length

To evaluate the effect of changes in camera focal length on the algorithm in this paper, under the simulation conditions of noise at 0.25 and camera height at 30 cm, the camera focal length was set to 8 mm, 12 mm, and 16 mm, respectively, and the absolute error under different focal lengths was calculated. The simulation results are shown in Figure 14a–f.

Figure 14a–c show the results of solving the absolute coordinates of the spherical buoys using the two methods at different focal lengths. As can be seen from the figures, when the focal length is 12 mm and 16 mm, the mean absolute error calculated by both methods is less than 1 m. This indicates that both methods can effectively and accurately solve the absolute coordinates when the focal length is 12 mm and 16 mm. However, when the focal length is 8 mm, the mean absolute error obtained by the method from reference [19] is 2.47 m, while the mean absolute error obtained by the method in this paper is 0.81 m. Compared with the method from reference [19], the mean absolute error is reduced by 67.2%, which indicates that the method in this paper has high adaptability under different camera focal length conditions.

Figure 14d–f represent the projection situations when the focal length is 8 mm, 12 mm, and 16 mm, respectively. As can be seen from the figures, as the focal length increases, the proportion of the projected area of the spherical buoy in the image increases. This is because the increase in focal length leads to a decrease in the camera’s field of view, which makes the area of the spherical buoy in the image relatively larger, which will improve the noise resistance of the two methods.

4.5. Impact of Rotation Angle

During the operation of the USV, its roll angle, pitch angle, and yaw angle will change. To evaluate the effect of changes in the rotation angle on the algorithm in this paper, the noise was set to 0.25, the focal length was set to 12 mm, and the camera height was set to 30 cm in the simulation. The rotation angles were set to 10°, 20°, and 30°, respectively, and the absolute error under different rotation angle disturbances was calculated. The simulation results are shown in Figure 15a–l.

Figure 15a–i show the results of locating spherical buoys using the two methods when the USV is affected by different magnitudes of yaw, roll, and pitch angles of 10°, 20°, and 30°. As can be seen from the figures, under the influence of rotation angles, the mean absolute error obtained by the method in this paper is smaller, indicating that the absolute coordinates of the spherical buoy obtained based on the method in this paper are closer to the true value. Especially under the influence of the roll angle, when the roll angle reaches 30°, the mean absolute error obtained by the method from reference [19] is 1.99 m, while the mean absolute error obtained by the method in this paper is 1.15 m. Compared with the method from reference [19], the mean absolute error is reduced by 42.2%. Thus, it can be concluded that under the influence of rotational disturbances, the method in this paper has a better absolute coordinate solution effect.

Figure 15j–l represent the projection scenarios when the yaw, roll, and pitch angles are 10 degrees, respectively. As can be seen from the figures, under the influence of the yaw angle, the projected position of the spherical buoy shifts left and right; under the influence of the pitch angle, the projected position of the spherical buoy shifts up and down; under the influence of the roll angle, the spherical buoy appears tilted.

5. Real-World Experiment

5.1. Hardware Configuration

Using the USV as a platform for onboard experiments, a high-resolution industrial camera with a resolution of 1920 × 1080 pixels, a focal length of 12 mm, and a height setting of 30 cm was mounted to capture images of the spherical buoys. Through calibration, the intrinsic parameter matrix $M_{12mm}$ and distortion coefficients $D_{12mm}$ of the onboard camera were obtained:

$$M_{12mm}=\left[\begin{array}{ccc}4470.8& 0& 928.496\\ 0& 4459.0& 526.385\\ 0& 0& 1\end{array}\right],$$

(28)

$$D_{12mm}=\left[\begin{array}{ccccc}0.3399& -3.8439& -0.0066& 0.0003& -1.52317629\end{array}\right].$$

(29)

5.2. USV Experiment

5.2.1. Ellipse Fitting Comparative Experiment

To verify the effectiveness of the ellipse fitting method proposed in this paper, multiple experiments were conducted (with all images captured by the imaging acquisition device carried by USV “Haiyue”). Three representative images were selected for demonstration. Figure 16a shows the original image of the spherical buoy. For these images, we used four different methods to perform ellipse fitting on the edge points of the spherical buoy: the first method filters the edge points based on gray-scale characteristic analysis and uses the least squares method to fit the ellipse, as shown in Figure 16b; the second method uses the RANSAC algorithm to filter the edge points and also uses the least squares method to fit the ellipse, as shown in Figure 16c; the third method applies the constrained randomized Hough transform-based ellipse fitting algorithm proposed in reference [39], as shown in Figure 16d; the fourth method employs the approach proposed in this paper, as shown in Figure 16e.

From the first row of Figure 16, it is evident that all four methods accurately fit the complete spherical buoy contour in the absence of water surface reflections.

In the second row of Figure 16, the larger reflection areas introduce more pseudo-edge points, which compromise the fitting performance of Method 2. This occurs because a higher proportion of pseudo-edge points increases the likelihood of random sampling selecting these erroneous points, leading to inaccurate ellipse model estimates. Method 3 also fails to reconstruct complete contours under reflection interference. In contrast, Method 1 and the method proposed in this paper effectively eliminate reflection interference through grayscale analysis, resulting in more accurate contour fitting.

The third row of Figure 16 shows that under varying illumination conditions, reduced grayscale contrast at reflection boundaries weakens the effectiveness of Method 1. Although Method 3 can filter some curvature anomalies through curvature constraints, it cannot completely remove all pseudo-edge points due to inconsistent curvature distributions introduced by reflections, leading to suboptimal ellipse fitting. Methods 2 and the method proposed in this paper demonstrate improved accuracy when reflection areas are small, as RANSAC effectively filters pseudo-edge points.

In summary, the experimental results validate the effectiveness of the elliptical fitting methods presented in this paper.

5.2.2. Spherical Buoys Localization Experiment

After completing the ellipse fitting comparison experiments, to validate the practicality of the positioning method proposed in this paper, we further conducted a spherical buoy localization experiment on an open lake surface as shown in Figure 17a. The experimental scenario involved 30 spherical buoys with a radius of 12.7 cm, arranged in a 5 × 6 matrix, covering a water surface area of 40 m in width and 80 m in length.

The experiment considered both external and internal interference factors. The external factors primarily refer to varying environmental conditions, while the internal factors pertain to camera distortions. The specific steps of the experiment are as follows: Firstly, a GNSS-equipped USV was manually controlled to visit each spherical buoy in sequence, collecting satellite observation data for each buoy. Subsequently, the USV, equipped with a camera, GNSS, and IMU, traveled along the route set in Figure 17b, synchronously recording image data, the USV’s position from satellite observation, heading information, and roll angle. Finally, the coordinates of each spherical buoy in the geodetic coordinate system were estimated using the method proposed in this paper, as well as the methods presented in references [19,40].

(1)

Strong Illumination, Reflections, and Water Obstruction

Figure 18a shows an image taken under strong noon illumination, where intense light causes color distortion in parts of the buoy and there are significant reflections and water obstructions. After image preprocessing, the result is shown in Figure 18b, where the contour is affected by false edges and local deformations. Further applying the proposed algorithm for ellipse fitting, the result is shown in Figure 18c. Although there are some contour mismatches, the fitted ellipse aligns well with the actual buoy overall, verifying the algorithm’s robustness against strong illumination.

In this environment, the comparison of absolute errors among the three methods is shown in Figure 19.

As shown in Figure 19, the mean absolute error of the spherical buoy positions calculated using the method proposed in this paper is 2.57 m. This represents a reduction of 14.6% compared to the method in reference [19] (3.01 m) and a reduction of 19.9% compared to the method in reference [40] (3.21 m).

(2)

Low Illumination, Reflections, and Water Obstruction

Figure 20a shows an image captured in a low-light evening scenario, where illumination intensity is significantly reduced but reflections and obstructions still exist. The preprocessing result is shown in Figure 20b, and the ellipse fitting result is shown in Figure 20c. The detection results indicate that the algorithm can accurately reconstruct the buoy contour even under weak light, reflections, and water obstructions.

In this environment, the comparison of absolute errors among the three methods is shown in Figure 21.

As shown in Figure 21, the mean absolute error of the spherical buoy positions calculated using the method proposed in this paper is 2.42 m. This represents a reduction of 16.3% compared to the method in reference [19] (2.89 m) and a reduction of 22.4% compared to the method in reference [40] (3.12 m).

(3)

Low Water Transparency, Reflections, and Water Obstruction

Figure 22a shows an image captured in a scenario with low water transparency, where reflections and obstructions exist. After preprocessing, the result is shown in Figure 22b, and the final contour reconstruction effect after applying the proposed ellipse fitting algorithm is shown in Figure 22c. The detection results demonstrate that the algorithm can effectively fit the complete contour of the spherical buoy in environments with low water transparency, reflections, and water obstructions.

In this environment, the comparison of absolute errors among the three methods is shown in Figure 23.

As shown in Figure 23, the mean absolute error of the spherical buoy positions calculated using the method proposed in this study is 2.37 m. This represents a reduction of 20.5% compared to the method in reference [19] (2.98 m) and a reduction of 30.5% compared to the method in reference [40] (3.41 m).

(4)

Comparison of Positioning Results Before and After Correction

To quantitatively analyze the impact of lens distortion on the positioning accuracy of the camera used in this study, we performed lens distortion correction on images collected under conditions of strong illumination, reflections, and water occlusions. We then applied the positioning algorithm proposed in this paper to both the original and corrected images for location calculations. The images before and after distortion correction are shown in Figure 24a,b, respectively, while the positioning results before and after distortion correction are presented in Figure 25.

From Figure 24a,b, it can be observed that the distortion phenomenon in the uncorrected image is not pronounced, resulting in no significant difference between the images before and after correction. According to the experimental results shown in Figure 25, the mean absolute error for the spherical buoy’s position before correction was 2.58 m, while after correction it was 2.57 m, with a difference of only 0.01 m between the two. This outcome indicates that the impact of the camera’s distortion on positioning accuracy is negligible.

In summary, the method proposed in this paper can stably detect spherical buoys and accurately reconstruct their contours under various complex environments, including high illumination, low illumination, low water transparency, water surface reflections, and partial occlusions. The experiments demonstrate that the impact of lens distortion on the positioning accuracy can be neglected. The coordinates obtained using the positioning algorithm proposed in this paper have an average absolute error of less than 2.57 m. Moreover, the algorithm achieves a processing speed of 109 milliseconds per frame, meeting both the requirements for positioning accuracy and real-time processing, thereby validating the practicality of the proposed method.

6. Conclusions

To address the issues of incomplete contours and weakened edge features of spherical buoys on the water surface due to water obstruction and strong illumination, this paper proposes a monocular vision-based localization method using elliptical fitting. This approach fits an ellipse to the target’s edges, leveraging prior knowledge of the spherical buoy’s radius to establish a monocular vision-based ranging model. It integrates the USV’s position from satellite observation in the scene to estimate the coordinates of each spherical buoy in the geodetic coordinate system. Additionally, to address the problem of pseudo-edge points introduced by water reflections affecting the performance of elliptical fitting, this paper presents a method based on grayscale feature analysis and RANSAC multiple outlier rejection for elliptical fitting, effectively filtering out pseudo-edge points to obtain accurate elliptical parameters. Simulation and onboard experiment results demonstrate that this method can accurately obtain the coordinates of spherical buoys in the geodetic coordinate system under disturbances such as water reflections, varying illumination, and water obstruction, with precision comparable to general satellite positioning. This verifies the effectiveness and practicality of the method, meeting the requirements for real-time applications. One limitation of this paper is the difficulty in effectively detecting the edges of spherical buoys under extremely low-light conditions. Therefore, designing algorithms for detecting spherical buoy edges in extremely low-light conditions will be a key focus of our future research.