Miniature Multi-Target Tracking in Sonar Images Using Dual Trajectory Storage Method

Abstract

To address the conflict between trajectory fragmentation and the trade-off between association efficiency and data integrity in underwater micro-scale multi-target sonar motion detection and tracking in video sequences, a multi-target motion detection and tracking algorithm based on a dual trajectory storage mechanism and adaptive trajectory association is proposed. The method first obtains target centroids through Gaussian mixture model foreground extraction, morphological post-processing, and connected region analysis. By employing a dual-storage structure consisting of real-time trajectories and complete trajectories, it dynamically adjusts association thresholds based on frame sampling rates to achieve adaptive distance calculation for trajectory tracking. Experimental results demonstrate that the proposed method achieves a completeness rate of 100% in recording valid trajectory point lengths. The adaptive threshold mechanism improves association accuracy to 96.07% while reducing trajectory fragmentation rate to 0.9%. The average association time is 0.28 ms per frame, enabling efficient real-time association while ensuring the integrity of motion trajectory tracking. This research contributes to enhancing real-time detection and tracking capabilities for micro-scale underwater targets and provides support for applications such as underwater security surveillance, marine resource exploration, and intelligent autonomous underwater vehicle navigation.

1. Introduction

The detection and tracking of multi-target motion trajectories in underwater sonar imagery hold significant value for fields such as marine ecological monitoring, fisheries resource assessment, and autonomous underwater vehicle navigation [1,2]. Translating this potential into practical applications relies on a shared prerequisite: the accurate detection and stable tracking of moving targets. Traditional moving target detection methods, such as background subtraction, frame differencing, and optical flow, have been widely applied in terrestrial and aerial vision tasks [3,4,5]. However, their effectiveness relies on relatively ideal imaging conditions and sufficient data support. In contrast, the acquisition of underwater sonar images is often costly and inefficient, resulting in datasets that are typically limited in both quantity and quality [6,7,8]. Moreover, due to interference from water column reverberation, environmental noise, and inherent device characteristics, sonar images commonly exhibit low resolution, pronounced speckle noise, and blurred target contours [9]. These factors contribute to the degradation and distortion of target motion features [10], posing challenges for underwater multi-target trajectory detection and tracking. These challenges include frequent trajectory fragmentation, unstable target association accuracy, and limited model generalization capability under data-scarce conditions.

To address these challenges, researchers have made various improvements to traditional methods [11,12] to enhance their robustness in underwater settings. For instance, Wang et al. [13] optimized optical flow computation and edge fusion strategies to overcome the computational inefficiency and real-time limitations of traditional optical flow methods, significantly improving their speed and denoising capability. However, this approach depends heavily on the quality of edge detection, leading to suboptimal performance for partially or fully occluded moving targets. Chen et al. [14] proposed an improved ViBe algorithm that effectively suppresses ghosting artifacts caused by moving objects in the initial frame by integrating a three-frame differencing method and constructs adaptive thresholds to improve adaptability to dynamic backgrounds. Nevertheless, this method still falls short of achieving fully adaptive parameter adjustment. This limitation, along with the work of Zheng [15], underscores the fundamental constraint of traditional background subtraction methods: their reliance on manual parameter tuning and lack of inherent adaptability.

To surpass the accuracy and generalization limitations of traditional methods, researchers have turned to data-driven paradigms, particularly deep learning, aiming for superior detection performance [16,17,18,19]. However, as Zhang [20] and Liu [21] notes in their study, although deep learning methods can achieve higher detection accuracy and robustness in complex environments through end-to-end training, their performance is highly dependent on training with large-scale, annotated datasets. This creates a fundamental contradiction with the scarcity of annotated underwater sonar datasets.

A comprehensive analysis reveals that while existing research has made progress in specific aspects, limitations remain in addressing underwater multi-target detection and tracking. On one hand, most methods focus on mitigating individual challenges such as noise or occlusion, with insufficient exploration of systematically maintaining trajectory continuity in long-term, multi-target scenarios. On the other hand, under data-scarce conditions, constructing efficient and highly adaptive lightweight frameworks remains a challenging area requiring further investigation. Furthermore, while Cao et al. [22] proposed an observation-centric trajectory correction method that uses the ORU module to correct accumulated Kalman filter errors upon trajectory reactivation, Maggiolino et al. [23] introduced EMA-based dynamic appearance feature fusion to upgrade the pure motion model to an appearance-motion joint model, and Du et al. [24] addressed missing associations and missing detections through AFLink and GSI modules, respectively, for offline post-processing repair. The aforementioned SORT-series methods all adopt a single-layer storage structure. Consequently, after trajectory loss, they primarily rely on prediction, post-correction, or offline remediation, making it difficult to retain complete historical information during the loss period.

To address the aforementioned multiple challenges, this paper proposes a solution that balances trajectory integrity and computational efficiency by optimizing the trajectory storage structure and adaptively adjusting the association threshold. Specifically, a micro-scale multi-target motion detection and tracking algorithm based on a dual trajectory storage mechanism and adaptive trajectory association is introduced. The method begins by employing an adaptively updated Gaussian Mixture Model (GMM) for dynamic foreground extraction, followed by morphological operations and connected component analysis to obtain target centroid positions. Building on this, a dual-storage structure combining real-time trajectories and complete trajectories is introduced. Real-time trajectories record the motion state of targets in the current frame, while complete trajectories serve as backup storage, preserving the historical motion paths of targets. When a target is temporarily lost due to occlusion or missed detection, the algorithm retrieves the missing trajectory segments from the complete trajectories, thereby preventing trajectory fragmentation and ensuring long-term tracking continuity. Furthermore, the association threshold is dynamically adjusted based on the frame sampling rate, which enhances association accuracy while achieving lightweight computation. This enables adaptive trajectory association and complete trajectory output, ultimately generating visualized motion trajectories and statistical information. By integrating local background modeling with an efficient data association mechanism, the proposed method aims to improve the stability and coherence of micro-scale multi-target tracking under limited data conditions. It provides a lightweight and practical technical approach for underwater motion analysis.

2. Data Acquisition and Methods

The sonar image data were collected at an experimental station located in the Duhekou Reservoir, Zhejiang Province, where the water level is consistently maintained between 27.0 m and 28.0 m. An M750d dual-frequency multi-beam imaging sonar (Blueprint Design Engineering Limited trading as Blueprint Subsea, Kendal, UK) was employed for data acquisition. This system integrates transmitting and receiving transducer arrays with a signal processing module, functioning as a complete underwater acoustic imaging device. The experiment primarily measured the relative radial distance and azimuth angle between the sonar transducer and underwater dynamic targets. The detection range was set to 0.5–50 m (with an imaging range of 5 m during the experiment). The horizontal scanning angle was 130°, the vertical scanning angle was 20°, the range resolution reached 10 mm, and the operating frequencies were 750 kHz and 1.2 MHz [25].

During the experiment, the target was rotated 360° at a constant speed using a turntable. The sonar recorded echo data in real time at a refresh rate of 15 Hz, while simultaneously capturing echo image data, as shown in Figure 1. The video sequence includes four targets (all “micro-scale” moving targets, with physical dimensions less than 0.5 m). An artificially suspended target (an aluminum three-cylinder structure, each cylinder approximately 10 cm in diameter and 40 cm in height) positioned beneath the turntable remains visible throughout the entire rotational motion. In addition, three moving targets (fish naturally active in the water, each approximately 10 cm in body length) appear at different time intervals. The first target appears from the 17th to the 41st second, lasting 21 s; the second appears from the 36th to the 51st second; and the third appears from the 94th to the 109th second. Each target remains continuously detectable during its corresponding period. A frame containing three simultaneous targets is shown in Figure 2, and a frame containing two simultaneous targets is shown in Figure 3.

2.1. Background Modeling and Foreground Extraction

A Gaussian Mixture Model (GMM) [26,27,28] was employed for background modeling to achieve foreground extraction by establishing a statistical distribution model of the background. Based on a pixel-wise probability density function model, $k$ Gaussian distributions are constructed for each pixel in the image to model multi-modal data, thereby adapting to complex background variations. For a current pixel $X_t$ at time $t$, its probability density function is defined as:

$$P(X_t)=\sum _{i=1}^k{\omega }_{i,t}\eta (X_t,{\mu }_{i,t},{\Sigma }_{i,t})$$

(1)

In Equation (1), ${\omega }_{i,t}$ represents the weight of the $i$-th Gaussian component at time $t$, satisfying ${\sum }_{i=1}^k{\omega }_{i,t}=1$; ${\mu }_{i,t}$ is the mean vector; ${\Sigma }_{i,t}={\sigma }_{i,t}^{2}I$ is the covariance matrix, where ${\sigma }_{i,t}$ is the standard deviation and I is the identity matrix. The probability density function for a single Gaussian component is:

$$\eta (X_t,{\mu }_{i,t},{\Sigma }_{i,t})={\displaystyle \frac{1}{(2\pi {)}^{\frac{n}{2}}|{\Sigma }_{i,t}{|}^{\frac{1}{2}}}}\exp \left(-{\displaystyle \frac{1}{2}}(X_t-{\mu }_{i,t}{)}^T{\Sigma }_{i,t}^{-1}(X_t-{\mu }_{i,t})\right)$$

(2)

Here, $n$ denotes the data dimensionality, which determines the mathematical structure of function $\eta$. $X_t$ and ${\mu }_{i,t}$ are $n$-dimensional vectors, and ${\Sigma }_{i,t}$ is an $n\times n$ matrix. In this study, the data dimensionality $n$ = 3. Following initialization, the model parameters are updated to determine the Gaussian components suitable for each pixel.

The background model selection utilizes the stability index ${\displaystyle \frac{{\omega }_{i,t}}{{\sigma }_{i,t}}}$. The set of background components is defined as:

$$B_{set}=\left\{i\mid {\displaystyle \frac{{\omega }_{i,t}}{{\sigma }_{i,t}}}>T\right\}$$

(3)

where $T$ is the background selection threshold. For the current pixel $X_t$, if it matches any component in $B_{set}$, it is classified as a background pixel; otherwise, it is classified as a foreground pixel. This classification produces a foreground binary image $f(x,y)$, where $f(x,y)=1$ denotes a foreground pixel corresponding to a moving target, and $f(x,y)=0$ denotes a background pixel representing the water body. Once the parameters remain relatively stable, the classification of foreground and background pixels is completed. The statistical distribution for each pixel is estimated through online updates.

2.2. Target Detection and Segmentation

2.2.1. Morphological Preprocessing

A three-step morphological procedure was applied to the binary image obtained from GMM-based foreground extraction. First, an opening operation was performed to remove noise. This was followed by a closing operation to connect broken points and fill small gaps. Finally, a hole-filling algorithm was used to address completely enclosed cavities within the detected targets.

Morphological filtering of the foreground binary map $f(x,y)$ involved both opening and closing operations. The opening operation is defined as:

$$O\left(f\right)=\left(f\right(x,y)\ominus S{E}_1)\oplus S{E}_1$$

(4)

In Equation (4), $S{E}_1$ is a $3\times 3$ rectangular structuring element, $\ominus$ denotes erosion, and $\oplus$ denotes dilation. The opening operation, which performs erosion followed by dilation, is used to eliminate small protrusions and noise from the foreground binary map.

Subsequently, a closing operation was applied to the result of Equation (4), defined as:

$$C\left(f\right)=\left(O\right(f)\oplus S{E}_2)\ominus S{E}_2$$

(5)

In Equation (5), $S{E}_2$ is a $10\times 10$ rectangular structuring element. The closing operation, performing dilation followed by erosion, serves to fill small indentations and smooth the surfaces within the foreground binary map. Let A denote the binary image after morphological filtering, i.e., $A=C\left(f\right)$.

To address the issue of cavities on the surfaces of targets arising from the image binarization process, a morphological hole-filling algorithm was employed. Given A as the morphologically filtered binary image and $A^c$ as its complement, with B as a $3\times 3$ rectangular structuring element, the iterative hole-filling process is:

$$X_k=(X_{k-1}\oplus B)\cap A^c\hspace{1em}k=\mathrm{1,2},3,\dots$$

(6)

In Equation (6), $X_0$ is the initial seed image, typically created by setting at least one pixel within each cavity to 1 (with all others set to 0). $X_k$ represents the binary image after the $k$-th iteration, where pixels with a value of 1 indicate the filled region. This iterative process expands the filling region through dilation and intersects it with the image complement $A^c$, ensuring the filled area does not exceed the cavity boundaries. Iteration continues until convergence, i.e., when $X_k=X_{k-1}$. At this point, all pixels with a value of 1 are extracted from $X_k$; the region formed by their positions constitutes the filled cavity area. This filled cavity region is then merged with the morphologically filtered foreground binary image to produce the final foreground binary map, $A_{final}(x,y)$, which serves as the foundation for subsequent connected component analysis. The original grayscale frame, as shown in Figure 4a, is input into the GMM to obtain the foreground binary image presented in Figure 4b. The final morphological processing result is illustrated in Figure 4c.

2.2.2. Connected Component Analysis

Connected component analysis was performed on the morphologically processed foreground binary image $A_{final}(x,y)$ to identify and extract distinct target regions. A connected component labeling algorithm (Blob Analysis) [29] was employed to segment the binary image, enabling rapid identification, localization, and feature extraction of independent connected components (blobs).

For each foreground pixel in $A_{final}(x,y)$, connectivity was determined using either a 4-connectivity or 8-connectivity criterion. The connected component labeling algorithm grouped all connected pixels, with each connected group forming an independent connected region. After labeling, N connected regions were obtained, denoted as $R=\{R_1,R_2,\dots ,R_N\}$, where N is the total number of connected regions and $g$ is an index variable with $g\in \mathrm{1,2},...,N$. Each element $R_g$ in the set represents the $g$-th connected region (i.e., the $g$-th blob), defined as the set of coordinates for all pixels within that connected pixel group.

For each connected region $R_g$, key geometric features were extracted, primarily area and centroid. The area $A_g$ of the $g$-th connected region is defined as the total number of pixels within that region. The centroid coordinates $(x_g,y_g)$ of the $g$-th connected region, representing its geometric center, are defined as:

$$x_g={\displaystyle \frac{1}{A_g}}\sum _{(x,y)\in R_g}x,\hspace{1em}{y}_g={\displaystyle \frac{1}{A_g}}\sum _{(x,y)\in R_g}y$$

(7)

The results of blob detection are shown in Figure 5a, which displays the individual blob target regions identified by the connected component labeling algorithm along with their corresponding area values.

An area thresholding strategy was applied to filter the detected connected regions, aiming to select genuine targets while eliminating noise and false detections. Only connected regions satisfying the area threshold condition were retained as valid targets, as illustrated in Figure 5b. This figure presents area statistics for five detected blobs, with the red dashed line indicating the minimum area threshold (160 pixels). The areas of all five blobs exceed this threshold, demonstrating that the area thresholding strategy effectively filters out noise while preserving real targets. The set of valid connected regions is defined as:

$$R_{valid}=\left\{R_g|A_g\ge A_{threshold},R_g\in R\right\}$$

(8)

Here, $A_{threshold}$ is the area threshold, set to $A_{threshold}=160$ in this study. After filtering, the valid connected region set $R_{valid}$ is obtained. The corresponding valid centroid set is defined as $C=\{C_j=(x_j,y_j)|R_j\in R_{valid}\}$, where $j$ is the index for valid connected regions. This valid centroid set C serves as the input for the subsequent dual trajectory storage and association algorithm.

2.2.3. Dual Trajectory Storage and Association Algorithm

To address the trade-off between real-time performance and data integrity in multi-target tracking, a dual trajectory storage mechanism coupled with an adaptive association algorithm is proposed. This method separates the storage of real-time trajectories from complete historical trajectories and incorporates the frame sampling rate into the association threshold calculation to enhance the algorithm’s adaptability under varying frame rates.

The dual trajectory storage structure consists of a Real-time Trajectory $T_m^{\left(t\right)}$ and a Complete Trajectory $C_m^{\left(t\right)}$, where $m$ is the trajectory index and $t$ is the temporal frame index. The algorithm takes the valid centroid set $C$ obtained from connected component analysis as its input and employs a synchronous update strategy. When a detected centroid $(x_j,y_j)\in C$ is successfully associated with trajectory $m$, this centroid is simultaneously appended to both $T_m$ and $C_m$. The real-time trajectory retains only the last valid centroid for fast Euclidean distance computation with newly detected centroids, while the complete trajectory preserves all centroids to ensure data integrity. Upon the detection of a new associated centroid, both trajectories are updated synchronously:

$$T_m^{\left(t+1\right)}=T_m^{\left(t\right)}\cup \left\{p_{new}\right\}$$

(9)

$$C_m^{(t+1)}=C_m^{\left(t\right)}\cup \left\{p_{new}\right\}$$

(10)

Here, $T_m^{\left(t+1\right)}$ and $C_m^{(t+1)}$ are the real-time and complete trajectories for target $m$ at time $(t+1)$, respectively, and $p_{new}=(x_j,y_j)\in C$ is the newly detected centroid coordinate. If no associated centroid is detected in the current frame, $p_{new}$ is a missing value (defined as ‘NaN’), and both trajectories synchronously add a ‘NaN’ placeholder to maintain temporal alignment. The advantage of this mechanism is twofold: trajectory association only requires processing the last valid centroid from the real-time trajectory, while the complete trajectory retains all centroid information, ensuring data integrity for subsequent analysis.

Association decisions are based on Euclidean distance computation [30]. The Euclidean distance between the last valid centroid $(x_m,y_m)$ of trajectory $m$ and a newly detected centroid $C_j=(x_j,y_j)\in C$ is given by:

$$d_{mj}=\sqrt{(x_m-x_j{)}^2+(y_m-y_j{)}^2}$$

(11)

To accommodate varying frame sampling rates, an adaptive association threshold is employed. If the target’s maximum velocity is $v_{\mathrm{m}\mathrm{a}\mathrm{x}}$ (pixels/frame) and the sampling interval is $f_s$ (i.e., processing one frame every $f_s$ frames), then the maximum possible displacement over $f_s$ frames is ${\Delta }_{\mathrm{m}\mathrm{a}\mathrm{x}}^{\left(f_s\right)}=v_{\mathrm{m}\mathrm{a}\mathrm{x}}\times f_s$. The adaptive threshold is defined as:

$$D_{adj}=D_{base}\times f_s$$

(12)

where $D_{adj}$ is the adaptive association distance threshold and $D_{base}$ is the base association distance threshold, set to $D_{base}=17$ pixels in this study. Regarding the determination basis of $D_{base}$, this threshold is derived from target kinematic constraints and sonar imaging parameters, with the fundamental relationship expressed as:

$$D_{base}\approx \mathsf{\sigma }·{\displaystyle \frac{v_{max}}{f·R}}$$

(13)

where $v_{max}$ is the maximum motion velocity, $f$ is the sonar frame rate, $R$ is the image spatial resolution, and $\mathsf{\sigma }$ is a relaxation coefficient compensating for detection errors. In the 750 kHz mode, considering the typical motion speed of micro-scale targets and the device sampling interval, the calculated theoretical displacement range falls between 10 and 20 pixels. The value $D_{base}=17$ pixels is precisely the optimal observation value under this physical constraint. When the sonar frequency changes, leading to variations in resolution $R$, $D_{base}$ scales linearly to maintain a constant physical search radius, thereby achieving adaptive adjustment for different sonar resolutions.

This adaptive formula ensures that $D_{adj}$ scales appropriately with the maximum possible displacement ${\Delta }_{\mathrm{m}\mathrm{a}\mathrm{x}}^{\left(f_s\right)}$ under different sampling rates. When $d_{mj}\le D_{adj}$, the newly detected centroid $C_j$ is successfully associated with trajectory $m$, and both trajectories are updated synchronously according to Equations (9) and (10). If the distance between $C_j$ and all existing trajectories exceeds $D_{adj}$, a new trajectory is initialized as $T_{N_{traj}+1}^{(t+1)}=\{p_{new}\}$ and $C_{N_{traj}+1}^{(t+1)}=\{p_{new}\}$, where $p_{new}$ is the coordinate of $C_j$, and the total trajectory count $N_{traj}$ is incremented by one.

After trajectory association, a complete trajectory set $C_{traj}=\{C_1^{\left(t\right)},C_2^{\left(t\right)},\dots ,C_{N_{traj}}^{\left(t\right)}\}$ is obtained, where each $C_m^{\left(t\right)}$ is iteratively updated. $N_{traj}$ is the total number of trajectories, which serves as the input for the subsequent trajectory filtering algorithm.

2.2.4. Trajectory Filtering

Factors such as background noise, occlusion, and detection failures often generate numerous short trajectory segments, which typically correspond to noise points or false detections and interfere with subsequent motion analysis. To extract valid motion trajectories, a post-processing algorithm based on a length threshold is applied to filter the complete trajectory set.

The algorithm takes the complete trajectory set $C_{traj}=\{C_1^{\left(t\right)},C_2^{\left(t\right)},\dots ,C_{N_{traj}}^{\left(t\right)}\}$ output by the dual trajectory storage and association algorithm as its input. For each complete trajectory $C_m^{\left(t\right)}$, its valid point set $P_m=\{(x,y)\in C_m^{(t)}|(x,y)\ne (NaN,NaN)\}$ is extracted, and its valid length $L_m=|P_m|$ is computed. Only trajectories with a valid length $L_m\ge \tau$ are retained. The filtered set of valid complete trajectories is defined as:

$$C^{\prime }=\{C_m^{\left(t\right)}|C_m^{\left(t\right)}\in C_{traj},L_m\ge \tau \}$$

(14)

where $\tau$ is the minimum length threshold and $C^{\prime }$ is the filtered set of valid complete trajectories. In this study, the video frame sampling rate is 1/3, and the minimum length threshold $\tau$ is determined based on specific conditions. This algorithm effectively filters out short trajectory segments caused by background noise and false detections while retaining motion trajectories with sufficient statistical significance.

3. Results

After applying foreground extraction, morphological processing, connected component analysis, dual trajectory storage, and adaptive association, multi-target trajectory data with reduced noise, significantly fewer fractures, and high integrity are obtained. These results are visualized below. Figure 6a,b present the results obtained from two different video segments. The artificially suspended target in the video consists of multiple structural units. In Figure 6a, Artificial Target I is composed of three unit structures, resulting in three displayed trajectories in the trajectory plot. Additionally, three longer motion paths corresponding to naturally swimming fish appear during video monitoring. In Figure 6b, Artificial Target I consists of two unit structures; thus, two trajectories are shown. Furthermore, the motion trajectories of two fish are accurately tracked. Each trajectory is distinguished by a different color, with a green circle marking the start point and a red square marking the end point to clearly indicate the motion’s initiation and termination. Overall, the movement path morphology, directional changes, displacement range, and distribution patterns (whether clustered or dispersed) of each target can be directly observed from the visualization.

The visual results indicate significant differences in trajectory length and motion patterns among different targets, as detailed in

Table 1. Statistics of motion trajectory in video 1.

Object ID		Trajectory Points	Start Point X	Start Point Y	End Point X	End Point Y
Object I	Subobject 1	244	505.15	272.93	554.43	212.77
	Subobject 2	71	511.47	219.85	567.89	254.48
	Subobject 3	285	483.93	269.93	519.00	195.96
Object II		235	130.48	215.18	387.72	77.42
Object III		113	402.15	364.97	620.26	395.45
Object IV		130	761.45	138.90	779.19	334.35

and

Table 2. Statistics of motion trajectory in video 2.

Object ID		Trajectory Points	Start Point X	Start Point Y	End Point X	End Point Y
Object I	Subobject 1	272	627.5	302.7	654.9	293.6
	Subobject 2	231	628.0	330.2	647.8	342.7
Object II		103	684.4	66.4	750.2	102.2
Object III		103	601.6	62.3	642.2	107.8

. For Video 1, the average trajectory length is 179.16 trajectory points, with the longest trajectory spanning 285 points. For Video 2, the average trajectory length is 191 points, with the longest trajectory comprising 272 points. In Figure 6a, the dense cluster of trajectories in the region with x-coordinates 450–600 pixels and y-coordinates 150–300 pixels corresponds to the activity area of the fixed rotating Target I. Since this target is formed by three connected similar subunits, three distinct target trajectories are obtained. The dispersed independent trajectories in areas II, III, and IV represent the motion paths of individual targets, such as fish trajectories. Some trajectories exhibit long continuous paths, as seen in the 272-point trajectory within region I of Figure 6b, indicating that these targets remained consistently within the sonar field of view and were effectively tracked. This target is constructed from two connected similar subunits, hence two trajectories are generated. These statistical results demonstrate that, under a frame sampling rate of 1/3, the tracking performance achieves good continuity and stability. The extracted trajectories can serve as a foundational input for subsequent recognition tasks, including target classification and behavior analysis.

4. Discussion

To objectively evaluate the effectiveness of the proposed method, the following metrics were adopted: trajectory completeness, trajectory fragmentation rate, association accuracy, and association time. Trajectory completeness measures the integrity of data recording and is defined as:

$$Q_{completeness}={\displaystyle \frac{1}{N_{traj}}}\sum _{m=1}^{N_{traj}}{\displaystyle \frac{L_m}{T_m}}$$

(15)

where $N_{traj}$ is the total number of trajectories in the complete trajectory set $C_{traj}=\{C_1^{\left(t\right)},C_2^{\left(t\right)},\dots$, $C_{N_{traj}}^{\left(t\right)}\}$; $L_m$ is the number of valid frames for the $m$-th complete trajectory, equal to the number of elements in its valid point set $P_m$, i.e., $L_m=\left|P_m\right|$; $T_m$ is the total number of frames (including NaN frames) for the $m$-th complete trajectory $C_m^{\left(t\right)}$.

Taking Video 2 as an example, the trajectory completeness was calculated. As shown in Figure 7a, all detected valid trajectory points were completely recorded, achieving a trajectory completeness of 100%. This indicates that the dual trajectory storage mechanism can fully record and preserve all detected target point information, ensuring that historical data are not physically lost due to filtering or elimination by real-time association logic.

The trajectory fragmentation rate is calculated as:

$$R_{break}={\displaystyle \frac{{\sum }_{m=1}^{N_{traj}}{G}_m}{{\sum }_{m=1}^{N_{traj}}{T}_m}}$$

(16)

where $G_m$ is the number of transitions from a NaN state to a valid state for the $m$-th trajectory, i.e., the count of trajectory breaks. $T_m$ and $N_{traj}$ are defined as in Equation (14). Figure 7b shows that the overall trajectory fragmentation rate is as low as 0.9%, with the fragmentation rate for individual trajectories remaining at relatively low levels (2.24% to 5.15%). This indicates that the proposed method effectively controls trajectory fragmentation while maintaining trajectory completeness.

It should be noted that due to the lack of precise ground truth annotations in underwater sonar images, this study did not employ classical multi-object tracking metrics that require frame-by-frame ground truth comparison (such as MOTA, MOTP, IDF1, etc.). Therefore, we adopt the trajectory fragmentation rate, which does not require external ground truth, as an evaluation metric for tracking continuity. However, it must be acknowledged that this metric cannot directly reflect identity switch errors. To compensate for this limitation, we introduce an adaptive association threshold mechanism at the algorithm level (Equation (12) and related derivation), which reduces unreasonable trajectory associations from the bottom layer through physical constraints, thereby lowering the probability of identity switches.

Presents two core metrics for algorithm performance evaluation: association accuracy and association time. The association accuracy is calculated as:

$$A_{trajectory}={\displaystyle \frac{N_{successful}}{N_{total}}}$$

(17)

where $N_{successful}$ is the number of successfully associated centroids satisfying $d_{mj}\le D_{adj}$; $N_{total}$ is the total number of centroids detected across all frames, i.e., the total number of elements in the input set $C=\left\{\right(x_j,y_j\left)\right\}$. Under a frame sampling rate of 1/3, Figure 7a compares the association accuracy using a fixed threshold (17.0 pixels) versus an adaptive threshold (51.0 pixels). The fixed-threshold method uses a constant distance threshold of 17.0 pixels, but when frames are skipped, targets may move more than 17 pixels, resulting in an average accuracy of only 45.81%, with a significant decline in the later part of the video. The red solid line in the figure shows gaps around frames 0–180 because no centroids were detected in those frames (targets left the field of view or were occluded), making instantaneous accuracy incalculable; NaN values are automatically skipped during plotting. The adaptive-threshold method dynamically adjusts the threshold based on the frame sampling rate ($D_{adj}=D_{base} \times f_s=51$ pixels), ensuring the threshold covers the maximum possible displacement of targets during skipped frames, thereby maintaining stable association performance in frame-skipping scenarios. The adaptive-threshold method achieves an average association accuracy of 96.07%, representing a relative improvement of 50.26%.

The association time is calculated as:

$$T_{association}={\displaystyle \frac{1}{N_{frames}}}\sum _{t=1}^{N_{frames}}{t}_{assoc}^{\left(t\right)}$$

(18)

where $t_{assoc}^{\left(t\right)}$ is the association computation time for frame $t$, and $N_{frames}$ is the total number of processed frames. The experiments in this study were conducted on a laptop equipped with a 13th Gen Intel Core i5-1340P CPU (1.90 GHz), 16 GB of RAM, and an integrated Intel Iris Xe graphics card, running a 64-bit Windows 11 operating system. The algorithm was implemented using MATLAB R2022a (64-bit) and its Computer Vision Toolbox, with all processing executed on the CPU in the form of single-threaded scripts. The reported association time of 0.28 ms per frame represents the average time consumption of the association module over the complete test sequence and can be directly reproduced under the aforementioned experimental environment.

Figure 8b shows that the average association time is approximately 0.28 ms/frame, well below the real-time target of 1.0 ms/frame. The experimental results demonstrate that the adaptive threshold mechanism significantly improves association accuracy and stability by dynamically adjusting the association distance threshold, effectively addressing the performance degradation of fixed thresholds in frame-skipping scenarios. Furthermore, the algorithm meets real-time requirements, providing reliable trajectory data for subsequent motion analysis.

Under the condition that all preprocessing parameters, detection parameters, visualization parameters, and ROI settings were kept completely consistent, a comparative experiment was conducted between the proposed algorithm and the classical “Kalman filter + nearest neighbor” tracker. The comparison results are shown in Figure 9a,b.

The recording results of the proposed algorithm align well with the actual motion logic observed in the experiment. In contrast, the “Kalman filter + nearest neighbor” tracker exhibited unstable association performance, failing to fully correspond to the actual presence of these targets and generating trajectories that did not correspond to real targets. The observed data deviated significantly from the true physical process. This discrepancy is primarily attributed to two major issues. First, its linear motion model struggles to match the nonlinear maneuvering behavior of targets such as fish, leading to systematic deviations between predicted and actual positions. Second, the tracker is highly sensitive to observation interruptions. Once a target is temporarily lost due to clutter interference, relying solely on open-loop prediction rapidly accumulates errors. When the target reappears, the deviation between the predicted and actual positions exceeds the association threshold, forcing the algorithm to mistakenly identify it as a “new target” rather than recovering the original trajectory. This results in frequent trajectory fragmentation.

The proposed method introduces a dual trajectory storage mechanism, which collaboratively manages trajectories by categorizing them into “real-time trajectories” and “complete trajectories.” When a target is temporarily lost due to signal interference, the algorithm retains its historical state. Upon target reappearance, the interrupted trajectory segments are effectively reconnected through logical backtracking, significantly improving trajectory recording completeness.

The selection of the frame sampling rate involves a trade-off among association accuracy, trajectory fragmentation rate, and computational efficiency. To systematically optimize this parameter, the following objective function is used to determine the optimal frame sampling rate:

$$f_{skip}^{\left(opt\right)}=\mathit{arg}\underset{f_s}{\min }\left(1-A_{trajectory}(f_s)+\alpha \cdot R_{break}(f_s)+\lambda \cdot {\displaystyle \frac{T_{association}\left(f_s\right)}{T_{\mathrm{m}\mathrm{a}\mathrm{x}}}}\right)$$

(19)

where $f_s$ is a candidate frame-skip interval; $T_{\max }=1.0$ ms/frame is the real-time target threshold; and $\alpha$ and $\lambda$ are weight parameters for the trajectory fragmentation rate and association time, respectively. Here, $\alpha =0.5$ assigns equal weight to accuracy and fragmentation rate to ensure tracking quality, while $\lambda =0.3$ serves as an efficiency penalty term to guarantee that the algorithm meets real-time requirements.

By iterating over different $f_s$ values and computing the objective function, the $f_s$ that minimizes the objective function is selected as the optimal frame-skip interval. Based on this optimization framework, a frame sampling rate of 1/3 is chosen as the optimal interval, demonstrating good stability across test sequences with varying target numbers and motion complexities.

5. Conclusions

(1)

To address the challenges of trajectory fragmentation, the trade-off between association computational efficiency and data integrity in tracking the motion trajectories of micro-scale targets in underwater sonar images, a multi-target tracking method based on a dual trajectory storage mechanism is proposed. This method employs Gaussian Mixture Model (GMM) foreground extraction, morphological preprocessing, and connected component analysis to obtain target centroids. It utilizes an adaptive distance threshold for trajectory association, effectively handling issues such as trajectory breakage, computational efficiency in association, and data integrity for micro-scale targets.

(2)

The dual trajectory storage mechanism uses real-time trajectories for fast association computation while maintaining complete trajectories that preserve all historical information, thereby achieving separation between computation and storage. The association algorithm adopts an adaptive distance threshold strategy, where the threshold is dynamically adjusted based on the frame sampling rate to effectively accommodate the maximum displacement of targets under different frame-rate scenarios. Experimental results show that the adaptive threshold mechanism significantly improves association accuracy. The algorithm meets real-time requirements while ensuring data integrity. Overall, the algorithm demonstrates clear advantages in both accuracy enhancement and trajectory completeness.

(3)

The proposed method successfully tracks all targets in the experiments, including artificial targets (aluminum three-cylinder and two-cylinder structures) and naturally swimming fish. False trajectories are filtered out based on trajectory length, outputting complete and quantifiable trajectory data. This provides high-quality trajectory feature data for the motion analysis of underwater micro-scale multi-targets, supporting the extraction of behavioral features such as movement distance, start/end positions, and other relevant metrics.

The method exhibits strong engineering transferability and can be applied to practical tasks such as marine ecological observation, underwater security surveillance, and fishery resource monitoring, offering robust support for micro-scale multi-target motion analysis in underwater environments. When targets are in extremely close proximity, causing overlapping detection centroids, the algorithm assigns the merged point to the nearest trajectory, while the other trajectory temporarily holds a NaN value and enters a protected state. Short-term overlap maintains trajectory integrity, whereas long-term overlap leads to removal by the length filtering mechanism due to an insufficient number of valid points. However, beyond overlap scenarios, the distance-threshold-based association strategy remains susceptible to trajectory fragmentation or ID switching under prolonged occlusion, sudden acceleration, or frequent target crossing. Regarding the memory overhead of the dual trajectory storage mechanism, which is twice that of single storage, quantitative analysis shows that the dual mechanism increases overhead by only approximately 320 KB, demonstrating good hardware adaptability. This design trades a small amount of space for improved processing efficiency and trajectory continuity, satisfying both real-time and stability requirements. It should be noted that “double memory” refers to logical functional partitioning rather than data redundancy. Finally, concerning acoustic artifacts, the proposed method effectively suppresses interference through basic sonar equipment filtering combined with an area-threshold-based foreground extraction algorithm. Future improvements will incorporate acoustic propagation models to address more complex environments.