DeepSORT-OCR: Design and Application Research of a Maritime Ship Target Tracking Algorithm Incorporating Hull Number Features

Abstract

Maritime ship target tracking plays an important role in applications such as maritime patrol and maritime surveillance. However, complex sea conditions, similar target appearances, and long-distance imaging often lead to target identity confusion and unstable trajectories. To address these issues, in this paper, a ship multi-object tracking algorithm, DeepSORT-OCR, that integrates hull number semantic features is proposed. Based on the YOLO detection framework and the DeepSORT tracking architecture, a CBAM-ResNet network is introduced to enhance the representation of ship appearance features. An Inner-SIoU metric is adopted to improve the geometric matching of slender ship targets, while an LSTM-Adaptive Kalman Filter is employed to model the nonlinear motion patterns of ships and improve trajectory prediction stability. In addition, a Hull Number Feature Extraction module is designed in order to recognize ship hull numbers using OCR and match them with a hull number database. The extracted hull number semantic features are dynamically fused with visual appearance features to strengthen identity constraints during target association. The experimental results show that the proposed method achieves an MOTA of 66.53% on the MOT16 dataset, representing an improvement of 5.13% over DeepSORT. On the self-constructed maritime ship dataset, the method achieves an MOTA of 70.89% and an MOTP of 80.84%. Furthermore, on the hull-number subset, the MOTA further increases to 77.18%, an improvement of 7.31% compared with DeepSORT, while the number of ID switches is significantly reduced. In addition, experiments conducted on pure real data, pure synthetic data, and cross-domain evaluation settings demonstrate the stability and strong generalization capability of the proposed algorithm under different data distributions. The proposed method effectively improves the stability and identity consistency of ship multi-object tracking in complex maritime environments.

1. Introduction

Maritime ship target tracking has been extensively applied in offshore patrol, maritime surveillance, and navigation safety, serving as a critical technical foundation for maritime situational awareness, early risk warning, and maritime law enforcement. With the rapid growth of maritime traffic, the navigational environment is becoming increasingly complex, and the demand for persistent ship identification and stable long-term tracking is continuing to rise. Nevertheless, existing multi-target ship detection and tracking approaches still encounter multiple technical bottlenecks in real-world marine environments, making it difficult to achieve robust performance under long-duration and strong-interference conditions. In particular, maritime scenes exhibit highly dynamic background characteristics, where wave fluctuations, sun glint, fog occlusion, and variations in sea-surface textures significantly degrade detection reliability, resulting in frequent missed detections and false alarms. Meanwhile, ships of the same category often present limited inter-class differences in scale, color, and contour structure, and conventional feature extraction networks show insufficient discriminative capability in modeling key structural regions such as the bridge and deck, leading to unstable appearance representations. Consequently, cross-frame identity association becomes unreliable and is prone to target confusion and frequent ID switches. Moreover, ship motion is jointly affected by ocean currents, wind direction, channel constraints, and maneuvering behaviors, exhibiting pronounced nonlinearity and uncertainty. Under dynamic scenarios, including sharp turns, speed variations, acceleration/deceleration, and short-term stopping, the linear motion assumption in conventional Kalman filtering fails to capture the true motion evolution, causing cumulative state estimation errors and further resulting in trajectory drift, track fragmentation, and identity inconsistency, particularly in cases involving occlusion, target interaction, or short-term detection loss. In addition, most existing multi-object tracking methods rely on geometric overlap metrics such as IoU for detection-to-track assignment; however, this strategy does not adequately account for the elongated geometric characteristics of ship targets. When target pose or scale changes or inter-target overlap occurs, the overlap ratio becomes highly unstable, leading to unreliable matching decisions and continuous accumulation of localization errors, thereby degrading overall tracking accuracy. Although general-purpose tracking frameworks such as DeepSORT have achieved promising results in pedestrian and vehicle tracking, their designs are not specifically tailored to ship appearance structures, maritime motion patterns, or complex sea-surface backgrounds, making it difficult to simultaneously guarantee tracking accuracy, occlusion robustness, and identity consistency under challenging sea conditions, and limiting the reliability of identity confirmation. Therefore, it is necessary to incorporate more discriminative appearance modeling strategies, introduce motion prediction mechanisms better aligned with ship dynamics, and leverage domain-specific information to strengthen identity constraints, thereby improving the robustness and long-term stability of maritime multi-object ship tracking in complex marine environments and providing reliable technical support for offshore patrol and maritime surveillance tasks.

2. Related Work

Multiple Object Tracking (MOT) aims to simultaneously track multiple targets in video sequences while performing identification and association of targets in each frame. For the object detection stage, the YOLO algorithm proposed by Joseph Redmon et al. [1] is a classic end-to-end method. With its high inference speed and excellent detection performance, YOLO provides a reliable detection foundation for multi-object tracking tasks. In the tracking stage, the classical tracking algorithm SORT [2] establishes a basic framework for target tracking. Its core components include the Hungarian matching algorithm [3] and the Kalman filtering algorithm [4], which jointly utilize target detection and motion prediction to achieve target tracking. Other commonly used multi-object tracking algorithms include TraDes [5], ByteTrack [6], and QDTrack [7], as well as StrongSORT [8], which exhibits strong robustness in complex scenarios, and OC-SORT [9], which introduces an observation-centric smoothing strategy for nonlinear motion scenarios.

To overcome the limitations of SORT, Wojke et al. [10] proposed the DeepSORT algorithm, which represents an important advancement in the field of multi-object tracking. Based on SORT, DeepSORT introduces deep appearance features by employing a pretrained convolutional neural network [11] to extract deep visual representations of targets and measure similarity between targets using cosine distance. Meanwhile, DeepSORT adopts a cascade matching strategy that prioritizes high-confidence detections and incorporates motion prediction to assist association for occluded targets. This improvement reduces the frequency of ID switches and enhances robustness in complex environments, demonstrating significant potential for multi-object tracking tasks and inspiring various subsequent improvements. Zhou et al. [12] introduced the IBN-Net architecture and GIoU matching into DeepSORT to alleviate tracking discontinuity and target loss in complex scenarios. Duan et al. [13] integrated the Lightweight OpenPose algorithm into DeepSORT to improve multi-target recognition accuracy. Abass M. et al. [14] proposed the SE-PDA algorithm, which optimizes the tracking process through a time-separation strategy. Liu et al. [15] expanded the receptive field in the re-identification module to enhance feature capture capability, while Yin et al. [16] proposed a robust fine-grained learning framework for re-identification, demonstrating that mining subtle local features is crucial for overcoming appearance confusion.

In the field of maritime target detection and tracking, complex ocean environments pose numerous challenges. Sea waves, cloud occlusion, and illumination variations may weaken discriminative target features and lead to missed detections. Interactions and occlusions among vessels can easily cause tracking drift, while maneuvering behaviors during ship navigation may change target shape and scale, making tracking algorithms relying on single features more prone to target loss [17,18]. In addition, obstacles such as islands or reefs may block ships and interrupt sensor observations, resulting in trajectory fragmentation and ID switching. Meanwhile, practical applications impose strict requirements on real-time performance, making it difficult to simultaneously maintain both accuracy and efficiency in complex scenarios. To address these challenges, extensive studies have been conducted. Ma et al. [19] improved YOLO-based networks to enhance ship detection accuracy, thereby providing a stronger foundation for subsequent tracking tasks. Under degraded imaging conditions, improving detector robustness has also been shown to be critical for reducing tracking drift and missed detections. Chen et al. [20] proposed GMS-YOLO, a lightweight real-time detection architecture designed for foggy environments, providing a transferable solution for ship detection in low-visibility conditions such as sea fog. Hu et al. [21] further proposed a lightweight robust detection framework based on dual-teacher feature alignment, which improves representation consistency under adverse weather conditions by aligning features during training. In addition, Sriram et al. [22] introduced DPAM and DALSM modules into the YOLOv10 framework to improve real-time underwater object detection. Their feature enhancement strategy for handling scattering, reflection, and highlight interference also provides valuable insights for dealing with complex maritime illumination conditions. To address the large proportion of small targets and significant scale variations in long-distance maritime observation, Ding et al. [23] proposed a cross-stage focused small object detection network for UAV-assisted maritime applications, improving small target detectability through cross-stage feature aggregation and small-object enhancement strategies, thereby providing more reliable detection inputs for subsequent cross-frame association and stable tracking.

Li et al. [24] proposed a ship tracking method based on graph matching, in which consecutive frames are modeled as graph structures with nodes and edges representing target positions. Wu et al. [25] combined YOLOv3 detection with the DeepSORT tracking framework for ship tracking in remote sensing images. Zhou et al. [26] introduced a multi-granularity network to extract appearance information for feature matching and assigned priorities according to the frequency of target appearance, thereby achieving efficient ship tracking. Guo et al. [27] addressed non-common-view tracking problems caused by island occlusions by integrating LSTM [28] with Kalman filtering to analyze common-view trajectories and predict target positions under non-common-view conditions, thereby reducing trajectory jumps and improving localization accuracy. Yu et al. [29] adopted a correlation filtering strategy that uses the maximum response position of the response map as the updated target location for real-time tracking. Qiu et al. [30] addressed tracking drift and loss caused by shape changes and scale variations by introducing multi-feature weighted fusion using HOG, LBP, and CN features, along with a five-scale target box joint search mechanism and maximum response peak localization to improve anti-interference capability. Chen et al. [31] further improved DeepSORT by introducing an average appearance cosine distance strategy, which effectively reduces ID switching during tracking and improves overall performance.

Meanwhile, multimodal data fusion has demonstrated stronger real-time performance and robustness in maneuvering target scenarios. Luan et al. [32] proposed a fast recognition and detection algorithm based on multimodal data fusion, demonstrating that heterogeneous data fusion can overcome the perception limitations of single sensors under rapid maneuvering conditions. Furthermore, Gülsoylu et al. [33] proposed a deep fusion technique combining image data and AIS information. By performing spatiotemporal alignment between visual detections and AIS messages, the method supplements detection boxes with velocity and vessel-type information, thereby improving the reliability of target identification and association.

To facilitate a more systematic and intuitive comparison among existing methods, representative approaches proposed in recent years are reviewed and summarized in this paper. These methods are compared from the perspectives of detector selection, tracking framework design, principal strengths, and inherent limitations, as presented in

Table 1. Comparison of representative detection–tracking combined methods.

Method	Detector	Tracker	Advantages	Limitations
YOLO+SORT	YOLO	SORT	High speed, easy deployment	Track fragmentation under occlusion
YOLO+ByteTrack	YOLO	ByteTrack	More complete trajectories	False positives may cause interference
YOLO+StrongSORT	YOLO	StrongSORT	Strong robustness	Slow inference speed
YOLO+OC-SORT	YOLO	OC-SORT	Stable under maneuvering motion	Highly dependent on detection quality
TraDes	Built-in detection	Built-in tracking	Tight detection–association integration	Complex training process
FasterR-CNN+DeepSORT	Faster R-CNN	DeepSORT	High accuracy	Weak real-time performance
YOLO+DeepSORT	YOLO	DeepSORT	Stable identity association	High computational cost

.

As can be observed from the table, existing detection–tracking combined methods involve inherent trade-offs among tracking accuracy, robustness, and real-time performance. Lightweight approaches achieve high inference efficiency but tend to suffer from unstable trajectories in occlusion or complex scenarios, while methods incorporating stronger appearance modeling or association strategies improve identity consistency at the cost of increased model complexity and still primarily rely on single-modal appearance or motion features. Overall, current methods do not sufficiently exploit the structural characteristics of maritime ships, their nonlinear motion patterns under complex sea conditions, or the uniquely discriminative semantic information provided by ship hull numbers. To address these limitations, a DeepSORT-OCR maritime multi-object tracking framework that couples hull number features is proposed in this paper, achieving stable ship tracking and enhanced identity consistency in complex maritime environments through multi-dimensional feature fusion and identity-constrained association.

3. Methodology

3.1. Design of DeepSORT-OCR Architecture Incorporating Hull Number Features

Ship targets in maritime environments are easily affected by factors such as sea clutter, illumination variations, and long-distance imaging, which may introduce uncertainty in visual-feature-based target recognition. However, the hull number, as an important identity marker of ships, possesses strong semantic discriminative capability and serves as a key feature for distinguishing vessels of the same type. Based on this observation, hull number semantic information is introduced into the DeepSORT tracking framework in this study. By fusing hull number features with visual appearance features, semantic-enhanced tracking of ship targets is achieved. The overall architecture of the improved DeepSORT framework is shown in Figure 1.

During the new trajectory initialization stage, the system first employs a YOLO detector to detect ship targets on the sea surface and generate initial detection bounding boxes. Before the detection results enter the matching process, multi-dimensional target features are extracted, including the appearance features obtained by the CBAM-ResNet network, motion features predicted by the motion model, and hull number information obtained by the hull number semantic feature extraction module. The hull number semantic feature extraction module uses OCR technology to recognize characters in the hull number region within the detection box and then retrieves the recognition results from the hull number database to determine whether the detected hull number exists in the known vessel database.

During the target association stage, the system dynamically adjusts the weight of hull number features according to the reliability of the recognition results. When the recognized hull number matches an entry in the database and the recognition confidence is high, the weight of the hull number semantic feature in the fused representation is increased to strengthen identity discrimination. Conversely, when the hull number cannot be matched in the database or the recognition confidence is low, the weight of the hull number feature is reduced so that it only acts as auxiliary information in the matching process. After dynamic weighted fusion, the hull number semantic feature and the visual appearance feature jointly form a comprehensive target representation, which is then fed into the Matching Cascade module for target association.

For detection targets that remain unmatched after the cascade matching stage, the Inner-SIoU matching strategy is further employed to supplement spatial association, thereby improving matching stability in complex scenarios. If the detection target still fails to be matched, it is initialized as a new candidate trajectory and assigned a new track ID. As the tracking process continues, the system uses an Adaptive Kalman Filter to predict and update the target state, while managing track lifetimes according to detection confidence and motion consistency, thereby maintaining stable ship trajectories.

By introducing a dynamic fusion mechanism of hull number semantic features and visual appearance features, the proposed method can fully exploit the identity discrimination capability of hull numbers when reliable hull number information is available. Meanwhile, when hull number information is missing or unreliable, the system can still rely on visual and motion features for target association, thereby improving the accuracy and robustness of multi-ship tracking in complex maritime environments.

3.2. Improvement of CBAM-ResNet for Ship Appearance Feature Extraction

As a classic neural network architecture in the field of deep learning, ResNet’s core design idea lies in introducing a residual learning mechanism, enabling the network to focus on learning the residual mapping relationship between inputs and target outputs rather than directly fitting the original input-output mapping. This design realizes the direct transmission of input information through residual connections, effectively addressing issues such as gradient vanishing and degradation in the training of deep networks.

CBAM is a lightweight convolutional attention module that can automatically identify the regions in an image that require special attention through learning, thereby enhancing the network’s ability to extract key features. Its core advantage lies in the efficiency and universality of its structure design. By merely adding a few parameters to the traditional convolutional network, it can increase the model’s sensitivity to important features. CBAM consists of a channel attention module and a spatial attention module, and its structure is shown in Figure 2.

In Figure 2, the symbol $⨂$ denotes element-wise multiplication, and the arrows represent the flow of feature information.

It can be seen that CBAM achieves feature enhancement through the channel attention mechanism and the spatial attention mechanism. Assuming the input feature map is F ∈ R^H×W×C (where H and W represent the height and width of the feature map, and C represents the number of channels), CBAM successively derives the channel attention mapping and the spatial attention mapping. The entire attention process can be summarized as

$$F^{\prime }=M_c\left(F\right)\otimes F$$

(1)

$$F_{refined}=M_s\left(F^{\prime }\right)\otimes F^{\prime }$$

(2)

In the formula, the symbol $⨂$ represents element-wise multiplication. The logic of CBAM for handling channel attention and spatial attention is as follows: The channel attention mechanism first performs global max pooling and global average pooling on the input feature map F and then inputs the results into the shared multi-layer perceptron (Shared MLP) for feature transformation and fusion. After this, it generates the channel attention mapping $Mc$ through Sigmoid activation. The structure of the channel attention module is shown in Figure 3.

In Figure 3, $\oplus$ denotes element-wise addition, and represents the Sigmoid activation function.

The spatial attention mechanism is based on the weighted features $F^{\prime }$ of channels. It performs maximum pooling and average pooling along the channel dimension, concatenates them, and then passes them through a convolutional layer and a Sigmoid activation function to generate a spatial attention map $Ms$. The structure of its spatial attention module is shown in Figure 4.

In Figure 4, represents the Sigmoid activation function.

In summary, through the stepwise weighting method of CBAM, key features can be highlighted, redundant information suppressed, and feature representation optimized.

The appearance model adopted by the traditional DeepSORT has a relatively simple design in terms of the feature extraction network, and its architecture is a basic CNN network composed of several convolutional layers and residual components. This lightweight design can meet basic application needs in environments with low scene complexity but has obvious limitations when facing complex scenarios. Especially when the appearance features of two tracked targets are highly similar, relying only on the information captured by this simple feature extraction network as a constraint condition can no longer meet the requirements of accurate tracking. To solve this problem, a new feature extraction network, CBAM-ResNet, is proposed in this study to optimize the appearance model structure of DeepSORT. Its structure is shown in Figure 5.

This network is based on the ResNet50 architecture. Input images first undergo convolution, normalization, activation, and pooling to complete basic feature extraction, then enter a four-layer structure stacked with residual blocks. Residual connections are used to alleviate gradient vanishing and enable multi-scale feature learning. The CBAM-ResNet is integrated into the residual blocks: channel attention captures global channel information through global pooling and learns weights via shared convolutions; spatial attention generates weights through convolution after channel-wise pooling. The two work together to precisely weight features in both channel and spatial dimensions, highlighting key ship features and suppressing background interference.

3.3. Hull Number Feature Extraction Module

The hull number semantic feature extraction module is designed to extract hull number character information from detected ship regions and construct semantic feature representations that can participate in target association. In application scenarios such as port channel monitoring, ship entry and departure management, and nearshore patrol, ship targets are typically observed in a far-to-near process within the monitoring field of view. When the target is far away, the hull number region has low resolution in the image, making the character information difficult to recognize reliably. At this stage, target association mainly relies on ship appearance features and motion features. As the ship gradually approaches the camera, the hull number characters become clearer and can provide additional semantic constraints for target identity discrimination. Therefore, the hull number information is introduced in this work as a supplementary identity feature, participating in target association when it becomes recognizable, thereby enhancing identity consistency during tracking.

Due to the presence of sea clutter interference, sea surface reflections, illumination variations, and long-distance imaging in maritime environments, directly performing character recognition on the original detection region may lead to unstable recognition results. To improve the reliability of hull number recognition, a CBAM-ResNet-based feature enhancement mechanism is introduced into the hull number recognition process to improve the discriminability of hull number regions. The overall structure of the Hull Number Feature Extraction module is shown in Figure 6.

Specifically, for each ship bounding box detected by YOLO, the system first extracts the corresponding ship image region and inputs it into the CBAM-ResNet feature enhancement network. Through channel attention and spatial attention mechanisms, the network recalibrates the input features to strengthen hull number character structures while suppressing complex sea-surface background noise, making the hull number character regions more prominent in the feature space. The enhanced image is then fed into the OCR recognition module, which outputs the corresponding hull number text and its recognition confidence.

To further improve the reliability of the recognition results, the hull number recognized by OCR is retrieved and matched against a pre-constructed hull number database. The database stores known ship hull numbers and is used to assist in verifying the validity of recognition results. If the recognition result successfully matches an entry in the database, the hull number is considered to have high credibility. Otherwise, the result may correspond to recognition errors or ships not included in the database. Based on this process, the system evaluates the reliability of hull number information by jointly considering the recognition confidence and the database matching result.

Since hull number recognition results are textual information, while appearance features are visual features, a unified representation is required to jointly utilize the two modalities. Therefore, the recognized hull number character sequence is first encoded into a fixed-dimensional semantic feature vector F_ocr, enabling it to reside in the same feature space as the visual features. Meanwhile, the visual appearance features extracted by the CBAM-ResNet network are denoted as F_app. Based on these representations, the fused feature representation of the target is defined as

$$F_{fusion}=\alpha F_{app}+\left(1-\alpha \right) F_{ocr}$$

(3)

where $\alpha$ is the feature fusion weight that controls the contribution of visual appearance features and hull number semantic features in the fused representation.

To dynamically adjust the fusion ratio according to the reliability of hull number recognition, a weight adjustment strategy based on recognition confidence and database matching results is designed. Let the OCR recognition confidence be denoted as conf, and the confidence threshold be T_c. When the recognition result satisfies conf > T_c and the hull number is successfully matched in the database, the hull number information is considered highly reliable. In this case, the weight of the hull number semantic feature in the fusion representation is increased to strengthen its identity discrimination capability. Conversely, when the recognition confidence is low or the hull number fails to match the database, the weight of the hull number semantic feature is reduced so that the system relies more on visual appearance features for target association. The weight adjustment strategy is expressed as

$$\alpha =\left\{\begin{array}{cc}{\alpha }_h,\hfill & conf>T_c and matched\hfill \\ {\alpha }_l,\hfill & otherwise\hfill \end{array}\right.$$

(4)

where ${\alpha }_h$ and ${\alpha }_l$ represent the fusion weights under high-confidence and low-confidence conditions, respectively. In the experiments, the recognition confidence threshold is set as T_c = 0.8, and the weight parameters are set as ${\alpha }_h$ = 0.4 and ${\alpha }_l$ = 0.8.

After dynamic weighted fusion, the resulting fused feature vector is used as the input to the Matching Cascade module, where it is combined with motion prediction information and spatial matching strategies to form the comprehensive criterion for target association. This mechanism strengthens semantic identity constraints when hull number recognition is reliable while reducing its influence when recognition is unstable or unavailable, thereby improving the stability and robustness of multi-ship tracking in complex maritime environments.

3.4. Improvement of LSTM-Adaptive Kalman Filter Tracking Module for Ship Motion Features

Long Short-Term Memory (LSTM) is a Recurrent Neural Network (RNN) variant. It is used to address the gradient vanishing or exploding problem that traditional RNNs tend to encounter when processing long-sequence data, and it can effectively capture long-term dependencies in sequences. Its structure is shown in Figure 7.

LSTM breaks through the limitations of traditional RNNs in temporal information transmission through the collaborative operation of gating mechanisms and cell states. When processing long sequences, it can effectively capture and utilize long-term dependencies, providing more robust feature modeling capabilities for tasks such as time-series prediction and sequence classification, and has become one of the core tools for deep learning modeling in complex temporal scenarios.

To address the characteristics of nonlinear and time-varying motion in ship tracking scenarios, an LSTM-Adaptive Kalman Filter (LSTM-AKF) tracking model is constructed. The model utilizes a Long Short-Term Memory (LSTM) network to capture temporal dependencies and nonlinear variations in ship motion data, while combining an Adaptive Kalman Filter to perform optimal estimation of the dynamic system state, thereby achieving stable tracking of ship motion. Specifically, the LSTM first models the temporal features of historical ship motion data to extract potential motion trends and provide temporally informed auxiliary parameters for the Kalman filter. Subsequently, the Adaptive Kalman Filter integrates observation information obtained from detection results to perform prediction and update of the target state, enabling dynamic estimation of ship trajectories. By combining deep learning with classical filtering methods, the proposed model effectively improves the accuracy and stability of ship tracking under complex maritime conditions. The overall structure is illustrated in Figure 8.

The core of this method consists of an Adaptive Kalman Filter main loop and an LSTM optimization loop. In the AKF main loop, the target position and aspect ratio (x, y, aspect) provided by the target detector serve as the observation basis, and the optimal estimation of the target state is performed through two steps: prediction and update. The prediction step estimates the target state at time k based on the posterior state at time k − 1:

$${\widehat{x}}_{k|k-1}=F_k{\widehat{x}}_{k-1|k-1}$$

(5)

$$P_{k|k-1}=F_k{P}_{k-1|k-1}{F}_k^T+Q_k$$

(6)

where ${\widehat{x}}_{k|k-1}$ denotes the prior state estimate at time k, $F_k$ is the state transition matrix, $P_{k|k-1}$ is the prior estimation error covariance matrix, and $Q_k$ is the process noise covariance matrix that characterizes uncertainty in the motion model.

During the update stage, measurement information is used to correct the predicted state, and the update process is calculated as

$${K_k=P_{k|k-1}{H}_k^T(H_k{P}_{k|k-1}{H}_k^T+R_k)}^{-1}$$

(7)

$${\widehat{x}}_{k|k}={{\widehat{x}}_{k|k-1}+K}_k(z_k^{fused}-H_k{\widehat{x}}_{k|k-1})$$

(8)

where $K_k$ represents the Kalman gain, $H_k$ denotes the observation matrix, $R_k$ is the observation noise covariance matrix representing sensor measurement reliability, $z_k^{fused}$ is the fused measurement value, and ${\widehat{x}}_{k|k}$ is the posterior state estimate at time k.

In the LSTM optimization loop, temporal modeling is used to provide adaptive parameters for the AKF. The LSTM module takes the 8-dimensional motion features from the previous five frames as an input sequence. The features are processed through a two-layer LSTM network (8→32, 32→32) followed by fully connected layers (32→16→6) to generate a 6-dimensional optimization parameter vector, including four position correction values and two noise adjustment coefficients. In this way, the LSTM learns the temporal variation patterns in ship motion and dynamically adjusts key parameters in the filtering process. The adaptive adjustment of parameters is implemented as

$$Q_k=q_k{Q}_0$$

(9)

$$R_k=r_k{R}_0$$

(10)

where $Q_0$ and $R_0$ represent the baseline noise covariance matrices, and $q_k$ and $r_k$ are the noise scaling coefficients predicted by the LSTM network. To ensure that the noise covariance matrices satisfy the constraints of Kalman filtering, activation functions are applied to constrain $q_k$ and $r_k$ within the positive range, ensuring they remain positive. Meanwhile, both the process noise and observation noise are constructed as diagonal covariance matrices formed from squared standard deviations, ensuring that $Q_k$ and $R_k$ remain symmetric and positive semi-definite. In practical implementation, if the LSTM prediction exceeds a reasonable range, the system reverts to the baseline noise parameters to maintain the numerical stability of the filtering process.

To further improve the stability of observation data, weighted fusion of measurement values is performed in this study, expressed as

$$z_k^{fused}=0.3·z_k^{measured}+0.7·z_k^{LSTM}$$

(11)

This strategy fuses the original measurement values and LSTM predictions with a 3:7 weighting ratio, balancing the real-time responsiveness of detection data with the smoothness of predicted trajectories. While maintaining responsiveness to sudden motion changes, this approach effectively reduces the influence of detection noise on tracking results.

It should also be noted that the LSTM model in this work learns the general temporal motion patterns of ships rather than individual vessel characteristics. During actual tracking, the same LSTM model is shared among all targets, while each target independently feeds its historical trajectory sequence into the network for prediction. Therefore, the proposed model can simultaneously support multi-target ship tracking scenarios and can generalize to new vessels that were not present in the training set.

Through the close integration of the LSTM optimization loop and the Adaptive Kalman Filter main loop, the proposed method achieves joint dynamic optimization of measurement information and filtering parameters, thereby significantly improving the stability and accuracy of ship target tracking in complex maritime environments.

3.5. Improvement of Inner-SIoU Module for Multi-Ship Target Tracking

The DeepSORT algorithm constructs a cost matrix and combines the Hungarian algorithm with the Kalman filter to perform target matching between predicted positions and detected positions based on IoU. Building on this, an improvement is proposed in this study through introducing a cutting-edge loss function that integrates Inner-IoU and SIoU—namely, Inner-SIoU—to replace the traditional IoU matching strategy. This loss function simultaneously leverages SIoU’s advantage in geometric constraint modeling and Inner-IoU’s flexibility in bounding box scale adjustment. It can significantly enhance the perception ability of small targets and effectively alleviate the error accumulation problem during large-scale target matching, thereby achieving more stable and efficient target detection and association performance in complex and variable scenarios.

Inner-IoU is a method used to enhance bounding box regression. It accelerates the regression convergence process and improves accuracy by introducing auxiliary bounding boxes and scale factor ratio. The parameters are shown in Figure 9. Let the true box and the predicted box (Anchor Box) be b^gt and b, with their centroid coordinates being $(x_c^{gt},y_c^{gt})$ and $(x_c,y_c)$, and the width and height being $w^{gt}$, $h^{gt}$ and $w$, $h$. The inner layer true box is obtained by scaling the true box proportionally, and it is used to calculate the IoU between the true box and the predicted box, serving as an auxiliary indicator for bounding box optimization. The derivation formula of Inner-IoU is as follows.

$$\left\{\begin{array}{cc}{b}_l^{gt}=x_c^{gt}-{\displaystyle \frac{w^{gt}\times ratio}{2}}& \\ b_r^{gt}=x_c^{gt}+{\displaystyle \frac{w^{gt}\times \mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}}{2}}& \end{array}\right.\left\{\begin{array}{cc}{b}_t^{gt}=y_c^{gt}-{\displaystyle \frac{h^{gt}\times ratio}{2}}& \\ b_b^{gt}=y_c^{gt}+{\displaystyle \frac{h^{gt}\times \mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}}{2}}& \end{array}\right.$$

(12)

In Equation (10), $b_l^{gt}$, $b_r^{gt}$, $b_t^{gt}$ and $b_b^{gt}$, respectively, represent the positions of the left, right, top, and bottom boundaries of the real box; $x_c^{gt}$ and $y_c^{gt}$ represent the horizontal and vertical center coordinates of the real box; $w^{gt}$ represents the width of the real box; $h^{gt}$ represents the height of the real box; and ratio represents the proportion coefficient used for dynamically adjusting the scaling ratio between the real box and the predicted box.

$$\left\{\begin{array}{cc}{b}_l=x_c-{\displaystyle \frac{w\times ratio}{2}},& b_r=x_c+{\displaystyle \frac{w\times ratio}{2}}\\ b_t=y_c-{\displaystyle \frac{h\times ratio}{2}},& b_b=y_c+{\displaystyle \frac{h\times ratio}{2}}\end{array}\right.$$

(13)

where $b_l, b_r, b_t$, and $b_b$, respectively, represent the positions of the left, right, top, and bottom boundaries of the prediction box; $x_c$ and $y_c$, respectively, represent the horizontal and vertical center coordinates of the prediction box; $w$ represents the width of the prediction box; and $h$ represents the height of the prediction box.

$$inter=\left(\mathrm{m}\mathrm{i}\mathrm{n}(b_r^{gt},b_r)-\max (b_l^{gt},b_l)\right)\times \left(\mathrm{m}\mathrm{i}\mathrm{n}(b_b^{gt},b_b)-\max (b_t^{gt},b_t)\right)$$

(14)

$$union=((w^{gt}+h^{gt})\times (ratio{)}^2)+(w\times h)\times (ratio{)}^2-inter$$

(15)

$$Io{U}_{inner}={\displaystyle \frac{inter}{union}}$$

(16)

In Equations (14)–(16), inter represents the overlapping area of the inner regions of the target box and the predicted box, and union represents the part of the total area of the inner regions of the target box and the predicted box after subtracting the overlapping area of the inner regions. IoU_inner measures the degree of overlap between the inner regions of the target box and the predicted box.

SIoU is a loss function and metric that improves upon IoU in the field of multi-object tracking. By incorporating more geometric factors, it enables more accurate modeling of the relationships between target bounding boxes. SIoU consists of four components: Angle cost, Distance cost, Shape cost, and IoU cost.

The angle loss function is used to measure the angle difference between the predicted bounding box and the real bounding box. Figure 10 shows the angle loss calculation method, and the expression is as follows:

In Figure 10, the dotted box represents the minimum enclosing box used to compute the angle difference.

$$\Lambda =1-2{\mathrm{s}\mathrm{i}\mathrm{n}}^2(\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{s}\mathrm{i}\mathrm{n}({\displaystyle \frac{c_h}{\mathsf{\sigma }}})-{\displaystyle \frac{\pi }{4}})=\mathrm{c}\mathrm{o}\mathrm{s}(2(\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{s}\mathrm{i}\mathrm{n}({\displaystyle \frac{c_h}{\mathsf{\sigma }}})-{\displaystyle \frac{\pi }{4}}))$$

(17)

Among them,

$${\displaystyle \frac{c_h}{\mathsf{\sigma }}}=\mathrm{s}\mathrm{i}\mathrm{n}(\alpha )$$

(18)

$$\mathsf{\sigma }=\sqrt{{(b_{cx}^{gt}-b_{cx})}^2+{(b_{cy}^{gt}-b_{cy})}^2}$$

(19)

$$c_h=max(b_{cy}^{gt} ,b_{cy})-min(b_{cy}^{gt} ,b_{cy})$$

(20)

Referring to Figure 10, for a given prediction and true box, the angle between the horizontal axis and the line connecting the center of each box is considered as $\alpha$, and the angle with the vertical axis is considered as $\beta$. The distance between the center points of the true box and the prediction box is $\sigma$; the height difference between the center points of the true box and the prediction box is $c_h$; $b_{cx}^{gt}$, $b_{cy}^{gt}$ denote the center coordinate of the true box; and $b_{cx}$, $b_{cy}$ denote the center coordinate of the prediction box. When the angle difference between the prediction box and the true box is small, the angle loss is small; otherwise, the angle loss is large.

The distance loss function is used to measure the distance between the center points of the prediction box and the true box. Figure 11 shows the calculation method of the distance loss, and the expression is as follows:

Figure 11. Method for calculating distance loss.

Figure 11. Method for calculating distance loss.

$$△=\sum _{t=x,y}(1-e^{-\gamma {\rho }_t})=2-e^{-\gamma {\rho }_x}-e^{-\gamma {\rho }_y}$$

(21)

Among them,

$${\rho }_x={({\displaystyle \frac{b_{cx}^{gt}-b_{cx}}{c_w}})}^2 , {\rho }_y={({\displaystyle \frac{b_{cy}^{gt}-b_{cy}}{c_h}})}^2 , \gamma =2-\Lambda$$

(22)

where $c_w$ and $c_h$ represent the width and height of the smallest enclosing rectangle of the real box and the predicted box. Optimizing the distance loss can make the center point of the predicted box closer to that of the real box, thereby improving the target tracking accuracy.

The shape loss function is used to measure the difference between the shapes of the predicted box and the real box. The expression of the shape loss is as follows:

$$\mathsf{\Omega }=\sum _{t=w,h}{(1-e^{-w_t})}^{\theta }={(1-e^{-w_w})}^{\theta }+{(1-e^{-w_h})}^{\theta }$$

(23)

Among them,

$$w_w={\displaystyle \frac{|w-w^{gt}|}{max(w,w^{gt})}} , w_h={\displaystyle \frac{|h-h^{gt}|}{max(h,h^{gt})}}$$

(24)

where $w$, $h$, $w^{gt}$, and $h^{gt}$ represent the width and height of the predicted box and the true box, respectively, and $\theta$ controls the degree of attention paid to the shape loss. The shape difference can be measured by calculating the aspect ratio or the area ratio of the width and height of the predicted box and the true box.

The IoU loss is the traditional IoU loss, and its expression is as follows:

$$L_{IoU}=1-IoU$$

(25)

The SIoU loss is defined using distance loss $\Delta$, shape loss $\mathrm{\Omega }$, and IoU loss, and the angle loss is incorporated into the distance loss. The SIoU expression is as follows:

$$L_{SIoU}=1-IoU+{\displaystyle \frac{△+\mathsf{\Omega }}{2}}$$

(26)

In summary, the formula for the Inner-SIoU loss function is defined as follows:

$$L_{Inner-SIoU}=L_{SIoU}+IoU-{IoU}_{Inner}$$

(27)

By comprehensively considering the above factors, Inner-SIoU can more fully reflect the similarity between two bounding boxes. The DeepSORT-OCR algorithm employed in this paper introduces Inner-SIoU to replace the original IoU matching, aiming to overcome some limitations of the traditional IoU and its derivative metrics.

4. Experimental Results and Analysis

4.1. Dataset and Experimental Configuration

To verify the effectiveness of the proposed method in complex scenarios, experiments were conducted using both a public dataset and a self-constructed maritime ship dataset. The public dataset used in this study is the MOT16 dataset, while the self-constructed dataset is used for experiments involving maritime ship detection, recognition, and tracking.

In the experiments, the MOT16 dataset is mainly used to evaluate the general multi-object tracking performance of the proposed tracking algorithm in complex scenes. It should be noted that the objects in the MOT16 dataset are pedestrians. Therefore, the related experiments are not specifically designed for ship targets; instead, they are intended to verify the algorithm’s capability in target association and trajectory maintenance under conditions involving dense targets, frequent occlusions, and complex motion patterns. Since multi-object tracking algorithms share common mechanisms in terms of association strategies, trajectory management, and motion modeling, the MOT16 dataset can effectively reflect the tracking robustness of the algorithm in complex dynamic environments.

In addition to the public dataset, a self-constructed maritime ship dataset was built for this study. The dataset consists of two parts: virtual simulation data and real maritime surveillance data. The virtual data are generated using the Unreal Engine (UE), which is used to expand hull number samples and construct a hull number feature database. The real data are collected from actual maritime surveillance videos and are used to evaluate the algorithm’s performance in detection, matching, and continuous tracking under real-world conditions. In the overall dataset, the ratio of virtual data to real data is approximately 3:7.

The dataset contains approximately 6700 images and several video sequences, with a unified image resolution of 1920 × 1080. Among them, about 30% of the samples contain clearly recognizable hull numbers. Some example images from the dataset are shown in Figure 12.

In terms of dataset partitioning, the ship detection dataset is divided into training, validation, and test sets according to a ratio of 8:1:1. The training set is used for training the object detection model, the validation set is used for parameter tuning, and the test set is used for final performance evaluation. For the feature extraction network (ReID) training, the dataset is divided based on target IDs, ensuring that the ship IDs in the training set and the test set do not overlap, thereby guaranteeing the model’s generalization ability to unseen targets. Tracking experiments are mainly evaluated on video sequences, where the detection results serve as inputs to further verify the performance of target association and trajectory maintenance.

The experimental environment is based on the Windows 11 operating system, with hardware configured as an Intel i9 processor and an NVIDIA RTX 3090 GPU with 24 GB of memory. The algorithm is implemented using the PyTorch 2.3.1 deep learning framework.

During the training of the object detection network, the official YOLOv5 pretrained weights are used for initialization and then fine-tuned on the ship detection dataset. During training, a cosine annealing strategy is employed to dynamically adjust the learning rate. In addition, data augmentation techniques such as Mosaic augmentation, random horizontal flipping, and scale transformation are introduced to improve the model’s generalization ability in complex maritime environments. The detailed training parameters of the detection network are shown in

Table 2. Training parameters of the YOLOv5 detection network.

Parameter	Setting
Input image size	640 × 640
Batch Size	16
Training epochs	300 epochs
Optimizer	SGD
Initial learning rate	0.01
Momentum	0.937
Weight decay	0.0005

.

During the training of the feature extraction network, the input images are uniformly resized to 128 × 64 resolution, and data augmentation strategies such as random cropping and random horizontal flipping are adopted to improve the robustness of the model. Meanwhile, the input images are normalized, with the mean and variance set to [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225], respectively, which are consistent with the settings used in ImageNet pretrained models. The detailed training parameters of the feature extraction network are shown in

Table 3. Training parameters of the feature extraction network.

Parameter	Setting
Batch Size	64
Training epochs	200 epochs
Optimizer	SGD
Initial learning rate	0.1
Momentum	0.9
Weight decay	0.0005
Loss function	Loss function

.

To ensure the fairness of experimental comparisons, all comparison methods adopt the same dataset partitioning and training configurations, with differences only in the feature extraction structures or tracking strategies.

4.2. Evaluation Indicators

In order to better evaluate the performance of the DeepSORT-OCR model, the selected evaluation metrics are crucial for measuring the model’s performance. They can comprehensively reflect the accuracy, robustness, and generalization ability of the model in the target detection and tracking tasks. In this study, MOTA (Multiple Object Tracking Accuracy), MOTP (Multiple Object Tracking Precision), MT (Mostly Tracked), ML (Mostly Lost), and the number of ID switches are used as evaluation metrics to assess the performance of the DeepSORT-OCR model.

The following is a detailed analysis of the evaluation indicators:

$$MOTA=1-{\displaystyle \frac{FN+FP+IDSW}{GT}}$$

(28)

$$MOTP=1-{\displaystyle \frac{{\sum }_{t,i}{d}_{t,i}}{{\sum }_t{c}_t}}$$

(29)

FN represents false negative examples, which is the sum of all missed detections in the entire video, while FP represents false positive examples, which is the sum of all false alarms in the entire video. IDSW is the total number of ID switches—that is, the number of target ID switches throughout the tracking process. GT is the total number of actual targets. C_t represents the number of matches in the t-th frame, and d_t,i represents the distance between the predicted bounding box and the real bounding box of the i-th target in the t-th frame.

MT represents the proportion of successfully tracked targets to the total number of targets.

ML is the proportion of targets with a loss time exceeding a given threshold in the entire tracking sequence to the total number of targets.

To evaluate the performance of the hull number recognition module, samples containing clearly identifiable hull numbers were selected from the self-constructed ship dataset to build a hull number recognition subset and the OCR recognition results were quantitatively evaluated. Since the hull number recognition task involves both the accuracy of individual character recognition and the correctness of the entire hull number string, the performance is evaluated from two levels: character-level recognition and hull-number-level recognition.

At the character level, the evaluation mainly measures the OCR model’s ability to correctly recognize individual characters. At the hull-number level, the evaluation determines whether the entire hull number string is completely and correctly recognized. To comprehensively evaluate the hull number recognition performance, four metrics are adopted: Accuracy, Precision, Recall, and F1-score.

The evaluation metrics are explained as follows.

In recognition tasks, commonly used evaluation metrics are mainly employed to measure the model’s performance in terms of recognition accuracy, recall capability, and overall recognition quality under complex maritime conditions.

Precision measures the proportion of correctly recognized characters among all characters predicted as correct by the OCR model. It is defined as

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(30)

where TP represents the number of correctly recognized characters, and FP represents the number of incorrectly recognized characters.

Recall measures the proportion of correctly recognized characters among all actual characters. It is defined as

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(31)

where FN represents the number of characters that were not correctly recognized.

The F1-score combines Precision and Recall and serves as an important indicator of recognition performance. It is defined as

$$F1=2·{\displaystyle \frac{Precision·Recall}{Precision+Recall}}$$

(32)

For character-level evaluation, Character Recognition Accuracy (Character Accuracy) is also used to measure the proportion of correctly recognized characters. It is defined as

$${Acc}_{char}={\displaystyle \frac{N_{correct}}{N_{total}}}$$

(33)

where $N_{correct}$ represents the number of correctly recognized characters and $N_{total}$ represents the total number of characters.

In addition, to evaluate the recognition performance of complete hull number strings, Hull Number Accuracy is introduced and defined as

$${Acc}_{hull}={\displaystyle \frac{N_{correct\_hull}}{N_{total\_hull}}}$$

(34)

where $N_{correct\_hull}$ represents the number of hull numbers that are completely recognized correctly, and $N_{total\_hull}$ represents the total number of hull numbers. A hull number is considered correctly recognized only when all characters in the string are correctly identified.

Through these evaluation metrics, the OCR module can be comprehensively evaluated from both character recognition accuracy and overall hull number recognition accuracy, thereby verifying the effectiveness of the CBAM-ResNet feature enhancement mechanism in improving hull number recognition performance.

4.3. Analysis of Experimental Results

A.

General Experimental Validation

To verify the general performance of the proposed tracking algorithm in complex scenarios, comparative experiments were conducted with SORT, DeepSORT, ByteTrack, QDTrack, and the improved DeepSORT algorithm proposed in this paper. The experiments were carried out on the MOT16 benchmark dataset. The target category in this dataset is pedestrians; however, its complex target interactions and frequent occlusion changes make it suitable for evaluating the performance of multi-object tracking algorithms in terms of target association, trajectory maintenance, and occlusion recovery. Therefore, the MOT16 dataset is used in this study to verify the general multi-object tracking capability of the proposed algorithm.

It should be noted that the MOT16 dataset does not contain ship targets or hull number information. Therefore, the Hull Number Match module proposed in this work is not involved in this part of the experiment. This module is mainly evaluated in the subsequent experiments conducted on the self-constructed maritime ship dataset, where it is used to verify ship identity recognition and long-term tracking performance.

In the experiments, all tracking algorithms adopt the same detector, i.e., the detection results generated by the trained YOLOv5 model, as the input to ensure the fairness of the comparative experiments. Under the same detection results and runtime environment, the performance of different tracking algorithms is quantitatively evaluated. The experimental results are shown in

Table 4. Experimental results of different target tracking models on the MOT16.

Method	MOTA/%	MOTP/%	MT/%	ML/%	IDSW	FPS	Latency (ms)	GPU Memory (MB)
Sort	59.98	77.30	25.40	22.70	1423	42.7	23.5	826.8
DeepSORT	61.40	79.10	32.80	18.20	781	38.4	26.7	924.3
ByteTrack	63.10	77.80	35.20	14.90	610	36.9	27.5	987.6
QDTrack	64.00	77.90	36.10	14.30	560	32.8	31.9	1328.5
Ours	66.53	79.48	38.54	13.89	528	36.5	27.4	962.9

.

As shown in Table 4, the improved DeepSORT algorithm proposed in this paper achieves better tracking performance on the MOT16 dataset. Specifically, the MOTA reaches 66.53%, which is an improvement of 5.13% compared with DeepSORT, indicating that the proposed method achieves higher overall tracking accuracy in complex scenarios. Meanwhile, the MT metric increases to 38.54%, while the ML metric decreases to 13.89%, indicating that the algorithm can successfully track more targets while reducing tracking failures. In addition, the number of ID Switches (IDSW) is reduced to 528, further demonstrating the advantage of the proposed method in improving target association stability.

In terms of computational efficiency, since all algorithms employ the same YOLOv5 detector, the overall runtime speed is mainly influenced by the detection stage. As a result, the differences in FPS among different tracking algorithms are relatively small. The proposed method maintains a real-time processing speed of 36.5 FPS, with an average processing latency of 27.4 ms and GPU memory consumption of 962.9 MB, while achieving higher tracking accuracy. This indicates that the method has good real-time performance and resource efficiency in practical applications.

In summary, the experiments on the MOT16 dataset are mainly designed to verify the general multi-object tracking capability of the algorithm in complex dynamic environments, while the detection and tracking performance for ship targets will be further validated in the subsequent experiments conducted on the self-constructed maritime ship dataset.

B.

Experiments on the Self-Constructed Dataset

To further verify the applicability and performance advantages of the proposed improved tracking algorithm in complex maritime environments, comparative experiments were conducted on the self-constructed maritime ship dataset. During the experiment, the YOLOv5 model trained on this dataset was first used to generate target detection results, which were then used as the common input for all tracking algorithms to ensure fairness in the comparison. Based on this setup, the proposed algorithm was compared with four representative multi-object tracking methods: SORT, DeepSORT, ByteTrack, and QDTrack. The experiments mainly evaluated the tracking accuracy and stability of each method under complex maritime conditions such as illumination variations, wave interference, and target occlusion. The experimental results are shown in

Table 5. Experimental results of different target tracking models on the self-built dataset.

Method	MOTA/%	MOTP/%	MT/%	ML/%	IDSW	FPS	Latency (ms)	GPU Memory (MB)
Sort	63.93	70.17	26.34	21.90	252	42.6	23.5	828.1
ByteTrack	66.17	74.51	28.16	19.21	213	36.8	27.2	988.7
QDTrack	67.79	78.87	28.98	18.05	186	32.7	30.6	1327.9
DeepSORT	65.74	75.12	33.24	18.65	198	38.3	26.1	925.4
Ours	70.89	80.84	36.58	16.82	124	34.7	28.8	985.5

.

It should be noted that unlike the previous MOT16 experiments, the Hull Number Match module was enabled in the self-constructed ship dataset experiments. This module recognizes ship hull numbers using OCR and matches the recognition results with the hull number feature database. In this way, semantic identity constraints based on hull number information are introduced during the target association stage to enhance identity discrimination and long-term tracking stability.

As shown in Table 5, the improved tracking algorithm proposed in this paper achieves the best overall performance on the self-constructed maritime ship dataset. Specifically, the MOTA reaches 70.89%, which is approximately 5.15% higher than that of DeepSORT, indicating that the improved algorithm achieves higher overall tracking accuracy in complex maritime environments. Meanwhile, the MOTP increases to 80.84%, demonstrating that the proposed method also improves target localization accuracy.

In terms of trajectory stability, the number of ID switches of the proposed method is reduced to 124, which is approximately 37.4% lower than that of DeepSORT. This result indicates that introducing hull number semantic constraints combined with multi-feature matching mechanisms can effectively reduce identity switching and improve the stability of target association. In addition, the MT metric increases to 36.58%, while the ML metric decreases to 16.82%, showing that the improved algorithm is able to successfully track more targets while reducing tracking failures.

Regarding computational efficiency, although the proposed method introduces the Hull Number Match module during the matching stage, the overall inference speed still maintains 34.7 FPS, with an average processing latency of 28.8 ms and GPU memory consumption of 985.5 MB, indicating that the method still maintains good real-time processing capability. Overall, the proposed method can effectively improve tracking accuracy and trajectory stability in complex maritime scenarios while maintaining real-time performance, demonstrating its practical value in maritime multi-ship tracking tasks.

To analyze the contribution of each improvement module to the overall tracking performance, ablation experiments were conducted on the self-constructed ship dataset. Taking the complete model as the baseline, each module was removed individually while keeping the remaining modules unchanged, including the CBAM-ResNet50 feature enhancement module, the LSTM-AKF motion prediction module, the Inner-SIoU matching strategy, and the Hull Number Match module. The experimental results are shown in

Table 6. Ablation study results.

ID	CBAM-ResNet50	LSTM-AKF	Inner-SIoU	Hull Number Match	MOTA/%	MOTP/%	IDSW
1	√	√	√	√	70.89	80.84	124
2	×	√	√	√	69.42	79.31	151
3	√	×	√	√	69.01	78.67	158
4	√	√	×	√	68.83	77.95	162
8	√	√	√	×	67.54	78.10	181

, where √ indicates the module is enabled and × indicates it is disabled.

As shown in Table 6, when the CBAM-ResNet50 module is removed, the MOTA decreases to 69.42%, and the number of ID switches increases to 151, indicating that the attention mechanism can enhance appearance feature representation and thus improve the accuracy of target matching. When the LSTM-AKF module is removed, the MOTA decreases to 69.01% and IDSW increases to 158, demonstrating that LSTM-based motion modeling and adaptive Kalman filtering effectively improve trajectory prediction accuracy and reduce association errors.

When the Inner-SIoU matching strategy is removed, the MOTA decreases to 68.83%, and IDSW increases to 162, indicating that the improved spatial matching mechanism enhances the stability of target localization and association. When the Hull Number Match module is removed, the performance degradation is the most significant, with the MOTA dropping to 67.54% and IDSW increasing to 181. This result shows that introducing hull number semantic constraints can significantly reduce identity switches and improve long-term tracking stability in maritime ship scenarios.

Overall, the proposed modules complement each other in feature representation, motion modeling, and target association, jointly improving the performance of multi-object tracking in complex maritime environments and enabling the full model to achieve the best experimental results.

To visually demonstrate the tracking performance of the proposed algorithm in complex scenarios, several consecutive frames from UE virtual environment video sequences and real maritime surveillance videos were selected for qualitative comparison. The results are shown in Figure 13 and Figure 14, where the images on the left represent the tracking results of the YOLOv5–DeepSORT algorithm, and the images on the right show the tracking results of the improved DeepSORT algorithm proposed in this paper. In the figures, the red boxes highlight typical regions where tracking differences, such as identity switches or missed associations, can be observed.

C.

Hull Number Subset Experiments

To further analyze the practical role of hull number semantic information in ship target tracking, and to verify the improvement of hull number recognition performance brought by the CBAM-ResNet feature enhancement module, a hull number recognition subset was constructed by selecting all samples containing clearly visible hull numbers from the self-constructed maritime ship dataset. This subset was used to evaluate both the hull number recognition module and the contribution of hull number semantic features to target association.

First, the hull number recognition module is evaluated on this subset. Two OCR recognition schemes are compared, namely, the original OCR model and the OCR model enhanced with the CBAM-ResNet feature enhancement module, inorder to analyze the effect of the attention mechanism on hull number recognition performance. The recognition performance is evaluated from two levels: character-level (Character-level) and complete hull number recognition level (Hull Number-level). The evaluation metrics include Character Accuracy (Char Accuracy), Precision, Recall, and F1-score. The experimental results are shown in

Table 7. Performance evaluation of the hull number recognition module.

Method	Accchar/%	Precision/%	Recall/%	F1/%	Acchull/%
OCR	90.8	89.5	88.7	89.1	83.6
OCR + CBAM-ResNet	94.7	93.6	92.9	93.2	88.9

.

As shown in Table 7, after introducing the CBAM-ResNet feature enhancement module, the hull number recognition performance improves significantly across all evaluation metrics. Specifically, the character-level recognition accuracy increases from 90.8% to 94.7%, while the overall hull number recognition accuracy increases from 83.6% to 88.9%. These results indicate that the CBAM attention mechanism can effectively enhance the feature representation of hull number character regions, making character structural information more distinguishable in complex maritime backgrounds, thereby improving the stability and accuracy of OCR recognition in challenging maritime environments.

After evaluating the hull number recognition performance, multi-object tracking experiments are further conducted on the hull number subset to analyze the influence of hull number semantic features on target association performance. In this experiment, all tracking methods use the detection results generated by the YOLOv5 detector trained on the self-constructed dataset as the unified input to ensure fair comparison. Consistent with the experiments conducted on the full dataset, the proposed method fuses visual appearance features and hull number semantic features during the target association stage and integrates them through a dynamic weighting strategy.

Since all targets in this subset contain recognizable hull numbers, the semantic features of these numbers can provide more stable identity constraints during the association process. The experimental results are shown in

Table 8. Tracking results on the hull number subset.

Method	MOTA/%	MOTP/%	MT/%	ML/%	IDSW
Sort	68.12	72.04	29.31	18.96	121
ByteTrack	71.08	75.64	31.84	16.78	89
QDTrack	72.26	79.58	33.17	16.02	76
DeepSORT	69.87	76.41	35.62	16.84	94
Ours	77.18	82.46	41.35	13.92	53

.

As shown in Table 8, when hull number information can be reliably obtained, the proposed tracking algorithm achieves further improvements in both Multiple Object Tracking Accuracy (MOTA) and Multiple Object Tracking Precision (MOTP). The MOTA reaches 77.18%, which is 7.31% higher than DeepSORT, and the MOTP increases to 82.46%. These results demonstrate that when hull number semantic information is fully available, the proposed method can further improve the accuracy of target association.

In terms of trajectory stability, the number of ID switches (IDSW) decreases to 53, indicating that hull number semantic features can provide additional identity cues when targets have similar appearances or experience occlusion, thereby effectively reducing identity switching. Meanwhile, the MT metric increases to 41.35%, while the ML metric decreases to 13.92%, showing that the improved algorithm can stably track more targets and reduce tracking failures.

To visually demonstrate the improvement in tracking stability brought by the Hull Number Feature Extraction module, a typical scenario from a video sequence is selected for qualitative visualization. Figure 15 presents three groups of comparison results. The first image shows the moment when the OCR module successfully recognizes the hull number for the first time, where both the target ID and the recognized hull number are displayed above the detection box. The second and third images compare the tracking results with and without the OCR module under the same frame. The comparison clearly shows that after introducing hull number semantic features, the system achieves more stable identity maintenance and trajectory continuity. The experimental visualization results are shown in Figure 15.

In summary, the CBAM-ResNet feature enhancement module effectively improves hull number recognition performance, and the hull number semantic features provide stable semantic constraints for ship identity discrimination during target association, thereby further improving the accuracy and stability of multi-ship tracking in maritime environments.

D.

Cross-Domain Generalization Experiment

To further evaluate the generalization capability of the proposed method across different data domains, cross-domain experiments were conducted on the self-constructed maritime ship dataset. According to the data source, the dataset is divided into three settings, namely, purely synthetic, purely real, and mixed data, and an additional cross-domain scenario (synthetic training–real testing) is constructed. The experimental results are shown in

Table 9. Cross-domain generalization experiment results.

Training Data	Testing Data	MOTA/%	MOTP/%	IDSW
Synthetic Data	Synthetic Data	70.42	80.21	131
Real Data	Real Data	69.87	80.03	138
Synthetic Data	Real Data	67.95	78.94	152
Mixed Data	Mixed Data	70.89	80.84	124

.

As shown in Table 9, the proposed method achieves the best performance under the mixed-data training and testing setting, indicating that combining real data and synthetic data can effectively improve the model’s adaptability to different scenarios. Under both pure synthetic data and pure real data domains, the model also maintains relatively stable tracking performance, demonstrating that the proposed method performs robustly within a single data domain. Furthermore, under the cross-domain testing condition, the MOTA still reaches 67.95%, showing only a moderate decrease. This result indicates that the proposed method maintains good generalization capability across different data distributions.

5. Conclusions

To address the challenges of similar target appearances, unstable trajectories, and frequent identity switches in multi-ship tracking under complex maritime environments, an improved DeepSORT-based ship multi-object tracking algorithm, DeepSORT-OCR, which integrates hull number semantic features, is proposed in this paper. Based on the DeepSORT framework, the proposed method enhances ship appearance feature representation using CBAM-ResNet, introduces Inner-SIoU to improve the spatial matching stability of elongated ship targets, and incorporates an LSTM-Adaptive Kalman Filter to model the nonlinear motion patterns of ships, thereby improving trajectory prediction accuracy and tracking stability. In addition, a Hull Number Feature Extraction module is designed to recognize ship hull numbers through OCR and match them with a hull number database, dynamically integrating hull number semantic information into the target association process to enhance identity discrimination capability.

The experimental results demonstrate that the proposed method achieves strong tracking performance across different datasets. On the MOT16 dataset, the algorithm achieves an MOTA of 66.53%, representing an improvement of 5.13% compared with DeepSORT. On the self-constructed maritime ship dataset, the proposed method achieves an MOTA of 70.89% and an MOTP of 80.84%, while significantly reducing the number of ID switches. In particular, in the subset containing valid hull number information, the MOTA further increases to 77.18%, which is 7.31% higher than DeepSORT, indicating that hull number semantic information can provide effective identity constraints when targets have similar appearances or experience occlusions, thereby significantly improving the stability of target association. Furthermore, the results of pure real-data experiments, pure synthetic-data experiments, and cross-domain experiments demonstrate that the proposed method maintains good robustness and generalization capability under different data distributions.

Overall, while maintaining good real-time performance, the proposed method effectively improves the accuracy, trajectory continuity, and identity consistency of multi-ship tracking in complex maritime environments. This provides a feasible technical solution for applications such as maritime traffic monitoring, intelligent patrol, and multi-source perception fusion. Future work will focus on improving hull number recognition methods with higher robustness and developing more efficient multimodal fusion mechanisms for long-distance, low-resolution, and heavily occluded scenarios, thereby further enhancing the applicability of the algorithm in real-world maritime environments.