Intelligent Logistics Sorting Technology Based on PaddleOCR and SMITE Parameter Tuning

Featured Application

Main findings: (1) The SMITE video image segmentation algorithm maintains stable and accurate segmentation under complex conditions, including object appearance variations, illumination changes, and viewpoint shifts, and the PaddleOCR text recognition algorithm achieves an average recognition accuracy exceeding 96.67%, significantly outperforming traditional methods. (2) The ROS-based unmanned vehicle system integrates SMITE segmentation, PaddleOCR recognition, Eye-to-Hand visual servoing, and hybrid A* + Dijkstra path planning, achieving an average sorting time of 18.4 s per parcel, a recognition accuracy of 95.0%, a sorting success rate of 92.0%, and an average localization error of 1.6 cm ± 0.4 cm. implication of the main finding: (1) The findings provide a feasible intelligent sorting solution to address the low efficiency, high operational costs, and high error rates of traditional manual and semi-automated logistics sorting operations. (2) The integrated technical framework and cost advantage (approximately 70% lower hardware costs than commercial systems) enable rapid deployment and secondary development, promoting the popularization of intelligent sorting technology in logistics scenarios such as pharmaceutical distribution.

Abstract

To address the current reliance on manual labor in traditional logistics sorting operations, which leads to low sorting efficiency and high operational costs, this study presents the design of an unmanned logistics vehicle based on the Robot Operating System (ROS). To overcome bounding-box loss issues commonly encountered by mainstream video-stream image segmentation algorithms under complex conditions, the novel SMITE video image segmentation algorithm is employed to accurately extract key regions of mail items while eliminating interference. Extracted logistics information is mapped to corresponding grid points within a map constructed using Simultaneous Localization and Mapping (SLAM). The system performs global path planning with the A* heuristic graph search algorithm to determine the optimal route, autonomously navigates to the target location, and completes the sorting task via a robotic arm, while local path planning is managed using the Dijkstra algorithm. Experimental results demonstrate that the SMITE video image segmentation algorithm maintains stable and accurate segmentation under complex conditions, including object appearance variations, illumination changes, and viewpoint shifts. The PaddleOCR text recognition algorithm achieves an average recognition accuracy exceeding 98.5%, significantly outperforming traditional methods. Through the analysis of existing technologies and the design of a novel parcel-grasping control system, the feasibility of the proposed system is validated in real-world environments.

1. Introduction

Logistics sorting is a fundamental component of China’s modern logistics development strategy. With the rapid advancement in smart cities and autonomous driving technologies, increasingly stringent requirements have been set for the efficiency of intelligent logistics sorting. Currently, traditional manual logistics operations suffer from low efficiency and high operational costs. Most workflows still rely either entirely on manual sorting or on a hybrid of manual and automated processes; however, these approaches generally exhibit low efficiency and limited accuracy, making them inadequate for large-scale distribution demands. Under a purely manual sorting model, a single worker can process only 1500 to 2000 parcels per day, with an error rate as high as 1.2% to 1.5%. Semi-automated sorting systems, which incorporate equipment such as barcode scanners and mechanical conveyors, can increase throughput to 3000 to 4000 items per hour. Nevertheless, human operators remain necessary for parcel classification and placement, resulting in challenges such as workflow disruptions during human–machine collaboration, insufficient equipment flexibility (with changeover times exceeding 30 min), and high energy consumption costs (power consumption per system ≥50 kW) [1,2,3,4]. This is especially critical in sorting scenarios involving high-value goods, such as pharmaceuticals and electronics, where traditional methods fail to meet fine-grained operational requirements, including precise temperature and humidity control (±2 °C deviation) and collision-avoidance protection (impact force ≤ 5 N). Consequently, the cargo damage rate remains persistently high, ranging between 0.8% and 1.0% [5,6].

Currently, the logistics industry widely adopts intelligent sorting methods, which offer clear advantages over traditional approaches but may also impact overall system performance. In particular, integrating technologies such as robotic arms, mobile chassis platforms, and autonomous driving significantly increases system complexity. If parameters are not optimally designed, systems may experience increased coupling and reduced robustness. To address these key technical challenges, both academia and industry have conducted extensive multi-dimensional research. In hardware system optimization, Lü Liqiang and Yang Shuangxing from Shanxi Jiaotong Technician College proposed a distributed control architecture based on industrial Ethernet. By employing modular design, they reduced system coupling by 40%, achieving a sorting unit positioning accuracy of ±2 mm and increasing throughput to 6000 items per hour [7]. This work provides an engineering-oriented solution for hardware integration in multi-robot collaborative sorting, but does not address the optimization of perception algorithms under complex environmental conditions. Regarding simulation-based verification, Yao Xin from Harbin Institute of Technology developed a digital-twin sorting environment using the NVIDIA Omniverse platform [8]. By integrating the Isaac Sim simulation tool, virtual-to-real transfer learning was applied to a parcel pose estimation model, yielding real-world testing errors of less than 5 cm in position and less than 5° in orientation. While this significantly shortened the algorithm development cycle, discrepancies between the simulated environment and actual operating conditions, such as sensor noise and mechanical vibrations, still require further calibration. In algorithmic innovation, Guan Xiangyang’s research team at the University of Washington proposed a hybrid heuristic algorithm combining genetic algorithms and local search to tackle the Capacitated Multi-Compartment Vehicle Routing Problem with Robotic Sorting Tasks (CMCVRP-RSTC) [9]. Empirical analyses of logistics networks in Chongqing showed that this method reduced vehicle travel distance by 12% and decreased sorting energy consumption by 15% [10]. However, the algorithm has limited adaptability to dynamic order insertion scenarios, with re-planning time increasing superlinearly as the number of nodes grows.

In automatic identification and data collection systems, barcode technology plays a pivotal role. Barcodes, including one-dimensional and two-dimensional formats, are extensively utilized in fields such as retail, warehousing, healthcare, and industrial automation. However, in practical applications, barcode images are frequently compromised by various factors, including ambient lighting, camera quality, camera shake, inaccurate focusing, rapid movement, and noise interference, resulting in blurred images [11]. Barcodes, while inexpensive and easy to implement, require direct line-of-sight and manual scanning, which limits throughput and scalability in fast-paced environments [12]. And some logistics systems have adopted Radio Frequency Identification (RFID) for contactless, multi-item tracking. However, RFID infrastructure remains costly and is often incompatible with paper-based postal workflows. As a result, Optical Character Recognition (OCR), combined with image processing and machine learning, has gained attention for its ability to automate text-based sorting tasks [13].

When a 2D barcode cannot be recognized due to physical damage, poor image quality, or obstruction, “human-readable label OCR” can serve as an important backup identification method. Robots can use OCR technology to recognize human-readable information on courier waybills (such as recipient name, address, phone number, postal code, etc.), ensuring that sorting tasks continue and avoiding efficiency losses caused by process interruptions and manual intervention. This redundant design significantly enhances the system’s reliability in real, complex logistics environments [14].

In summary, although two-dimensional barcode recognition technology has advantages in efficiency, it still has inherent limitations in practical applications regarding robustness, compliance, error correction, and information completeness. Therefore, integrating a ‘human-readable label OCR’ solution into logistics sorting robots is not merely a repetition of existing technology, but an innovative strategy to enhance overall system reliability, expand its application boundaries, meet regulatory requirements, and optimize human–machine collaboration. This will make logistics sorting systems more resilient, smarter, and better suited to the demands of complex real-world environments.

The above literature presents various implementation schemes for intelligent logistics sorting systems from multiple perspectives, providing valuable support for the advancement in mature intelligent sorting solutions. However, when applied to complex real-world environments, challenges such as significant gaps between simulation and reality, prolonged planning and decision-making times, and relatively low recognition efficiency still persist. In response to the urgent demands of modern intelligent logistics sorting technology, developing an algorithm that balances system complexity with recognition accuracy has become especially critical. Accordingly, this study refines and optimizes parameters based on the insights gained from prior research, aiming to maximize recognition accuracy while minimizing resource consumption and reducing overall system complexity. This paper proposes an integrated approach that combines visual segmentation, text recognition, hand–eye coordination, and path planning. By innovatively utilizing the SMITE image segmentation algorithm for feature extraction, together with PaddleOCR-based text recognition and precise hand–eye calibration, the system achieves accurate grasping and delivery of target parcels with high recognition efficiency and rapid decision-making.

2. System Design

2.1. Hardware Architecture

The modified logistics vehicle used by the project team is shown in Figure 1, and the system architecture is illustrated in Figure 2. The vehicle’s primary parameters are summarized in

Table 1. Vehicle Parameters.

Parameter	Description
Drive type	Differential control
Laser radar	EAI G1 high-precision LiDAR
Motor	DC brushless motor
Obstacle-avoidance sensor	Ultrasonic modules × 2
Robotic arm	DOBOT magician robotic arm (optional)
Battery type	Power lithium battery
Camera	RGB camera
IMU module	MEMS gyroscope and 3-axis accelerometer
Hardware interfaces	USB 3.0, USB 2.0, LAN, WAN

:

2.2. Software Functions

The unmanned logistics vehicle utilizes the ROS open-source framework, running on Ubuntu 18.04, as depicted in Figure 3. Through a unified communication architecture, it tightly integrates devices such as the robotic arm, camera, and LiDAR. This seamless integration not only simplifies coordination among components but also provides a robust foundation for algorithm implementation, ensuring high efficiency and reliability in the vehicle’s task execution.

The logistics vehicle primarily performs the following functions: map construction, navigation, express label information recognition, sorting, and robotic-arm-based grasping. The detailed implementation workflow is illustrated in Figure 4.

3. Technical Scheme

The parcel sorting and delivery workflow of the logistics vehicle is illustrated in Figure 5. Upon system startup, the vehicle first uses the IMU module, odometry data, and laser-based SLAM technology to construct a two-dimensional grid map via the GMapping algorithm. The A* algorithm is employed to plan the globally optimal path, while the Dijkstra algorithm manages robust local path planning to navigate complex road conditions, guiding the vehicle to the parcel sorting station. A pose recalibration routine is then initiated to ensure the vehicle’s position and orientation remain fixed. Following recalibration, the camera mounted on the robotic arm is activated, and a vision-based recognition algorithm captures the text printed on parcels. Through a combination of OpenCV-based image preprocessing, SMITE image segmentation, and the PaddleOCR text recognition framework, the system accurately identifies key parcel information, such as the destination province. This data is transmitted to the control system, which matches the destination, plans the delivery route accordingly, and directs the robotic arm to deliver each parcel precisely to its designated mailbox.

Throughout this process, the Adaptive Monte Carlo Localization (AMCL) algorithm maintains accurate vehicle localization even in complex environments. To correct AMCL drift, often caused by odometry errors or wheel slippage, the system uses LiDAR-based localization by assessing the alignment between scanned laser points and the map. This significantly enhances real-time localization accuracy and supports reliable autonomous navigation [15]. After completing each delivery task, the vehicle automatically returns to the sorting platform and repeats the workflow, continuously sorting and delivering parcels to ensure efficient and accurate handling. Once the final parcel is delivered or the process is manually stopped, the vehicle ceases its current task, returns to the designated standby area, and awaits the next logistics sorting session.

4. Optimized Sorting Method Design

4.1. Mapping and Navigation Technology

Mapping and navigation technologies play a pivotal role in the development of robotics. By leveraging SLAM techniques and integrating data from LiDAR sensors and high-precision wheel encoders, robots can construct maps and perform real-time localization within previously unknown environments. Laser-based SLAM enables robots to achieve localization and three-dimensional map construction in uncharted areas by fusing LiDAR data with machine learning algorithms, thus laying the foundation for safe navigation and task execution. Among these methods, the open-source GMapping algorithm utilizes a particle-filtering framework, improving localization accuracy and map-update efficiency through feature matching and map optimization. The AMCL algorithm updates the robot’s pose probability distribution via a set of particles, serving as a key technique for real-time localization. For path planning, the system comprehensively accounts for map information, obstacles, and target positions, employing both the A* and Dijkstra algorithms to compute optimal routes. By integrating global and local planning strategies, the intelligent vehicle navigates safely and efficiently through complex environments,as illustrated in Figure 6.

4.2. Sided Visual Inspection

4.2.1. Camera Calibration

In this study, a monocular camera model is adopted, and the chessboard calibration method is employed to compute the camera’s intrinsic and extrinsic parameters. Using these parameters, image undistortion, specifically perspective transformation, is performed to obtain corrected images that provide accurate relative positional information of the target with respect to the camera. Distortion occurs due to lens characteristics that deform the raw images, making image undistortion necessary to eliminate these effects. Following the pinhole camera imaging model, the intrinsic and extrinsic parameters are derived by relating real-world coordinates to pixel coordinates. During calibration, a chessboard pattern is used as the calibration object [16], as its regular geometric structure and well-defined corner points are easily detected and recognized by the camera, making it an ideal choice. The implementation steps are as follows:

• Acquisition of chessboard image sequences: Capture multiple images of the chessboard from various angles and positions.
• Chessboard corner extraction: Detect and extract the corner points of the chessboard in each image.
• Pixel coordinate computation: Calculate the pixel coordinates corresponding to each detected corner point within the images.
• Intrinsic parameter calibration: Using the 2D pixel coordinates of the chessboard corners alongside the known 3D model coordinates, solve for the camera’s intrinsic parameters, such as focal length and principal point, by minimizing the reprojection error. This process yields the camera intrinsic matrix and distortion coefficients.

After completing camera extrinsic calibration [17], the Perspective-n-Point (PnP) problem is solved to determine the camera’s position and orientation within the world coordinate system, thereby calculating the extrinsic parameters for each captured image. Following calibration, OpenCV’s camera calibration evaluation functions are used to assess metrics such as reprojection error and chessboard corner detection accuracy, ensuring that the calibration precision meets system requirements and allowing for further optimization if needed. Within the logistics vehicle’s operating system, the workflow for obtaining continuous real-time undistorted images includes: performing camera calibration using the camera-calibration package; launching the corresponding ROS nodes to subscribe to raw image streams (Figure 7) and execute undistortion processing; and publishing the corrected images (Figure 8) to designated topics. After calibration, OpenCV evaluation tools are again employed to assess reprojection errors and chessboard localization accuracy, verifying calibration quality and facilitating any necessary refinements.

Through the above steps, accurate camera calibration and image undistortion are achieved, providing a precise data foundation for subsequent computer vision applications.

4.2.2. Hand–Eye Coordination Algorithm

Text Recognition Module: The PaddlePaddle framework, integrated with ROS and Python 3.6.9, is utilized, and the PaddleOCR text recognition system is installed. Building upon the OCR capabilities, the overall grasping logic is developed. The design workflow of this module is illustrated in Figure 9.

OpenCV and PaddlePaddle are widely used libraries for visual algorithms, with PP-OCR being an OCR (Optical Character Recognition) toolkit developed on the PaddlePaddle deep learning framework [18]. PP-OCR primarily consists of three components: DB-based text detection, detection-frame correction, and CRNN text recognition. PaddleOCR is chosen as the core algorithm for visual detection due to its efficient text detection and recognition capabilities, fast processing speed while maintaining high accuracy, and its simple, user-friendly APIs with multilingual support.

The image-processing workflow is illustrated in Figure 10. Once the robot reaches the target parcel location, the camera mounted on the end of the robotic arm dynamically captures each frame of image data, extracting the current frame for processing. The image is first preprocessed using the OpenCV library [19], followed by the application of a pre-trained text-detection model to locate text regions within the image. A deep learning–based recognition model then performs text recognition. Simultaneously, the center coordinates of the detected text bounding box are extracted and used as input parameters for the robotic arm’s pose adjustment via coordinate-system transformation.

For text-region localization, information filtering is a crucial step. During this process, object color characteristics are leveraged for positioning; specifically, areas likely to contain parcel boxes are extracted from the HSV image based on a predefined blue color range. Concurrently, the median filtering algorithm from OpenCV is applied, which replaces each pixel value with the median of its neighboring pixels to effectively reduce noise in the image. Figure 11 illustrates the theoretical concept of the median filtering algorithm [20].

In mail recognition and localization, mask-based processing is employed to binarize the blue-color region, isolating only the blue portions of the image. This approach reduces the computational load for subsequent processing steps while enhancing the efficiency and accuracy of target object extraction. By effectively removing image noise and preserving the edge information of the target, these methods minimize interference and improve both the precision and reliability of object recognition.

Coordinate Transformation Module:

In the designed system, the hand–eye coordination follows an “Eye-to-Hand” configuration, where the camera is fixed to the end-effector of the robotic arm. Consequently, visual processing involves transforming coordinates between the camera coordinate system and the robotic arm coordinate system. Utilizing the pinhole camera imaging model, image data is acquired and processed accordingly. According to the pinhole camera model [21], the camera, with known focal length and distortion parameters, locates the target by converting extracted pixel coordinates into camera coordinates. The camera’s intrinsic parameter matrix K is defined as follows:

$$\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]$$

(1)

where f_x and f_y represent the camera’s focal lengths in the x- and y-directions, respectively, that is, the distances from the camera’s optical center to the imaging plane. These values are expressed in pixel units within the intrinsic matrix and determine both the camera’s field of view and the size of the captured image. The parameters c_x and c_y denote the coordinates of the principal point, which correspond to the projection of the optical axis onto the image plane, indicating the origin of the image coordinate system within the image.

Accordingly, given the image coordinate point (u, v) corresponding to the detected target object, the camera coordinate point (x, Y) can be computed as

$$\left\{\begin{array}{c}x={\displaystyle \frac{u-c_x}{f_x}}\\ Y={\displaystyle \frac{v-C_y}{f_y}}\end{array}\right\}$$

(2)

To transform the camera-coordinate point into the robotic arm coordinate system, yielding (Arm_x,Arm_y,Arm_z), the fixed offsets between the camera and the robotic arm (offset_x,offset_y,offset_z) and the object’s depth Z relative to the camera must be considered.

Based on these factors, a comprehensive hand–eye coordination algorithm was developed. By leveraging the spatial relationships among the parcel, the camera, and the robotic arm, along with the conversion ratio between pixel dimensions and real-world distances, the algorithm converts camera coordinates into robotic arm pose information, enabling precise arm movement to grasp the parcel accurately.

Principle of Hand–Eye Coordination Parameter Tuning:

Based on actual testing conditions, the observation position of the robotic arm (arm _x,arm _y,arm _z) is first determined.

$$\left\{\begin{array}{c}arm\_x=1.73\\ arm\_y=-233.67\\ arm\_z=50\end{array}\right\}$$

(3)

Next, based on the pixel offsets and from the camera’s center coordinate to the target coordinate, and the scaling ratio between actual physical size and pixel units, the real-world distances from the target to the image center, denoted as and are calculated.

$$dx=\mathsf{\Delta }x\times proportion$$

(4)

$$dy=\mathsf{\Delta }y\times proportion$$

(5)

Finally, using the observation height, the actual distance between the target and the camera, the fixed offset between the camera and the robotic arm, and the grasping parameters (grab _x,grab _y) are calculated and transmitted to the robotic arm. These parameters represent the target pose to which the arm should adjust after the camera has recognized the parcel in real time, enabling accurate extraction of textual information at the correct position.

$$\left\{\begin{array}{l}grab\_x=arm\_x+dx*10(mm)\\ grab\_y=arm\_y-(dy+3.5)*10\end{array}\right\}$$

(6)

If the parcel’s text information cannot be recognized, a four-point region is defined around the current pose, prompting the robotic arm to adjust its position and attempt recognition again.

4.2.3. SMITE Video Image Segmentation Algorithm

1. Key-Region Segmentation for Target Images:

When the robotic arm adjusts its pose, complex motions and variations in the target within the camera’s field of view, such as occlusion or changes in orientation, can cause the bounding box around key object parts to be lost. This often results in failure to recognize critical information printed on the parcel.

To address this challenge, we innovatively adopt the SMITE model proposed by DeepMind for image segmentation [22]. Architecturally, the SMITE model employs a diffusion-based segmentation approach that processes video frames sequentially, achieving efficient video segmentation. Additionally, it incorporates a temporal-attention mechanism to maintain label consistency across frames, effectively reducing flicker and noise in the segmentation results. The core principle of diffusion models involves a forward-diffusion process followed by a reverse-denoising process. In the forward diffusion stage, applied to a single image frame (equivalent to each frame in video segmentation), the process can be described as follows:

Let $x_t$ denote the image data at time $t$, $x_0$ represent the original image (the initial frame state), ${\beta }_t$ be a predefined noise scheduling parameter controlling the rate at which noise is added, and $\epsilon ~N(0,1)$ represent standard Gaussian noise. The forward diffusion process can then be expressed as

$$x_t=\sqrt{1-{\beta }_t}{x}_{t-1}+\sqrt{{\beta }_t}\epsilon$$

(7)

By iteratively applying this formulation (with $t-1$ starting from the initial step), the image becomes progressively “corrupted” by noise.

In the reverse denoising process, the model learns a conditional probability distribution $p_{\theta }(x_{t-1}|x_t)$ to restore the image. This distribution is typically modeled as Gaussian, with mean $\mu (x_t,t)$ and variance ${\sigma }_{\theta }^2(x_t,t)$ determined by the model parameter $\theta$. During training, the model optimizes these parameters by minimizing the negative log-likelihood, aiming to make the predicted denoised image as close as possible to the true noise-free image. During inference, the model progressively removes noise from a corrupted image to generate the segmented output (Figure 12). Additionally, the SMITE model introduces a novel tracking-based voting mechanism that enhances segmentation stability and accuracy by tracking pixel positions within the attention maps. This mechanism effectively addresses the bounding-box loss problem, ensuring reliable segmentation accuracy across a wide range of challenging conditions.

2. Temporal-Attention Mechanism-Related Formula Derivation:

Suppose the video sequence contains $T$ frames, and the $i$-th frame produces a feature vector $F_i\in R^d$ after feature extraction, where $d$ represents the feature dimension.

3. Compute attention scores:

To capture inter-frame correlations, the attention score matrix $A\in R^{T\times T}$ is computed, where each element $j$ represents the attention score of frame $j$ with respect to frame $i$. A dot-product attention mechanism can be used, for example:

$$A_{ij}={\displaystyle \frac{\exp (s(F_i,F_j))}{{\displaystyle {\sum }_{k=1}^T\exp (s(F_i,F_k))}}}$$

(8)

where $s(F_i,F_j)=F_i^T{F}_j$ is a function measuring the similarity between feature vectors.

4. Obtain weighted features:

By performing a weighted summation of the features from all frames using the attention scores, a temporally contextualized feature vector is obtained for segmentation, expressed as

$$C_i={\displaystyle \sum _{j=1}^T{A}_{ij}{F}_j}$$

(9)

In this way, the resulting feature vector $C_i$ incorporates information from other frames, helping to ensure label consistency across the entire sequence.

5. Tracking-and-Voting Mechanism-Related Formula Derivation:

Suppose that in the attention map, the position of pixel $p$ in frame $t$ is denoted as $(x_p^t,y_p^t)$, and its corresponding feature vector is $f_p^t$.

6. Position similarity computation:

The position similarity between pixels in adjacent frames is computed, for example, using a Gaussian kernel to measure positional variation. Let ${\sigma }_1$ be the scale parameter controlling position similarity. Then, the position similarity $S_{l,pq}^{t,t+1}$ (representing the similarity between pixel $p$ in frame $t$ and pixel $q$ in frame $t+1$) can be expressed as

$$S_{l,pq}^{t,t+1}=\exp \left(-{\displaystyle \frac{{(x_p^t-x_q^{t+1})}^2+{(y_p^t-y_q^{t+1})}^2}{2{\sigma }_l^2}}\right)$$

(10)

7. Feature similarity computation:

Feature similarity $S_{f,pq}^{t,t+1}$ is computed simultaneously, for example, using cosine similarity. Let $f_p^t$ and $f_q^{t+1}$ denote the feature vectors corresponding to two pixels, then

$$S_{f,pq}^{t,t+1}={\displaystyle \frac{f_p^{tT}{f}_q^{t+1}}{‖f_p^t‖‖f_q^{t+1}‖}}$$

(11)

Voting weight computation: The voting weight $w_{pq}^{t,t+1}$ is obtained by combining position similarity and feature similarity, expressed as

$$w_{pq}^{t,t+1}=S_{l,pq}^{t,t+1}\times S_{f,pq}^{t,t+1}$$

(12)

Final voting score: For pixel $q$ in frame $t+1$, the final voting score $V_q^{t+1}$ is obtained by summing the voting weights from all corresponding pixels $p$ in frame $t$, expressed as

$$V_q^{t+1}={\displaystyle \sum w_{pq}^{t,t+1}}$$

(13)

Based on this voting score, the segmentation label of the pixel can be determined. For example, given a threshold $\tau$, if $V_q^{t+1}>\tau$, the pixel $q$ is classified as part of the target object; otherwise, it is labeled as background.

8. Algorithm Advantages:

Compared to other video segmentation algorithms such as Baseline-I and Grounded SAM 2, the SMITE model can accurately predict and segment objects in video frames using only one or a few reference images. This significantly reduces reliance on large amounts of annotated data, thereby enhancing the flexibility and efficiency of segmentation tasks. Furthermore, through its tracking and temporal voting mechanisms, SMITE ensures temporal consistency of segmentation results throughout the entire video sequence. Even under challenging conditions, such as variations in object appearance, changes in illumination, or shifts in viewpoint, the model maintains stable and precise segmentation, greatly improving the temporal continuity and reliability of the results (Figure 13).

9. Deployment of the SMITE Algorithm:

Environment Preparation: Due to the substantial computational load of the SMITE model, the use of an NVIDIA GPU is recommended. The GPU should provide sufficient memory; processing high-resolution video typically requires at least 8 GB of VRAM or more. As the hardware on the logistics vehicle is insufficient for training and inference, cloud-based training is chosen for the SMITE model. A suitable deep learning framework, such as PyTorch or TensorFlow, should be selected according to the model implementation, with the corresponding versions installed. Additionally, NVIDIA CUDA and cuDNN libraries must be installed. Other essential libraries for data processing and visualization, including NumPy and OpenCV-Python, are also required. Pretrained SMITE model weights should be obtained from the official code repository linked in the corresponding research publication. Video data for segmentation must be collected and preprocessed by adjusting parameters such as resolution and frame rate to meet the model’s input specifications. Reference images should be prepared as per the model’s requirements, serving to guide the segmentation of objects appearing in the video frames.

Video Processing and Segmentation: Video frames are read sequentially using OpenCV-Python. Each frame undergoes preprocessing, such as normalization and resizing, to align with the model’s input requirements. The preprocessed frames are then passed through the SMITE model for segmentation inference, which outputs a segmentation mask for each frame. Finally, post-processing operations, including noise removal and hole filling, are applied to these masks to enhance the overall quality and accuracy of the segmentation results.

5. Materials and Methods

5.1. Experimental Platform and Hardware Configuration

This study is conducted on a custom-modified, ROS-based unmanned vehicle platform that integrates visual recognition, robotic arm control, and path-planning modules. The specific hardware components are listed in

Table 2. Hardware Parameter Table.

Module	Model/Parameter Description
Main control unit	NVIDIA Jetson Xavier NX (ARM architecture, 8 GB RAM)
Camera	Intel RealSense D435 (1920 × 1080, 30 fps)
Robotic arm	Robot Magician (4-axis, repeatability ±0.2 mm)
Laser radar	EAI G1 (360°, measurement range 0.15–12 m)
Drive type	Differential-drive wheel system, maximum speed 0.8 m/s
Battery	24 V lithium battery, approximately 2 h of operation
Communication method	USB3.0, Ethernet, Wi-Fi

.

5.2. Software Environment and Versions

The software environment and library versions used in this study are summarized in

Table 3. Software Environment.

Software/Library	Version	Description of Use
Ubuntu	18.04 LTS	Operating system
ROS	Melodic	Communication and node management
OpenCV	4.5.5	Image preprocessing
PaddleOCR	2.7.0	Text recognition
SMITE	1.0.1	Video image segmentation
Python	3.6.9	Primary programming language
PyTorch	1.12.1	SMITE model inference

Note: The SMITE model is trained in the cloud (NVIDIA A100). Only the inference module (FP16 precision) is deployed on the edge device.

.

5.3. Experimental Dataset and Testing Environment

A total of 120 parcel images of various sizes were collected for the experiments, containing key information such as handwritten or printed addresses, barcodes, and province abbreviations. The dataset included images captured under three lighting conditions: natural light, LED lighting, and low light, and introduced disturbances such as label wrinkling, reflections, and occlusions of up to 30%. Parcel placement angles ranged from 0° to 45°. Province information was manually annotated, resulting in a dataset of 1200 images, which was split into training and testing subsets at an 8:2 ratio.

The experimental site consisted of a 4 m × 6 m indoor open area with six mailboxes arranged along the perimeter, each representing a different province as the sorting destination. Using a combination of overhead spotlights and blackout curtains, tests were conducted under three lighting conditions: normal illumination (300–500 lux), strong illumination (>1000 lux), and low illumination (<100 lux). To simulate real logistics scenarios, parcels were either randomly stacked or laid flat on the sorting platform. The camera height and angle were fixed to ensure each recognition task reflected realistic operating conditions.

At the start of each experiment, the vehicle departed from the starting point and used SLAM to construct an environmental map. It then autonomously navigated to the sorting platform, docking with a positioning error within 2 cm. After the onboard camera captured images of the parcels, SMITE segmentation and PaddleOCR text recognition were performed sequentially. The extracted province information was matched with the corresponding mailbox, and the optimal delivery path was planned. The robotic arm completed the grasping operation, placing each parcel into its assigned mailbox. Finally, the vehicle returned to the starting point to prepare for the next task. Each experimental round processed 10 parcels, and the procedure was repeated for 10 rounds, resulting in a total of 100 sorting tasks.

Figure 14 shows a sample of the recognition results, including extracted text information such as tracking numbers and addresses.

5.4. Evaluation Indicators

The performance evaluation metrics used in this study are summarized in

Table 4. Evaluation Indicators.

Indicator	Definition
Recognition accuracy	Number of parcels correctly recognized by province/Total number of parcels
Sorting success rate	Number of parcels successfully delivered to the correct mailbox/Total number of parcels
Average sorting time	Average time consumed (in seconds) from recognition to completed delivery
Average path-planning time	Average time consumed (in seconds) from completion of recognition to the start of movement
Localization error	Euclidean distance (cm) between the actual docking position and the target point
Failure type statistics	Recognition failure/Grasping failure/Delivery failure/Path-planning failure

.

6. Results and Analysis

6.1. Recognition Accuracy Test

In 100 sorting tasks, 98 parcels were correctly recognized. The recognition accuracy is presented in

Table 5. Parcel Recognition Accuracy Under Different Lighting Conditions.

Lighting Condition	Number of Test Parcels	Number Correctly Recognized	Accuracy
Normal Illumination	40	39	98.5%
Strong-light interference	30	29	96.7%
Low-light environment	30	27	90.0%
Total	100	95	95.0%

Note: Recognition failures mainly occurred on parcels with wrinkled labels, partial occlusion, or strong reflections.

.

6.2. Analysis of Sorting Success Rate and Efficiency

According to the statistical results, the system demonstrates an average sorting time of 18.4 s, with a standard deviation of only 2.1 s. The maximum and minimum sorting times recorded were 24.3 s and 14.9 s, respectively. This narrow fluctuation range, combined with performance significantly surpassing the industry benchmark of 30 s, reflects a stable and efficient operational rhythm. The average path-planning time is 1.2 s, with a standard deviation of 0.3 s; extreme values did not exceed 2.0 s. These results indicate that the combined A* + Dijkstra algorithms provide excellent real-time performance and high consistency under the experimental conditions. The system achieved a sorting success rate of 92%, demonstrating strong overall reliability. The remaining 8% of failures primarily resulted from occlusion-related recognition errors and grasp slippage, issues that could be mitigated through the addition of multi-view cameras or the adoption of force-controlled grippers. Localization accuracy was also impressive, with an average error of 1.6 cm, a standard deviation of 0.4 cm, and a maximum error of 2.3 cm. These values are significantly smaller than the robotic arm’s ±0.2 mm repeatability margin, remaining well within safety tolerances. Consequently, navigation accuracy has virtually no negative impact on subsequent grasping and provides a robust spatial foundation for the system.

Consequently, navigation accuracy has virtually no negative impact on subsequent grasping and provides a robust spatial foundation for the system (

Table 6. System Sorting Efficiency and Localization Accuracy Statistics.

Indicator	Average Value (s)	Standard Deviation	Maximum	Minimum
Average sorting time	18.4 s	2.1 s	24.3 s	14.9 s
Average path-planning time	1.2 s	0.3 s	2.0 s	0.8 s
Sorting success rate	93%	,	,	,
Localization error	1.6 cm	0.4 cm	2.3 cm	0.9 cm

).

6.3. Failure Case Analysis

The seven failures observed during the experiment are summarized in

Table 7. Statistics of Failure Types and Causes.

Failure Type	Number of Cycles	Cause Description
Recognition failure	3	Label wrinkling and strong reflections prevented OCR from identifying the province
Grasping failure	2	Parcel center of mass shifted, causing unstable grasping by the robotic arm
Delivery failure	1	Delivery-angle deviation led to parcel slippage
Path-planning failure	1	LiDAR misdetected an obstacle, causing the path to be interrupted

.

Among the seven failures recorded, recognition failures accounted for three cases, primarily caused by label wrinkling and strong reflections that hindered OCR from accurately reading the province information. Grasping and delivery failures occurred twice each: grasping failures resulted from a shifted parcel center of mass, leading to unstable gripping, while delivery failures were due to angle deviations causing parcel slippage. One path-planning failure occurred when the LiDAR sensor falsely detected an obstacle, interrupting the navigation process. These findings indicate that the recognition stage is most vulnerable to environmental interference, grasping and delivery precision still rely on coordinated optimization between the vision system and robotic arm, and path-planning errors are rare with minimal overall impact.

6.4. Comparison with Traditional Methods

We implemented a traditional-method comparison group using:

• YOLOv5 for object detection.
• Tesseract OCR for text recognition.
• Fixed-coordinate grasping (without visual servoing).

Compared with traditional methods, the proposed approach improves recognition accuracy from 84.0% to 95.0% and increases the sorting success rate from 78.0% to 92.0%. Additionally, the average sorting time is reduced from 26.7 s to 18.4 s, a decrease of 8.3 s, representing an efficiency improvement of approximately 31%. These significant gains across all three metrics validate the effectiveness and efficiency of the SMITE segmentation and PaddleOCR joint strategy in real-world sorting scenarios (

Table 8. Performance Comparison Between the Proposed Method and Traditional Methods (Note: ↑ indicates increase/improvement).

Method	Recognition Accuracy	Sorting Success Rate	Average Sorting Time
Traditional methods	84.0%	78.0%	26.7 s
Proposed method	95.0%	92.0%	18.4 s
Improvement ratio	+11.0%	+14.0%	−8.3 s (↑31%)

).

6.5. Robustness Testing

To comprehensively assess the system’s robustness, a series of extreme testing scenarios was designed, including simulated sudden illumination changes and environments with dense obstacles, to evaluate the vehicle’s stability and task-execution capability. The results demonstrate that the vehicle operated reliably, exhibiting strong adaptability and self-recovery abilities (

Table 9. Robustness Test Results Under Extreme Scenarios.

Test Scenario	Number of Tests	Success Rate	Remarks
Sudden illumination change	10	90%	Lights switched on/off abruptly
Obstacle occlusion	10	80%	Human-induced path blocking
Parcel stacking	10	70%	Parcels stacked in two layers; lower-layer recognition difficult
Continuous operation for 30 min	1	Success	Successfully sorted 98 consecutive parcels without system crash

). For efficiency assessment, extensive data were collected on sorting time, error rate, and system stability. Statistical comparisons between traditional vision algorithms and the optimized vision algorithm clearly show that conventional methods suffer from low efficiency and significant instability [23,24]. After integrating the optimized vision algorithm, both recognition and grasping efficiency of the logistics vehicle improved substantially, thereby enhancing the system’s overall robustness.

7. Discussion

The experimental results demonstrate that the proposed SMITE segmentation and PaddleOCR recognition framework significantly outperforms the traditional YOLO + Tesseract baseline under laboratory conditions. Recognition accuracy and sorting success rate improved by 11 and 14 percentage points, respectively, while the average sorting time decreased by 31%. These performance gains stem primarily from two factors: (1) The temporal attention mechanism integrated into SMITE effectively suppresses noise caused by label wrinkling, reflections, and partial occlusions, substantially increasing the proportion of usable regions in the OCR input images. (2) The Eye-to-Hand visual servoing approach directly maps image-feature deviations to robotic arm velocity commands, eliminating the cumulative errors inherent in the conventional “detection → pose estimation → motion planning” pipeline. Consequently, the total execution time for grasping and delivery operations is reduced by approximately 40%. However, despite these improvements, the achieved 92% sorting success rate remains below the industrial benchmark of >99%. Failures primarily occur in three scenarios: (1) Under low illumination, LiDAR point clouds become sparse, resulting in 1–2 cm of AMCL drift and corresponding deviations in the robotic arm’s base coordinate system. (2) When the parcel’s center of mass deviates more than 2 cm from its geometric center, the two-finger gripper, lacking force feedback, fails to maintain a stable grip, leading to slippage. (3) When over 50% of the province-abbreviation region is occluded, OCR confidence falls below 0.85, preventing activation of the secondary imaging mechanism. These issues highlight limitations in the current system’s perception redundancy and grasping robustness.

Compared with recent advances in warehouse logistics robotics, the 18.4 s single-item sorting time achieved here outperforms the 23.6 s reported in 2024 by The Chinese University of Hong Kong (using an RGB-D + VRP routing pipeline), though it remains slower than the 12 s cycle time of Amazon Robotics’ sixth-generation system (which employs a high-speed Delta arm and full barcode coverage). Notably, the latter relies on global QR-code labeling and a servo conveyor infrastructure, incurring hardware costs approximately seven times higher than the present platform. Therefore, the proposed scheme offers a more cost-effective and modular alternative. The SMITE model contains 113 million parameters, and single-frame inference on the Jetson Xavier NX takes approximately 280 ms, representing a primary source of latency. Replacing the backbone network with a Mobile-SAM or Edge-TPU–compatible EfficientNet-SAM variant could reduce inference latency to under 90 ms, potentially decreasing the overall sorting cycle time to below 15 s.

Future work will focus on three main directions: First, multispectral image fusion will be introduced by incorporating 850 nm near-infrared illumination combined with polarizing filters to suppress reflections while maintaining invisible, non-intrusive lighting for human operators. This enhancement is expected to improve recognition robustness under low-light and high-reflection conditions. Second, a six-axis force/torque sensor will be integrated into the robotic arm’s end-effector to enable an impedance-control strategy, allowing dynamic adjustment of gripping force and posture. This upgrade is projected to reduce slippage caused by center-of-mass shifts from 2% to below 0.5%. Third, a large-scale multimodal dataset comprising 10,000 parcels will be constructed, alongside an online hard-example mining strategy to continually fine-tune the SMITE and OCR models. This approach will allow the system to adapt to seasonal packaging variations (e.g., winter fogging of tape, summer reflective films). With these improvements, the goal is to raise the end-to-end sorting success rate above 99% within two years and achieve 72 h of continuous, human-free operation verification in a real e-commerce warehouse.

8. Conclusions

This study addresses the demand for high-efficiency, low-cost logistics sorting by proposing an ROS-based unmanned vehicle system integrating SMITE video segmentation, PaddleOCR text recognition, Eye-to-Hand visual servoing, and a hybrid A* + Dijkstra path-planning framework. Experiments conducted in a 4 m × 6 m indoor environment with 100 sorting tasks demonstrate an average sorting time of 18.4 s per parcel, recognition accuracy of 95.0%, and a sorting success rate of 92.0%. These results represent improvements of 11 and 14 percentage points in accuracy and success rate, respectively, and a 31% increase in efficiency compared to the traditional YOLO + Tesseract baseline. Localization error averaged 1.6 cm ± 0.4 cm, confirming the feasibility and superiority of the proposed approach under laboratory conditions.

The main contributions of this work are threefold: (1) the novel introduction of SMITE into logistics-oriented video segmentation, where its temporal attention mechanism effectively suppresses noise from label wrinkling, reflections, and occlusions; (2) the design of a lightweight Eye-to-Hand coordination algorithm that directly uses image-feature deviations as control inputs, eliminating the traditional pose-estimation stage and reducing cumulative errors; (3) the development of a complete open-source ROS-based hardware–software framework with hardware costs approximately 70% lower than commercial systems, enabling rapid deployment and facilitating secondary development.

Despite these advances, limitations remain. Recognition accuracy decreases under low-light conditions or when over 50% of the label is occluded. The two-finger gripper, lacking force feedback, struggles with parcels whose centers of mass are shifted, leading to slippage. Additionally, the SMITE model’s large parameter size causes an inference delay of approximately 280 ms at the edge. Future work will focus on incorporating multispectral illumination, integrating a six-axis force-controlled gripper, and lightweighting the model to enhance robustness and real-time performance. A 72 h continuous operation test in a real e-commerce warehouse is planned, aiming to increase the sorting success rate beyond 99% and provide a highly cost-effective autonomous sorting solution for intelligent logistics.