A Structure-Aware Masked Autoencoder for Sparse Character Image Recognition

Abstract

Conventional vehicle character recognition methods often treat detection and recognition as separate processes, resulting in limited feature interaction and potential error propagation. To address this issue, this paper proposes a structure-aware self-supervised Masked Autoencoder (CharSAM-MAE) framework, combined with an independent region extraction preprocessing stage. A YOLOv8n detector is employed solely to crop the region of interest (ROI) from full-frame vehicle images using 50 single bounding-box annotated samples. After cropping, the detector is discarded, and subsequent self-supervised pre-training and recognition are fully executed using MAE without any involvement of YOLO model parameters or labeled data. CharSAM-MAE incorporates a structure-aware masking strategy and a region-weighted reconstruction loss during pre-training to improve both local structural representation and global feature modeling. During fine-tuning, a multi-head attention-enhanced CTC decoder (A-CTC) is applied to mitigate issues such as sparse characters, adhesion, and long-sequence instability. The framework is trained on 13,544 ROI images, with only 5% of labeled data used for supervised fine-tuning. Experimental results demonstrate that the proposed method achieves 99.25% character accuracy, 88.6% sequence accuracy, and 0.85% character error rate, outperforming the PaddleOCR v5 baseline (98.92%, 85.2%, and 1.15%, respectively). These results verify the effectiveness of structure-aware self-supervised learning and highlight the applicability of the proposed method for industrial character recognition with minimal annotation requirements.

1. Introduction

With the rapid advancement of industrial automation and intelligent manufacturing, character recognition technology has found increasingly widespread applications in industrial production, particularly in scenarios such as automatic acquisition of vehicle identification numbers (VINs) and detection of engraved characters [1]. This technology not only supports product traceability and anti-counterfeiting mechanisms but also significantly enhances production quality management and informatization levels. However, character images captured in industrial environments face formidable challenges: on the one hand, character regions typically occupy only a small proportion of the entire image, resulting in high information density and sparse spatial distribution; on the other hand, the background areas are highly variable and cluttered with interferences such as textures, scratches, reflections, and stains. These factors severely constrain the feature extraction and classification accuracy of character recognition algorithms, thereby undermining model robustness, especially in small-batch or customized production settings.

In recent years, self-supervised learning methods based on deep learning have emerged as an effective paradigm for enhancing performance in visual tasks [2]. Among these, the Masked Autoencoder (MAE) captures rich and generalizable global features by randomly masking input images and reconstructing the masked regions, demonstrating superior efficacy in image classification and detection tasks [3]. However, in industrial images featuring sparse and structured characters, the fully random masking strategy is prone to over-masking critical character regions, thereby impairing the model’s learning of structural character features and constraining the performance of downstream recognition tasks.

To address the aforementioned challenges, this study proposes a structure-aware region masking strategy and introduces a region-weighted reconstruction loss function to enhance the model’s self-supervised learning capabilities in character-sparse scenarios [4]. Specifically, leveraging the character bounding box information output by the object detection model (e.g., YOLOv8 [5]), differential masking ratios are applied to character and background regions: a lower masking ratio (e.g., 25%) is adopted for character regions to maximize the preservation of structural features, while a higher masking ratio (e.g., 75%) is applied to background regions to suppress irrelevant interferences. Concurrently, a region-weighted mechanism is incorporated into the reconstruction loss computation, assigning higher weights to character regions (with a penalty factor $\alpha =3$), thereby further strengthening the model’s sensitivity and discriminability toward local character structures.

Following the self-supervised pre-training based on the structure-aware MAE, this study integrates the optimized encoder with a CTC (Connectionist Temporal Classification) decoding module incorporating a fusion attention mechanism [6], enabling end-to-end fine-tuning with minimal annotated samples. This approach substantially reduces manual annotation costs while achieving high-precision character sequence recognition.

The main contributions of this study are as follows:

1.

A structure-aware region masking strategy is proposed to effectively alleviate the over-masking of character regions associated with random masking;

2.

A region-weighted reconstruction loss function is designed to significantly enhance the model’s representational capabilities for critical character regions;

3.

Self-supervised pre-training is integrated with few-shot supervised fine-tuning to construct an efficient vehicle character recognition framework;

4.

Systematic experimental validation is conducted on a self-constructed industrial vehicle character dataset, which encompasses diverse vehicle types, complex backgrounds, and varying imaging conditions.

The results demonstrate that the proposed method maintains high accuracy and robustness in industrial complex scenarios, exhibiting substantial application potential.

2. Related Work

Vehicle character recognition (VCR) plays a crucial role in industrial automation and intelligent manufacturing, particularly for vehicle identification number (VIN) acquisition and quality traceability [7]. Existing studies can mainly be categorized into three methodological directions: CNN-based recognition, Transformer-driven models, and self-supervised representation learning.

2.1. CNN-Based Recognition Methods

Traditional VCR approaches were dominated by handcrafted feature extraction (e.g., edge detection and template matching), which are highly sensitive to illumination variation and background interference [8]. With the development of deep learning, convolutional neural networks (CNNs) enabled end-to-end optimization, such as the CRNN integrated with CTC loss [9], significantly improving recognition performance. Object detection frameworks (e.g., YOLOv8 [10]) have also been introduced for character localization, thus boosting processing speed. However, most CNN-based methods treat detection and recognition as separate tasks, which often leads to insufficient feature interaction and cumulative errors in sparse-character scenarios.

2.2. Transformer-Based and Attention-Driven Methods

Recent advancements in attention mechanisms and Transformer architectures have facilitated improved multi-scale feature fusion, particularly for low-contrast industrial images [11]. These models are capable of long-range dependency modeling, which enhances recognition robustness. Nevertheless, they typically rely on strong semantic priors, which may be less suitable for VIN code recognition where the character structure is fixed, semantics are weak, and background complexity is constrained.

2.3. Self-Supervised Learning and Masked Autoencoders

Self-supervised learning (SSL) has become a prominent paradigm due to its ability to learn transferable representations from unlabeled data [12,13]. The Masked Autoencoder (MAE) [14] achieves scalable SSL by randomly masking image patches and reconstructing them through a lightweight decoder, demonstrating excellent performance in visual pre-training. MAE-based adaptations in OCR tasks, such as MaskOCR and masked text modeling [15,16], provide a unified encoder–decoder framework for enhancing sequential character representation. In challenging scenarios, degradation-invariant autoencoders further improve robustness to noisy text inputs [17,18]. However, random masking often leads to excessive occlusion of critical character regions when applied to character-sparse industrial images, resulting in reconstruction bias and degraded feature consistency.

2.4. Structure-Aware Masking Strategies

To address this issue, structure-aware masking approaches have been proposed to guide SSL toward task-relevant regions. For example, SSTAG selectively masks non-essential regions in text alignment tasks [19], while RoMAE performs region-level masking to better focus on semantic structures [14]. Weighted reconstruction and penalty-based learning strategies have also shown effectiveness in suppressing background interference and improving representation stability [17]. However, existing methods do not explicitly differentiate between character and non-character regions in vehicle character recognition tasks, limiting their applicability in industrial environments.

In addition, Raisi and Zelek [20] investigated character-level occlusion handling for text recognition in natural scene images. Their approach focuses on free-form text with strong semantic dependencies, addressing complex occlusion patterns through robustness-oriented modeling. However, VIN characters in industrial settings exhibit fixed-length spatial structures, weak semantic relevance, and consistent imaging conditions, making direct adaptation of such occlusion-based methods non-trivial without substantial architectural modification.

In summary, while recent deep learning-based methods have significantly improved VCR performance, their generalization capability and annotation efficiency remain limited under real-world industrial conditions. To this end, this study introduces a structure-aware masked autoencoder that integrates region-weighted reconstruction and targeted masking strategies with YOLOv8n-based character localization and A-CTC decoding, providing a practical SSL-based paradigm for industrial character recognition.

3. Materials and Methods

To achieve efficient recognition of vehicle identification number (VIN) characters in industrial environments, this study proposes a self-supervised pre-training framework based on a structure-aware Masked Autoencoder (CharSAM-MAE). The framework exploits unlabeled ROI images for pre-training to learn both global and local structural features inherent to character-sparse industrial imagery, followed by few-shot fine-tuning to achieve end-to-end sequence recognition.

In designing the workflow, the selection of individual modules is guided by the characteristics of VIN recognition rather than the convenience of implementation. VIN codes generally exhibit a fixed structural format, weak semantic dependency, and sparse spatial distribution under complex imaging conditions. Consequently, a lightweight YOLOv8n detector is employed exclusively for region-of-interest extraction, while self-supervised MAE-based modeling is used for representation learning without the use of YOLO-derived parameters.

The overall pipeline—illustrated in Figure 1—includes character region detection, structure-aware masking during pre-training, region-weighted reconstruction loss optimization, and fine-tuning with a CTC decoder enhanced by a multi-head attention mechanism. This design effectively reduces annotation requirements while preserving structural consistency in the reconstructed features.

The suitability of the architectural design is further validated by comparative experiments in Section 4.3 and in-depth ablation studies in Section 4.4, which demonstrate the effectiveness of the structure-aware masking strategy and attention-based decoding under low-sample supervision.

To provide a concise and operational understanding of the complete framework, Algorithm 1 summarizes the end-to-end workflow from ROI extraction to structure-aware self-supervised learning and fine-tuning with attention-enhanced decoding. The following subsections provide detailed descriptions of each component.

Algorithm 1: Overall Training Workflow of the Proposed CharSAM-MAE

3.1. VIN Recognition System

The proposed VIN recognition system has been successfully deployed on an industrial automotive assembly line to enable high-accuracy real-time identification for light and dark vehicles [21]. It comprises high-speed industrial area-scan cameras, edge computing units, and dedicated recognition software, realizing fully automated end-to-end VIN acquisition while substantially reducing manual intervention and improving production traceability.

To address challenging factory conditions—including conveyor-induced motion blur, partial occlusion, varying illumination, and extremely low character contrast on dark body surfaces—images are captured continuously with adaptive exposure control. A lightweight YOLOv8n detector, trained on only 50 annotated full-frame images, performs offline ROI cropping to extract VIN character regions, which are then used exclusively for all subsequent model training and evaluation.

The practical deployment is shown in Figure 2: (a) illustrates the assembly-line environment with fixed camera positions above the moving conveyor, and (b) demonstrates the integrated software interface on industrial displays with real-time monitoring and alarm functions. This solution requires minimal modification to existing production infrastructure, facilitating large-scale industrial adoption [22].

The VIN recognition workflow is outlined in Figure 3, commencing with a trigger signal for image capture: the camera acquires the image, followed by preprocessing (including grayscale conversion and noise suppression), YOLOv8n-based character region detection, CharSAM-MAE feature extraction, and CTC sequence decoding, culminating in the output of recognition results and an end signal. With vehicles arriving at a rate of one per minute, the total process latency is controlled within 10 s, supporting continuous production rates.

To address illumination inconsistency and image degradation caused by vehicle body color variations, an adaptive gain adjustment mechanism was integrated into the imaging system, as shown in Figure 4. The mechanism evaluates scene brightness by monitoring the average grayscale value of incoming frames in real time. When the grayscale value drops below 20, the camera automatically increases the gain parameter from the default 4 to 18.9, while keeping exposure time unchanged to prevent motion-induced blur. This adjustment enhances character contour visibility and local contrast under low-light conditions, thereby facilitating more reliable structural feature learning during MAE pre-training and improving downstream recognition performance.

The software interface of the system, as depicted in Figure 5, integrates functionalities for displaying detection processes, presenting recognition results, monitoring status, and issuing anomaly alerts. The upper section of the interface displays the current captured image alongside corresponding character recognition outcomes, including YOLOv8n-cropped character regions and the VIN sequence output from the CharSAM-MAE model, with annotations for confidence scores and judgment status (OK/NG) [23]. The middle module visualizes judgment results and NG root cause analysis, enabling quality tracking and process retrospection; the bottom status area monitors the real-time heartbeat and connectivity of key system components (industrial camera, I/O card, database, PLC, and production signaling). Through this visualization interface, operators can monitor detection processes in real time, trace anomalous events, and execute device interlocks, thereby ensuring highly reliable operation of the VIN recognition system in industrial settings [24].

3.2. YOLOv8n-Based Character Region Detection

In vehicle character recognition tasks, raw industrial images typically contain extensive irrelevant regions, while the VIN character strip occupies only a minor portion. To eliminate redundant input and improve the efficiency of the subsequent recognition pipeline, a YOLOv8n model is employed for rapid localization and cropping of the character region, which has demonstrated strong performance in small-object detection with high inference speed [25].

As a stable evolution of the YOLO series, YOLOv8n adopts an anchor-free pipeline that reduces redundant bounding box proposals and improves detection efficiency. Its lightweight architecture is well-suited to the compact and fixed spatial characteristics of VIN regions in industrial images, thereby meeting real-time deployment requirements. In this study, the YOLOv8n (nano) variant is trained using 50 raw industrial images, each manually annotated with a single bounding box, following the default Ultralytics training configuration without any further adjustment.

The annotated dataset used to train YOLOv8n is completely independent from the 13,544 cropped character images later used for both the self-supervised pre-training and the fine-tuning stages of CharSAM-MAE. After region extraction, the YOLOv8n model is discarded entirely, and no parameters, predicted features, or detection results are used in subsequent learning as illustrated in Figure 6. Only the cropped character regions serve as the actual input dataset for CharSAM-MAE, ensuring strict data isolation and preventing any potential leakage.

This preprocessing strategy significantly reduces annotation cost while providing clean and focused visual input for downstream modeling, thereby facilitating the integration of structure-aware masking strategies within the MAE framework [26].

3.3. Structure-Aware Masking Strategy for Pretraining

In the pre-training stage for character recognition, self-supervised learning offers a promising solution to alleviate the challenge of limited labeled data. Masked Autoencoders (MAEs), which reconstruct missing image regions by randomly masking patches, have demonstrated strong feature learning capabilities in various visual tasks [27]. However, in vehicle character recognition scenarios, character regions typically occupy only a small portion of the image but carry a high information density and significant recognition value. In contrast, background areas, while pixel-rich, often introduce noise and redundant features. When conventional MAE randomly masks image patches, it is likely that a large portion of the character region is obscured, making it difficult to learn effective structural representations of the characters.

To address this limitation, we propose a structure-aware masking strategy that enables the MAE to retain more informative content in character regions during pre-training, while applying a higher masking ratio to background regions. This strategy leverages the bounding boxes of character regions provided by a YOLOv8n-based detection module, allowing the input image to be segmented into character and background zones. Different masking probabilities are applied accordingly: a lower masking ratio (e.g., 25%) is used for character regions to preserve essential structural details, while a higher ratio (e.g., 75%) is applied to background regions to suppress redundant information and reduce the computational burden of pre-training.

Specifically, the input image is partitioned into a set of patches $P=\{p_1,p_2,\dots ,p_N\}$, which is then divided into a character patch subset $P_{\mathrm{char}}$ and a background patch subset $P_{\mathrm{bg}}$, based on the character region masks provided by the YOLOv8n detection model. The proposed structure-aware masking strategy can be formally expressed as follows:

$$M\left(p\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}p\in P_{\mathrm{char}}\mathrm{and}r<m_{\mathrm{char}}\hfill \\ 1,\hfill & \mathrm{if}p\in P_{\mathrm{bg}}\mathrm{and}r<m_{\mathrm{bg}}\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(1)

Here, $\mathrm{M}\left(p\right)$ indicates whether a patch p is masked (1 for masked, 0 for retained); $r\sim U(0,1)$ denotes a random variable uniformly sampled from the range $[0,1]$; $m_{\mathrm{char}}$ and $m_{\mathrm{bg}}$ represent the masking ratios for the character and background regions, respectively.

This strategy enables the model to focus more effectively on local structural features in the character regions, while suppressing interference from background areas, as illustrated in Figure 7. As a result, it significantly enhances the feature learning performance during the pre-training stage and lays a solid foundation for downstream character recognition tasks.

3.4. Region-Weighted Reconstruction Loss

Building upon the structure-aware masking strategy, this work further introduces a region-weighted reconstruction loss function to better guide the model to focus on the reconstruction quality of character regions during the pre-training process, as illustrated in Figure 8. Traditional MAE frameworks typically adopt the mean squared error (MSE) loss as the reconstruction objective, treating all masked regions equally during optimization.

$${\mathcal{L}}_{\mathrm{MSE}}={\displaystyle \frac{1}{\left|M\right|}}\sum _{i\in M}{\parallel x_i-{\widehat{x}}_i\parallel }^2$$

(2)

However, in industrial scenarios, character images often exhibit a significant regional imbalance: character regions are densely informative and carry high recognition value. In contrast, background regions tend to be redundant and contribute little to the recognition task [28]. If a uniform reconstruction loss is applied across all areas, the model may focus excessively on background details during training, while neglecting the structural features of character regions.

To address this issue, a region-weighted reconstruction loss is introduced, where different weights are assigned to other regions during pixel-level error computation. Specifically, higher weights are applied to reconstruction errors in character regions to emphasize their structural importance and enhance feature learning. This weighting mechanism encourages the model to be more attentive to the reconstruction of character areas, enabling it to learn finer and more discriminative character representations.

$$L=\sum _{i\in \mathrm{char}}\alpha {\left(x_i-{\widehat{x}}_i\right)}^2+\sum _{j\in \mathrm{bg}}\beta {\left(x_j-{\widehat{x}}_j\right)}^2$$

(3)

In the above equation, $x_i$ and ${\widehat{x}}_i$ denote the original and reconstructed values of the i-th pixel, respectively. The coefficients $\alpha$ and $\beta$ represent the reconstruction weights for the character and background regions, respectively, where $\alpha >\beta$. In our experiments, we empirically set $\alpha =3.0$ and $\beta =1.0$ to emphasize the fidelity of character region reconstruction.

By integrating the structure-aware masking strategy, the proposed region-weighted reconstruction loss can effectively enhance the model’s ability to capture character region features during the pre-training stage, thereby providing a more robust feature foundation for downstream character recognition tasks.

3.5. Character Recognition in the Fine-Tuning Stage

After the structure-aware self-supervised pre-training, the MAE encoder is further fine-tuned in a supervised manner to achieve accurate recognition of vehicle character regions. Taking into account challenges such as character adhesion, blurring, and confusion between similar characters, traditional CTC often struggles with alignment issues in long sequences.

To address this, we propose an attention-constrained CTC (A-CTC) decoding structure that integrates a self-attention mechanism to enhance contextual modeling, as illustrated in Figure 9. This design is inspired by recent work that combines attention with CTC to improve sequence alignment and recognition robustness [29]. In this module, the output features from the MAE encoder are first enhanced by a multi-head self-attention mechanism, and then passed to a CTC decoder for final sequence prediction.

In the mathematical formulation of CTC, let the input characteristic sequence be denoted as $X=\{x_1,x_2,\dots ,x_T\}$, and the target character sequence as $Y=\{y_1,y_2,\dots ,y_L\}$. Since the length of the input sequence T is usually much greater than the length of the target sequence L, CTC introduces a blank symbol and allows repetition of characters to generate an alignment path $\pi$ along the time axis. Each path $\pi$ is sampled from the set of all valid alignments ${\mathcal{B}}^{-1}\left(Y\right)$ that can be mapped to the target sequence Y. Accordingly, the CTC-based probability model can be formulated as:

$$p(Y\mid X)=\sum _{\pi \in {\mathcal{B}}^{-1}\left(Y\right)}p(\pi \mid X)$$

(4)

Here, $\mathcal{B}$ denotes the mapping from the alignment path to the target character sequence by collapsing repeated characters and removing blank symbols. The traditional CTC loss is defined as:

$$L_{\mathrm{CTC}}=-logp(Y\mid X)$$

(5)

In the proposed Attention-enhanced CTC, the input feature sequence X is first processed using a multi-head self-attention mechanism to model contextual dependencies.

$$\tilde{X}=\mathrm{MultiHeadAttention}(Q=X,K=X,V=X)$$

(6)

This results in a contextually enriched feature sequence $\tilde{X}$, which is then used to compute the refined CTC loss.

$$L_{A-\mathrm{CTC}}=-log\sum _{\pi \in {\mathcal{B}}^{-1}\left(Y\right)}p(\pi \mid \tilde{X})$$

(7)

The attention mechanism enables the model to assign dynamic weights to different time steps, effectively mitigating the contextual modeling limitations of traditional CTC in long sequence tasks while preserving its flexible alignment property.

The fine-tuning stage consists of the following steps:

1.

Input images containing character regions are cropped using YOLOv8n;

2.

High-level feature sequences are extracted via the structure-aware MAE encoder;

3.

These features are contextually enhanced using a multi-head self-attention module;

4.

The enriched sequence is passed to a CTC decoder for final character prediction;

5.

The CTC loss is computed and backpropagated to optimize the network.

By integrating multi-head self-attention into the CTC framework, the model maintains strong alignment capability while significantly improving contextual understanding of long character sequences, leading to enhanced robustness in vehicle character recognition.

4. Experiments

4.1. Data Preprocessing

The original vehicle character dataset contained 7967 images. To enhance model robustness and increase training diversity, data augmentation was performed by randomly selecting 30% of the images for $\pm 20$% linear brightness adjustment and 10% for mild Gaussian blur, simulating illumination variation and defocus effects. After augmentation, the dataset was expanded to 13,544 images.

Before augmentation, all images were preprocessed using a YOLOv8n cropping stage to remove irrelevant background and retain only the VIN character region. Only these cropped ROI images were used for both pre-training and fine-tuning of the CharSAM-MAE model. It should be emphasized that the 50 raw annotated images used to train YOLOv8n were not included in this dataset and were discarded immediately after training the detector, ensuring no interaction between YOLO-based preprocessing and the subsequent self-supervised learning process.

Two distinct data partitions were used during model training. In the self-supervised pre-training stage, all 13,544 unlabeled ROI images were utilized to train the structure-aware MAE for generalizable representation learning. During the fine-tuning stage, only 5% of the original labeled data (approximately 400 images) were employed for supervised adaptation [30], further divided into training (70%), validation (20%), and test (10%) subsets.This strategy significantly reduces dependence on labeled data while maintaining strong generalization capability, as summarized in

Table 1. Comparison of Label Requirements.

Scheme	Total Samples	Labeled Samples	Training Set	Validation Set	Test Set
Conventional supervised	7967	7967	5577	1593	797
Proposed method	13,544	150	105	30	15
Annotation workload reduced compared to conventional scheme					94.9%

.

To enhance the quality of vehicle character images and improve recognition performance, a multi-stage image preprocessing pipeline was designed. First, the original RGB images were converted to grayscale and normalized to ensure uniform pixel distribution. Then, contrast-limited adaptive histogram equalization (CLAHE) was applied at multiple scales (8 × 8, 16 × 16, and 32 × 32 grid sizes) to simultaneously enhance local character details and global contrast. The enhanced results from different scales were fused with weighted averaging, followed by bilateral filtering to preserve edges while smoothing. In addition, a combination of detail enhancement and non-local means denoising was employed to suppress noise and highlight character contours and textures. This preprocessing significantly improved the clarity and signal-to-noise ratio of character regions, providing a solid foundation for downstream deep feature extraction, The full preprocessing pipeline is illustrated in Figure 10.

Given that most raw images had a resolution of 1300 × 300, black padding was applied to maintain the original structure while resizing them to a square format. All images were finally rescaled to 256 × 256 pixels to standardize input dimensions, ensuring centered character regions without distortion and improving training efficiency.

4.2. Experimental Setup

The proposed method first performs self-supervised pretraining on the full unlabeled dataset using the structure-aware MAE to learn key structural features of vehicle characters. Then, 15% of the labeled samples are used for supervised fine-tuning, with a 70%-20%-10% split for training, validation, and testing, respectively. During fine-tuning, high-level features are extracted via the pretrained MAE encoder, further enhanced by a multi-head self-attention module, and decoded by an attention-integrated CTC decoder for character sequence prediction. The model is optimized using the AdamW optimizer, with class weights incorporated into the loss function to address class imbalance.

Model performance is evaluated using multiple metrics: character-level accuracy (CharAcc), character error rate (CER), and sequence-level accuracy (SeqAcc), providing a comprehensive assessment of recognition quality. The corresponding metric formulas are defined as follows:

$$\mathrm{CharAcc}={\displaystyle \frac{N_{\mathrm{correct}}}{N_{\mathrm{total}}}}\times 100\%$$

(8)

$$\mathrm{CER}={\displaystyle \frac{N_{\mathrm{substitutions}}+N_{\mathrm{insertions}}+N_{\mathrm{deletions}}}{N_{\mathrm{total}}}}\times 100\%$$

(9)

$$\mathrm{SeqAcc}={\displaystyle \frac{N_{\mathrm{correct}\mathrm{sequences}}}{N_{\mathrm{total}\mathrm{sequences}}}}\times 100\%$$

(10)

where $N_{\mathrm{correct}}$ denotes the number of correctly predicted characters, $N_{\mathrm{total}}$ represents the total number of characters, $N_{\mathrm{substitutions}}$, $N_{\mathrm{insertions}}$, and $N_{\mathrm{deletions}}$ indicate the number of character substitutions, insertions, and deletions, respectively, and L denotes the length of the character sequence.

Considering the practical industrial production workflow, the VIN acquisition cycle is approximately 60 s per vehicle, during which image capturing and manufacturing data synchronization dominate total processing time. In contrast, the inference stage of the proposed method accounts for only a negligible fraction of this period and does not impose any delay on the overall pipeline. Therefore, model efficiency is not a limiting factor for deployment, and the recognition module fully meets real-time operational requirements under factory conditions.

4.3. Comparison with Baselines

To validate the effectiveness of CharSAM-MAE in the VIN code recognition task, we conducted two groups of comparative experiments: one comparing the performance of different models on VIN recognition, and another evaluating the impact of data augmentation on model performance.

As shown in the experimental results, CharSAM-MAE clearly outperforms both CRNN and PaddleOCR v5 in terms of CharAcc and SeqAcc.Specifically, CharSAM-MAE achieves a CharAcc of 99.25%, compared to 98.61% for CRNN and 98.92% for PaddleOCR v5 [31]. The improvement in SeqAcc is even more pronounced, reaching 88.6%, while CRNN and PaddleOCR v5 obtain 79.6% and 85.2%, respectively. Regarding the CER, CharSAM-MAE records the lowest value at 0.85%, further confirming its superior recognition capability.

The results demonstrate that CharSAM-MAE exhibits a clear advantage in recognizing complete VIN sequences. The significant improvement in SeqAcc indicates that the structure-aware masking strategy effectively mitigates cumulative errors caused by sparse character sequences, enabling more reliable recognition in industrial continuous-shooting scenarios. To ensure that these improvements are not due to random fluctuations, all experiments were repeated under identical deployment settings, and the variance of CharAcc, SeqAcc, and CER across repeated evaluations remained within ±0.05%. This level of stability is considerably below the performance gains observed in

Table 2. VIN Recognition Performance Across Different Models.

Model	Pre-Training Loss	CharAcc (%)	CER (%)	SeqAcc (%)
CRNN (ResNet34)	0.0238	98.61	1.41	79.6
ViT-Base	0.0187	98.72	1.28	82.3
PaddleOCR v5	0.0116	98.92	1.15	85.2
MAE (original)	0.0138	98.80	1.20	83.7
PARSeq	0.0080	99.02	0.98	87.0
ABINet	0.0074	99.05	0.96	87.4
Ours	0.0047	99.25	0.85	88.6

, confirming the robustness and statistical reliability of the proposed method.

4.4. Ablation Study

To comprehensively assess the contribution of each component within the proposed CharSAM-MAE framework to VIN recognition, multiple groups of ablation experiments were performed. In addition, training convergence behavior was evaluated using loss and accuracy curve analysis.

Furthermore, to investigate the influence of contrast-enhanced preprocessing on recognition robustness, the original VIN images were processed using contrast enhancement and subsequently evaluated on CharSAM-MAE without modifying any other parameters. As shown in

Table 3. Impact of Contrast Enhancement on VIN Recognition.

Case	CharAcc (%)	CER (%)	SeqAcc (%)
w/o enhancement	87.46	12.54	11.72
w/ contrast enhancement	99.25	0.85	88.3

, this preprocessing step led to a substantial improvement in recognition performance: character accuracy increased from 87.46% to 99.25%, sequence accuracy rose from 11.72% to 88.3%, and CER was reduced from 12.54% to 0.85%. These results indicate that contrast enhancement effectively strengthens visual feature extraction on sparse character input, particularly for complete VIN sequence recognition.

As shown in

Table 4. Effect of Proposed Modules on VIN Recognition Performance.

Configuration	Pre-Training Loss	CharAcc (%)	CER (%)	SeqAcc (%)
MAE (origin)	0.0138	98.80	1.20	83.7
w/o Structure-Aware Masking	0.0096	99.06	1.05	86.1
w/o Region-Weighted Loss	0.0089	99.02	1.01	86.4
($\alpha =3$)
Ours	0.0047	99.25	0.85	88.6

, both the Structure-Aware Masking strategy and the Region-Weighted Reconstruction Loss independently contribute to performance improvement over the baseline MAE configuration, increasing the sequence accuracy to 86.1% and 86.4%, respectively. When the two components are jointly applied, the sequence accuracy further improved to 88.6%, indicating a clear synergistic enhancement rather than a simple additive effect. This confirms that the observed performance gain arises from the intrinsic effectiveness of the proposed architectural design, rather than from model scaling or hyperparameter tuning alone.

To further quantify the effectiveness of the proposed attention-enhanced decoding scheme, an additional comparison was conducted between the standard CTC and A-CTC decoders while all other components were kept unchanged. As summarized in

Table 5. Comparison Between Standard CTC and Attention-enhanced CTC (A-CTC).

Decoder	CharAcc (%)	CER (%)	SeqAcc (%)
Standard CTC	99.11	1.03	86.9
A-CTC (Ours)	99.25	0.85	88.6

, the introduction of multi-head self-attention leads to a consistent improvement in recognition performance, particularly on long VIN sequences.

This confirms that the attention mechanism effectively improves contextual dependency modeling while maintaining the alignment flexibility of the CTC paradigm.

The experiments also examined the effects of model scale, character retention ratio, and loss penalty strategy, as illustrated in Figure 11. Specifically, CharSAM-MAE was tested in three configurations—base, large, and huge—with parameter sizes of 110.92 M, 328.35 M, and 655.90 M, respectively. Results show that the model capacity increases and the recognition performance consistently improves: the base model achieves a CharAcc of 99.25% and a SeqAcc of 88.6%, the large model reaches 99.43% and 91.2%, and the huge model further attains 99.52% and 92.6%.

It can be observed that enlarging the model scale improves recognition accuracy but also leads to a substantial increase in parameters, requiring a trade-off between performance and computational cost for industrial deployment.

To examine the adaptability of the structure-aware masking strategy to sparse character images, different character retention ratios were explored during pre-training, as shown in

Table 6. Effect of Character Retention Ratio on Pretraining and Recognition.

Character Retention Ratio	Pre-Training Loss	CharAcc (%)	CER (%)	SeqAcc (%)
30%/70%	0.0064	98.90	0.85	88.2
25%/75%	0.0047	99.25	0.85	88.6
20%/80%	0.0083	98.84	0.87	88.1
15%/85%	0.0115	98.15	0.91	87.0

. The results show that a 25%/75% retention ratio achieves the best balance, giving a CharAcc of 99.25% and a SeqAcc of 88.6%. Higher or lower ratios cause a decrease in SeqAcc—for example, 30%/70% results in 88.2%, while 15%/85% drops to 87.0%.

This experiment investigates the impact of different loss penalty schemes (w/o penalty, $\alpha$ = 2, $\alpha$ = 3, $\alpha$ = 4) on model performance, as shown in

Table 7. Effect of Penalty Schemes on Pretraining and Recognition.

Penalty Scheme	Pre-Training Loss	CharAcc (%)	CER (%)	SeqAcc (%)
w/o penalty	0.0054	98.80	1.10	82.6
w/loss $\times 2$	0.0049	99.08	0.92	85.8
w/loss $\times 3$	0.0047	99.25	0.85	88.6
w/loss $\times 4$	0.0046	99.17	0.81	87.9

. The results indicate that the introduction of a penalty factor of $\alpha$ = 3 yields the best results, achieving a CharAcc of 99.25% and a SeqAcc of 88.6%, significantly outperforming the no-penalty setting (SeqAcc 82.6%). Meanwhile, the CER decreases from 1.10% to 0.85%, demonstrating the effectiveness of the penalty strategy in improving the robustness of the recognition.

Comprehensive comparison and ablation experiments demonstrate that the proposed CharSAM-MAE consistently outperforms existing industrially deployable approaches for VIN character recognition, exhibiting superior robustness under low-contrast conditions and in the presence of complex imaging artifacts. The ablation findings further confirm that both the character-region retention ratio and the region-weighted reconstruction strategy are essential to improve the quality of the feature reconstruction and maintain sequence integrity.

To complement the quantitative analysis, a qualitative comparison is provided to visually illustrate the effectiveness of the proposed modules. As shown in Figure 12, both PaddleOCR v5 and the original MAE framework suffer character-level recognition errors (highlighted in red), particularly in cases involving weak contrast or character adhesion. In contrast, CharSAM-MAE demonstrates improved structural consistency and significantly reduces misrecognition across all illustrated samples.

It should be noted that, in the third example, all models—including ours—produce duplicated leading characters. This behavior is attributed to motion-induced streaking and optical defocus in the source image, which distorts structural boundaries and impairs separability between adjacent characters at the feature level. Although such degradation lies beyond the modeling scope of self-supervised pre-training, CharSAM-MAE still produces results that are most closely aligned with the ground truth. This observation indicates that the proposed structure-aware masking strategy and the attention-enhanced decoding mechanism substantially improve resilience against extreme imaging degradation, while the remaining inconsistency mainly originates from inherent limitations in the acquisition process rather than from the recognition architecture itself.

5. Conclusions

This study proposes CharSAM-MAE, a structure-aware self-supervised pre-training framework tailored for industrial VIN character recognition. By integrating a masking strategy guided by character-region structure and a region-weighted reconstruction loss, the method effectively enhances feature representation in sparse-character scenarios while maintaining sequence consistency. Unlike conventional OCR-based solutions, which heavily rely on large-scale manual annotations, the proposed approach substantially reduces labeling cost by fully utilizing unlabeled ROI data during pre-training and requiring only 5% supervised fine-tuning samples.

Experimental results confirm that CharSAM-MAE consistently outperforms existing deployable methods under low-contrast imaging and background interference. Ablation studies further validate the individual contributions of both the structure-aware masking and region-weighted reconstruction modules, as well as the effectiveness of the attention-enhanced CTC decoder. Qualitative analysis demonstrates that the method is particularly robust against character adhesion and structural distortion, achieving superior sequence accuracy compared to representative industrial and MAE-based baselines.

However, in extreme cases involving severe motion-induced streaking or optical defocus, the recognition performance of all models, including CharSAM-MAE, may still be affected due to the physical loss of feature separability in the original image. Although such degradation lies beyond the learnable scope of the recognition architecture, the proposed method generates the closest output to ground truth among all the approaches tested, highlighting its enhanced resilience under challenging acquisition conditions.

In summary, CharSAM-MAE provides both theoretical and practical contributions for industrial character recognition, offering a low-annotation-cost and highly robust framework suitable for real-world deployment. Future work will explore extending the method to real-time edge computing, as well as investigating degradation-aware self-supervised strategies to improve robustness under extreme visual perturbations further and broaden cross-scenario generalization capability.