A Deep Learning Framework of Super Resolution for License Plate Recognition in Surveillance System

Abstract

Recognizing low-resolution license plates from real-world scenes remains a challenging task. While deep learning-based super-resolution methods have been widely applied, most existing datasets rely on artificially degraded images, and common quality metrics poorly correlate with OCR accuracy. We construct a new paired low- and high-resolution license plate dataset from dashcam videos and propose a specialized super-resolution framework for license plate recognition. Only low-resolution images with OCR accuracy ≥5 are used to ensure sufficient feature information for effective perceptual learning. We analyze existing loss functions and introduce two novel perceptual losses—one CNN-based and one Transformer-based. Our approach improves recognition performance, achieving an average OCR accuracy of 85.14%.

1. Introduction

In real-world scenarios, police rely on license plate recognition (LPR) [1,2] for crime investigations, such as identifying hit-and-run vehicles, tracking suspect cars, and other law enforcement applications. However, LPR accuracy is often compromised by motion blur and low-resolution images resulting from fast-moving vehicles [3]. To overcome these challenges, super-resolution techniques serve as a vital image preprocessing step, enhancing the clarity of license plate images and improving recognition performance. For instance, in hit-and-run accidents [4], LPR can be used to identify fleeing vehicles by extracting license plate details from traffic or security camera footage, even when the image is affected by degraded motion blur [5], poor lighting conditions [6] (low light, shadows, or overexposure), low camera resolution [7], oblique viewing angles [8], and weather conditions (rain, fog, or snow) [9]. Similarly, in suspect vehicle tracking [10], police surveillance systems equipped with LPR can cross-check captured license plates against criminal databases, enabling real-time alerts and swift action when a suspect car is detected in monitored areas in Figure 1.

Despite these advancements, current public datasets for license plate recognition primarily consist of synthetic low-resolution images with unrealistic noise, limiting their effectiveness in real-world applications. Additionally, the evaluation of super-resolution models for LPR remains constrained by traditional image quality assessment metrics, such as the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) [11,12], which may not fully capture the impact of image enhancement on recognition accuracy.

To address these limitations, we propose a framework that leverages real-world black box video footage to construct a more realistic dataset. Specifically, we capture high-resolution images (I^HR) from near-distance recordings and low-resolution images (I^LR) from far-distance recordings, forming paired training data with license plate characters as ground truth labels (Figure 2). To ensure sufficient feature information for effective perceptual learning, only low-resolution images with OCR accuracy ≥5 are used.

Using this dataset, the super-resolution model SwinFIR [13] is trained to enhance low-resolution images by leveraging a combination of pixel-wise Mean Squared Error (MSE) loss and perceptual losses [14]. This approach preserves both structural details and visual fidelity during backpropagation. The restored high-resolution images are then processed by a pretrained CRNN-based OCR recognizer [15], improving license plate character extraction accuracy.

To train the super-resolution model for reconstructing license plate characters, five types of perceptual losses are utilized: multi-task MSE loss for OCR [16], Deep Image Structure and Texture Similarity (DISTS) [17] loss with VGG16, VGG loss with VGG19 [18], loss with Swin Transformer [19], and loss with a Convolutional Recurrent Neural Network (CRNN) [20], along with ensemble models combining these losses. Training with a single perceptual loss improved character recognition accuracy from 75.75% to 82.57% (a 7% increase). Furthermore, the ensemble of Swin Transformer and DISTS perceptual losses further boosted accuracy to 85.14%, achieving a 9.75% improvement.

This study makes four key contributions: 1. The framework of the surveillance system with an extendable dataset and license plate super resolution and license plate detection in Figure 3. 2. Introduction of a novel LR-HR paired license plate dataset sourced from real-world driving scenes, which can be shared for further research purposes. 3. Detailed analysis of various loss function performances in the super-resolution task tailored to improve license plate text recognition outcomes. 4. Demonstration of the significant enhancement in the overall performance of visualization and text recognition results on license plates through the effective combination of different loss functions.

2. Related Work

2.1. License Plates Dataset

There are various public license plate datasets, including LSV-LP [21], CCPD [22], SSIG-SegPlates [23], UFPR-ALPR [24], and RodoSol [25], each designed for different applications in license plate recognition (LPR) and automatic license plate recognition (ALPR). These datasets vary in terms of image quantity, geographic region, distance variability, tilt angle, motion blur, illumination conditions, and resolution.

In

Table 1. Comparison of public license plate datasets.

Dataset	Number of Images	Country	Distance Variability	Tilt Angle Variability	Blur	Illumination Variability	Resolution
LSV-LP [21]	1,175,390	China	High	High	Yes	Yes	1920 × 1080
CCPD [22]	290,000	China	Medium	Medium	Yes	Yes	720 × 1160
SSIG-SegPlates [23]	2000	Brazil	Low	Low	No	Limited	1920 × 1080
UFPR-ALPR [24]	4500	Brazil	Medium	Low	Yes	Yes	1920 × 1080
RodoSol [25]	20,000	Brazil	Medium	Medium	Yes	Yes	1920 × 1080

, we compare these datasets based on key attributes, such as the number of images, the country of origin, variability in distance and tilt angles, the presence of motion blur, illumination conditions, and image resolution. The LSV-LP dataset stands out due to its large dataset size, high variability in camera angles and distances, and challenging real-world conditions, making it a strong benchmark for super-resolution and LPR models.

From the comparison, we can see that LSV-LP provides the largest number of images and the most diverse range of real-world conditions, making it well-suited for evaluating deep learning-based super-resolution and LPR models. Unlike some datasets that rely on synthetically generated distortions, LSV-LP captures real-world variations in terms of distance, angle, and motion blur, which are crucial for practical deployment.

• Distance variability: The dataset includes images captured from close range (e.g., toll booths) to long-range surveillance (e.g., highway cameras at 50+ meters).
• Tilt angle variability: Plates are captured from various viewpoints, including frontal, oblique, extreme tilt angles, and even partially occluded perspectives due to real-world road conditions.
• Motion blur: Unlike artificially blurred datasets, LSV-LP contains natural motion blur caused by high-speed vehicles, making it a realistic benchmark for LPR models operating under dynamic conditions.

These characteristics make LSV-LP particularly suitable for training and evaluating super-resolution techniques, ensuring that models are robust to real-world distortions encountered in traffic monitoring, law enforcement, and intelligent transportation systems.

2.2. License Plate Recognition with OCR Recognizer

License Plate Recognition (LPR) using Optical Character Recognition (OCR) heavily relies on high-resolution images, as they are crucial for accurate character recognition. Multi-task OCR models [26,27], which address tasks such as plate detection, character recognition, and plate classification, benefit significantly from high-resolution images. These images allow the models to capture finer details, leading to the better localization and recognition of characters, which is essential for LPR. Such models typically use shared CNN backbones for feature extraction, followed by task-specific branches for each task. The increased image resolution helps in extracting more precise features, enhancing the performance of both detection and recognition tasks. However, training these models requires substantial computational resources due to their complexity. PaddleOCR [28] is particularly effective with high-resolution images, as it can better handle multilingual and multi-oriented text, improving its overall performance in license plate recognition. Furthermore, CRNN [29] models are specifically designed for recognizing characters in sequences, making them ideal for OCR tasks. The combination of convolutional layers for feature extraction and recurrent layers for sequence modeling allows CRNNs to recognize text more accurately, especially when trained on high-resolution images that provide more detailed character information. In conclusion, OCR recognizers, including Multi-task OCR, PaddleOCR, and CRNN-based models, require high-resolution images for optimal character recognition, as they capture the necessary details for accurate LPR performance.

2.3. Licenese Plate Restoration with Super Resolution (SR)

Super-resolution techniques [30,31,32] strive to enhance the quality of images by generating higher-resolution versions from lower-resolution inputs. They employ both traditional methods, such as bilinear or bicubic interpolation, and more recent deep-learning approaches. Traditional techniques employ mathematical formulas to estimate pixel values between neighboring pixels with similar colors, whereas deep learning treats super resolution as a regression problem. In this paradigm, a deep neural network is trained to forecast high-resolution images from low-resolution inputs by learning from pairs of low-resolution (LR) and high-resolution (HR) images. The network endeavors to minimize the disparity between its predictions and the actual high-resolution images.

2.4. Perceptual Loss for Super Resolution

Perceptual loss is a technique used in image super resolution that focuses on high-level features rather than pixel-wise differences like MSE or MAE. Traditional pixel-wise loss functions often fail to capture perceptual quality, leading to blurry or unrealistic results. Perceptual loss compares feature maps extracted from pre-trained neural networks, such as VGG, which are capable of capturing complex structures and semantic content [28,29]. This helps maintain important textures and structures, improving the visual quality of the generated image.

Perceptual loss is often combined with other loss functions like pixel-wise loss to balance feature alignment and pixel accuracy. By using models like VGG, it ensures the generated image is perceptually closer to the ground truth [30].

In addition to traditional methods, advanced techniques like a CNN OCR model of multi-task MSE loss [16,26], Deep Image Structure and Texture Similarity (DISTS) loss [17] with VGG16 [33], VGG loss with VGG19 [33], loss with Swin Transformer [19], and loss with Convolutional Recurrent Neural Network (CRNN) [34] can also be used as perceptual loss functions. These methods help in preserving both high-level content and fine details across different image features.

• The multi-task MSE loss function supports any OCR model for LPR, allowing the seamless integration of new models. The approach follows the multi-task model proposed by Gonçalves et al. [26] for its efficiency. By combining MSE and L1 loss, it balances structural preservation and error minimization—MSE maintains image integrity, while L1 enhances robustness and edge sharpness.
• DISTS combines structure and texture similarity, preserving both high-level content and fine details.
• Swin Transformer captures long-range dependencies, offering a strong perceptual loss by maintaining high-level semantics and global image structure.
• CRNN loss is used for text recognition tasks, helping the model align with the structural features of characters, ensuring accurate reconstruction and recognition of textual information.

3. Methodology

3.1. Overview

Figure 4 illustrates the workflow of the SwinFIR super-resolution (SR) model training using a perceptual loss function. The process begins with a low-resolution input image ($I^{LR}$), which is fed into the SwinFIR model to generate a super-resolution output ($I^{SR}$). To ensure accurate training, the actual high-resolution image ($I^{HR}$) is used as a ground-truth reference, and the low-resolution image ($I^{LR})$ is captured from a video frame with a cropped license plate region, and only frames with an OCR accuracy of ≥5 are used, ensuring the presence of sufficient feature information for effective perceptual feature extraction.

The perceptual loss function plays a crucial role in optimizing the model’s performance. Both $I^{SR}$ and $I^{HR}$ are processed through a pretrained network to extract high-level features, which are then compared using a loss computation module. This computed perceptual loss guides the training process, encouraging the model to generate high-quality images that closely resemble the ground truth. Unlike traditional pixel-wise losses such as mean squared error (MSE) or L1 loss, perceptual loss enhances image reconstruction quality by leveraging multi-level feature comparisons, leading to more visually realistic results.

3.2. Dataset Generation

• China license plate dataset of LSV-LP [21] with seven characters.
• Data with relative motion of: move vs. move, static vs. move.
• Data pair of HR/LR images:

  ○

  HR image: Frame of near distance.

  ○

  LR image: Frame of far distance of image with OCR recognizer accuracy ≥5.

• Total images: 783 pairs of LR/HR.
• Rescale resolution: 110 × 40.
• Train, validation, test split: 7, 2, 1.

Examples of image pair in Figure 5:

3.3. Aablation Study on PixelShuffle Three-Fold Attention Module

In this study, we evaluate the performance of the PixelShuffle Three-Fold Attention module proposed by Nascimento et al. (2023) [16] for license plate super resolution. The model applies artificial Gaussian noise to simulate low-resolution images, aiming to improve robustness to real-world distortions. It uses a combination of MSE and perceptual loss to balance pixel-level accuracy and visual quality, both of which are important for OCR performance. We conducted an ablation study using our own dataset and assessed OCR accuracy at different match levels: exact (7/7), near (6/7), and partial (5/7).

However, the results reveal two main limitations: (1) the model struggles with real-world noise, as synthetic Gaussian noise fails to capture real degradation patterns, and (2) the loss function focuses on multi-task objectives (e.g., detection, segmentation) rather than OCR-specific perceptual fidelity. This leads to low exact-match OCR accuracy (3.85%) and only moderate partial recognition (67.95%).

To address these issues, we adopt SwinFIR, a more advanced architecture for image restoration, and design a new OCR-oriented perceptual loss tailored specifically to enhance character-level details in license plate images. This approach aims to better bridge the gap between visual super-resolution and accurate text recognition.

3.4. Super Resolution Model of SwinFIR

The SwinFIR [13] super-resolution (SR) model shown in Figure 6 is specifically designed to enhance the quality of low-resolution license plate images, improving their clarity for Optical Character Recognition (OCR) tasks. SwinFIR adopts a hierarchical architecture based on the Swin Transformer, which captures both local textures and global structural information through shifted window self-attention. It integrates convolutional layers for fine-grained detail enhancement and leverages feature interaction across multiple scales to preserve edge sharpness and character structure—crucial for license plate readability.

As illustrated in the pipeline in Figure 7, the process begins with a low-resolution input image (I^LR), which passes through the SwinFIR network to produce a super-resolved output (I^SR). During training, the model uses a perceptual loss function to guide reconstruction, comparing the ISR to the high-resolution ground-truth image (I^HR). This loss function incorporates deep feature similarities to better retain semantic content and texture consistency, ensuring the SR output is visually coherent and OCR-friendly.

3.5. Perceptual Loss Calculation

Figure 8 illustrates the super-resolution framework for License Plate Recognition (LPR) using a perceptual loss function. The framework takes a high-resolution ground truth image (I^HR) and a super-resolved image (I^SR) as inputs and feeds them into a loss network composed of multiple pretrained models, including convolutional neural networks [35], including VGG-16, VGG-19, a transformer-based Swin transformer, and a recurrent neural network (CRNN). These networks extract deep feature representations from both images, which are then compared to compute the perceptual loss. The final loss is a combination of several components, including Multi-task-MSE, DISTS loss, VGG loss, Swin Transformer loss, and CRNN loss. This perceptual loss guides the super-resolution model to produce visually accurate and semantically meaningful reconstructions, thereby enhancing the performance of license plate recognition tasks.

The original SwinFIR model used Charbonnier loss [34], a smooth variant of L1 loss, for color image reconstruction. However, per-pixel losses failed to capture perceptual differences, leading to unsatisfactory results. To improve performance, we adopted perceptual loss, which leverages high-level features from a pre-trained network (

Table 2. The perceptual loss calculated from different pretrained networks for SwinFIR model training.

Network Architecture	Layers for Feature Extraction	Losses	Network Weight from Pretrained Dataset
CNN [36]	last layer	Multi-task MSE loss [26] (MSE + perceptual loss)	SSIG-ALPR [26]
VGG-16 [13]	{conv1_2, conv2_2, conv3_3, conv4_3, conv5_3}	DISTS loss [16] (Structural loss + Texture loss)	KADID-10k [37]
VGG-19	{conv1_2, conv2_2, conv3_4, conv4_4, conv5_4}	VGG loss [17] (Perceptual loss + style loss)	ImageNet-10k [38]
Swin Transformer [13]	{layers.0.blocks.1,layers.1.blocks.1, layers.2.blocks.5,layers.3.blocks.1}	SwinT loss [19] (proposed) (Perceptual loss + style loss)	ImageNet-1k [39]
CRNN [14]	{conv3_2, conv4_1, conv4_2}	CRNN loss [40] (proposed) (Perceptual loss)	CCPD [20] and CRPD [22]

) to better align with human visual perception. This approach enhanced the super-resolution training of SwinFIR.

3.5.1. Multi-Task MSE Loss

Figure 8 illustrates the Multi-Task MSE Loss computation described in the Appendix A.1 process for training the SwinFIR super-resolution model with a focus on license plate character recognition. The framework consists of the following key components in Figure 9.

By combining both terms, the Multi-Task MSE Loss balances pixel-level reconstruction accuracy with OCR consistency, making it particularly effective for license plate super resolution [41] and character recognition enhancement [42].

3.5.2. Deep Image Structure and Texture Similarity (DISTS) Loss

The DISTS loss [17] is calculated with Equation (A2) of Appendix A.2 for the summation of the texture similarity difference and structural similarity difference of the LR and HR images. The network extracted the feature outputs of specific layers in VGG16 [13], which are {conv1_2, conv2_2, conv3_3, conv4_3, and conv5_3} with weights {0.1, 0, 1, 1, 1, 1}, and ${\mathrm{c}}_1$ = 10⁻⁶, ${\mathrm{c}}_1$ = 10⁻⁶, where ${\tilde{\mathrm{x}}}_{\mathrm{j}}^{\left(\mathrm{i}\right)}$ denotes that the j-th feature map of the i-th convolutional layer of the super-resolved image, and ${\tilde{\mathrm{y}}}_{\mathrm{j}}^{\left(\mathrm{i}\right)}$ denotes that the j-th feature map of the i-th convolutional layer of the ground-truth image. ${\mu }_{{\tilde{x}}_j}^{\left(i\right)}, {\mu }_{{\tilde{y}}_j}^{\left(i\right)}, {\sigma }_{{\tilde{x}}_j}^{\left(i\right)}, {\sigma }_{{\tilde{y}}_j}^{\left(i\right)}$ represent the global means and variance of ${\tilde{x}}_j^{\left(i\right)}$ and ${\tilde{y}}_j^{\left(i\right)}$.

The extracted perceptual information of the LR and HR images is extracted from the pretrained VGG16 [13] on KADID-10k [40], as shown in Figure 10, as perceptual loss is used as a loss for back propagation to the SwinFIR SR model.

3.5.3. VGG Loss

The VGG loss [18] is a summation of perceptual loss and style loss from the pretrained VGG-19 network on the ImageNet-10k. The perceptual loss is extracted from the feature map from {conv1_2, conv2_2, conv3_4, conv4_4,conv5_4}, with weights {0.1, 0.1, 1, 1, 1} of L1 loss of the feature map from the i-th max-pooling layer and the j-th convolution layer of ${\mathrm{I}}^{\mathrm{SR}}$ and ${\mathrm{I}}^{\mathrm{HR}}$ in Equations (A5) and (A6) in Appendix A.3.

The extracted perceptual information of SR and HR images is extracted from the pretrained VGG19 network on ImageNet-10k, as shown in Figure 11, as perceptual loss that is used as a loss for the back propagation to the SwinFIR SR model.

3.5.4. Swin Transformer Loss

We introduce a novel perceptual loss named Swin Transformer (SwinT) [19] loss with Swin Transformer feature extraction with 96 channels in the initial stage and comprising four stages structured of 2,2,6,2 layers. In the experiment, we chose {layers.0.blocks.1, layers.1.blocks.1, layers.2.blocks.5, layers.3.blocks.1} with weights {0.1, 0.1, 1, 1} as the extracted layers of features.

The perceptual and style features are summed to obtain the key features of an image, as described in Equation (A7) of Appendix A.4, and the difference between the key features of the low-resolution (LR) and high-resolution (HR) images is defined as the SwinT loss in Equation (A8). These features are extracted using a pretrained Swin Transformer network on ImageNet-1K [26], as shown in Figure 12. The SwinT loss, computed as the difference in perceptual features between the LR-HR image pair, is then backpropagated to optimize the SwinFIR super-resolution (SR) model.

3.5.5. Convolutional Recurrent Neural Network (CRNN) Loss

A Convolutional Recurrent Neural Network (CRNN) [40] is effective for Optical Character Recognition (OCR) as it combines CNNs for feature extraction and RNNs [43] for sequential text modeling. The CNN captures character shapes and local patterns, while the RNN (e.g., LSTM [44] or GRU [45]) learns dependencies between characters. Unlike traditional CNN-based OCR models, CRNNs handle variable-length text without character segmentation and use Connectionist Temporal Classification (CTC) loss for alignment-free text prediction. This makes CRNNs ideal for recognizing scene text [46], handwriting [47], license plates [48], and historical manuscripts [49].

When configuring the CRNN loss, we evaluated various layer combinations drawn from both convolutional and ReLU activation stages. Using CRNN loss alone, the optimal set of layers was {relu1_4, relu2_2, relu3_2, relu4_2}. However, when pairing CRNN loss with other losses, the combination {conv3_2, conv4_1, conv4_2} yielded superior results—except in the case of DISTS loss, where {relu3_2, relu4_1, relu4_2} proved most effective. In all experiments, we assigned equal weight {1} to each selected layer. The CRNN perceptual L1 loss itself is computed as the element-wise L1 difference between corresponding high-resolution and super-resolution feature maps, as defined in Equations (A9) and (A10) of Appendix A.5.

The extracted perceptual information of LR and HR images is extracted from the pretrained CRNN network on the CCPD [20] and the CRPD [22] of Chinese license plate datasets on the GitHub [15], as shown in Figure 13, as differences in the features of the LR and HR image pairs is used as a loss for the back propagation to the SwinFIR SR model.

3.6. Validation and Test Flow

We partition the self-built dataset into training (70%), validation (20%), and testing (10%) sets. The training set is used to optimize the SwinFIR model by minimizing the perceptual loss. The validation set helps prevent overfitting by incorporating the Structural Similarity Index (SSIM) [50] maximization in Figure 14 for SwinT loss as an example. The maximum SSIM is at 5000 epochs and has the best OCR recognition result.

Finally, the test set evaluates the license plate reconstruction after super-resolution and measures the OCR character recognition accuracy. The test dataset distribution, illustrated in Figure 15, includes numbers (0–9), uppercase English letters (A–Z, excluding ‘O’), and Chinese province abbreviations of others.

The test flow assesses the SwinFIR super-resolution model in enhancing license plate recognition accuracy. A low-resolution image ($I^{LR}$) is processed by SwinFIR to generate a high-resolution output ($I^{SR}$), which is then analyzed by a pretrained CRNN OCR recognizer [10]. The recognized characters are compared to the ground truth to evaluate OCR accuracy, demonstrating SwinFIR’s effectiveness in improving license plate clarity in Figure 16.

4. Result and Discussion

4.1. OCR License Plate Recognition Accuracy by Tiers

To evaluate image super-resolution (SR) models for license plate restoration, we assessed OCR performance across four accuracy tiers, as shown in

Table 3. OCR accuracy tier differentiation with references.

Metric Type	Description	Implication
7/7 Accuracy	Strictest—all characters correct	Best for law enforcement, tolling, legal ID.
6/7 Accuracy	Tolerates 1 character error	Good for real-time monitoring, alerting where minor errors are acceptable.
5/7 Accuracy	Still useful in noisy environments	Moderate-level tasks—fleet tracking, logistics where identity can be inferred with tolerable errors.
Average Accuracy	General OCR efficiency (across all characters and samples)	Overall system health indicator—used for model selection and benchmarking.

. These metrics reveal how well each model supports text extraction under challenging conditions, such as blurring or occlusion. This tiered analysis highlights both peak accuracy (7/7) and robustness in error-tolerant scenarios (6/7, 5/7), offering a practical view of model performance for real-world applications in transportation and surveillance.

We compare the OCR accuracy of different metrics in Figure 17 and

Table 4. License plate OCR recognition accuracy across SR methods of different perceptual loss.

Super-Resolution Model	Loss from Pretrained Model		Architecture Type	Function/Strength	OCR Accuracy
					7 out of 7	6 out of 7	5 out of 7
Pixel Shuffle (Repro.) [16]	Single loss	Multi-Task MSE	MSE + OCR perceptual loss	Multi-task for license plate detection, character segmentation, and OCR proposed by Nascimento et al. (2023) [16]	3.85%	38.46%	67.95%
SwinFIR	Single loss	Multi-task MSE	MSE + OCR perceptual loss	Multi-task proposed by Nascimento et al. (2023) [16]	19.23%	42.31%	66.67%
		CRNN	Recurrent Neural Network (RNN)	Excels At Sequence Learning, Beneficial For Character Continuity.	25.64%	61.54%	82.05%
		SwinT	Transformer (VisionTransformer)	Leverages Global Attention, Ideal For Holistic Pattern Enhancement.	28.21%	61.54%	83.33%
		DISTS	CNN-Based Perceptual Loss	Emphasizes Visual Similarity Using Deep Features, Enhancing Human-Perceived Quality.	32.05%	65.38%	84.62%
		VGG	CNN-Based Perceptual Loss	Captures Localized Textures, Aiding Character Shape Restoration.	33.33%	73.08%	88.46%
	Ensemble loss	CRNN + VGG	CNN + RNN (Ensemble)	Combines spatial feature extraction (VGG) with sequence modeling (CRNN); improves character-level consistency.	39.74%	69.23%	89.74%
		SwinT + DISTS	Transformer + CNN Loss (Ensemble)	uses global attention (Swin Transformer) with deep perceptual similarity (DISTS); excels in semantic restoration.	47.44%	75.64%	89.74%

for observation:

• Trend discovery reveals a clear performance scaling across tiers, with all models showing improved accuracy as tolerance increases from 7/7 to 6/7 to 5/7. This indicates that even models with lower full-plate accuracy retain partial character information that supports approximate recognition. For example, CRNN (grey line) improves from 25.64% at 7/7 accuracy to 61.54% at 6/7 and 82.05% at 5/7, demonstrating strong robustness despite slight degradations in image quality.
• Model ranking and pattern clustering:
  • Lower-tier models: Multi-task MSE and raw LR input fall behind.
    They demonstrate limited ability to enhance OCR-relevant features.
  • Mid-tier models: DISTS, SwinT, and VGG.
    All CNN- or Transformer-based—offer reliable OCR improvement. Models leveraging perceptual or deep features, as described in Table 4 (e.g., CRNN, SwinT), group in the higher performance band.
  • Top-tier models: CRNN + VGG and SwinT + DISTS.
    They consistently outperform others, indicating the value of multi-loss and hybrid-architecture training strategies.
• Correlation with feature integration: The inclusion of semantic losses (e.g., CRNN, VGG, SwinT) or ensemble losses (e.g., CRNN + VGG, SwinT + DISTS) shows strong correlation with better OCR performance. This suggests that deeper representations and hybrid loss strategies better preserve text-relevant features.

This visual analysis enables data-driven decisions in model selection for OCR-based license plate restoration. The multi-loss ensemble models stand out as the most reliable, especially SwinT + DISTS, which offers the best balance of full and partial recognition performance. Clustering and trend analysis reinforce that feature-rich loss functions significantly boost OCR readiness, making them ideal for deployment in intelligent traffic systems, surveillance, and smart cities.

4.2. OCR Character Recognition Accuracy on Average

We analyzed the average OCR character recognition accuracy (Ri) to evaluate the overall effectiveness of each SR model in enhancing degraded license plates for OCR systems, as shown in

Table 5. Effect of loss models on OCR performance in license plate SR.

Loss Model for Feature Extraction		OCR Average Accuracy (Ri)	RSR − RLR	Avg. OCR Characters Recognized
LR		75.57%	0.00%	5.29
SwinFIR original loss	Charbonnier Loss	68.68%	−6.89%	4.81
Single perceptual loss	Multi-Task MSE	70.14%	−5.43%	4.91
	CRNN	78.14%	2.57%	5.47
	SwinT	79.57%	4.00%	5.57
	DISTS	80.57%	5.00%	5.64
	VGG	82.57%	7.00%	5.78
Ensemble perceptual losses	Multi-Task MSE + SwinT	80.14%	4.57%	5.61
	DISTS + VGG	80.14%	4.57%	5.61
	Multi-Task MSE + CRNN	81.86%	6.29%	5.73
	SwinT + VGG	81.86%	6.29%	5.73
	CRNN + DISTS	82.86%	7.29%	5.80
	SwinT + CRNN	82.86%	7.29%	5.80
	CRNN + VGG	83.43%	7.86%	5.84
	SwinT + DISTS	85.14%	9.57%	5.96
HR		99.00%	23.43%	6.93

:

• LR as baseline:
  • The LR baseline yields an average OCR accuracy of 75.57%, correctly recognizing about 5.29 characters per plate.
  • Any model exceeding this benchmark illustrates the positive impact of super-resolution (SR) techniques on OCR capability.
• Simple loss:
  • Charbonnier loss leads to a notable performance drop (−6.89%), proving inadequate in capturing perceptual fidelity.
  • Multi-task MSE, while slightly better, still falls below baseline, revealing the limitations of traditional pixel-based loss functions in restoring visually coherent character structures.
• Individual perceptual loss:
  • Models with CRNN, SwinT, DISTS, and VGG as perceptual losses all surpass the baseline: Indicates the importance of architectural alignment between the SR model and perceptual loss.
  • VGG stands out with 82.57% accuracy and 5.78 recognized characters, emphasizing the power of deep CNNs in capturing semantic features for OCR. This phenomenon was proposed by Y. Liu et al. [51] that generic perceptual loss is applicable to structured learning, including super-resolution, style transfer, image segmentation.
• Ensemble of Perceptual Losses:
  • Combining perceptual loss functions further boosts OCR accuracy:

    ○

    CRNN + VGG: 83.43%, 5.84 characters.

    ○

    SwinT + DISTS: 85.14%, 5.96 characters, which is the best-performing combination.

  • These combinations capitalize on complementary strengths:

    ○

    Spatial fidelity from CNNs and DISTS.

    ○

    Sequential structure from CRNN.

    ○

    Global context understanding from SwinT transformers

They closely mimic human perception, which integrates both detail and context for character recognition.

Incorporating perceptual losses significantly improves character recognition in license plate SR and OCR tasks. Ensembles like SwinT + DISTS and CRNN + VGG deliver the best results by combining spatial, sequential, and contextual strengths. Perceptual losses help restore structural details from degraded images affected by motion blur, low brightness, skew, and resolution, bringing performance closer to that of high-resolution ground truth.

4.3. Super-Resolution Images by Perceptual Losses

We applied trained super-resolution models with various perceptual losses to restore low-resolution images and assessed their impact on license plate recognition using an OCR recognizer, as illustrated in Figure 18. Notably, the VGG-based model and the CRNN + VGG ensemble achieved the highest OCR accuracy, demonstrating the effectiveness of perceptual supervision in improving both visual quality and character recognition. A detailed analysis of the other models can be found in Appendix B.

1.

LR image:

In low-resolution (LR) license plate images, character substitution is common with frequent misrecognitions, such as “1C” being interpreted as “T”, “D” as “0”, or Chinese characters, which are logographic symbols used in the Chinese writing system, like “甘” misread as “鲁”. These errors are especially prevalent in the initial region of the plate, where province codes are typically located—likely due to edge blur and resolution fall-off at the image boundaries. Interestingly, numbers are generally better preserved than letters, possibly because of their simpler and more distinct visual structures. Overall, semantic coherence is lacking, as no domain-specific correction or contextual filtering has been applied to refine or validate the OCR output based on plate syntax or geographic consistency.

2.

VGG loss:

Super resolution using VGG perceptual loss produces high-quality, visually pleasing images with enhanced texture and structural fidelity. It effectively reduces noise and artifacts, preserves character spacing, and maintains the overall layout and background consistency of license plates. However, in character-sensitive applications like license plate recognition, VGG loss may prioritize aesthetics over semantic accuracy. This can lead to misrecognitions, such as “8” being interpreted as “B” or “0” as “D” due to the lack of a character-level understanding. While the results appear sharp and realistic, the loss function lacks semantic awareness, meaning it does not account for the meaning or identity of the characters. To address this limitation, it is advisable to combine VGG loss with recognition-aware or character-level constraints to improve OCR robustness without sacrificing visual quality.

3.

Ensemble loss of SwinT + DISTS:

The SwinT model excels at reconstructing clean, well-defined character structures and preserving license plate layouts, including accurate spacing and background details. It enhances the sharpness of number sequences, improving readability, though it can misinterpret similar characters (e.g., “1” as “T”) due to a lack of semantic correction. DISTS maintains visual and structural integrity with a natural appearance and balanced detail but also struggles with character confusions like “8” as “B”, as it prioritizes perceptual quality over textual accuracy.

The SwinT + DISTS ensemble effectively combines SwinT’s structural clarity with DISTS’s perceptual refinement, producing sharp, uniform outputs with consistent layout and improved readability. While the ensemble mitigates some individual weaknesses, integrating a recognition-based loss such as CRNN could further enhance character-level accuracy for OCR tasks.

4.4. Character Recognition Accuracy for Single Loss on Each Position

We also analyzed the recognition accuracy of license plates from left to right in Figure 17 as positions 0 to 6 with different perceptual losses, as listed in Figure 19.

The character-wise OCR accuracy from Figure 20a–g reveals how different super-resolution loss models affect recognition across license plate positions. At position 0, SwinT leads with 76.92%, highlighting its strength in recovering early characters where the LR baseline struggles (28.77%). CRNN performs best at position 1 (91.03%), while VGG dominates mid-sequence positions (2–4), likely due to its texture-preserving perceptual loss. DISTS also peaks in central positions, reinforcing the importance of structure-based losses. Notably, Multi-Task MSE underperforms across all positions, possibly due to diluted focus on fine details.

At positions 5 and 6, performance trends shift—CRNN and LR baselines outperform super-resolved results, suggesting some models introduce artifacts that hurt recognition in later characters. Surprisingly, LR achieves the highest accuracy at position 6 (81.19%), while complex models like Multi-Task MSE drop sharply.

In summary, no single model excels across all positions: SwinT is strongest early, VGG in mid-sequence, and CRNN in later characters. These findings highlight the potential of position-aware or hybrid loss strategies for better overall OCR accuracy.

4.5. Evaluation Result on Other Datasets

We evaluated our model using a publicly available Chinese license plate generator [52], which creates synthetic images with a blue background, white alphanumeric characters, and a leading Chinese symbol. To simulate real-world conditions, the high-resolution (HR) images undergo augmentations such as perspective distortion, HSV adjustments, Gaussian noise, and stain artifacts, as shown in Figure 21 (“HR img”).

To generate low-resolution (LR) images, we added Gaussian noise to the HR images, downscaled them by a factor of two, and then upscaled them using bicubic interpolation. This simulates degradation from low-quality sensors or compression, resulting in blurred edges and distorted characters (Figure 21, “LR img”).

We evaluated two SwinFIR-based super-resolution (SR) models: one trained with VGG perceptual loss and the other with a combined Swin Transformer (SwinT) + DISTS loss. The VGG loss model improves the structure but retains artifacts, while the SwinT + DISTS model produces sharper, more accurate characters with fewer distortions (Figure 21, “SR img”). These results highlight the effectiveness of perceptual losses that balance structure and texture, improving OCR accuracy and visual clarity.

The SR (super-resolution) model demonstrates notable improvements over the LR (low-resolution) model in terms of OCR accuracy. The average OCR accuracy for SR is 88.14%, surpassing the 87.89% achieved by LR. Additionally, SR shows a slight but consistent advantage in character accuracy per license plate, with a score of 6.17 compared to LR’s 6.15. This indicates that the SR model is more effective at accurately recognizing characters, making it a stronger candidate for OCR tasks that require precision in character recognition in

Table 6. OCR performance comparison of HR, LR, and SR with VGG loss. SR with SwinT + DISTS loss with 59 test license plate images.

	GT	HR	LR	SROF VGG LOSS	SR OF SWINT + DISTS LOSS
OCR ACCURACY OF TOTAL CHARACTERS	413	389	300	363	364
OCR ACCURACY OF CHARACTERS PER LICENSE PLATE	7	6.59	5.08	6.15	6.17
OCR AVERAGE ACCURACY (%)	-	94.19%	72.64%	87.89%	88.14%

.

Beyond raw accuracy, the SR model stands out in its ability to perform advanced perceptual feature extraction, which is critical for handling variations and noise in the input data. The artificial noise introduced during data augmentation closely mimics real-world scenarios, where images often suffer from imperfections such as distortion, low contrast, and environmental interference. This alignment with real-world conditions enables the SR model to capture subtle details and better generalize to noisy environments. In OCR applications, where the input quality can fluctuate significantly due to lighting, camera quality, or weather conditions, this enhanced feature extraction helps the model understand the structure and context of the text more effectively, improving its real-world performance.

In terms of training efficiency and error reduction, the SR model also outperforms LR by achieving lower VGG Loss and SwinT + DISTS Loss scores. These reduced loss values indicate that the SR model is more proficient in learning essential visual features and minimizing image quality-related errors. The combination of perceptual loss-guided training and noise handling allows the SR model to recover critical details from degraded images, making it more reliable and accurate. Overall, the SR model’s ability to handle artificial noise that closely mirrors real-world degradation, coupled with improved feature extraction and training performance, makes it a far more robust solution for OCR tasks, especially in noisy or suboptimal environments, compared to LR.

5. Conclusions and Future Work

Our study presents a comprehensive approach to improving license plate recognition (LPR) under real-world conditions such as motion blur, poor lighting, and low resolution. We introduced a novel LR-HR paired dataset sourced from real surveillance footage, enabling more realistic super-resolution (SR) training. Our SwinFIR-based SR model, trained with diverse perceptual loss functions, significantly enhances degraded plate images and boosts OCR performance.

Extensive evaluations revealed key insights: the CRNN + VGG ensemble and SwinT + DISTS models achieved the highest OCR accuracy (89.74%) on full seven-character plates, demonstrating the effectiveness of combining sequential modeling with perceptual or attention-based loss strategies. On moderately degraded six-character plates, VGG, DISTS, and CRNN achieved strong OCR results—88.46%, 84.62%, and 82.05%, respectively—highlighting the critical role of perceptual loss selection in character restoration.

To ensure deployability, we applied model distillation [53,54] to transfer knowledge from high-capacity teacher models to lightweight student networks, reducing model size and computational demand while preserving accuracy. This makes our system suitable for real-time, on-device applications [55], such as traffic cameras and mobile patrol units.

In summary, we contribute a realistic dataset, an effective SR framework, and a deployable LPR pipeline. Future work will (1) investigate OCR performance for samples with initial LR accuracy below 5%, and (2) evaluate how such challenging samples respond to SR+OCR processing. We also aim to explore domain adaptation for varied weather and lighting conditions, and further compress the model to enable faster inference at the edge.