Research on a Method for Recognizing Text on Book Spines in Libraries Based on Improved YOLOv11 and Optimized PaddleOCR

Abstract

The growing scale of libraries necessitates intelligent management solutions, particularly for book inventory tasks. To address the challenge of book spine recognition in dense, text-heavy environments, this study proposes an integrated approach combining an enhanced YOLOv11 model with a hyperparameter-optimized PaddleOCR framework. The methodology involves augmenting the YOLOv11 object detector with a Channel-Spatial Dual Attention Mechanism (CBAM) to better extract spine texture features and suppress interference from adjacent books. For the text recognition stage, PaddleOCR’s hyperparameters were task-optimized by adopting the RecAug data augmentation strategy, adjusting the curved text detection loss weight, expanding the character dictionary, and modifying the input image size. Experimental results on a self-constructed Book Spine Dataset show that the improved YOLOv11 achieved a segmentation accuracy of 97.4%, a 2.1% increase over the baseline, while reducing computational load and parameters. The optimized PaddleOCR saw its character error rate drop from 8.6% to 3.2%. Consequently, the end-to-end system attained a 96.8% single-book recognition accuracy in real bookshelf scenarios, demonstrating that this targeted strategy significantly enhances performance for intelligent library management.

1. Introduction

With the accelerated development of smart libraries, precise and efficient management of library collections has become a core requirement. However, the continuous growth in the size of library collections and readership makes traditional methods that rely on manual inventory or barcode scanners increasingly inadequate. In particular, in open-shelf reading areas, the dense arrangement of books causes overlapping spines, text distortion, and other issues that severely limit the accuracy of automated recognition technologies. According to the 2024 “China Library Yearbook,” the average mis-shelving rate in university libraries across the country ranges from 4.7% to 8.3%, with a significant portion of mis-shelving originating from errors in spine information recognition. This not only results in books being “missing” for long periods, severely affecting readers’ borrowing experience, but also causes the library’s database to gradually become distorted, undermining the foundation of smart library services. Therefore, developing a technology that can stably and accurately recognize book spine text in complex scenarios has urgent practical value and significant research importance.

Researchers both domestically and internationally have explored solutions to the problem of book spine recognition for many years, primarily forming two technical approaches. One is RFID [1]-based (Radio Frequency Identification) systems, such as the unmanned library system proposed by KHO. Hao Yuan’s team [2] in the early years. This method involves placing RFID tags on each book for positioning, but it is costly and susceptible to environmental interference, such as metal bookshelves. The other approach is based on computer vision, which, due to its low cost and flexible deployment, is more suitable for small- and medium-sized libraries and is the focus of this paper. Early studies in this area mostly employed traditional image processing techniques. For instance, Fang Jianjun’s team [3] used wavelet analysis and Hough transformation for spine contour detection, and Yin Ce [4] proposed a gradient-based edge growth algorithm to improve the spine segmentation effect. These methods laid the foundation for the field, but their robustness to lighting changes and complex backgrounds is relatively poor.

In recent years, with the rise in deep learning, neural network-based methods have significantly improved book spine recognition performance. For example, Cui Chen [5] applied the Mask R-CNN [6,7,8] instance segmentation network for spine segmentation, combined with text detection algorithms, effectively improving recognition accuracy. Despite these advances, practical deployment still faces significant challenges. In the detection stage, mainstream models like YOLOv11 have the advantage of real-time processing, but they struggle to differentiate features in densely arranged book spines, often resulting in false positives and missed detections. In the recognition stage, the direct application of the generic PaddleOCR model also has limitations, as its default parameters and training strategies are not well-suited for curved, perspective-distorted text or special publishing symbols, and the recognition recall rate fails to meet practical application requirements.

In conclusion, the current research gap in book spine recognition technology lies in the lack of a lightweight solution that is optimized end-to-end for densely packed, curved spine scenarios. Existing methods either rely on expensive hardware or face limitations in detection and recognition accuracy. To overcome this, this paper proposes a book spine recognition solution that integrates dual optimizations: First, in the detection phase, a Channel-Spatial Dual Attention Mechanism CBAM [9] is injected into YOLOv11 to enhance the model’s ability to extract key texture features from spines and effectively suppress interference from adjacent books. Second, in the recognition phase, task-oriented hyperparameter optimization is applied to PaddleOCR, including adjustments to data augmentation strategies, loss function weights, and character dictionaries, to enhance its robustness in recognizing curved text and special symbols. Finally, these improvements are integrated into an efficient end-to-end recognition system, achieving full-process optimization from image input to text structuring without increasing computational costs. Experimental results show that the system performs excellently in real-world scenarios, with the spine detection recall rate improved to 94.7%, the character error rate reduced to 3.2%, and the end-to-end recognition accuracy reaching 96.8%, providing a practical technical path for intelligent library management.

2. Algorithm Improvement Introduction

2.1. YOLOv11

YOLO (You Only Look Once) [10,11] is an algorithm first introduced by Joseph Redmon and others [10] in 2015. It innovatively transforms the object detection task into a single regression problem, predicting bounding box coordinates and class probabilities directly from image pixels using a convolutional neural network. This design discards traditional two-stage detection methods (such as the R-CNN series), significantly simplifying the detection process and greatly improving detection speed.

Since its inception, the YOLO series has undergone multiple iterations and improvements. The original YOLOv1, based on the Darknet framework, had 24 convolutional layers and 2 fully connected layers. YOLOv2 introduced anchor boxes (Anchors), improving the accuracy of bounding box predictions. YOLOv3 further optimized the network structure, enhancing detection performance. YOLOv4 introduced the CSP (Cross-Stage Partial Connections) architecture, improving feature extraction capabilities. YOLOv5 was optimized within the PyTorch (≥1.7, preferably 1.8+ LTS)framework, improving both development efficiency and model performance. YOLOv6 and YOLOv7 further improved performance through reparameterization and other techniques. YOLOv10 introduced improvements such as a dual label assignment mechanism.

YOLOv11 is the latest version in this series. It inherits the advantages of previous versions and further optimizes the architecture. For instance, its backbone network introduces the C3k2 module and the C2PSA module, the neck network uses the PAN structure, and the C3k2 module is also incorporated. These improvements have resulted in significant gains in both detection accuracy and speed, while reducing the model’s parameter count. Compared to previous versions, YOLOv11 achieves higher mAP50-95 scores on the COCO dataset and faster inference speeds. These advances make YOLOv11 more advantageous in real-time object detection tasks, making it better suited to adapt to complex detection scenarios. Therefore, this paper chooses YOLOv11 as the algorithm for book spine detection and segmentation and makes specific improvements to meet the detection needs of the complex library book spine environment. The network architecture of YOLOv11 is shown in Figure 1.

2.2. PaddleOCR

PaddleOCR [6,12] is a multi-language, high-performance Optical Character Recognition (OCR) [13,14] tool developed and open-sourced by Baidu based on its self-developed deep learning framework PaddlePaddle. It was first released in 2020 with the goal of solving text recognition challenges in complex scenarios and promoting the widespread adoption of industrial-grade OCR technology. Its development has demonstrated rapid iteration and breakthroughs in technology. The basic modular structure of PaddleOCR is shown in Figure 2.

The first version (2020) laid the foundational framework, supporting both Chinese and English recognition, and integrated text detection models based on the DB algorithm [15], CRNN recognition models, and direction classification models, along with a lightweight MobileNetV3 backbone network solution. The release of PaddleOCR v2 in 2021 saw significant upgrades, introducing five key technologies such as CML knowledge distillation, LCNet backbone network, and CopyPaste data augmentation [16]. Compared to the original model, its accuracy in Chinese scenarios increased by over 7%, and inference speed improved by 220%.

In 2022, PaddleOCR v3 underwent major innovations in its recognition module: the recognizer was upgraded from CRNN [17] to the SVTR architecture [18], and the detection module introduced nine optimizations, such as LK-PAN and RSE-FPN. This resulted in an additional improvement of over 5% in Chinese recognition accuracy compared to v2, and significantly enhanced robustness in complex scenarios such as curved text and occlusions. In 2024, v4 moved towards even higher accuracy by introducing multimodal large model technology for training assistance and continuously optimizing the model structure and training strategies, achieving new heights in accuracy for challenging tasks involving complex scenes, artistic fonts, and small text, while maintaining high speed. In 2025, v5 single-model support expanded to five types of text and complex handwriting recognition, improving overall recognition accuracy by 13 percentage points compared to the previous generation, making it a more comprehensive OCR model suitable for large model applications in complex document parsing. The model network structure of PP-OCRv5 is shown in Figure 3:

2.3. Challenges in the Research

Based on the actual arrangement of books in the Huaibei Normal University Library, the following challenges in library book recognition have been identified that need to be overcome:

(1)

The dense arrangement of book spines makes it difficult for traditional detection models to distinguish between adjacent target boundaries, especially when the spine materials are similar, causing a weakening of edge gradient features and resulting in severe misdetections due to adhesion;

(2)

The diversity in spine thickness, background color, text shape, text size, and language type significantly increases the difficulty of recognition;

(3)

Some spines have blurred boundaries or damage, making text detection and recognition more difficult;

(4)

Occasionally, spines are bent or poorly arranged, leading to text deformation, such as curved shapes, which are challenging to recognize.

2.4. YOLOv11 Improvement Method with Embedded CBAM Attention Mechanism

2.4.1. Introduction to CBAM

CBAM, the Convolutional Block Attention Module [9], is a simple yet effective attention module suitable for feedforward convolutional neural networks. For intermediate feature maps, this module sequentially infers attention maps along two independent dimensions—channel and spatial—then multiplies these attention maps with the input feature maps to perform adaptive feature enhancement. Since CBAM is a lightweight and universal module, it can be seamlessly integrated into any CNN architecture [19] with minimal additional cost and can be end-to-end trained alongside the base CNN.

The core idea of CBAM is to enhance the network’s representation ability by focusing on important features and suppressing unnecessary ones. The module first applies channel attention, focusing on “important” features, then applies spatial attention to focus on the “important locations” of these features. In this way, CBAM effectively helps the network focus on key information in the image, enhancing the representation of features. Figure 4 below illustrates the basic principle structure of CBAM.

The Channel Attention Module (CAM) assigns weights to feature maps along the channel dimension, while the Spatial Attention Module (SAM) focuses on positional information within the feature maps. These two mechanisms operate in a complementary manner, as illustrated in Figure 5.

Given an input feature map: $F\in R^{C\times H\times W}$; CBAM sequentially infers a 1D channel attention map: $Mc\in R^{C\times 1\times 1}$ and a 2D spatial attention map: $Ms\in R^{1\times H\times W}$; the overall procedure can be summarized as follows:

$$\begin{array}{c}{\mathrm{F}}^{\prime }={\mathrm{M}}_{\mathrm{c}}\left(\mathrm{F}\right)⨂\mathrm{F}\end{array}$$

(1)

$$\begin{array}{c}{\mathrm{F}}^{″}={\mathrm{M}}_{\mathrm{s}}\left(\mathrm{F}\right)⨂{\mathrm{F}}^{\prime }\end{array}$$

(2)

$$\begin{array}{c}{\mathrm{M}}_{\mathrm{c}}\left(\mathrm{F}\right)=\sigma \left(\mathrm{M}\mathrm{L}\mathrm{P}\left(\mathrm{A}\mathrm{u}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}\left(\mathrm{f}\right)\right)+\mathrm{M}\mathrm{L}\mathrm{P}\left(\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}\left(\mathrm{F}\right)\right)\right)=\\ \sigma \left({\mathrm{W}}_1\left({\mathrm{W}}_0\left({\mathrm{F}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{c}}\right)\right)+{\mathrm{W}}_1\left({\mathrm{W}}_0\left({\mathrm{F}}_{\mathrm{m}\mathrm{a}\mathrm{x}}^{\mathrm{c}}\right)\right)\right)\end{array}$$

(3)

$$\begin{array}{c}{\mathrm{M}}_{\mathrm{s}}\left(\mathrm{F}\right)=\sigma \left({\mathrm{f}}^{7\times 7}\left(\left[\mathrm{A}\mathrm{u}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}\left(\mathrm{F}\right);\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}\left(\mathrm{f}\right)\right]\right)\right)=\sigma \left({\mathrm{f}}^{7\times 7}\left(\left[{\mathrm{F}}_{\mathrm{a}\mathrm{v}\mathrm{g}}^{\mathrm{s}};{\mathrm{F}}_{\mathrm{m}\mathrm{a}\mathrm{x}}^{\mathrm{s}}\right]\right)\right)\end{array}$$

(4)

where σ denotes the sigmoid function, $W_0\in R^{C/r\times C}$ and $W_1\in R^{C\times C/r}$. The weights $W_0$ and $W_1$ of the MLP are shared, and a ReLU activation function is applied before $W_0$. Here, $f^{7\times 7}$ represents a convolutional kernel with a size of 7 × 7.

2.4.2. Advantages of CBAM

The core focus of this research is to address the challenge of book spine localization in complex backgrounds, including background interference, foreground blurriness, and scale variations. The Channel-Spatial Dual Attention Mechanism of CBAM can systematically tackle these challenges: channel attention filters key features and suppresses background noise; spatial attention further focuses on the text region along the spine’s central axis. This structure aligns with visual cognitive logic and offers dual focus, computational efficiency, plug-and-play capability, and strong interpretability. Compared to other attention mechanisms, CBAM significantly improves detection accuracy with minimal computational cost, making it especially suitable for library scenarios where book spines are densely packed and features are easily confused. It is an ideal solution for balancing performance and efficiency.

2.4.3. Specific Placement of Embedded CBAM

Based on the four C3k2 module structure of YOLOv11, the first CBAM module is embedded after the second C3k2 module and before the fourth Conv module. This position corresponds to mid-level features where the original C3k2 module primarily performs multi-branch feature extraction and local fusion. Replacing it with CBAM enables the model to actively enhance discriminative spine texture channels while suppressing irrelevant background interference in the spatial dimension, thereby better preserving spatial details of slender spine regions. The second CBAM module is embedded after the fourth C3k2 module and before the SPPF, where the deepest-level features contain rich semantic information. The original structure focuses on high-level semantic fusion, while embedding CBAM here strengthens global perception of the spine’s overall structure and long-range contextual dependencies, effectively addressing challenges such as partial occlusion and curvature deformation. The YOLOv11 network structure with embedded CBAM attention modules is shown in Figure 6.

2.5. PaddleOCR-Based Hyperparameter Optimization for Text Recognition

The complete workflow of the proposed book spine recognition system is depicted in Figure 7. As an integrated framework, it connects the CBAM-enhanced YOLOv11 segmentation model introduced in Section 2.4 with the hyperparameter-optimized PaddleOCR recognition module to be elaborated in this section. The pipeline begins with an input image of bookshelves, which is first processed by the improved YOLOv11 model to locate and segment all detectable book spines. Each segmented spine region is then cropped into a single image and passed sequentially through a series of optimized OCR components—including RecAug data enhancement, loss.alpha tuning, threshold parameter adjustment, input size optimization, and character set expansion—collectively improving PaddleOCR’s detection and recognition accuracy. The final output is structured textual information extracted from the book spines, ready for further application. This integrated approach effectively addresses common challenges such as dense arrangement, curved text, and special symbols in real library environments.

2.5.1. Introduction to PaddleOCR Hyperparameters

The text recognition component employs the PP-OCRv5 general scene text recognition model from the PaddleOCR v3.0 framework. This model’s hyperparameter system covers two main modules: text detection and recognition. The tuning of PaddleOCR Hyperparameters [20,21] directly affects its performance in the library book spine scenario.

In the text detection part (det_xxx.yml), key parameters include the following:

loss.alpha (binary image loss weight, enhancing the ability to detect curved text edges, but too high a value can cause instability during training);

loss.beta (threshold image loss weight, adjusting this helps improve the model’s adaptability to uneven lighting);

loss.ohem_ratio (hard example mining ratio, controlling the extent to which the model focuses on difficult samples);

Data augmentation strategy (rotation, cropping, and other transformations must be prioritized to simulate spine tilt and local occlusion);

Post-processing parameters (including postprocess.thresh to control binary confidence, box_thresh to constrain the average confidence of the detection box, and max_candidates to limit the maximum number of detections—in dense spine scenes, these need to be increased to avoid missed detections).

In the text recognition part (rec_xxx.yml), core parameters to focus on include:

Learning rate decay strategy (Cosine decay can smooth convergence and improve training effectiveness);

RecAug data augmentation (its built-in random perspective transformation and motion blur significantly improve robustness in recognizing curved text);

Character dictionary path (must be extended to include common special symbols found on book spines, such as ISBN separators);

Input image size (rec_image_shape) (adjusting height and width can impact the balance between detail perception and inference speed).

These hyperparameters together form the key dimensions of model-specific performance. To address the challenges present in library book spine images, such as curved text, dense arrangements, and lighting variations, systematic joint optimization must be performed to improve both detection accuracy and recognition precision.

2.5.2. Targeted Optimization Strategy for Key Hyperparameters in the Library Scenario

For the library book spine text recognition scenario, considering the characteristics and roles of each hyperparameter, the following optimizations are proposed:

(1)

Data augmentation is a top priority: Geometric transformations should be enhanced in the transform section of the detection stage, and the default data augmentation strategy, Basic Rec Augment, should be replaced with the RecAug strategy in the dataset.transform section of the recognition stage to perfectly simulate the curvature and deformation of the book spine.

(2)

Adjusting loss.alpha repeatedly: The loss weight should be optimized to find the most effective value for improving the model’s ability to detect curved text. This optimization theoretically allows the detection model to “forcefully” learn how to generate clear text boundaries during training, thus better handling the edge blurring caused by the curvature of the book spine.

(3)

Testing and raising postprocess.thresh and box_thresh: The best parameters should be identified to filter out many misdetections caused by paper texture or light reflection, theoretically making the results cleaner.

(4)

Expanding the character dictionary in the dataset.character_dict_path parameter: Adding commonly used characters in the library scene, as well as artistic fonts from popular publishers, will significantly improve the recognition accuracy of special characters.

(5)

Book spine text is usually narrow: Therefore, increasing the height value in rec_image_shape should be tested. Increasing the height will allow the model to see more vertical context, helping to differentiate characters that look similar.

2.5.3. Targeted Optimization Process for Key Hyperparameters in the Library Scenario

To ensure that PaddleOCR achieves optimal performance in complex library book spine scenarios, this paper adopts the control variable method, using Character Error Rate (CER) and F1 score for curved text line recognition as core evaluation metrics. The optimization process involves multiple rounds of iterative optimization of key hyperparameters, based on the baseline model pretrained on the book spine dataset, which has the following performance: CER = 8.6%, F1 = 89.7%.

The baseline model hyperparameter configuration: loss.alpha = 5; rec_image_shape = [3, 32, 320]; no use of RecAug data augmentation strategy; character dictionary = the official default dictionary (ppocr_keys_v1.txt).

The first round of optimization chooses to switch to the RecAug data augmentation strategy to enhance the model’s ability to adapt to curved deformations and complex lighting in book spines. Specifically, this involves enabling the RecAug strategy under dataset.transform in the configuration file rec_xxx.yml.

After optimization, the ability to recognize curved text significantly improved, with CER dropping from 8.6% to 5.8%, and the F1 score of the model increased by 2.4%, proving that this optimization strategy has a significant effect on recognizing curved, deformed book spines.

The second round of optimization involved adjusting the loss weight, loss.alpha. The baseline parameter was 5, and adjusting it to an appropriate value can significantly enhance the model’s focus on curved text edges, improving the accuracy of the detection boxes, and thus improving the F1 score. The results from multiple tests are shown in

Table 1. Effect of loss.alpha values on character error rate and F1-score in spine text recognition.

loss.alpha	CER/%	F1-Score/%
4	5.8	91.3
5 (Baseline)	5.8	92.1
6	5.9	92.4
7	5.8	92.9
8	5.8	93.7
9	5.8	93.2

:

As evidenced by the table above, in the library scenario, the model’s F1-score peaks when the loss.alpha parameter is optimally adjusted to 8. Since this parameter primarily governs the weight of the binarization map loss, it does not significantly impact the CER. These results demonstrate that this optimization strategy markedly mitigates missed detections.

In the third optimization round, adjustments were made to the postprocess.thresh and box_thresh parameters in the configuration file det_r50_db.yml. Fine-tuning these parameters to their optimal values was observed to substantially enhance the quality of the final detection boxes. The results from multiple adjustment tests are shown in

Table 2. Performance comparison of different threshold parameters in text detection post-processing.

postprocess.thresh	box_thresh	CER/%	F1-Score/%
0.20	0.50	5.8	93.5
0.25	0.55	5.9	93.5
0.30 (Baseline)	0.60 (Baseline)	5.8	93.7
0.35	0.65	5.6	94.2
0.40	0.70	5.3	94.8
0.45	0.75	5.5	93.8

:

Test results indicate that increasing postprocess.thresh from the baseline of 0.3 to 0.4 and box_thresh from 0.6 to 0.7 effectively filters out low-confidence false detection boxes caused by paper textures and light reflections. This optimization significantly purifies the detection output, reducing the Character Error Rate (CER) by 0.5% and increasing the F1-score by 1.1%, demonstrating substantial model improvement.

The fourth optimization round involved expanding the dictionary character set to address Out-of-Vocabulary (OOV) issues in PaddleOCR when processing library spine-specific characters (such as special symbols, publisher logos, and artistic fonts). This aims to eliminate inevitable recognition errors caused by dictionary limitations. We first extensively analyzed recognition results from the baseline general model applied to library spine content, identifying the most frequent recognition error cases. After examining approximately 200 real-world library scene recognition results, we found the following characters most prone to misrecognition: (¬, ©, ^®, ™, §, ¶, 【, 】, ※, /, ○, ◎, ●, etc.). These characters were added to the official PaddleOCR dictionary file ppocr_keys_v1.txt. As shown in Figure 8, symbols such as (○, ◎) that were previously misidentified are now correctly recognized:

Next, a large online Unicode character table was referenced, and about 6000 characters from the CJK symbols and punctuation (Unicode range: U+3000–U+303F), general punctuation (Unicode range: U+2000–U+206F), letter-like symbols (Unicode range: U+2100–U+214F), and mathematical operators (Unicode range: U+2200–U+22FF) were added to the ppocr_keys_v1.txt file to expand the character dictionary. After expansion, tests on the same data showed that the average CER decreased from 5.3% to 3.6%, and the F1 score improved from 94.8% to 95.2%, indicating that this optimization had significant effectiveness.

The Fifth Round of Optimization involves adjusting the input image size (rec_image_shape). The baseline parameter is [3, 32, 320], and multiple tests with different height configurations were performed. The results are shown in

Table 3. Impact of input image dimensions on recognition accuracy.

rec_image_shape	CER/%	F1-Score/%
[3, 16, 320]	4.1	93.3
[3, 32, 320]	3.6	95.2
[3, 48, 320]	3.2	95.7
[3, 64, 320]	4.1	94.9

:

From the data, it is evident that since book spines are typically narrow and elongated, the performance of the model is maximized when the rec_image_shape is adjusted from [3, 32, 320] to [3, 48, 320]. In this case, the F1 score increased from 95.2% to 95.7%, and the CER decreased by 0.4%.

The detailed comparison between baseline and optimized hyperparameter values is summarized in

Table 4. Comparison of PaddleOCR Hyperparameter Configurations.

Hyperparameter	Baseline Value	Optimized Value
data augmentation strategy	——	RecAug strategy
loss.alpha	5	8
postprocess.thresh	0.3	0.4
box_thresh	0.6	0.7
rec_image_shape	[3, 32, 320]	[3, 48, 320]

Through iterative experimentation, we found that increasing loss.alpha to 0.8 better balanced the detection and recognition tasks in our book spine scenario. Similarly, adjusting box_thresh to 0.7 effectively reduced false positives while maintaining recall. The modification of rec_image_shape from [3, 48, 320] to [3, 48, 320] better accommodated the aspect ratio characteristics of book spine text.

After the five rounds of targeted optimization, the performance of the PaddleOCR model in the book spine recognition task significantly improved, with the character error rate reduced by 5.4% and the F1 score increased by 6%. This provides an effective technical solution for constructing an efficient and accurate library automation management system.

3. Experimental Results and Analysis

3.1. YOLO Training Dataset Introduction

The Book Spine Dataset consists of 1400 annotated images systematically divided into training, validation, and test sets with a ratio of 60:20:20, as detailed in

Table 5. Dataset Split.

Split	Images	Ratio
Training	840	60%
Validation	280	20%
Test	280	20%

. The dataset was collected in January 2025 at a local library using a mobile device. Original images at 1920 × 1080 resolution were resized to 640 × 480 to optimize training efficiency. The collection encompasses diverse lighting conditions—including morning, noon and evening natural light—to better simulate real-world environments, and covers multiple book genres such as literature, science, and social sciences. While efforts were made to minimize obstructions and perspective variations during capture, the dataset intentionally retains challenging cases with complex backgrounds, reflections, or oblique angles to enhance model robustness.

3.2. Experimental Environment

The experiment was conducted using the Windows 11 operating system and the PyTorch deep learning framework as the network architecture. The specific experimental platform parameters are shown in

Table 6. Hardware and software configuration of the experimental platform.

Parameter	Configuration
Operating System	Windows 11
GPU	NVIDIA RTX 4060Ti
Memory (GPU)	16 GB
Network Framework	PyTorch 2.4.0
GPU Acceleration	CUDA12
Programming Language	Python 3.9

. To ensure effective training and performance optimization of the YOLO model, the experiment parameters were uniformly configured: the learning rate was set to 0.001, batch size was set to 8, the number of training epochs was set to 300, the number of worker threads was set to 4, and the input image size was set to 640. Other parameters followed the default settings of the YOLOv11n-seg model. The text recognition part used the PP-OCRv5 general scene text recognition model from the targeted optimization of Paddle OCR v3.0.

Mask-Level Precision ($\mathrm{M}\mathrm{a}\mathrm{s}\mathrm{k}-\mathrm{P}$) is determined based on the predicted masks, representing the proportion of correctly predicted pixels (or instances) among those predicted as positive. The pixels predicted as positive fall into two categories: those that are actually part of the positive mask are denoted as ${TP}_{mask}$, while those that are actually part of the negative mask are denoted as ${FP}_{mask}$. This relationship can be expressed with the formula:

$$\begin{array}{c}\mathrm{Mask}-\mathrm{P}={\displaystyle \frac{{\mathrm{T}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}}{{\mathrm{T}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}+{\mathrm{F}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}}}\end{array}$$

(5)

Mask-Level Recall ($\mathrm{M}\mathrm{a}\mathrm{s}\mathrm{k}-\mathrm{R}$) is evaluated based on the ground truth masks, representing the proportion of correctly predicted positive pixels (or instances) among all actual positive pixels. For the pixels that are actually positive, they are either correctly predicted as positive (${TP}_{mask}$) or missed in the prediction (${FN}_{mask}$). This can be formulated as:

$$\begin{array}{c}\mathrm{Mask}-\mathrm{R}={\displaystyle \frac{{\mathrm{T}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}}{{\mathrm{T}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}+{\mathrm{F}\mathrm{N}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}}}\end{array}$$

(6)

Mask-Level Average Precision (${\mathrm{A}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}$) is a metric used to evaluate the performance of object detection models on mask-level object detection. In practice, to assess model performance across multiple categories, the average of AP_mask values over all classes is typically computed, known as ${\mathrm{m}\mathrm{A}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}$. As a global metric, mAP_mask provides a more comprehensive reflection of the model’s overall capability in mask-based tasks. The specific formula is shown below:

$$\begin{array}{c}{\mathrm{A}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}=\underset{0}{\overset{1}{\int }}{\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}\left(\mathrm{R}\right)\mathrm{d}\mathrm{R}\end{array}$$

(7)

$$\begin{array}{c}{\mathrm{m}\mathrm{A}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}={\displaystyle \frac{{\sum }_{\mathrm{j}=1}^{\mathrm{C}}{{\mathrm{A}\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}}_{\mathrm{j}}}{\mathrm{C}}}\end{array}$$

(8)

F1-Score (Detection F1-Score) is the harmonic mean of Precision and Recall, serving as a core metric for comprehensively evaluating detection model performance. It provides greater reference value than pure accuracy, especially when positive and negative samples are unevenly distributed. The specific formula is shown below:

$$\begin{array}{c}{\mathrm{F}}_1=2·{\displaystyle \frac{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{n}·\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{n}+\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}\end{array}$$

(9)

Character Error Rate (CER) is defined as a metric that quantifies the discrepancy between predicted text and ground truth text. It is evaluated by calculating the minimum number of edit operations (insertion, deletion, substitution) required to transform the predicted text into the ground truth. The specific formula is as follows, where $S$ represents the number of substitutions, $D$ the number of deletions, $I$ the number of insertions, and $N$ the total number of characters in the reference text:

$$\begin{array}{c}CER={\displaystyle \frac{S+D+I}{N}}\end{array}$$

(10)

End-to-End Accuracy (E2E-Acc) serves as an end-user experience metric. Based on the entire line of text as the unit of evaluation, it measures the perfect correctness rate after combining both the detection and recognition stages. In practice, for each line-level text sample, the outcome is either a perfect match with the ground truth (${TP}_{line}$) or contains at least one discrepancy (${FN}_{line}$). The formula is as follows:

$$\begin{array}{c}End-to-EndAccuracy={\displaystyle \frac{{TP}_{line}}{{TP}_{line}+{FN}_{line}}}\end{array}$$

(11)

Detection Recall measures the proportion of actual text bounding boxes that are successfully predicted by the model. It quantifies the model’s ability to be “thorough”—that is, its capacity to find all text instances. The formula is as follows:

$$\begin{array}{c}Recall={\displaystyle \frac{TP}{TP+FN}}\end{array}$$

(12)

Real-time Processing Frame Rate (Frames Per Second, FPS) indicates the number of image frames a system can process per second, and is used to measure the real-time performance of a recognition system. The specific formula is as follows, where TotalFrames represents the total number of frames processed, and TotalTime denotes the total time consumed (in seconds) to process these frames:

$$\begin{array}{c}FPS={\displaystyle \frac{TotalFrames}{TotalTime}}\end{array}$$

(13)

3.3. YOLO Model Improvement Ablation Experiment Results

To verify the performance improvements brought by different attention modules for the YOLOv11n-seg model required for this paper’s scenario, ablation experiments were designed for comparative validation. The results are shown in

Table 7. Performance comparison of different attention mechanisms on the book spine segmentation task.

rec_image_shape	Improved Method	Mask-P/%	Mask-P/%	mAP@0.5/%	GFLOPS/G	Params/M
YOLOv11n-seg	--	89.7	89.8	95.3	8.2	3.0
YOLOv11n-seg	+SE	89.7	93.2	96.0	8.2	3.0
YOLOv11n-seg	+ECA	92.1	93.2	96.5	6.4	2.3
YOLOv11n-seg	+CA	93.4	92.5	96.3	8.2	3.0
YOLOv11n-seg	+CBAM	94.8	94.7	97.4	6.4	2.3

.

As can be seen from the above table, the baseline YOLOv11n-seg model achieves an mAP@0.5 of 95.3%. After integrating SE, ECA, CA, and CBAM attention mechanisms into the baseline model, the mAP@0.5 improves by 0.7%, 1.2%, 1.0%, and 2.1%, respectively. Among these, the incorporation of the CBAM attention mechanism contributes most significantly to model performance, yielding the highest improvement in mAP@0.5. Moreover, the model with CBAM reduces computational complexity (GFLOPS) and the number of parameters (Params) by 1.8 G and 0.7 M, respectively, compared to the original network.

3.4. Comparison Experiment Results Between Different YOLO Models

To comprehensively evaluate the actual performance of different object detection and segmentation models in real-time book spine segmentation tasks in a library, experiments were conducted using YOLOv5n-seg, YOLOv7-seg, YOLOv8n-seg, YOLOv10n-seg, and YOLOv11n-seg models (considering the real-time processing requirements and selecting lightweight n-series models). The comparison results are shown in

Table 8. End-to-end performance evaluation of different YOLO versions on the book spine detection task using mask-level metrics.

Model	Mask-P/%	Mask-P/%	mAP@0.5/%	GFLOPS/G	Params/M
YOLOv5n-seg	84.3	83.9	87.4	4.2	2.0
YOLOv7n-seg	86.8	86.5	90.1	6.1	3.3
YOLOv8n-seg	88.1	87.9	92.6	6.5	3.2
YOLOv10n-seg	88.1	88.7	94.0	7.1	3.1
YOLOv11n-seg	89.7	89.8	95.3	8.2	3.0
Proposed model	94.8	94.7	97.4	6.4	2.3

.

As shown in Table 8, the improved model proposed in this paper maintains a relatively small number of parameters while outperforming YOLOv5n-seg, YOLOv7n-seg, YOLOv8n-seg, YOLOv10n-seg, and YOLOv11n-seg models. Compared with these baseline models, our approach achieves improvements in average segmentation accuracy for library book spines by 10.0, 7.3, 4.8, 3.4, and 2.1 percentage points, respectively. Furthermore, the proposed model exhibits a reduced parameter count, lower computational complexity, enhanced real-time capability, and greater suitability for deployment in resource-constrained real-time computing scenarios.

3.5. Comparison Experiment Results of Paddle OCR Optimization

To verify the effectiveness of optimizations based on the functions of various hyperparameters of PP-OCR for the library book spine recognition scenario, optimizations were performed sequentially (with each optimization preserving the previous improvements). The results are shown in

Table 9. Step-by-step improvement of PaddleOCR performance through hyperparameter optimization on Book Spine Datasets.

Baseline Model	Improved Method	F1-Score/%	CER/%	E2E-Acc/%	Recall/%	FPS
PP-OCR v5	———	89.7	8.6	85.1	90.3	38
PP-OCR v5	+RecAug	92.1	5.8	89.3	91.2	36
PP-OCR v5	+Increased loss.alpha	93.7	5.8	91.5	95.2	36
PP-OCR v5	+increased postprocess.thresh and box_thresh	94.8	5.3	93.3	94.9	36
PP-OCR v5	+Expanded character dictionary	95.2	3.6	95.4	94.9	36
PP-OCR v5	+Modified rec_image_shape	95.7	3.2	96.8	95.1	36

:

As evidenced in the table above, after performing task-oriented hyperparameter optimization for the library book spine recognition scenario, PP-OCR v5 demonstrates notable improvements: the detection F1-Score, End-to-End Accuracy (E2E-Acc), and detection Recall have increased by 6.0%, 11.7%, and 4.8%, respectively, while the Character Error Rate (CER) has decreased by 5.4%. Notably, the inference speed (FPS) saw only a minimal decrease of 2 frames per second.

These results indicate that our optimization approach delivers substantial enhancements across nearly all key accuracy metrics. A particularly remarkable achievement is the 62.8% reduction in the Character Error Rate (CER), which is crucial for accurately reading critical information such as ISBN codes. Meanwhile, the inference speed experiences only a marginal decline, preserving efficient real-time processing capabilities. This outcome substantiates the method’s exceptional balance between precision and efficiency.

3.6. Visualization Results Analysis

To visually demonstrate the performance differences before and after the improvements in this paper’s algorithm, we conducted extensive testing on the images, selecting representative scene images with densely arranged book spines and complex and varied text for performance comparison. As shown in Figure 9, we can clearly observe that in the segmentation part, the YOLOv11n-seg model with the CBAM attention module shows a significant improvement in edge detection performance, allowing for more precise segmentation of book spine edges in dense scenes. In the text recognition part, the hyperparameter-optimized PP-OCR v5 significantly improved the issue of missed and false detections in complex text scenarios. The Mask-P-R curves of the YOLOv11n-seg baseline model and the improved model proposed in this paper are shown in Figure 10:

4. Conclusions

This study systematically proposes a book spine recognition solution that integrates attention mechanisms and hyperparameter optimization. By introducing the CBAM module to enhance YOLOv11′s feature selection ability for dense book spines and combining targeted PaddleOCR parameter tuning and dictionary expansion, this approach effectively solves key challenges such as curved text recognition and the omission of special symbols. The experiments demonstrate that this method significantly improves core metrics such as spine detection recall rate, character error rate, and end-to-end accuracy, validating the effectiveness and advancement of the technical solution. This research provides reliable technical support for the development of smart libraries and offers an optimized paradigm that can be referenced for fine-grained visual recognition tasks in complex scenarios. Future work will focus on deploying the proposed algorithm on a wheeled mobile robot equipped with a manipulator to enable autonomous book retrieval and placement tasks. This integration will serve as a foundational step toward developing a new generation of intelligent library robotics systems, further validating the algorithm’s generalization capability and practical applicability in real-world scenarios.