A Three-Stage Generative Adversarial Image Inpainting Framework for Broken-Stroke Restoration in Historical Rubbings: A Case Study on Oracle Bone Rubbings
Abstract
1. Introduction
- We reformulate oracle bone rubbing restoration as a structural and texture-aware image inpainting problem and propose a three-stage generative adversarial framework specialized for oracle bone rubbing completion while retaining design ideas that may be informative for other degraded inscription images.
- We design a staged restoration pipeline in which an LBP-guided coarse completion module supplies local structural priors, a spatial-attention refinement module improves damaged-region feature aggregation, and a Swin-based refinement network enhances long-range consistency.
- We introduce a dual-discriminator supervision strategy that evaluates restored results from complementary global and local perspectives, thereby reducing artifacts and improving the realism of repaired stroke regions.
- We provide qualitative, quantitative, and ablation-based evaluations on oracle bone rubbing images, and we further report model size and inference cost to support a more transparent experimental discussion.
2. Related Work
2.1. CNN- and GAN-Based Image Inpainting
2.2. Transformer-Based Inpainting and Image Restoration
2.3. Oracle Bone and Historical Inscription Image Analysis
3. Method
3.1. Problem Formulation
3.2. Overall Architecture and Inter-Stage Workflow
3.3. Three-Stage Restoration Pipeline
3.3.1. Stage I: LBP-Guided Coarse Completion
3.3.2. Stage II: Spatial-Attention Refinement with Dual Discriminators
3.3.3. Stage III: Swin-Based Global Refinement
3.4. Objective Functions and Optimization
3.4.1. Stage I Loss
3.4.2. Stage II Loss
3.4.3. Stage III Loss and Training Schedule
4. Experiment
4.1. Dataset and Experimental Protocol
4.2. Qualitative Comparison
4.3. Quantitative Comparison
4.4. Ablation Study
4.5. Computational Cost and Limitation Analysis
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Buades, A.; Coll, B.; Morel, J.M. A non-local algorithm for image denoising. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05); IEEE: Piscataway, NJ, USA, 2005; Volume 2, pp. 60–65. [Google Scholar]
- Pathak, D.; Krahenbuhl, P.; Donahue, J.; Darrell, T.; Efros, A.A. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2016; pp. 2536–2544. [Google Scholar]
- Yu, J.; Lin, Z.; Yang, J.; Shen, X.; Lu, X.; Huang, T.S. Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2018; pp. 5505–5514. [Google Scholar]
- Yu, J.; Lin, Z.; Yang, J.; Shen, X.; Lu, X.; Huang, T.S. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision; IEEE: Piscataway, NJ, USA, 2019; pp. 4471–4480. [Google Scholar]
- Zeng, Y.; Fu, J.; Chao, H.; Guo, B. Learning pyramid-context encoder network for high-quality image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2019; pp. 1486–1494. [Google Scholar]
- Ren, Y.; Yu, X.; Zhang, R.; Li, T.H.; Liu, S.; Li, G. Structureflow: Image inpainting via structure-aware appearance flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision; IEEE: Piscataway, NJ, USA, 2019; pp. 181–190. [Google Scholar]
- Li, J.; Wang, N.; Zhang, L.; Du, B.; Tao, D. Recurrent feature reasoning for image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2020; pp. 7760–7768. [Google Scholar]
- Zeng, Y.; Lin, Z.; Lu, H.; Patel, V.M. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision; IEEE: Piscataway, NJ, USA, 2021; pp. 14164–14173. [Google Scholar]
- Guo, X.; Yang, H.; Huang, D. Image inpainting via conditional texture and structure dual generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision; IEEE: Piscataway, NJ, USA, 2021; pp. 14134–14143. [Google Scholar]
- Quan, W.; Zhang, R.; Zhang, Y.; Li, Z.; Wang, J.; Yan, D.M. Image inpainting with local and global refinement. IEEE Trans. Image Process. 2022, 31, 2405–2420. [Google Scholar] [CrossRef] [PubMed]
- Wu, H.; Zhou, J.; Li, Y. Deep generative model for image inpainting with local binary pattern learning and spatial attention. IEEE Trans. Multimed. 2021, 24, 4016–4027. [Google Scholar]
- Xu, Z.; Zhang, X.; Chen, W.; Yao, M.; Liu, J.; Xu, T.; Wang, Z. A Review of Image Inpainting Methods Based on Deep Learning. Appl. Sci. 2023, 13, 11189. [Google Scholar] [CrossRef]
- Elharrouss, O.; Damseh, R.; Belkacem, A.N.; Badidi, E.; Lakas, A. Transformer-based image and video inpainting: Current challenges and future directions. Artif. Intell. Rev. 2025, 58, 124. [Google Scholar] [CrossRef]
- Dong, Q.; Cao, C.; Fu, Y. Incremental transformer structure enhanced image inpainting with masking positional encoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2022; pp. 11358–11368. [Google Scholar]
- Liu, T.J.; Chen, B.W.; Liu, K.H. Lightweight image inpainting by stripe window transformer with joint attention to CNN. arXiv 2023, arXiv:2301.00553. [Google Scholar] [CrossRef]
- Ko, K.; Kim, C.S. Continuously masked transformer for image inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision; IEEE: Piscataway, NJ, USA, 2023; pp. 13169–13178. [Google Scholar]
- Liu, Q.; Jiang, Y.; Tan, Z.; Chen, D.; Fu, Y.; Chu, Q.; Hua, G.; Yu, N. Transformer Based Pluralistic Image Completion With Reduced Information Loss. IEEE Trans. Pattern Anal. Mach. Intell. 2024, 46, 6652–6668. [Google Scholar] [CrossRef] [PubMed]
- Chen, S.; Atapour-Abarghouei, A.; Shum, H.P.H. HINT: High-Quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention. IEEE Trans. Multimed. 2024, 26, 7649–7660. [Google Scholar] [CrossRef]
- Guo, J.; Wang, C.; Roman-Rangel, E.; Chao, H.; Rui, Y. Building hierarchical representations for oracle character and sketch recognition. IEEE Trans. Image Process. 2015, 25, 104–118. [Google Scholar] [CrossRef] [PubMed]
- Wang, M.; Deng, W.; Liu, C.L. Unsupervised structure-texture separation network for oracle character recognition. IEEE Trans. Image Process. 2022, 31, 3137–3150. [Google Scholar] [CrossRef] [PubMed]
- Jiang, R.; Liu, Y.; Zhang, B.; Chen, X.; Li, D.; Han, Y. Oraclepoints: A hybrid neural representation for oracle character. In Proceedings of the 31st ACM International Conference on Multimedia; Association for Computing Machinery: New York, NY, USA, 2023; pp. 7901–7911. [Google Scholar]
- Chang, X.; Chao, F.; Shang, C.; Shen, Q. Sundial-gan: A cascade generative adversarial networks framework for deciphering oracle bone inscriptions. In Proceedings of the 30th ACM International Conference on Multimedia; Association for Computing Machinery: New York, NY, USA, 2022; pp. 1195–1203. [Google Scholar]
- Wang, S.; Guo, W.; Xu, Y.; Liu, D.; Li, X. Coarse-to-Fine Generative Model for Oracle Bone Inscriptions Inpainting. In Proceedings of the 1st Workshop on Machine Learning for Ancient Languages (ML4AL 2024); Association for Computational Linguistics: Austin, TX, USA, 2024; pp. 107–114. [Google Scholar]
- Li, J.; Chen, Z.; Chen, T.; Liu, Z.; Wang, C. OBIFormer: A fast attentive denoising framework for oracle bone inscriptions. Displays 2025, 89, 103059. [Google Scholar] [CrossRef]
- Ma, Y.; Li, F.; Chen, S. OraGAN: A deep learning based model for restoring Oracle Bone Script Images. npj Herit. Sci. 2025, 13, 681. [Google Scholar] [CrossRef]
- Ren, H.; Sun, K.; Zhao, F.; Zhu, X. Dunhuang murals image restoration method based on generative adversarial network. Herit. Sci. 2024, 12, 39. [Google Scholar] [CrossRef]
- Ziran, Z.; Mecella, M.; Marinai, S. AI-driven enhancement of historical documents. Int. J. Digit. Libr. 2025, 26, 22. [Google Scholar] [CrossRef]
- Wang, N.; Ma, S.; Li, J.; Zhang, Y.; Zhang, L. Multistage attention network for image inpainting. Pattern Recognit. 2020, 106, 107448. [Google Scholar] [CrossRef]
- Wang, Y.; Guo, D.; Zhao, H.; Yang, M.; Zheng, H. Image inpainting via Multi-scale Adaptive Priors. Pattern Recognit. 2025, 162, 111410. [Google Scholar] [CrossRef]
- Johnson, J.; Alahi, A.; Fei-Fei, L. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 694–711. [Google Scholar]
- Gatys, L.A.; Ecker, A.S.; Bethge, M. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; IEEE: Piscataway, NJ, USA, 2016; pp. 2414–2423. [Google Scholar]






| Method | PSNR ↑ | SSIM ↑ | L1 ↓ |
|---|---|---|---|
| PEN | 11.46 | 0.7445 | 0.0890 |
| RFR | 12.55 | 0.7812 | 0.0823 |
| LBPLSA | 30.89 | 0.7812 | 0.0050 |
| LG | 33.42 | 0.9898 | 0.0521 |
| SWT-CNN | 28.03 | 0.9819 | 0.0104 |
| Ours | 35.18 | 0.9906 | 0.0030 |
| Method | PSNR ↑ | SSIM ↑ | L1 ↓ |
|---|---|---|---|
| PEN | 8.67 | 0.6377 | 0.1628 |
| RFR | 14.67 | 0.8057 | 0.0567 |
| LBPLSA | 25.67 | 0.9719 | 0.0091 |
| LG | 27.01 | 0.9781 | 0.0066 |
| CFGM | 29.61 | 0.9826 | 0.0058 |
| SWT-CNN | 26.02 | 0.9610 | 0.0169 |
| Ours | 29.90 | 0.9807 | 0.0059 |
| Method | PSNR ↑ | SSIM ↑ | L1 ↓ |
|---|---|---|---|
| PEN | 9.29 | 0.5979 | 0.1427 |
| RFR | 15.94 | 0.8802 | 0.0441 |
| LBPLSA | 22.74 | 0.9232 | 0.0169 |
| LG | 29.46 | 0.9818 | 0.0053 |
| CFGM | 23.43 | 0.9567 | 0.0143 |
| SWT-CNN | 20.40 | 0.9055 | 0.0499 |
| Ours | 25.68 | 0.9829 | 0.0064 |
| Metric | NO | CN | DP+Swin |
|---|---|---|---|
| PSNR ↑ | 25.67 | 26.59 | 29.90 |
| SSIM ↑ | 0.9719 | 0.8841 | 0.9857 |
| L1 ↓ | 0.0091 | 0.0126 | 0.0059 |
| Method | Parameters (M) | Inference Time (s/Image) | MAE |
|---|---|---|---|
| LBPLSA | 114.87 | 0.0908 | 0.0060 |
| CFGM | 120.82 | 0.0915 | 0.0043 |
| Ours | 194.07 | 0.1164 | 0.0190 |
| Item | Value |
|---|---|
| Operating system/framework | Linux, Python 3.10, PyTorch 1.13.1 (CUDA 11.7) |
| Additional libraries | torchvision 0.14.1, torchaudio 0.13.1 |
| GPU | NVIDIA GeForce RTX 3090 |
| VRAM | 23.69 GB |
| Batch size | 1 |
| Precision | FP32 (mixed precision disabled) |
| Training epochs | 500 |
| Total iterations | ≈1,000,000 |
| Total training time | ≈292,062.83 s (≈81.13 h) |
| Peak allocated memory | 10.00 GB |
| Peak reserved memory | 14.79 GB |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Shen, W.; Xu, Y.; Zhang, C.; Yan, J.; Wang, S. A Three-Stage Generative Adversarial Image Inpainting Framework for Broken-Stroke Restoration in Historical Rubbings: A Case Study on Oracle Bone Rubbings. Appl. Sci. 2026, 16, 4306. https://doi.org/10.3390/app16094306
Shen W, Xu Y, Zhang C, Yan J, Wang S. A Three-Stage Generative Adversarial Image Inpainting Framework for Broken-Stroke Restoration in Historical Rubbings: A Case Study on Oracle Bone Rubbings. Applied Sciences. 2026; 16(9):4306. https://doi.org/10.3390/app16094306
Chicago/Turabian StyleShen, Wenhan, Yubo Xu, Chaoqing Zhang, Juan Yan, and Shibin Wang. 2026. "A Three-Stage Generative Adversarial Image Inpainting Framework for Broken-Stroke Restoration in Historical Rubbings: A Case Study on Oracle Bone Rubbings" Applied Sciences 16, no. 9: 4306. https://doi.org/10.3390/app16094306
APA StyleShen, W., Xu, Y., Zhang, C., Yan, J., & Wang, S. (2026). A Three-Stage Generative Adversarial Image Inpainting Framework for Broken-Stroke Restoration in Historical Rubbings: A Case Study on Oracle Bone Rubbings. Applied Sciences, 16(9), 4306. https://doi.org/10.3390/app16094306
