FontFusionGAN: Refinement of Handwritten Fonts by Font Fusion
Abstract
:1. Introduction
2. Methodology
2.1. FFGAN Architecture
2.2. Font Fusion at Inference
2.3. Design of Fusion Encoder
2.4. Loss Functions
2.4.1. Adversarial Loss
2.4.2. Image Reconstruction Loss
2.4.3. Feature Matching Loss
3. Experiment
3.1. Datasets
3.2. Training Details
3.3. Qualitative Results
3.4. Baseline Evaluation
3.5. Quantivate Results
4. Conclusions
5. Future Work
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Cha, J.; Chun, S.; Lee, G.; Lee, B.; Kim, S.; Lee, H. Few-Shot Compositional Font Generation with Dual Memory. In Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020; Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer: Berlin/Heidelberg, Germany, 2020; Volume 12364 LNCS. [Google Scholar]
- Gao, Y.; Guo, Y.; Lian, Z.; Tang, Y.; Xiao, J. Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning. ACM Trans. Graph. 2019, 38, 6. [Google Scholar] [CrossRef]
- Ko, D.H.; Hassan, A.U.; Suk, J.; Choi, J. SKFont: Skeleton-Driven Korean Font Generator with Conditional Deep Adversarial Networks. Int. J. Doc. Anal. Recognit. 2021, 24, 325–337. [Google Scholar] [CrossRef]
- Li, C.; Taniguchi, Y.; Lu, M.; Konomi, S. Few-Shot Font Style Transfer between Different Languages. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, WACV, Waikoloa, HI, USA, 3–8 January 2022. [Google Scholar]
- Park, S.; Chun, S.; Cha, J.; Lee, B.; Shim, H. Few-Shot Font Generation with Localized Style Representations and Factorization. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, AAAI, Virtual, 2–9 February 2021; Volume 3B. [Google Scholar]
- Xie, Y.; Chen, X.; Sun, L.; Lu, Y. DG-Font: Deformable Generative Networks for Unsupervised Font Generation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar]
- Tian, Y. Zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks. Available online: https://github.com/kaonashi-tyc/zi2zi (accessed on 3 June 2023).
- Hassan, A.U.; Memon, I.; Choi, J. Real-Time High Quality Font Generation with Conditional Font GAN. Expert Syst. Appl. 2023, 213, 118907. [Google Scholar] [CrossRef]
- Liu, M.-Y.; Huang, X.; Mallya, A.; Karras, T.; Aila, T.; Lehtinen, J.; Kautz, J. Few-Shot Unsupervised Image-to-Image Translation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 27 October–2 November 2019. [Google Scholar]
- Choi, Y.; Choi, M.; Kim, M.; Ha, J.W.; Kim, S.; Choo, J. StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- Isola, P.; Zhu, J.Y.; Zhou, T.; Efros, A.A. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the Proceedings 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Honolulu, HI, USA, 21–26 July 2017; Volume 2017. [Google Scholar]
- Shrivastava, A.; Pfister, T.; Tuzel, O.; Susskind, J.; Wang, W.; Webb, R. Learning from Simulated and Unsupervised Images through Adversarial Training. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Honolulu, HI, USA, 21–26 July 2017; Volume 2017. [Google Scholar]
- Wang, T.C.; Liu, M.Y.; Zhu, J.Y.; Tao, A.; Kautz, J.; Catanzaro, B. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- Zhu, J.Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; Volume 2017. [Google Scholar]
- Baek, K.; Choi, Y.; Uh, Y.; Yoo, J.; Shim, H. Rethinking the Truly Unsupervised Image-to-Image Translation. In Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Virtual, 11–17 October 2021. [Google Scholar]
- Benaim, S.; Wolf, L. One-Sided Unsupervised Domain Mapping. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017. [Google Scholar]
- Chen, X.; Xu, C.; Yang, X.; Song, L.; Tao, D. Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer. IEEE Trans. Image Process. 2019, 28, 546–560. [Google Scholar] [CrossRef] [PubMed]
- Huang, X.; Belongie, S. Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; Volume 2017. [Google Scholar]
- Kim, T.; Cha, M.; Kim, H.; Lee, J.K.; Kim, J. Learning to Discover Cross-Domain Relations with Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning, ICML, Sydney, Australia, 6–11 August 2017; Volume 4. [Google Scholar]
- Gu, Z.; Li, W.; Huo, J.; Wang, L.; Gao, Y. LoFGAN: Fusing Local Representations for Few-Shot Image Generation. In Proceedings of the IEEE International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021. [Google Scholar]
- Ul Hassan, A.; Ahmed, H.; Choi, J. Unpaired Font Family Synthesis Using Conditional Generative Adversarial Networks. Knowl. Based Syst. 2021, 229, 107304. [Google Scholar] [CrossRef]
- Muhammad, A.U.H.; Choi, J. FontNet: Closing the gap to font designer performance in font synthesis. arXiv 2022, arXiv:2205.06512. [Google Scholar]
- Karras, T.; Laine, S.; Aila, T. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 4401–4410. [Google Scholar]
- Karras, T.; Laine, S.; Aittala, M.; Hellsten, J.; Lehtinen, J.; Aila, T. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 8110–8119. [Google Scholar]
- Choi, Y.; Uh, Y.; Yoo, J.; Ha, J.W. StarGAN v2: Diverse Image Synthesis for Multiple Domains. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015–Conference Track Proceedings, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; Volume 2016. [Google Scholar]
- Mescheder, L.; Geiger, A.; Nowozin, S. Which Training Methods for GANs Do Actually Converge? In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 10–15 July 2018; Volume 8. [Google Scholar]
- Kingma, D.P.; Ba, J.L. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015—Conference Track Proceedings, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; Chen, X. Improved Techniques for Training GANs. In Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain, 5–10 December 2016. [Google Scholar]
- Muhammad, A.U.H.; Lee, H.; Choi, J. Exploiting Mixing Regularization for Truly Unsupervised Font Synthesis. Pattern Recognit. Lett. 2023, 169, 35–42. [Google Scholar] [CrossRef]
Korean (LPIPS) | Chinese (LPIPS) | |||
---|---|---|---|---|
Seen Characters | Unseen Characters | Seen Characters | Unseen Characters | |
FUNIT | 0.14 | 0.18 | 0.21 | 0.26 |
FMGAN | 0.9 | 0.12 | 0.12 | 0.14 |
FFGAN | 0.8 | 0.11 | 0.9 | 0.13 |
Korean (FID) | Chinese (FID) | |||
---|---|---|---|---|
Seen Characters | Unseen Characters | Seen Characters | Unseen Characters | |
FMGAN | 21.44 | 21.75 | 61.85 | 72.55 |
FFGAN | 20.15 | 20.98 | 22.86 | 23.14 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kumar, A.; Kang, K.; Muhammad, A.u.H.; Choi, J. FontFusionGAN: Refinement of Handwritten Fonts by Font Fusion. Electronics 2023, 12, 4246. https://doi.org/10.3390/electronics12204246
Kumar A, Kang K, Muhammad AuH, Choi J. FontFusionGAN: Refinement of Handwritten Fonts by Font Fusion. Electronics. 2023; 12(20):4246. https://doi.org/10.3390/electronics12204246
Chicago/Turabian StyleKumar, Avinash, Kyeolhee Kang, Ammar ul Hassan Muhammad, and Jaeyoung Choi. 2023. "FontFusionGAN: Refinement of Handwritten Fonts by Font Fusion" Electronics 12, no. 20: 4246. https://doi.org/10.3390/electronics12204246
APA StyleKumar, A., Kang, K., Muhammad, A. u. H., & Choi, J. (2023). FontFusionGAN: Refinement of Handwritten Fonts by Font Fusion. Electronics, 12(20), 4246. https://doi.org/10.3390/electronics12204246