Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

SkelGAN: A Font Image Skeletonization Method

  • Ko, Debbie Honghee (School of Computer Science and Engineering, Soongsil University) ;
  • Hassan, Ammar Ul (School of Computer Science and Engineering, Soongsil University) ;
  • Majeed, Saima (School of Computer Science and Engineering, Soongsil University) ;
  • Choi, Jaeyoung (School of Computer Science and Engineering, Soongsil University)
  • Received : 2020.07.14
  • Accepted : 2020.11.08
  • Published : 2021.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this research, we study the problem of font image skeletonization using an end-to-end deep adversarial network, in contrast with the state-of-the-art methods that use mathematical algorithms. Several studies have been concerned with skeletonization, but a few have utilized deep learning. Further, no study has considered generative models based on deep neural networks for font character skeletonization, which are more delicate than natural objects. In this work, we take a step closer to producing realistic synthesized skeletons of font characters. We consider using an end-to-end deep adversarial network, SkelGAN, for font-image skeletonization, in contrast with the state-of-the-art methods that use mathematical algorithms. The proposed skeleton generator is proved superior to all well-known mathematical skeletonization methods in terms of character structure, including delicate strokes, serifs, and even special styles. Experimental results also demonstrate the dominance of our method against the state-of-the-art supervised image-to-image translation method in font character skeletonization task.

Keywords

Acknowledgement

This work was supported by Institute of Information & communications Technology Planning and Evaluation (IITP) grant funded by the Korea government (MSIP) (No. 2016-0-00166, Technology Development Project for Information, Communication, and Broadcast).

References

  1. F. Zhao and X. Tang, "Preprocessing and postprocessing for skeleton-based fingerprint minutiae extraction," Pattern Recognition, vol. 40, no. 4, pp. 1270-1281, 2007. https://doi.org/10.1016/j.patcog.2006.09.008
  2. M. P. Sampat and A. C. Bovik, "Detection of spiculated lesions in mammograms," in Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No. 03CH37439), Cancun, Mexico, 2003, pp. 810-813.
  3. W. Shen, K. Zhao, J. Yuan, Y. Wang, Z. Zhang, and X. Bai, "Skeletonization in natural images and its application to object recognition," in Skeletonization: Theory, Methods and Applications. St. Louis, MO: Academic Press, 2017, pp. 259-285.
  4. N. Li, "An implementation of OCR system based on skeleton matching," Computing Laboratory, University of Kent, Canterbury, UK, 1993.
  5. X. Bai, L. Ye, J. Zhu, L. Zhu, and T. Komura, "Skeleton filter: A self-symmetric filter for skeletonization in noisy text images," IEEE Transactions on Image Processing, vol. 29, pp. 1815-1826, 2019. https://doi.org/10.1109/tip.2019.2944560
  6. R. Vemulapalli, F. Arrate, and R. Chellappa, "Human action recognition by representing 3D skeletons as points in a lie group," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 588-595.
  7. C. Shu and Y. Mo, "Morphological thinning based on image's edges," in Proceedings of the International Conference on Communication Technology (IEEE Cat. No. 98EX243), Beijing, China, 1998.
  8. J. W. Brandt and V. R. Algazi, "Continuous skeleton computation by Voronoi diagram," CVGIP: Image Understanding, vol. 55, no. 3, pp. 329-338, 1992. https://doi.org/10.1016/1049-9660(92)90030-7
  9. G. Borgefors, "Distance transformations in digital images," Computer Vision, Graphics, and Image Processing, vol. 34, no. 3, pp. 344-371, 1986. https://doi.org/10.1016/s0734-189x(86)80047-0
  10. P. K. Saha, G. Borgefors, and G. S. di Baja, "A survey on skeletonization algorithms and their applications," Pattern Recognition Letters, vol. 76, pp. 3-12, 2016. https://doi.org/10.1016/j.patrec.2015.04.006
  11. T. Y. Zhang and C. Y. Suen, "A fast parallel algorithm for thinning digital patterns," Communications of the ACM, vol. 27, no. 3, pp. 236-239, 1984. https://doi.org/10.1145/357994.358023
  12. Z. Guo and R. W. Hall, "Parallel thinning with two-subiteration algorithms," Communications of the ACM, vol. 32, no. 3, pp. 359-373, 1989. https://doi.org/10.1145/62065.62074
  13. T. C. Lee, R. L. Kashyap, and C. N. Chu, "Building skeleton models via 3-D medial surface axis thinning algorithms," CVGIP: Graphical Models and Image Processing, vol. 56, no. 6, pp. 462-478, 1994. https://doi.org/10.1006/cgip.1994.1042
  14. S. Wei, B. Xiang, Y. X. Wei, and L. L. Jan, "Skeleton pruning as trade-off between skeleton simplicity and reconstruction error," Science China Information Sciences, vol. 53, pp. 1-14, 2013. https://doi.org/10.1007/s11432-010-0001-4
  15. P. Vincent, H. Larochelle, Y. Bengio, and P. A. Manzagol, "Extracting and composing robust features with denoising autoencoders," in Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008, pp. 1096-1103.
  16. D. P. Kingma and M. Welling, "Auto-encoding variational Bayes," 2014 [Online]. Available: https://arxiv.org/abs/1312.6114.
  17. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial networks," Advances in Neural Information Processing System, vol. 27, pp. 2672-2680, 2014.
  18. P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 5967-5976.
  19. W. P. Choi, K. M. Lam, and W. C. Siu, "Extraction of the Euclidean skeleton based on a connectivity criterion," Pattern Recognition, vol. 36, no. 3, pp. 721-729, 2003. https://doi.org/10.1016/S0031-3203(02)00098-5
  20. W. Shen, X. Bai, R. Hu, H. Wang, and L. J. Latecki, "Skeleton growing and pruning with bending potential ratio," Pattern Recognition, vol. 44, no. 2, pp. 196-209, 2011. https://doi.org/10.1016/j.patcog.2010.08.021
  21. Z. Yu and C. Bajaj, "A segmentation-free approach for skeletonization of gray-scale images via anisotropic vector diffusion," Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, 2004, pp. 415-420.
  22. Q. Zhang and I. Couloigner, "Accurate centerline detection and line width estimation of thick lines using the radon transform," IEEE Transactions on Image Processing, vol. 16, no. 2, pp. 310-316, 2007. https://doi.org/10.1109/TIP.2006.887731
  23. T. Lindeberg, "Edge detection and ridge detection with automatic scale selection," International Journal of Computer Vision, vol. 30, no. 2, pp. 117-156, 1998. https://doi.org/10.1023/A:1008097225773
  24. S. Xie and Z. Tu, "Holistically-nested edge detection," in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 2015, pp. 1395-1403.
  25. S. Chen, X. Tan, B. Wang, and X. Hu, "DeepSkeleton: learning multi-task scale-associated deep side outputs for object skeleton extraction in natural images," IEEE Transactions on Image Processing, vol. 26, no. 11, pp. 5298-5311, 2017. https://doi.org/10.1109/TIP.2017.2735182
  26. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, 2015, pp. 3431-3440.
  27. O. Panichev and A. Voloshyna, "U-Net based convolutional neural network for skeleton extraction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, 2019, pp. 1186-1189.
  28. S. Nathan and P. Kansal, "Skeletonnet: shape pixel to skeleton pixel," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, 2019, pp. 1181-1185.
  29. R. Atienza, "Pyramid U-network for skeleton extraction from shape points," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, 2019, pp. 1177-1180.
  30. W. Tang, Y. Su, X Li, D. Zha, W. Jiang, N. Gao, and Ji Xiang, "CNN-based chinese character recognition with skeleton feature," in Neural Information Processing. Cham, Switzerland: Springer, 2018, pp. 461-472.
  31. K. Kurach, M. Lucic, X. Zhai, M. Michalski, and S. Gelly, "The GAN landscape: losses, architectures, regularization, and normalization," 2018 [Online]. Available: https://openreview.net/forum?id=rkGG6s0qKQ.