Atypical Character Recognition Based on Mask R-CNN for Hangul Signboard

  • Received : 2019.07.31
  • Accepted : 2019.08.12
  • Published : 2019.09.30


This study proposes a method of learning and recognizing the characteristics that are the classification criteria of Hangul using Mask R-CNN, one of the deep learning techniques, to recognize and classify atypical Hangul characters. The atypical characters on the Hangul signboard have a lot of deformed and colorful shapes beyond the general characters. Therefore, in order to recognize the Hangul signboard character, it is necessary to learn a separate atypical Hangul character rather than the existing formulaic one. We selected the Hangul character '닭' as sample data and constructed 5,383 Hangul image data sets and used them for learning and verifying the deep learning model. The accuracy of the results of analyzing the performance of the learning model using the test set constructed to verify the reliability of the learning model was about 92.65% (the area detection rate). Therefore we confirmed that the proposed method is very useful for Hangul signboard character recognition, and we plan to extend it to various Hangul data.


Atypical Character Recognition;Hangul Signboard;Mask R-CNN;Hangul signboard character


Supported by : Dong Yang University


  1. Y. Sun, Y. Chen, X. Wang and X. Tang, "Deep learning face representation by joint identification-verification," Advances in neural information processing systems, pp. 1988-1996, 2014.
  2. T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," arXiv preprint, 2016.
  3. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee and A. Y. Ng, "Multimodal deep learning," in Proc. 28th international conference on machine learning(ICML-11), pp. 689-696, 2011.
  4. X. B. Zhang, F. C. Chen and R. Y. Huaug, "A Combination of RNN and CNN for Attention-based Relation Classification," Procedia Computer Science, Vol. 131, pp. 911-917, 2018. DOI:
  5. H. C. Moon, A. N. Yang and J. G. Kim, "CNN-Based Hand Gesture Recognition for Wearable Applications," The Korean Society of Broad Engineers, Vol. 23, No. 2, pp. 246-252, 2018. DOI:
  6. A. Krizhevsky, I. Sutskever and GE. Hinton, "Imagenet classification with deep convolution neural networks," Advances in Neural Information Processing Systems, pp.1106-1114, 2012.
  7. K, Simonyan, A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint, 2014.
  8. C. Szegedy, W. Liu, Y. Jia and P. Sermanet, "Going deeper with convolutions," in Proc. IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
  9. J. R. Uijlings, K. E. Van De Sande, T. Gevers and A. W. Smeulders, "Selective search for object recognition," International journal of computer vision, Vol. 104, No. 2, pp. 154-171, 2013.
  10. H. Drucker, C. J. Burges, L. Kaufman, A. J. Smola and V. Vapnik, "Support vector regression machines," Advances in neural information processing systems, pp. 155-161. 1997.
  11. R. Girshick, "Fast R-CNN," in Proc. IEEE international conference on computer vision, pp. 1440-1448, 2015.
  12. S. Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, pp. 91-99, 2015.
  13. K. He, G. Gkioxari, P. Dollar and R. Girshick, "Mask r-cnn," in Proc. IEEE international conference on computer vision, pp. 2961-2969, 2017.
  14. Z. Wang, J. Yang, H. Jin, E. Shechtman, A. Agarwala, J. Brandt and T. S. Huang, "Deepfont: Identify your font from an image," in Proc. 23rd ACM international conference on Multimedia, pp. 451-459, 2015. DOI:
  15. I. K Hwang, Study on Hangul font characteristics using CNN, Doctoral dissertation, Seoul National Univ., 2017.
  16. I. J. Kim, C. Choi and S. H. Lee, "Improving discrimination ability of convolutional neural networks by hybrid learning," International Journal on Document Analysis and Recognition, Vol. 19, No. 1, pp. 1-9, 2016. DOI:
  17. J. H. Yang, H. B. Kwak and I. J. Kim, "Large-Scale Hangul Font Recognition Using Deep Learning," in Proc. Annual Conference on Human and Language Technology, pp. 8-12. 2017.
  18. T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, "Microsoft COCO: Common objects in context," in Proc. European conference on computer vision, pp. 740-755, 2014. DOI:
  19. S. Lim, "Emotional Communication on Interactive Typography System," International Journal of Contents, Vol. 14, No. 2, pp. 41-44, 2018. DOI:
  20. S. Lim, "3D Spatial Interaction Method using Visual Dynamics and Meaning Production of Character," International journal of advanced smart convergence, Vol. 7, No. 3, pp. 130-139, 2018. DOI: