Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Recognition of dog's front face using deep learning and machine learning

딥러닝 및 기계학습 활용 반려견 얼굴 정면판별 방법

  • Kim, Jong-Bok (National Institute of Animal Science, Rural Development Administration) ;
  • Jang, Dong-Hwa (National Institute of Animal Science, Rural Development Administration) ;
  • Yang, Kayoung (National Institute of Animal Science, Rural Development Administration) ;
  • Kwon, Kyeong-Seok (National Institute of Animal Science, Rural Development Administration) ;
  • Kim, Jung-Kon (National Institute of Animal Science, Rural Development Administration) ;
  • Lee, Joon-Whoan (Division of Computer Science and Engineering, Jeonbuk National University)
  • 김종복 (농촌진흥청 국립축산과학원) ;
  • 장동화 (농촌진흥청 국립축산과학원) ;
  • 양가영 (농촌진흥청 국립축산과학원) ;
  • 권경석 (농촌진흥청 국립축산과학원) ;
  • 김중곤 (농촌진흥청 국립축산과학원) ;
  • 이준환 (전북대학교 컴퓨터공학부)
  • Received : 2020.10.30
  • Accepted : 2020.12.04
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

As pet dogs rapidly increase in number, abandoned and lost dogs are also increasing in number. In Korea, animal registration has been in force since 2014, but the registration rate is not high owing to safety and effectiveness issues. Biometrics is attracting attention as an alternative. In order to increase the recognition rate from biometrics, it is necessary to collect biometric images in the same form as much as possible-from the face. This paper proposes a method to determine whether a dog is facing front or not in a real-time video. The proposed method detects the dog's eyes and nose using deep learning, and extracts five types of directional face information through the relative size and position of the detected face. Then, a machine learning classifier determines whether the dog is facing front or not. We used 2,000 dog images for learning, verification, and testing. YOLOv3 and YOLOv4 were used to detect the eyes and nose, and Multi-layer Perceptron (MLP), Random Forest (RF), and the Support Vector Machine (SVM) were used as classifiers. When YOLOv4 and the RF classifier were used with all five types of the proposed face orientation information, the face recognition rate was best, at 95.25%, and we found that real-time processing is possible.

반려견을 키우는 가구 수가 급격하게 증가함에 따라 유기, 유실견도 많이 증가하고 있다. 국내에서는 2014년부터 반려동물 등록제를 시행하고 있지만, 안전성과 실효성 문제로 등록률이 높지 않은 실정이다. 이러한 문제를 해결할 방법으로 반려견 생체인식 기술이 주목을 받고 있다. 생체인식률을 높이기 위해서는 최대한 정면에서 같은 형태로 생체정보 이미지를 수집해야 한다. 하지만 반려견은 사람과 달리 비협조적이기 때문에 생체정보 이미지 수집이 어렵다. 본 논문에서는 반려견 생체인식에 적합한 생체정보 이미지 수집을 위해 실시간 영상에서 반려견 얼굴 방향이 정면인지를 판별하는 방법을 제안한다. 제안 방법은 딥러닝을 활용하여 반려견 눈과 코를 검출하고, 검출된 눈과 코의 상대적 크기와 위치를 통해 5가지의 얼굴 방향 정보를 추출하여 기계학습 분류기로 정면 여부를 판별한다. 2,000개의 반려견 이미지를 분류하여 학습, 검증 및 테스트에 사용하였다. 눈과 코 검출에는 YOLOv3와 YOLOv4를 사용하였고, 분류기는 MLP(Multi-layer Perceptron), RF(Random Forest), SVM(Support Vector Machine)을 사용하였다. YOLOv4와 RF 분류기를 사용하고 제안하는 5가지 얼굴 방향 정보 모두를 적용하였을 때 얼굴 정면 판별 성능이 95.25%로 가장 좋았으며, 실시간 처리도 가능한 것으로 나타났다.

Keywords

References

  1. Ministry of Agriculture, Food and Rural Affairs, Rescue and protection of 100,000 lost and abandoned animals in 2017, Press Release, Ministry of Agriculture, Korea, pp. 4-5.
  2. E. Murphy-Chutorian and M. M. Trivedi, "Head pose estimation in computer vision: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 607-626, Apr. 2009. https://doi.org/10.1109/TPAMI.2008.106
  3. H. Wilson, F. Wilkinson, L. Lin, and M. Castillo, "Perception of Head Orientation," Vision Research, vol. 40, no. 5, pp. 459-472, 2000. https://doi.org/10.1016/S0042-6989(99)00195-9
  4. A. Nikolaidis and I. Pitas, "Facial Feature Extraction and Pose Determination," Pattern Recognition, vol. 33, no. 11, pp. 1783-1791, 2000 https://doi.org/10.1016/S0031-3203(99)00176-4
  5. A. Gee and R. Cipolla, "Determining the Gaze of Faces in Images," Image and Vision Computing, vol. 12, no. 10, pp. 639-647, 1994. https://doi.org/10.1016/0262-8856(94)90039-6
  6. T. Horprasert, Y. Yacoob, and L. Davis, "Computing 3-D Head Orientation from a Monocular Image Sequence," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 242-247, 1996.
  7. J.-G. Wang and E. Sung, "EM Enhancement of 3D Head Pose Estimated by Point at Infinity," Image and Vision Computing, vol. 25, no. 12, pp. 1864-1874, 2007. https://doi.org/10.1016/j.imavis.2005.12.017
  8. V. Drouard, R. Horaud, A. Deleforge, S. Ba, and G. Evangelidis, "Robust head-pose estimation based on partially-latent mixture of linear regressions," IEEE Trans. Image Process., vol. 26, no. 3, pp. 14281440, Mar. 2017.
  9. N. Ruiz, E. Chong, and J. M. Rehg, "Fine-grained head pose estimation without keypoints," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2018, pp. 2074-2083.
  10. J. Xia, L. Cao, and J. Liao, "Head pose estimation in the wild assisted by facial landmarks based on convolutional neural networks", IEEE Access, vol. 7, Apr. 2019.
  11. A. F. Abate, P. Barra, C. Bisogni, M. Nappi, and S. Ricciardi, "Near real-time three axis head pose estimation without training", IEEE Access, vol. 7, May. 2019
  12. Tzutalin LabelImg GitHub [Internet]. Available : https://github.com/tzutalin/labelImg/
  13. J. Redmon, A. Farhadi, "Yolov3: An incremental improvement", arXiv preprint arXiv:1804.02767, 2018.
  14. A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4:Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
  15. D. Krstajic, L. J. Buturovic, D. E. Leahy, and S. Thomas, "Cross-validation pitfalls when selecting and assessing regression and classification models," J. Cheminformatics, vol. 6, no. 1, p. 10, Dec. 2014. DOI: https://doi.org/10.1186/1758-2946-6-10
  16. S. Arlot and A. Celisse, "A survey of cross-validation procedures for model selection," Statist. Surv., vol. 4, pp. 40-79, Jul. 2010. DOI: https://doi.org/10.1214/09-SS054
  17. AlexeyAB darknet GitHub [Internet]. Available : https://github.com/AlexeyAB/darknet/
  18. "Scikit-learn: Machine Learning in Python", Pedregosa et al., JMLR 12, pp. 2825-2830, 2011.