방송공학회논문지 (Journal of Broadcast Engineering)

  • 제22권2호
  • /
  • Pages.162-172
  • /
  • 2017
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

깊은 Convolutional Neural Network를 이용한 얼굴표정 분류 기법

Facial Expression Classification Using Deep Convolutional Neural Network

  • 최인규 (광운대학교 전자공학과) ;
  • 송혁 (한국전자부품연구원) ;
  • 이상용 (광운대학교 전자공학과) ;
  • 유지상 (광운대학교 전자공학과)
  • Choi, In-kyu (Department of Electrical Engineering, KwangWoon University) ;
  • Song, Hyok (Department of Electronic Engineering) ;
  • Lee, Sangyong (Department of Electrical Engineering, KwangWoon University) ;
  • Yoo, Jisang (Department of Electrical Engineering, KwangWoon University)
  • 투고 : 2017.01.10
  • 심사 : 2017.03.20
  • 발행 : 2017.03.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문에서는 딥러닝 기술 중의 하나인 CNN(Convolutional Neural Network)을 이용한 얼굴 표정 인식 기법을 제안한다. 기존의 얼굴 표정 데이터베이스의 단점을 보완하고자 질 좋은 다양한 데이터베이스를 이용한다. 제안한 기법에서는 '무표정', '행복', '슬픔', '화남', '놀람', 그리고 '역겨움' 등의 여섯 가지 얼굴 표정 data-set을 구축한다. 효율적인 학습 및 분류 성능을 향상시키기 위해서 전처리 및 데이터 증대 기법(data augmentation)도 적용한다. 기존의 CNN 구조에서 convolutional layer의 특징지도의 수와 fully-connected layer의 node의 수를 조정하면서 여섯 가지 얼굴 표정의 특징을 가장 잘 표현하는 최적의 CNN 구조를 찾는다. 실험 결과 제안하는 구조가 다른 모델에 비해 CNN 구조를 통과하는 시간이 가장 적게 걸리면서도 96.88%의 가장 높은 분류 성능을 보이는 것을 확인하였다.

In this paper, we propose facial expression recognition using CNN (Convolutional Neural Network), one of the deep learning technologies. To overcome the disadvantages of existing facial expression databases, various databases are used. In the proposed technique, we construct six facial expression data sets such as 'expressionless', 'happiness', 'sadness', 'angry', 'surprise', and 'disgust'. Pre-processing and data augmentation techniques are also applied to improve efficient learning and classification performance. In the existing CNN structure, the optimal CNN structure that best expresses the features of six facial expressions is found by adjusting the number of feature maps of the convolutional layer and the number of fully-connected layer nodes. Experimental results show that the proposed scheme achieves the highest classification performance of 96.88% while it takes the least time to pass through the CNN structure compared to other models.

키워드

참고문헌

  1. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," arXiv preprint arXiv:1512.03385, 2015.
  2. Mollahosseini, Ali, David Chan, and Mohammad H. Mahoor, "Going deeper in facial expression recognition using deep neural networks." Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016.
  3. Jung, Heechul, et al. "Joint fine-tuning in deep neural networks for facial expression recognition." Proceedings of the IEEE International Conference on Computer Vision, 2015.
  4. Lopes, Andre Teixeira, Edilson de Aguiar, and Thiago Oliveira-Santos, "A facial expression recognition system using convolutional networks," Graphics, Patterns and Images (SIBGRAPI), 2015 28th SIBGRAPI Conference on, IEEE, 2015.
  5. Hamester, Dennis, Pablo Barros, and Stefan Wermter. "Face expression recognition with a 2-channel convolutional neural network," Neural Networks (IJCNN), 2015 International Joint Conference on. IEEE, 2015.
  6. W. Bainbridge, P. Isola, and A. Oliva, "The intrinsic memorability of face photographs," Journal of Experimental Psychology: General, 142(4):1323-1334, 2013. https://doi.org/10.1037/a0033872
  7. S. Setty and et al, "Indian Movie Face Database: A Benchmark for FaceRecognition Under Wide Variation," In NCVPRIPG, 2013.
  8. P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression," in Proceedings of the IEEE Workshop on CVPR for Human Communicative Behavior Analysis, 2010.
  9. Ma, Correll, and Wittenbrink, The Chicago Face Database: A Free Stimulus Set of Faces and Norming Data, Behavior Research Methods, 47, 1122-1135. https://doi.org/10.3758/s13428-014-0532-5
  10. ESRC 3D Face Database, http://pics.stir.ac.uk/ESRC/
  11. J. Van der Schalk, S. T. Hawk, A. H. Fischer, and B. J. Doosje, Moving faces, looking places: The Amsterdam Dynamic Facial Expressions Set (ADFES), Emotion, 11, 907-920. DOI: 10.1037/a0023853, 2011.
  12. D. Lundqvist, A. Flykt, and A.Ohman (1998), The Karolinska Directed Emotional Faces - KDEF, CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, ISBN 91-630-7164-9.
  13. H. O'Reilly, D. Pigat, S. Fridenson, S. Berggren, S. Tal, O. Golan, S. B"olte, S. Baron-Cohen and D. Lundqvist, The EU-Emotion Stimulus Set: A Validation Study, Behavior Research Methods. DOI: 10.3758/s13428-015-0601-4, 2015.
  14. M. Olszanowski, G. Pochwatko, K. Kuklinski, M. Scibor-Rylski, P. Lewinski and RK. Ohme, Warsaw Set of Emotional Facial Expression Pictures: A validation study of facial display photographs, Front. Psychol, 5:1516. doi: 10.3389/fpsyg.2014.01516, 2015.
  15. A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, 2012.
  16. Learn facial expressions from an image, https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognitionchallenge/data
  17. Viola and Jones, "Rapid object detection using a boosted cascade of simple features," Computer Vision and Pattern Recognition, 2001.
  18. M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," In European Conference on Computer Vision , Springer International Publishing, pp. 818-833, September 2014.
  19. K. Simonyan, and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," In Proc. International Conference on Learning Representations, http://arxiv.org/abs/1409.1556 (2014).
  20. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus and Y. LeCun, "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks." In Proc. ICLR, 2014.
  21. P. Burkert, F. Trier, M. Z. Afzal, A. Dengel, and M. Liwicki. Dexpression: "Deep convolutional neural network for expression recognition," .CoRR, abs/1509.05371, 2015.