Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Extreme Learning Machine Ensemble Using Bagging for Facial Expression Recognition

  • Ghimire, Deepak (Department of Computer Engineering, Chonbuk National University) ;
  • Lee, Joonwhoan (Department of Computer Engineering, Chonbuk National University)
  • Received : 2013.10.04
  • Accepted : 2014.03.03
  • Published : 2014.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

An extreme learning machine (ELM) is a recently proposed learning algorithm for a single-layer feed forward neural network. In this paper we studied the ensemble of ELM by using a bagging algorithm for facial expression recognition (FER). Facial expression analysis is widely used in the behavior interpretation of emotions, for cognitive science, and social interactions. This paper presents a method for FER based on the histogram of orientation gradient (HOG) features using an ELM ensemble. First, the HOG features were extracted from the face image by dividing it into a number of small cells. A bagging algorithm was then used to construct many different bags of training data and each of them was trained by using separate ELMs. To recognize the expression of the input face image, HOG features were fed to each trained ELM and the results were combined by using a majority voting scheme. The ELM ensemble using bagging improves the generalized capability of the network significantly. The two available datasets (JAFFE and CK+) of facial expressions were used to evaluate the performance of the proposed classification system. Even the performance of individual ELM was smaller and the ELM ensemble using a bagging algorithm improved the recognition performance significantly.

Keywords

References

  1. A. Mehrabian, "Communication without words," Psychology Today, vol. 2, no. 4, pp. 53-56, 1968.
  2. P. Ekman, "Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique," Psychological Bulletin, vol. 115, no. 2, pp. 268-287, Mar. 1994. https://doi.org/10.1037/0033-2909.115.2.268
  3. L. K. Hansen and P. Salamon, "Neural network ensemble," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 10, pp. 993-1001, Oct. 1990. https://doi.org/10.1109/34.58871
  4. L. Breiman, "Bagging predictors," Machine Learning, vol. 24, no. 2, pp. 123-140, Aug. 1996.
  5. R. Schapire, "The strength of weak learnability," Machine Learning, vol. 5, no. 2, pp. 197-227, Jun. 1990.
  6. Y. Freund, "Boosting a weak learning algorithm by majority," Information and Computation, vol. 121, no. 2, pp. 256-285, Sep. 1995. https://doi.org/10.1006/inco.1995.1136
  7. G. B. Huang, Q. Y. Zhu, and C. K. Siew, "Extreme learning machine: theory and applications," Neurocomputing, vol. 70, no. 1-3, pp. 489-501, Dec. 2006. https://doi.org/10.1016/j.neucom.2005.12.126
  8. H. X. Tian and Z. Z. Mao, "An ensemble ELM based on modified AdaBoost.RT algorithm for predicting the temperature of molten steel in ladle furnace," IEEE Transactions on Automation Science and Engineering, vol. 7, no. 1, pp. 73-80, Jan. 2010. https://doi.org/10.1109/TASE.2008.2005640
  9. Y. Lan, Y. C. Soh, and G. B. Huang, "Ensemble of online sequential extreme learning machine," Neurocomputing, vol. 72, no. 13-15, pp. 3391-3395, Aug. 2009. https://doi.org/10.1016/j.neucom.2009.02.013
  10. Y. Liu, X. Xu, and C. Wang, "Simple ensemble of extreme learning machine," in Proceedings of the 2nd International Congress on Image and Signal Processing, Tianjin, China, October 17-19, 2009, pp. 1-5.
  11. N. Liu and H. Wang, "Ensemble based extreme learning machine," IEEE Signal Processing Letters, vol. 17, no. 8, pp. 754-757, Aug. 2010. https://doi.org/10.1109/LSP.2010.2053356
  12. M. van Heeswijk, Y. Miche, T. Lindh-Knuutila, P. J. Hilbers, T. Honkela, E. Oja, and A. Lendasse, "Adaptive ensemble models of extreme learning machines for time series prediction," in Artificial Neural Networks-ICANN 2009, Lecture Notes in Computer Science Volume 5769, C. Alippi, M. Polycarpou, C. Panayiotou, and G. Ellinas, Eds., Heidelberg: Springer Berlin, 2009, pp. 305-314.
  13. A. Samal and P. A. Iyengar, "Automatic recognition and analysis of human faces and facial expressions: a survey," Pattern Recognition, vol. 25, no. 1, pp. 65-77, Jan. 1992. https://doi.org/10.1016/0031-3203(92)90007-6
  14. M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba "Coding facial expressions with Gabor wavelets," in Proceedings of the 3rd IEEE International Conference on Face and Gesture Recognition, Nara, Japan, April, 14-16, 1998, pp. 200-205.
  15. M. Pantic and L. J. M. Rothkrantz, "Automatic analysis of facial expressions: the state of the art," IEEE Transactions of Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1434-1445, Dec. 2000.
  16. P. Michel and R. E. Kaliouby, "Real time facial expression recognition in video using support vector machines," in Proceedings of the 5th International Conference on Multimodal Interfaces, Vancouver, Canada, November 5-7, 2003, pp. 258-264.
  17. G. Littlewort, M. S. Bartlett, I. Fasel, J. Susskind, and J. Movellan, "Dynamics of facial expression extracted automatically from videos," Image and Vision Computing, vol. 24, no. 6, pp. 615-625, Jun. 2006. https://doi.org/10.1016/j.imavis.2005.09.011
  18. I. Kotsia and I. Pitas, "Facial expression recognition in image sequences using geometric deformation features and support vector machines," IEEE Transactions on Image Processing, vol. 16, no. 1, pp. 172-187, Jan. 2007. https://doi.org/10.1109/TIP.2006.884954
  19. D. Ghimire and J. Lee, "Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines," Sensors, vol. 13, no. 6, pp. 7714-7734, Jun. 2013. https://doi.org/10.3390/s130607714
  20. D. Ghimire and J. Lee, "Automatic facial expression recognition based on features extracted from tracking of facial landmarks," Proceedings of SPIE, vol. 9069, pp. 90691O, Jan. 2014.
  21. C. C. Chibelushi and F. Bourel, "Facial expression recognition: a brief tutorial overview," [Online]. Available: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/CHIBELUSHI1/CCC_FB_FacExprRecCVonline.pdf.
  22. B. Lee, J. Chun, and P. Park, "Classification of facial expressions using SVM for emotion care service system," in Proceedings of the 9th ASIS International Conference on Software, Engineering, Artificial Intelligence, Phuket, Thiland, August 6-8, 2008, pp. 8-12.
  23. A. Sanchez, J. V. Ruiz, A. B. Moreno, A. S. Montemayor, J. Hernandez, and J. J. Pantrigo,, "Differential optical flow applied to automatic facial expression recognition," Neurocomputing, vol. 74, no. 8, pp. 1272-1282, Mar. 2011. https://doi.org/10.1016/j.neucom.2010.07.017
  24. T. Jabid, H. Kabir and O. Chae, "Robust facial expression recognition based on local directional pattern," ETRI Journal, vol. 32, no. 5, pp. 784-794, Oct. 2010. https://doi.org/10.4218/etrij.10.1510.0132
  25. S. Zhang, X. Zhao, and B. Lei, "Robust facial expression recognition via compressive sensing," Sensors, vol. 12, no. 3, pp. 3747-3761, Mar. 2012. https://doi.org/10.3390/s120303747
  26. M. F. Valstar, M. Mehu, B. Jiang, M. Pantic, K. Scherer, and K. Scherer, "Meta-analysis of the first facial expression recognition challenge," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 42, no. 4, pp. 966-979, Aug. 2012. https://doi.org/10.1109/TSMCB.2012.2200675
  27. P. Viola and M. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  28. D. Ghimire and J. Lee, "A robust face detection method based on skin color and edges," Journal of Information Processing Systems, vol. 9, no. 1, pp. 141-156, Mar. 2013. https://doi.org/10.3745/JIPS.2013.9.1.141
  29. C. F. Juang and S. J. Shiu, "Using self-organizing fuzzy network with support vectors learning for face detection in color images," Neurocomputing, vol. 71, no. 16-18, pp. 3409-3420, Oct. 2008. https://doi.org/10.1016/j.neucom.2007.11.007
  30. S. J. Wang, C. G. Zhou, N. Zhang, X. J. Peng, Y. H. Chen, and X. Liu, "Face recognition using second-order discriminant tensor subspace analysis," Neurocomputing, vol. 74, no. 12-13, pp. 2142- 2156, Jun. 2011. https://doi.org/10.1016/j.neucom.2011.01.024
  31. W. Zong and G. B. Huang, "Face recognition based on extreme learning machine," Neurocomputing, vol. 74, no. 16, pp. 2541-2551, Sep. 2011. https://doi.org/10.1016/j.neucom.2010.12.041
  32. N. Dalal and B. Triggs, "Histogram of orientation gradients for human detection," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, June 25, 2005, pp. 886-893.
  33. P. Viola and M. Jones, "Rapid object detection using boosted cascade of simple features," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, December 8-14, 2001, pp. 511-518.
  34. P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, "The extended Cohn- Kanade dataset (CK+): a complete dataset for action unit and emotion-specific expressions," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, June 13-18, 2010, pp. 94-101.
  35. S. Zafeiriou and I. Pitas, "Discriminant graph structures for facial expression recognition," IEEE Transactions on Multimedia, vol. 10, no. 8, pp. 1528-1540, Dec. 2008. https://doi.org/10.1109/TMM.2008.2007292
  36. M. J. Lyons, J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, Dec. 1999. https://doi.org/10.1109/34.817413
  37. L. H. Thai, N. D. T. Nguyen, and T. S. Hai, "A facial expression classification system integrating canny, principal component analysis and artificial neural network," International Journal of Machine Learning and Computing, vol. 1, no. 4, pp. 388-393, Oct. 2011.

Cited by

  1. Distributed dynamic target tracking method by block diagonalization of topological matrix vol.72, pp.7, 2016, https://doi.org/10.1007/s11227-015-1499-4
  2. Output Effect Evaluation Based on Input Features in Neural Incremental Attribute Learning for Better Classification Performance vol.7, pp.1, 2015, https://doi.org/10.3390/sym7010053
  3. Image classification using label constrained sparse coding vol.75, pp.23, 2016, https://doi.org/10.1007/s11042-015-2626-1
  4. Facial expression recognition based on local region specific features and support vector machines vol.76, pp.6, 2017, https://doi.org/10.1007/s11042-016-3418-y
  5. Mouse operation on monitor by interactive analysis of intuitive hand motions vol.75, pp.23, 2016, https://doi.org/10.1007/s11042-014-2357-8
  6. Vehicle detection and recognition for intelligent traffic surveillance system vol.76, pp.4, 2017, https://doi.org/10.1007/s11042-015-2520-x
  7. Contourlet domain SAR image de-speckling via self-snake diffusion and sparse representation vol.76, pp.4, 2017, https://doi.org/10.1007/s11042-015-2560-2
  8. Recognition of facial expressions based on salient geometric features and support vector machines vol.76, pp.6, 2017, https://doi.org/10.1007/s11042-016-3428-9
  9. Boosted NNE collections for multicultural facial expression recognition vol.55, 2016, https://doi.org/10.1016/j.patcog.2016.01.032
  10. An improved collaborative recommendation algorithm based on optimized user similarity vol.72, pp.7, 2016, https://doi.org/10.1007/s11227-015-1518-5
  11. Emotion recognition from geometric fuzzy membership functions pp.1573-7721, 2019, https://doi.org/10.1007/s11042-018-6954-9