KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Noisy-Robust Approach for Facial Expression Recognition

  • Tong, Ying (College of Communications Engineering, PLA University of Science and Technology) ;
  • Shen, Yuehong (College of Communications Engineering, PLA University of Science and Technology) ;
  • Gao, Bin (College of Communications Engineering, PLA University of Science and Technology) ;
  • Sun, Fenggang (College of Communications Engineering, PLA University of Science and Technology) ;
  • Chen, Rui (Department of Communication Engineering, Nanjing Institute of Technology) ;
  • Xu, Yefeng (College of Communications Engineering, PLA University of Science and Technology)
  • Received : 2016.05.29
  • Accepted : 2017.02.09
  • Published : 2017.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Accurate facial expression recognition (FER) requires reliable signal filtering and the effective feature extraction. Considering these requirements, this paper presents a novel approach for FER which is robust to noise. The main contributions of this work are: First, to preserve texture details in facial expression images and remove image noise, we improved the anisotropic diffusion filter by adjusting the diffusion coefficient according to two factors, namely, the gray value difference between the object and the background and the gradient magnitude of object. The improved filter can effectively distinguish facial muscle deformation and facial noise in face images. Second, to further improve robustness, we propose a new feature descriptor based on a combination of the Histogram of Oriented Gradients with the Canny operator (Canny-HOG) which can represent the precise deformation of eyes, eyebrows and lips for FER. Third, Canny-HOG's block and cell sizes are adjusted to reduce feature dimensionality and make the classifier less prone to overfitting. Our method was tested on images from the JAFFE and CK databases. Experimental results in L-O-Sam-O and L-O-Sub-O modes demonstrated the effectiveness of the proposed method. Meanwhile, the recognition rate of this method is not significantly affected in the presence of Gaussian noise and salt-and-pepper noise conditions.

Keywords

References

  1. K.T. Song and S.C. Chien, ''Facial Expression Recognition Based on Mixture of Basic Expressions and Intensities,' in Proc. of IEEE International Conf. on Systems,Man,and Cybernetics, pp. 3123-3128,Oct. 2012.
  2. L.H.ZHAO, C.K.YANG, F.PAN et al., ''Face Recognition Based on Gabor with 2DPCA and PCA,' in Proc. of 24th Chines Control and Decision Conference, pp.2632-2635, 2012.
  3. C.F.Beckmann, S.M. Smith., ''Probabilistic independent component analysis for functional magnetic resonance imaging," IEEE Trans on Medical. Imaging Magazine, vol. 23, no. 2, pp. 137-152, Feb. 2004. https://doi.org/10.1109/TMI.2003.822821
  4. S.Nikitidis, A.Tefas et.al., ''Maximum Margin Projection Subspace Learning for Visual Data Analysis,'' IEEE Trans on Image ProcessingMagazine,vol.23, no.10,pp.4413-4425, Oct. 2014. https://doi.org/10.1109/TIP.2014.2348868
  5. Q.X. Gao, J.J. Liu et al., "Enhanced fisher discriminant criterion for image recognition," Pattern Recognition Magazine, vol.45, no.10, pp. 3717-3724, Oct. 2012. https://doi.org/10.1016/j.patcog.2012.03.024
  6. G.Lu,Z.Lin et al., ''Face recognition using discriminant locality preserving projections based on maximum margin criterion,' 'PatternRecognitionMagazine, vol.43, no.10, pp.3572-3579, Oct. 2010.
  7. J.X.Ruan, ''Study on Key Technology for Multi-Pose Face Detection and Facial Expression Recognition," South China University of Technology-Thesis, pp.64-83, Jun. 2010.
  8. M.Kyperountas, A.Tefas, I. Pitas., ''Salient feature and reliable classifier selection for facial expression classification," PatternRecognitionMagazine, vol. 43, no. 3, pp.972-986, Mar. 2010.
  9. T. Ojala, M. Pietikainen, D. Harwood., "A comparative study of texture measures with classification based on feature distributions," PatternRecognitionMagazine, vol. 29, no. 1, pp. 51-59, Jan. 1996.
  10. D. Huang,C.F.Shan,M. Ardabilian,et al., ''Local Binary Patterns and Its Application to Facial Image Analysis:A Survey,'' IEEE Transactions on Systems Man and Cybernetics-Part C:Applications and ReviewMagazine, vol. 41, no. 6, pp. 765-781, Nov. 2011. https://doi.org/10.1109/TSMCC.2011.2118750
  11. G. Zhao and M. Pietikainen, ''Dynamic texture recognition using local binary patterns with an application to facial expressions,'' IEEE Transactions on Pattern Analysis and Machine IntelligenceMagazine, vol. 29, no. 6, pp. 915-928, Jun. 2007. https://doi.org/10.1109/TPAMI.2007.1110
  12. J.Caoand C. Tong, "Facial expression recognition based on LBP-EHMM," in Proc. of Congress on Image and Signal Processing, vol. 2, pp. 371-375, May. 2008.
  13. T. Jabid, M.H.Kabir, O. Chae, "Robust Facial Expression Recognition Based on Local Directional Pattern," ETRIJournalMagazine, vol. 32, no. 5, pp. 784-794, Oct. 2010.
  14. N. Dalal,B. Triggs, ''Histograms of oriented gradients for human detection," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR), pp. 886-893, June. 2005.
  15. F.Y. Song, X.Y.Tan,X.Liu,S.C. Chen, "Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients," Pattern Recognition Magazine, vol. 47, no. 9, pp. 2825-2838, Sep. 2014. https://doi.org/10.1016/j.patcog.2014.03.024
  16. R.Minetto, N.Thome, M. Cord et al., ''T-HOG: An effective gradient-based descriptor for single line text regions," Pattern Recognition Magazine, vol. 46, no. 3, pp. 1078-1090, Mar. 2013. https://doi.org/10.1016/j.patcog.2012.10.009
  17. G.Gilanie, M.Attique, S.Naweed, "Object extraction from T2 weighted brain MR image using histogram based gradient calculation," Pattern Recognition Letters Magazine, vol. 34, no. 12, pp. 1356-1363, Sep. 2013. https://doi.org/10.1016/j.patrec.2013.04.010
  18. L.P.LI., "A Research on Image De-noising Methods,'' Yangtze University-Thesis, Jun. 2012.
  19. P. Perona and J. Malik, ''Scale space and edge detection using anisotropic diffusion," IEEE Transactions on Pattern Analysis and Machine Intelligence Magazine, vol. 12, no. 7, pp. 629-639,Jul. 1990. https://doi.org/10.1109/34.56205
  20. Z.YAN, H.M. GU,C.G.CAI, "Enhancement Processing on Seismic Image based on anisotropic Diffusion," Chinese Journal of Geophysics Magazine, vol. 48, no. 3, pp. 390-394,Oct. 2013.
  21. J.L. CHEN, X.J. LIU, "On Digital Image Watermarking Based on Anisotropic Diffusion,'' Computer Applications and Software Magazine, vol. 30, no. 7, pp. 151-153, Jun. 2013.
  22. L.J. FU, Y. YAO, Z.L.FU, ''Filtering method for medical images based on median filtering and anisotropic diffusion,'' Journal of Computer ApplicationsMagazine, vol. 34, no. 1, pp. 145-148, May. 2014.
  23. O.Deniz, G.Bueno, J.Salido, etal, "Face recognition using Histograms of Oriented Gradients," Pattern Recognition Letters Magazine, vol. 32, no. 12, pp. 1598-1603, Sep. 2011. https://doi.org/10.1016/j.patrec.2011.01.004
  24. B.C.Zhang, Y.S.Gao, S.Q.Zhao, et al., "Local Derivative Pattern Versus Local Binary Pattern : Face Recognition with High-Order Local Pattern Descriptor," IEEE Transactions on Image Processing Magazine, vol. 19, no. 2, pp. 533-544, Feb. 2010. https://doi.org/10.1109/TIP.2009.2035882
  25. Y.B. Zheng, X.S. Huang, S.J.Feng, ''An Image Matching Algorithm Based on Combination of SIFT and the Rotation Invariant LBP,'' Journal of Computer 2 Aided Design & Computer Graphics Magazine, vol. 22, no. 2, pp. 286-292, Sep. 2010.
  26. X. Tan and B. Triggs, ''Enhanced local texture feature sets for facerecognition under difficult lighting conditions,'' in Proc. of Anal. Model Faces Gestures, pp.168-182, Oct. 2007.
  27. Z. Xiang,H.L. Tan,Z.M.Ma, ''Performance Comparison of Improved HOG,Gabor and LBP,'' Journal of Computer-Aided Design & Computer Graphics Magazine, vol. 24, no. 6, pp. 787-792, Feb. 2012.
  28. A.R. Rivera, J.R. Castillo, O.Chae, ''Local Directional Number Pattern for Face Analysis : Face and Expression Recognition,'' IEEE Transactions on Image processing Magazine, vol. 22, no. 5, pp. 1740-1752, May. 2013. https://doi.org/10.1109/TIP.2012.2235848
  29. F.Ahmed, ''Gradient directional pattern: a robust feature descriptor for facial expression recognition,'' Electronics Letters Magazine, vol. 48, no. 19, pp. 1203-1204, Sep. 2012. https://doi.org/10.1049/el.2012.1841
  30. C.X. Ding, J.Y. Choi, D.C. Tao, et al., ''Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition,'' IEEE Transactions on Pattern Analysis and Machine Intelligence Magazine, vol. 38, no. 3, pp. 518-531, Mar. 2016. https://doi.org/10.1109/TPAMI.2015.2462338
  31. C.X. Ding and D.C. Tao., 'Robust Face Recognition via Multimodal Deep Face Representation,'' IEEE Transactions on Multimedia Magazine, vol. 17, no. 2, pp.2049-2058, Nov. 2015. https://doi.org/10.1109/TMM.2015.2477042
  32. Wang Q, Ma L, Gao Q, et al., ''Adaptive maximum margin analysis for image recognition,'' Pattern Recognition, vol. 61, pp.339-347, 2017. https://doi.org/10.1016/j.patcog.2016.07.025
  33. Zhang L, Wang L, Lin W., ''Semi-supervised biased maximum margin analysis for interactive image retrieval,'' IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 2294-2308, 2012. https://doi.org/10.1109/TIP.2011.2177846
  34. Gao Q, Liu J, Cui K, et al., ''Stable locality sensitive discriminant analysis for image recognition,'' Neural Networks, vol. 54, pp. 49-56, 2014. https://doi.org/10.1016/j.neunet.2014.02.009
  35. Wong W K, Zhao H T., ''Supervised optimal locality preserving projection,'' Pattern Recognition, vol. 45, no. 1, pp. 186-197, 2012. https://doi.org/10.1016/j.patcog.2011.05.014
  36. Chao W L, Ding J J, Liu J Z., ''Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection,'' Signal Processing, vol. 117, pp. 1-10, Mar. 2015. https://doi.org/10.1016/j.sigpro.2015.04.007
  37. Wolf L, Hassner T, Taigman Y., ''Effective unconstrained face recognition by combining multiple descriptors and learned background statistics,'' IEEE Transactions on pattern analysis and machine intelligence, vol. 33, no. 10, pp. 1978-1990, 2011. https://doi.org/10.1109/TPAMI.2010.230
  38. Wolf L, Hassner T, Taigman Y., ''Descriptor based methods in the wild,'' in Proc. of Workshop on faces in 'real-life' images: Detection, alignment, and recognition. 2008.
  39. Vu N S, Caplier A., ''Enhanced patterns of oriented edge magnitudes for face recognition and image matching,'' IEEE Transactions on Image Processing, vol. 21, no. 3, pp. 1352-1365, 2012. https://doi.org/10.1109/TIP.2011.2166974
  40. Lei Z, Pietikainen M, Li S Z., ''Learning discriminant face descriptor,''IEEE Transactions on pattern analysis and machine intelligence, vol. 36, no. 2, pp. 289-302, 2014. https://doi.org/10.1109/TPAMI.2013.112
  41. Shan C, Gong S, McOwan P W., ''Facial expression recognition based on local binary patterns: A comprehensive study,'' Image and Vision Computing, vol. 27, no. 6, pp. 803-816, 2009. https://doi.org/10.1016/j.imavis.2008.08.005
  42. ZHAO De, HE Chuan-Jiang, CHEN Qiang, ''Anisotropic Diffusion Model Combined with Local Entropy," PR & AI, vol. 25, no. 4, pp. 642-647, Aug. 2012.
  43. Schroder J B., ''Smoothed aggregation solvers for anisotropic diffusion,'' Numerical Linear Algebra With Applications, vol. 19, no. 2, pp. 292-312, 2012.
  44. Wang H, Wang Y, Ren W, et al., ''Image denoising using anisotropic second and fourth order diffusions based on gradient vector convolution,'' Computer Science and Information Systems, vol. 9, no. 4, pp. 1493-1511, Mar. 2012. https://doi.org/10.2298/CSIS120219060W