Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Segmentation of the Lip Region by Color Gamut Compression and Feature Projection

색역 압축과 특징치 투영을 이용한 입술영역 분할

  • Kim, Jeong Yeop (School of Sungsim College of General Education, Youngsan Universitty)
  • Received : 2018.07.27
  • Accepted : 2018.10.30
  • Published : 2018.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, a new type of color coordinate conversion is proposed as modified CIEXYZ from RGB to compress the color gamut. The proposed segmentation includes principal component analysis for the optimal projection of a feature vector into a one-dimensional feature. The final step adopted for lip segmentation is Otsu's threshold for a two-class problem. The performance of the proposed method was better than that of conventional methods, especially for the chromatic feature.

Keywords

MTMDCW_2018_v21n11_1279_f0001.png 이미지

Fig. 1. The color gamut mapping for x compression.

MTMDCW_2018_v21n11_1279_f0002.png 이미지

Fig. 2.The color gamut of conventional and proposed xy of CIEXYZ system.

MTMDCW_2018_v21n11_1279_f0003.png 이미지

Fig. 3. Sample image and u-v chromatic distribution of modified CIELuv: (a) input image, (b) uv data of image, (c) uv with xy, (d) uv with modified xy.

MTMDCW_2018_v21n11_1279_f0004.png 이미지

Fig. 4. Flow diagram of proposed lip segmentation method.

MTMDCW_2018_v21n11_1279_f0005.png 이미지

Fig. 5. Example test images: upper row from Cal. Tech and lower row from Georgia Tech.

MTMDCW_2018_v21n11_1279_f0006.png 이미지

Fig. 6. 5 Examples of detection result for #50 images: (a) input image, (b) ground truth image, (c) result of rg, (d) result of xy, (e) result of uv, and (f) result of modified uv-proposed.

Table 1. Conditions for the color gamut compression with regard to each terms

MTMDCW_2018_v21n11_1279_t0001.png 이미지

Table 2. Average results for the region segmentation [%]

MTMDCW_2018_v21n11_1279_t0002.png 이미지

Table 3. Comparison between conventional and proposed method [%]

MTMDCW_2018_v21n11_1279_t0003.png 이미지

References

  1. G. Potamianos, C. Neti, J. Luettin, and I. Matthews, "Audio-Visual Automatic Speech Recognition: An Overview," Issues in Audio-Visual Speech Processing, Massachusetts Institute of Technology Press, Massachusetts, 2004.
  2. A.L. Yuille, P.W. Hallinan, and D.S. Cohen, “Feature Extraction from Faces Using Deformable Templates,” International Journal of Computer Vision, Vol. 8, No. 2, pp. 99-111, 1992. https://doi.org/10.1007/BF00127169
  3. T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Grahan, “Active Shape Models - Their Training and Application,” Computer Vision and Image Understanding, Vol. 61, No. 2, pp. 38-59, 1995. https://doi.org/10.1006/cviu.1995.1004
  4. C. Bregler and Y. Konig, "Eigenlips for Robust Speech Recognition," Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 669-672, 1994.
  5. G.I. Chiou and J.N. Hwang, “Lip Reading from Color Video,” IEEE Transactions on Image Processing, Vol. 6, No. 8, pp. 1192-1195, 1997. https://doi.org/10.1109/83.605417
  6. M. Vogt, "Fast Matching of a Dynamic Lip Model to Color Video Sequences under Regular Illumination Conditions," North Atlantic Treaty Organization Advanced Study Institute Series F, Vol. 150, pp. 399-407, 1996.
  7. M.U.R. Sanchez, J. Matas, and J. Kittler, "Statistical Chromaticity-based Lip Tracking with B-Splines," Proceeding of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2973-2976, 1997.
  8. D. Sanger, Y. Miyake, H. Haneishi, and N. Tsumura "Algorithm for Face Extraction Based on Lip Detection," Journal of Imaging and Science and Technology, Vol. 41, No. 1, pp. 71-80, 1997.
  9. T. Wark, S. Sridharan, and V. Chandran, "An Approach to Statistical Lip Modelling for Speaker Identification via Chromatic Feature Extraction," Proceeding of the 14th International Conference on Pattern Recognition, Vol. 1, pp. 123-125, 1998.
  10. T.W. Lewis and D.M.W. Powers, "Lip Feature Extraction Using Red Exclusion," Proceeding of Pan-Sydney Workshop on Visual Information Processing, pp. 61-67, 2001.
  11. V. Vezhnevets, S. Soldatov, A. Degtiareva, and I.K. Park, "Automatic Extraction of Frontal Facial Features," Proceeding of 6th Asian Conference on Computer Vision, Vol. 2, pp. 1020-1025, 2004.
  12. M. Wimmer and B. Radig, "Adaptive Skin Color Classificator," Proceeding of the first International Congress for Global Science and Technology International Conference on Graphics, Vision and Image Processing GVIP-05, Vol. 1, pp. 324-327, 2005.
  13. A.W. Liew, S.H. Leung, and W.H. Lau, “Segmentation of Color Lip Images by Spatial Fuzzy Clustering,” IEEE Transactions on Fuzzy Systems, Vol. 11, No. 4, pp. 542-549, 2003. https://doi.org/10.1109/TFUZZ.2003.814843
  14. Y. Wu, R. Ma, W. Hu, T. Wang, Y. Zhang, J. Cheng, et al., "Robust Lip Localization on Multi-View Faces In Video," Proceeding of IEEE International Conference on Image Processing, pp. 481-484, 2007.
  15. S. Sridharan, S. Lucey, and V. Chandran, “Adaptive Mouth Segmentation Using Chromatic Features,” Pattern Recognition Letters, Vol. 23, No. 11, pp. 1293-1302, 2002. https://doi.org/10.1016/S0167-8655(02)00078-8
  16. O. Erol, Y. Berkay, K. Harun, E. Hakan, and U. Mustafa, "Lip Segmentation Using Adaptive Color Space Training," Proceeding of the International Conference on Auditory-Visual Speech Processing, pp. 219-222, 2008.
  17. R. Rohani, F. Sobhanmanesh, S. Alizadeh, and R. Boostani, “Lip Processing and Modeling based on Spatial Fuzzy Clustering in Color Images,” International Journal of Fuzzy Systems, Vol. 13, No. 2, pp. 65-73, 2011.
  18. J. Kim, "Extraction of Lip Region Using Chromaticity Transformation and Fuzzy Clustering," Journal of Korea Multimedia Society, Vol. 17, No. 7, pp. 806-817, 2014. https://doi.org/10.9717/KMMS.2014.17.7.806
  19. G. Wyszecki and W.S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae, John Wiley & Sons, New York, 1982.
  20. J. Kim, “Segmentation of Lip Region in Color Images by Fuzzy Clustering,” International Journal of Control, Automation and Systems, Vol. 12, No. 3, pp. 652-661, 2014. https://doi.org/10.1007/s12555-013-0245-z
  21. I.T. Jolliffe, C. Jorge, "Principal Component Analysis: a Review and Recent Developments," Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences Vol. 374, Issue 2065, pp. 20150202, 2016. https://doi.org/10.1098/rsta.2015.0202
  22. N. Otsu, “A Threshold Selection Method from Grey Level Histograms,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, pp. 62-66, 1979. https://doi.org/10.1109/TSMC.1979.4310076
  23. California Institute of Technology Face Database http://www.vision.caltech.edu/Image_Datasets/faces/faces.tar (accessed Jul., 25, 2018).
  24. Georgia Tech Face Database http://www.anefian.com/research/face_reco.htm. (accessed Jul., 25, 2018).
  25. M. Nilsson, J. Nordberg, and I. Claesson. "Face Detection Using Local SMQT Features and Split up Snow Classifier," Proceeding of 2007 IEEE International Conference on Acoustic, Speech, and Signal Processing, Vol. 2, pp. 589-592, 2007.