Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Tracking and Face Recognition of Multiple People Based on GMM, LKT and PCA

  • Lee, Won-Oh (Division of Electronics and Electrical Engineering, Dongguk University) ;
  • Park, Young-Ho (Division of Electronics and Electrical Engineering, Dongguk University) ;
  • Lee, Eui-Chul (Department of Computer Science, Sangmyung University) ;
  • Lee, Hee-Kyung (Broadcasting and Telecommunications Convergence Research Laboratory, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Park, Kang-Ryoung (Division of Electronics and Electrical Engineering, Dongguk University)
  • Received : 2011.09.23
  • Accepted : 2012.01.27
  • Published : 2012.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In intelligent surveillance systems, it is required to robustly track multiple people. Most of the previous studies adopted a Gaussian mixture model (GMM) for discriminating the object from the background. However, it has a weakness that its performance is affected by illumination variations and shadow regions can be merged with the object. And when two foreground objects overlap, the GMM method cannot correctly discriminate the occluded regions. To overcome these problems, we propose a new method of tracking and identifying multiple people. The proposed research is novel in the following three ways compared to previous research: First, the illuminative variations and shadow regions are reduced by an illumination normalization based on the median and inverse filtering of the L*a*b* image. Second, the multiple occluded and overlapped people are tracked by combining the GMM in the still image and the Lucas-Kanade-Tomasi (LKT) method in successive images. Third, with the proposed human tracking and the existing face detection & recognition methods, the tracked multiple people are successfully identified. The experimental results show that the proposed method could track and recognize multiple people with accuracy.

Keywords

References

  1. W. Hu, T. Tan, L. Wang, and S. Maybank, "A Survey on Visual Surveillance of Object Motion and Behaviors," IEEE Trans. on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol.34, No.3, pp. 334-352, 2004. https://doi.org/10.1109/TSMCC.2004.829274
  2. A. Dore, M. Soto, and C.S. Regazzoni, "Bayesian Tracking for Video Analytics," IEEE Signal Processing Magazine, Vol.27, Issue 5, pp. 46-55, 2010. https://doi.org/10.1109/MSP.2010.937395
  3. V. Saligrama, J. Konrad, and P. Jodoin, "Video Anomaly Identification," IEEE Signal Processing Magazine, Vol.27, Issue 5, pp. 18-33, 2010. https://doi.org/10.1109/MSP.2010.937393
  4. J. Houser and L. Zong, "The ARL Multimodal Sensor: A Research Tool for Target Signature Collection, Algorithm Validation, and Emplacement Studies," Proc. International Conference on Computer Vision and Pattern Recognition, 2007. (페이지 정보가 없습니다)
  5. Z. Zhu, W. Li, E. Molina, and G. Wolberg, "LDV Sensing and Processing for Remote Hearing in a Multimodal Surveillance System," Proc. International Conference on Computer Vision and Pattern Recognition, 2007. (페이지 정보가 없습니다)
  6. P. Smaragdis, B. Raj, and K. Kalgaonkar, "Sensor and Data Systems, Audio-Assisted Camera, and Acoustic Doppler Sensors," Proc. International Conference on Computer Vision and Pattern Recognition, 2007. (페이지 정보가 없습니다)
  7. C. Town, "Sensor Fusion and Environmental Modeling for Multimodal Sentient Computing," Proc. International Conference on Computer Vision and Pattern Recognition, 2007. (페이지 정보가 없습니다)
  8. T. Zhao, M. Aggarwal, T. Germano, L. Roth, A. Knowles, R. Kumar, H. Sawhney, and S. Samaraskera, "Toward a Sentient Environment: Real-time Wide Area Multiple Human Tracking with Identities," Machine Vision and Applications, Vol.19, No.5-6, pp. 301-314, 2008. https://doi.org/10.1007/s00138-008-0154-y
  9. I. Haritaoglu, D. Harwood, and L.S. Davis, "W4 : Real-time Surveillance of People and Their Activities," IEEE Trans. on Pattern Analysis Machine Intelligence, Vol.22, Issue 8, pp. 809- 830, 2000. https://doi.org/10.1109/34.868683
  10. B. Wu and R. Nevatia, "Tracking of Multiple, Partially Occluded Humans Based on Static Body Part Detection," Proc. International Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 951-958, 2006.
  11. H. Lim, V.I. Morariu, O.I. Camps, and M. Sznaier, "Dynamic Appearance Modeling for Human Tracking," Proc. International Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 751-757, 2006.
  12. T. Zhao, R. Nevatia, and F. Lv, "Segmentation and Tracking of Multiple Humans in Complex Situations," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.26, No.9, pp. 1208-1221, 2004. https://doi.org/10.1109/TPAMI.2004.73
  13. Y. Wum and T. Yu, "A Field Model for Human Detection and Tracking," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.28, No.5, pp. 753-765, 2006. https://doi.org/10.1109/TPAMI.2006.87
  14. J. Varona, J. Gonzalez, I.R. Juan, and J. Villanueva, "Importance of Detection for Video Surveillance Applications," Optical Engineering, Vol.47, No.8, pp. 087201-1-087201-9, 2008. https://doi.org/10.1117/1.2965548
  15. W. Lin, M.T. Sun, R. Poovendran, and Z. Zhang, "Activity Recognition using a Combination of Category Components and Local Models for Video Surveillance," IEEE Trans. on Circuits and Systems for Video Technology, Vol.18, No.8, pp. 1128-1139, 2008. https://doi.org/10.1109/TCSVT.2008.927111
  16. T. Xiang and S. Gong, "Activity Based Surveillance Video Content Modeling," Pattern Recognition, Vol.41, Issue 7, pp. 2309-2326, 2008. https://doi.org/10.1016/j.patcog.2007.11.024
  17. D. Makris, T. Ellis, and J. Black, "Intelligent Visual Surveillance: Towards Cognitive Vision Systems," The Open Cybernetics and Systemics Journal , Vol.2, No.??, pp. 219-229, 2008. (호 기입 요함!!!) (본 저널의 형식상 호가 없습 니다.) https://doi.org/10.2174/1874110X00802010219
  18. Z. Liu, K. Huang, T. Tan, and L. Wang, "Cast Shadow Removal with GMM for Surface Reflectance Component," Proc. International Conference on Pattern Recognition, Vol.1, pp. 727-730, 2006.
  19. Y. Matsushita, K. Nishino, K. Ikeuchi, and M. Sakauchi, "Shadow Elimination for Robust Video Surveillance," Proc. Workshop on Motion and Video Computing, pp. 15-21, 2002.
  20. R. Liu, X. Gao, R. Chu, X. Zhu, and S. Z. Li, "Tracking and Recognition of Multiple Faces at Distances," Advances in Biometrics, Vol.4642, pp. 513-522, 2007. https://doi.org/10.1007/978-3-540-74549-5_54
  21. L. Jiangwei and W. Yunhong, "Video-Based Face Tracking and Recognition on Updating Twin GMMs," Advances in Biometrics, Vol.4642, pp. 848-857, 2007. https://doi.org/10.1007/978-3-540-74549-5_89
  22. N. Friedman and S. Russell, "Image Segmentation in Video Sequences: a Probabilistic Approach," Proc. the 13th Conf. Uncertainty in Artificial Intelligence, pp. 1-3, 1997.
  23. D. Koller, J. Weber, T. Huang, J. Malik, G. Ogasawara, B. Rao, and S. Russel, "Toward Robust Automatic Traffic Scene Analysis in Real-time," Proc. International Conference on Pattern Recognition, pp. 126-131, 1994.
  24. M. Kohle, D. Merkl, and J. Kastner, "Clinical Gait Analysis by Neural Networks: Issues and Experiences," Proc. IEEE Symposium on Computer-Based Medical Systems, pp. 138-143, 1997.
  25. H. Z. Sun, T. Feng, and T. N. Tan, "Robust Extraction of Moving Objects from Image Sequences," Proc. Asian Conf. Computer Vision, pp. 961-964, 2000.
  26. W.E.L. Grimson, C. Stauffer, R. Romano, and L. Lee, "Using Adaptive Tracking to Classify and Monitor Activities in a Site," Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 22-31, 1998.
  27. S. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, and H. Wechsler, "Tracking Groups of People," Computer Vision Image Understanding, Vol.80, No.1, pp. 42-56, 2000. https://doi.org/10.1006/cviu.2000.0870
  28. A.J. Lipton, H. Fujiyoshi, and R.S. Patil, "Moving Target Classification and Tracking from Real-time Video," Proc. IEEE Workshop Applications of Computer Vision, pp. 8-14, 1998.
  29. D. Meyer, J. Denzler, and H. Niemann, "Model Based Extraction of Articulated Objects in Image Sequences for Gait Analysis," Proc. IEEE International Conference on Image Processing, pp. 78-81, 1998.
  30. J. Barron, D. Fleet, and S. Beauchemin, "Performance of Optical Flow Techniques," International Journal of Computer Vision, Vol.12, No.1, pp. 42-77, 1994. https://doi.org/10.1016/0262-8856(94)90054-X
  31. D. Meyer, J. Psl, and H. Niemann, "Gait Classification with HMM's for Trajectories of Body Parts Extracted by Mixture Densities," Proc. British Machine Vision Conference, pp. 459-468, 1998.
  32. C. Stauffer and W.E.L. Grimson, "Adaptive Background Mixture Models for Real-time Tracking," Proc. International Conference on Computer Vision and Pattern Recognition, Vol.2, pp. 246-252, 1999.
  33. Y. Tian, M. Lu, and A. Hampapur, "Robust and Efficient Foreground Analysis for Realtime Video Surveillance," Proc. International Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 1182-1187, 2005.
  34. G.P. Nam, B.J. Kang, and K.R. Park, "Robustness of Face Recognition to Variations of Illumination on Mobile Devices Based on SVM," KSI I Transactions on Internet and Information Systems, Vol.4, No.1, pp. 25-44, 2010. https://doi.org/10.3837/tiis.2010.01.002
  35. G.P. Nam, B.J. Kang, E.C. Lee, and K.R. Park, "New Fuzzy-based Retinex Method for the Illumination Normalization of Face Recognition on Mobile Device," IEEE Transactions on Consumer Electronics, in submission.
  36. D. Swets and J. Weng, "Discriminant Analysis and Eigenspace Partition Tree for Face and Object Recognition from Views," Proc. International Conference on Automatic Face and Gesture Recognition, pp. 182-187, 1996.
  37. B. Moghaddam, W.Wahid, and A. Pentland, "Beyond Eigenfaces: Probabilistic Matching for Face Recognition," Proc. International Conference on Automatic Face and Gesture Recognition, pp. 30-35, 1998.
  38. G. Guo, S. Li, and K. Chan, "Face Recognition by Support Vector Machines," Proc. International Conference on Automatic Face and Gesture Recognition, pp. 196-201, 2000.
  39. H. Rowley, S. Baluja, and T. Kanade, "Neural Network Based Face Detection," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.20, Issue 1, pp. 23-38, 1998. https://doi.org/10.1109/34.655647
  40. B. Menser and M. Wien, "Segmentation and Tracking of Facial Regions in Color Image Sequences," Proc. SPIE Visual Communications and Image Processing, Vol.4067, pp. 731-740, 2000.
  41. A. Saber and A.M. Tekalp, "Frontal-view Face Detection and Facial Feature Extraction using Color, Shape and Symmetry Based Cost Functions," Pattern Recognition, Vol.19, No.8, pp. 669-680, 1998. https://doi.org/10.1016/S0167-8655(98)00044-0
  42. P. KaewTraKulPong, and R. Bowden, "An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection," Proc. the 2nd European Workshop on Advanced Video Based Surveillance Systems, pp. 1-5, 2001.
  43. G.D. Hines, Z. Rahman, D.J. Jobson, and G.A. Woodell, "Single-scale Retinex Using Digital Signal Processors," Proc. Global Signal Processing Expo, pp. 335-343, 2004.
  44. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, "Detecting Moving Objects, Ghosts, and Shadows in Video Streams," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.25, No.10, pp. 1337-1342, 2003. https://doi.org/10.1109/TPAMI.2003.1233909
  45. B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
  46. C. Tomasi and T. Kanade, Detection and Tracking of Point Features, Technical Report CMU-CS-91-132, Carnegie Mellon University, 1991.
  47. J.Y. Bouguet, Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the Algorithm, Intel Corporation, Microprocessor Research Labs. OpenCV Documents, 2003.
  48. W.O. Lee, E.C. Lee, and K.R. Park, "Blink Detection Robust to Various Facial Poses," Journal of Neuroscience Method, Vol.193, No.2, pp. 356-372, 2010. https://doi.org/10.1016/j.jneumeth.2010.08.034
  49. J. Shi, C. Tomasi, "Good Features to Track," Proc. Int Conference on CVPR, pp. 593-600, 1994.
  50. Y. Freund and R.E. Schapire, "A Short Introduction to Boosting," Journal of Japanese Society for Artificial Intelligence, Vol.14, No.5, pp. 771-780, 1999
  51. P. Viola and M.J. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol.57, No.2, pp. 137-154, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  52. H. Abdi and L.J. Williams, "Principal Component Analysis," Computational Statistics, Vol. 2, issue 4, pp. 433-459, 2010 https://doi.org/10.1002/wics.101
  53. M. Turk and A. Pentland, "Eigenfaces for Recognition," Journal of Cognitive Neuroscience, Vol.3, No.1, pp. 71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71
  54. H.H. Nam, B.J. Kang, and K.R. Park, "Comparison of Computer and Human Face Recognition According to Facial Components," Journal of Korea Multimedia Society, Vol.15, No.1, pp. 40-50, 2012 https://doi.org/10.9717/kmms.2012.15.1.040
  55. Logitech Camera, http://www.logitech.com (accessed on December 24, 2011)
  56. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/ (accessed on December 24, 2011)
  57. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-Based Object Tracking", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.25, No.5, pp. 564-577, 2003. https://doi.org/10.1109/TPAMI.2003.1195991
  58. D. Comaniciu, V. Ramesh, and P. Meer, "Real- Time Tracking of Non-Rigid Objects using Mean Shift," Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR '00) , Vol.2, pp. 142-149, 2000.
  59. G.R. Bradski, Computer Video Face Tracking for Use in a Perceptual User Interface, Technical report, Intel Technology Journal, Q2, 1998.
  60. D. Thomas, K.W. Bowyer, and P.J. Flynn "Strategies for Improving Face Recognition from Video," Advances in Biometrics, pp. 339-361, 2008.

Cited by

  1. A Secure Face Cryptogr aphy for Identity Document Based on Distance Measures vol.16, pp.10, 2013, https://doi.org/10.9717/kmms.2013.16.10.1156
  2. Human Face Recognition used Improved Back-Propagation (BP) Neural Network vol.21, pp.4, 2012, https://doi.org/10.9717/kmms.2018.21.4.471