KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time Eye Tracking Using IR Stereo Camera for Indoor and Outdoor Environments

  • Lim, Sungsoo (Department of Electronics and Radio Engineering, Kyung Hee University) ;
  • Lee, Daeho (Humanitas College, Kyung Hee University)
  • Received : 2016.09.02
  • Accepted : 2017.05.02
  • Published : 2017.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a novel eye tracking method that can estimate 3D world coordinates using an infrared (IR) stereo camera for indoor and outdoor environments. This method first detects dark evidences such as eyes, eyebrows and mouths by fast multi-level thresholding. Among these evidences, eye pair evidences are detected by evidential reasoning and geometrical rules. For robust accuracy, two classifiers based on multiple layer perceptron (MLP) using gradient local binary patterns (GLBPs) verify whether the detected evidences are real eye pairs or not. Finally, the 3D world coordinates of detected eyes are calculated by region-based stereo matching. Compared with other eye detection methods, the proposed method can detect the eyes of people wearing sunglasses due to the use of the IR spectrum. Especially, when people are in dark environments such as driving at nighttime, driving in an indoor carpark, or passing through a tunnel, human eyes can be robustly detected because we use active IR illuminators. In the experimental results, it is shown that the proposed method can detect eye pairs with high performance in real-time under variable illumination conditions. Therefore, the proposed method can contribute to human-computer interactions (HCIs) and intelligent transportation systems (ITSs) applications such as gaze tracking, windshield head-up display and drowsiness detection.

Keywords

References

  1. S. A. Sirohey and A. Rosenfeld, "Eye detection in a face image using linear and nonlinear filters," Pattern recognition, vol. 34, no. 7, pp. 1367-1391, 2001. https://doi.org/10.1016/S0031-3203(00)00082-0
  2. S. Tripathi, V. Sharma and S. Sharma, "Face detection using combined skin color detector and template matching method," International Journal of Computer Applications, vol. 26, no. 7, pp. 5-8, July, 2011. https://doi.org/10.5120/3119-4290
  3. J. Dowdall, I. Pavlidis and G. Bebis, "Face detection in the near-IR spectrum," Image and Vision Computing, vol. 21, no. 7, pp. 565-578, July, 2003. https://doi.org/10.1016/S0262-8856(03)00055-6
  4. G. C. Feng and P. C. Yuen, "Multi-cues eye detection on gray intensity image," Pattern recognition, vol. 34, no. 5, pp. 1033-1046, May, 2001. https://doi.org/10.1016/S0031-3203(00)00042-X
  5. K. Peng, L. Chen, S. Ruan and G. Kukharev, "A robust algorithm for eye detection on gray intensity face without spectacles," Jounal of Computer Science & Technology, vol. 5, no. 3, pp. 127-132 2005.
  6. A. Krolak and P. Strumillo, "Eye-blink detection system for human-computer interaction," Universal Access in the Information Society, vol. 11, no. 4, pp. 409-419, 2012. https://doi.org/10.1007/s10209-011-0256-6
  7. W. Zhong and Z. Huang, "The real-time eye detection for single user based on template matching," in Proc. of IEEE Int. Conf. on Computational Science and Engineering, pp. 831-834, December 19-21, 2014.
  8. M. Rezaei and R. Klette, "Adaptive Haar-like classifier for eye status detection under non-ideal lighting conditions," in Proc. of 27th Conf. on Image and Vision Computing New Zealand, pp. 512-526, 2012.
  9. A. Majumder, L. Behera and V. K. Subramanian, "Automatic and robust detection of facial features in frontal face images," in Proc. of UKSim 13th Int. Conf. on Computer Modelling and Simulation, pp. 331-336, March 30-April 1, 2011.
  10. D. W. Hansen and R. I. Hammoud, "An improved likelihood model for eye tracking," Computer Vision and Image Understanding, vol. 106, no. 2-3, pp. 220-230, May-June 2007. https://doi.org/10.1016/j.cviu.2006.06.012
  11. B. Bhowmick and K. S. C. Kumar, "Detection and classification of eye state in IR camera for driver drowsiness identification," in Proc. of IEEE Int. Conf. on Signal and Image Processing Applications, November 18-19, 2009.
  12. S. W. Shih and J. Liu, "A novel approach to 3-D gaze tracking using stereo cameras," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 34, no. 1, pp. 234-245, February, 2004. https://doi.org/10.1109/TSMCB.2003.811128
  13. C. Ma, S. J. Baek, K. A. Choi and S. J. Ko, "Improved remote gaze estimation using corneal reflection-adaptive geometric transforms," Optical Engineering, vol. 53, no. 5, 053112, May, 2014. https://doi.org/10.1117/1.OE.53.5.053112
  14. D. Beymer and M. Flicker, "Eye gaze tracking using an active stereo head," in Proc. of IEEE Conf. Computer Vision and Pattern Recognition, June 18-20, 2003.
  15. Z. Zhu, K. Fujimura and Q. Ji, "Real-time eye detection and tracking under various light conditions," in Proc. of ACM Symp. on Eye Tracking Research & Applications, pp. 139-144, 2002.
  16. Z. Zhu and Q. Ji, "Robust real-time eye detection and tracking under variable lighting conditions and various face orientations," Computer Vision and Image Understanding, vol. 98, no. 1, pp. 124-154, April, 2005. https://doi.org/10.1016/j.cviu.2004.07.012
  17. Y. Park and D. Lee, "3D vision-based security monitoring for railroad stations," Journal of the Optical Society of Korea, vol. 14, no. 4, pp. 451-457, December, 2010. https://doi.org/10.3807/JOSK.2010.14.4.451
  18. O. Nikisins and M.Greitans, "Local binary patterns and neural network based technique for robust face detection and localization," in Proc. of Int. Conf. of IEEE Biometrics Special Interest Group, pp. 1-6, September 6-7, 2012.
  19. B. Jun, I. Choi and D. Kim, "Local transform features and hybridization for accurate face and human detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1423-1436, June, 2013. https://doi.org/10.1109/TPAMI.2012.219
  20. Y. He and N. Sang, "Robust illumination invariant texture classification using gradient local binary patterns," in Proc. of Int. Works. on IEEE Multi-Platform/Multi-Sensor Remote Sensing and Mapping , pp. 1-6, January 10-12, 2011.
  21. Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vo. 22, no. 11, pp. 1330-1334, November, 2000. https://doi.org/10.1109/34.888718
  22. C. Kim, S. Choi, M. Turk and C. Choi, "A new biased discriminant analysis using composite vectors for eye detection," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 42, no 4, pp. 1095-1106, August, 2012. https://doi.org/10.1109/TSMCB.2012.2186798
  23. D. Petrisor, C. Fosalau, M. Avila and F. Mariut, "Algorithm for face and eye detection using colour segmentation and invariant features," in Proc. of 34th Int. Conf. on Telecommunications and Signal Processing , pp. 564-569, August 18-20, 2011.
  24. S. Zhao and R. R. Grigat, "Robust eye detection under active infrared illumination," in Proc. of Int. Conf. on Pattern Recognition, August 20-24, 2006.

Cited by

  1. Drowsiness Driving Prevention System using Bone Conduction Device vol.13, pp.9, 2017, https://doi.org/10.3837/tiis.2019.09.011