Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

3D Pointing for Effective Hand Mouse in Depth Image

깊이영상에서 효율적인 핸드 마우스를 위한 3D 포인팅

  • 주성일 (숭실대학교 글로벌미디어학과) ;
  • 원선희 (숭실대학교 글로벌미디어학과) ;
  • 최형일 (숭실대학교 글로벌미디어학과)
  • Received : 2014.08.11
  • Accepted : 2014.08.24
  • Published : 2014.08.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a 3D pointing interface that is designed for the efficient application of a hand mouse. The proposed method uses depth images to secure high-quality results even in response to changes in lighting and environmental conditions and uses the normal vector of the palm of the hand to perform 3D pointing. First, the hand region is detected and tracked using the existing conventional method; based on the information thus obtained, the region of the palm is predicted and the region of interest is obtained. Once the region of interest has been identified, this region is approximated by the plane equation and the normal vector is extracted. Next, to ensure stable control, interpolation is performed using the extracted normal vector and the intersection point is detected. For stability and efficiency, the dynamic weight using the sigmoid function is applied to the above detected intersection point, and finally, this is converted into the 2D coordinate system. This paper explains the methods of detecting the region of interest and the direction vector and proposes a method of interpolating and applying the dynamic weight in order to stabilize control. Lastly, qualitative and quantitative analyses are performed on the proposed 3D pointing method to verify its ability to deliver stable control.

본 논문에서는 효율적인 핸드 마우스를 위한 3D 포인팅 인터페이스에 대해 제안한다. 제안하는 방법에서는 조명과 환경 변화에 강건하기 위해 깊이영상을 이용하고, 손바닥의 법선벡터를 이용하여 3D 포인팅을 구성한다. 먼저, 손 영역 검출과 추적은 기존 방법을 이용하고, 이로부터 획득한 정보를 바탕으로 손바닥의 영역을 예측하여 관심 영역을 획득한다. 관심 영역을 획득하면, 해당 영역을 평면의 방정식으로 근사시키고, 법선 벡터를 추출한다. 다음으로, 안정적인 제어를 위해 추출한 법선 벡터를 이용하여 보간을 수행하고, 교점을 검출한다. 검출된 교점은 안정성과 효율성을 위해 시그모이드 함수를 이용한 동적 가중치가 적용되고, 최종적으로 2D 좌표계로 변환된다. 본 논문에서는 관심 영역, 방향 벡터 검출 방법에 대해 설명하고 안정적인 제어를 위한 보간 방법과 동적 가중치 적용방법에 대해 제안한다. 마지막으로 제안된 3D 포인팅의 정성적, 정량적 분석을 통해 안정적인 제어 가능성을 입증한다.

Keywords

References

  1. http://www.cyberglovesystems.com/products/cyberglove-ii/overview
  2. D. J. Sturman and D. Zeltzer, "A Survey of Glove-based Input", IEEE Computer Graphics and Applications, Vol. 14, No. 1, pp. 30-39, January 1994. https://doi.org/10.1109/38.250916
  3. T. G. Zimmerman, J. Lanier, C. Blanchard, S. Bryson, and Y. Harvill, "A Hand Gesture Interface Device", CHI'87 In proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, pp. 189-192, May 1987.
  4. http://www.dh.aist.go.jp/en/research/centered/dhand-link2/
  5. R. Y. Wang and J. Popovic, "Real-Time Hand-Tracking with a Color Glove", ACM Transaction on Graphics(TOG), Vol. 28, No. 3, Article 63, August 2009.
  6. D. Kim, O. Hilliges, S. Izadi, A. Butler, J. Chen, I. Oikonomidis and P. Olivier, "Digits : Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor", In proceedings of the 25th annual ACM symposium on User Interface Software and Technology, pp. 167-176, October 2012.
  7. S. P. Priyal and P. K. Bora, "A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments", International Journal on Pattern Recognition, Vol. 46, No. 8, pp. 2202-2219, August 2013. https://doi.org/10.1016/j.patcog.2013.01.033
  8. Z. Song, H. Yang, Y. Zhao and F. Zheng, "Hand Detection and Gesture Recognition Exploit Motion Times Image in Complicate Scenarios", In proceedings of Advances in Visual Computing - 6th International Symposium, ISVC 2010, Vol. 6454, pp. 628-636, November 2010.
  9. S. Malassiotis and M. G. Strintzis, "Real-time hand posture recognition using range data", International Journal on Image and Vision Computing, Vol. 26, No. 7, pp. 1027-1037, July 2008. https://doi.org/10.1016/j.imavis.2007.11.007
  10. F. Kirac, Y. E. Kara and L. Akarun, "Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data", International Journal on Pattern Recognition Letters- Special Issue on Depth Image Analysis, September 2013.
  11. P. Suryanarayan, A. Subramanian and D. Mandalapu, "Dynamic Hand Pose Recognition using Depth Data", In proceedings of the 2010 International Conference on Pattern Recognition, pp. 3105-3108, August 2010.
  12. F. Dominio, M. Donadeo and P. Zanuttigh, "Combining multiple depth-based descriptors for hand gesture recognition", International Journal on Pattern Recognition Letters, October 2013.
  13. N. D. Binh, E. Shuichi and T. Ejima, "Real-time hand tracking and gesture recognition system", In proceedings of International Conference on Graphics, Vision and Image Processing, pp. 362-368, December 2005.
  14. A. Kurakin, Z. Zhang and Z. Liu, "A Real Time System for Dynamic Hand Gesture Recognition with a Depth Sensor", In proceedings of the 20th European Signal Processing Conference, pp. 1975-1979, August 2012.
  15. M. Elmezain, A. Al-Hamadi and B. Michaelis, "Hand trajectory-based gesture spotting and recognition using HMM", In proceedings of the International Conference on Image Processing, pp. 3577-3580, November 2009.
  16. X. Liu and K. Fujimura, "Hand Gesture Recognition using Depth Data", In proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 529-534, May 2004.
  17. H. J. Park, "A Method for Controlling Mouse Movement using a Real-Time Camera", Brown University, Providence, RI, USA, Department of Computer Science, 2010.
  18. D. D. Luong, S. Lee and T. S. Kim, "Human Computer Interface Using the Recognized Finger Parts of Hand Depth Silhouette via Random Forests", In proceedings of the 2013 13th International Conference on Control Automation and Systems, pp. 905-909, October 2013.
  19. A. Aksac, O. Ozturk and T. Ozyer, "Real-time Multi-Objective Hand Posture/Gesture Recognition by Using Distance Classifiers and Finite State Machine for Virtual Mouse Operations", In proceedings of the 2011 7th International Conference on Electrical and Electronics Engineering, pp. II-457-II-461, December 2011.
  20. S. I. Joo, "Dynamic Soft Cascade : Application to Gesture Recognition", Ph.D. Thesis, University of Soongsil, June 2014.
  21. S. I. Joo, S. H. Weon and H. I. Choi, "Real-Time Depth-Based Hand Detection and Tracking", The Scientific World Journal, Vol. 2014, Article ID 284827, pp. 1-17, March 2014.
  22. S. J. Miller, "The Method of Least Squares", Mathematics Department Brown University (2006): 1-7.
  23. D. Eberly, "Least Squares Fitting of Data", http://www.geometrictools.com/, 1999.

Cited by

  1. 깊이 영상 기반 실내 공간 인식 vol.19, pp.11, 2014, https://doi.org/10.9708/jksci.2014.19.11.053