• ETRI Journal
• /
• v.34 no.6
• /
• pp.932-941
• /
• 2012

A powerful interaction mechanism is one of the key elements for the success of smart TVs, which demand far more complex interactions than traditional TVs. This paper proposes a novel interface based on the famous touch interaction model but utilizes long-range bare hand tracking to emulate touch actions. To satisfy the essential requirements of high accuracy and immediate response, the proposed hand tracking algorithm adopts a fast color-based tracker but with modifications to avoid the problems inherent to those algorithms. By using online modeling and motion information, the sensitivity to the environment can be greatly decreased. Furthermore, several ideas to solve the problems often encountered by users interacting with smart TVs are proposed, resulting in a very robust hand tracking algorithm that works superbly, even for users with sleeveless clothing. In addition, the proposed algorithm runs at a very high speed of 82.73 Hz. The proposed interface is confirmed to comfortably support most touch operations, such as clicks, swipes, and drags, at a distance of three meters, which makes the proposed interface a good candidate for interaction with smart TVs.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.4 no.4
• /
• pp.618-632
• /
• 2010

This paper presents a novel approach to control the augmented reality (AR) objects robustly in a marker-less AR system by fingertip tracking and hand pattern recognition. It is known that one of the promising ways to develop a marker-less AR system is using human's body such as hand or face for replacing traditional fiducial markers. This paper introduces a real-time method to manipulate the overlaid virtual objects dynamically in a marker-less AR system using both hands with a single camera. The left bare hand is considered as a virtual marker in the marker-less AR system and the right hand is used as a hand mouse. To build the marker-less system, we utilize a skin-color model for hand shape detection and curvature-based fingertip detection from an input video image. Using the detected fingertips the camera pose are estimated to overlay virtual objects on the hand coordinate system. In order to manipulate the virtual objects rendered on the marker-less AR system dynamically, a vision-based hand control interface, which exploits the fingertip tracking for the movement of the objects and pattern matching for the hand command initiation, is developed. From the experiments, we can prove that the proposed and developed system can control the objects dynamically in a convenient fashion.

• Journal of the HCI Society of Korea
• /
• v.10 no.2
• /
• pp.65-71
• /
• 2015

In this paper, we present binocular augmented reality system with smartphones. With improvements of performance and resolution of recent smartphones, various virtual reality devices using smartphones and applications are appearing in the market. But the augmented reality system with smartphones are not that popular because of the lack of required technologies. In our work, we discuss about 3D mapping and tracking using computer vision. Also, we provide hybrid tracking using sensors in the smartphone which can handle the computer vision failure cases. Bare hand 3D interaction method in our system will be also presented.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.50 no.6
• /
• pp.201-211
• /
• 2013

In this paper, we propose a hand tracking and gesture recognition system. Our system employs a depth capture device to obtain 3D geometric information of user's bare hand. In particular, we build a flexible tracking volume and restrict the hand tracking area, so that we can avoid diverse problems caused by conventional object detection/tracking systems. The proposed system computes running average of the hand position, and tracking volume is actively adjusted according to the statistical information that is computed on the basis of uncertainty of the user's hand motion in the 3D space. Once the position of user's hand is obtained, then the system attempts to detect stretched fingers to recognize finger gesture of the user's hand. In order to test the proposed framework, we built a NUI system using the proposed technique, and verified that our system presents very stable performance even in the case that multiple objects exist simultaneously in the crowded environment, as well as in the situation that the scene is occluded temporarily. We also verified that our system ensures running speed of 24-30 frames per second throughout the experiments.

• Journal of Korea Multimedia Society
• /
• v.15 no.3
• /
• pp.323-335
• /
• 2012

In this paper, we propose an AR(augmented reality) system using user's bare hand based on computer vision. It is important for registering a virtual object on the real input image to detect and track correct feature points. The AR systems with markers are stable but they can not register the virtual object on an acquired image when the marker goes out of a range of the camera. There is a tendency to give users inconvenient environment which is limited to control a virtual object. On the other hand, our system detects fingertips as fiducial features using adaptive ellipse fitting method considering the geometric characteristics of hand. It registers the virtual object stably by getting movement of fingertips with determining the shortest distance from a palm center. We verified that the accuracy of fingertip detection over 82.0% and fingertip ordering and tracking have just 1.8% and 2.0% errors for each step. We proved that this system can replace the marker system by tacking a camera projection matrix effectively in the view of stable augmentation of virtual object.

• Proceedings of the Korean Information Science Society Conference
• /
• 2005.07b
• /
• pp.766-768
• /
• 2005

최근 대형 디스플레이 및 웨어러블 컴퓨터의 등장과 함께 키보드와 마우스를 사용하는 일반 데스크탑 환경에서 벗어난 컴퓨터와의 자연스러운 상호 작용 연구가 활발히 진행되고 있다. 본 논문에서는 스크린 정면에 놓인 프로젝터가 스크린과 그 위에 놓인 사용자의 손 위에 화면을 투영할 때 PC 카메라로 입력된 프레임 속에서 손의 영역을 인식하여 컴퓨터와 상호작용하게 하고자 한다. 이 경우에 투영된 빛이 사용자의 손 위에도 합쳐짐으로 인하여 피부의 고유색이 사라진다. 또한, 투영되는 화면이 사용자와 컴퓨터의 상호 작용에 따라 추정할 수 없이 변함에 따른 적응적 인식 방법이 필요하다. 따라서, 본 논문에서는 손을 인식하기에 앞서 스크린에 투영될 원본 이미지에 대해 칼라 보정을 수행하여 추정되는 카메라 입력 프레임을 생성한다. 이를 위해 우선 백색 영상을 투영하여 프레임 내의 자기 오차 맵을 생성한 후 R,G,B 채널 별로 원본 값에 대한 카메라 반응 값들을 룩업 테이블에 저장한다. 이를 통해 원본 이미지에 대해 칼라보정을 수행하고, 생성된 카메라 추정 프레임과 실제 카메라로 입력된 프레임 간 자기 성분을 비교하여 손 영역을 검출한다. 실험 결과, 주변의 조명 상태나 프로젝터 및 카메라의 위치에 관계없이 안정적인 인식 결과를 보였다.

• Journal of Korea Multimedia Society
• /
• v.16 no.6
• /
• pp.720-734
• /
• 2013

In this paper, we suggest an augmented reality authoring tool system that users can easily make augmented reality contents using hand gesture and marker-based interaction methods. The previous augmented reality authoring tools are focused on augmenting a virtual object and to interact with this kind of augmented reality contents, user used the method utilizing marker or sensor. We want to solve this limited interaction method problem by applying marker based interaction method and gesture interaction method using depth sensing camera, Kinect. In this suggested system, user can easily develop simple form of marker based augmented reality contents through interface. Also, not just providing fragmentary contents, this system provides methods that user can actively interact with augmented reality contents. This research provides two interaction methods, one is marker based method using two markers and the other is utilizing marker occlusion. In addition, by recognizing and tracking user's bare hand, this system provides gesture interaction method which can zoom-in, zoom-out, move and rotate object. From heuristic evaluation about authoring tool and compared usability about marker and gesture interaction, this study confirmed a positive result.