• Journal of Internet of Things and Convergence
• /
• v.10 no.5
• /
• pp.1-11
• /
• 2024

This paper describes a hybrid gesture interface based on MediaPipe that recognizes facial gestures and hand gestures using the MediaPipe Hands model and MediaPipe Face Mesh model, and then combines them. First, the presence of hands and faces is determined by individually detecting 3D hand landmarks of MediaPipe Hands model and 3D face landmarks of MediaPipe Face Mesh model from camera input frames, and then the face cursor position and face gestures, as well as the hand cursor position and hand gestures are recognized. Then, these are mixed in a user-friendly way to implement a user interface based on hand-face hybrid gestures. The proposed hand-face hybrid gesture interface based on MediaPipe has the advantage that the gesture mode is set to either the hand or the face, but the interface can be controlled freely using the face and hands without additional gestures for mode switching. In addition, the practicality and usefulness of the proposed hand-face hybrid gesture interface were confirmed through software operation experiments in Windows environment.

• Journal of KIISE:Software and Applications
• /
• v.32 no.8
• /
• pp.759-768
• /
• 2005

This paper describes an algorithm that can recognize gestures constructing subspace gesture symbols with hybrid feature information. The previous popular methods based on geometric feature and appearance have resulted in ambiguous output in case of recognizing between similar gesture because they use just the Position information of the hands, feet or bodily shape features. However, our proposed method can classify not only recognition of motion but also similar gestures by the partial feature information presenting which parts of body move and the global feature information including 2-dimensional bodily motion. And this method which is a simple and robust recognition algorithm can be applied in various application such surveillance system and intelligent interface systems.