• Journal of Internet Computing and Services
• /
• v.18 no.6
• /
• pp.85-92
• /
• 2017

Recently, in the field of video surveillance, a Deep Learning based learning method has been applied to a method of detecting a moving person in a video and analyzing the behavior of a detected person. The human activity recognition, which is one of the fields this intelligent image analysis technology, detects the object and goes through the process of detecting the body keypoint to recognize the behavior of the detected object. In this paper, we propose a method for Body Keypoint Localization based on Object Detection using RGB-D information. First, the moving object is segmented and detected from the background using color information and depth information generated by the two cameras. The input image generated by rescaling the detected object region using RGB-D information is applied to Convolutional Pose Machines for one person's pose estimation. CPM are used to generate Belief Maps for 14 body parts per person and to detect body keypoints based on Belief Maps. This method provides an accurate region for objects to detect keypoints an can be extended from single Body Keypoint Localization to multiple Body Keypoint Localization through the integration of individual Body Keypoint Localization. In the future, it is possible to generate a model for human pose estimation using the detected keypoints and contribute to the field of human activity recognition.

• Smart Media Journal
• /
• v.1 no.2
• /
• pp.40-46
• /
• 2012

In this paper, we describe interactive contents which is used the result of the inputted interface recognizing vision-based body gesture. Because the content uses the imp which is the common culture as the subject in Asia, we can enjoy it with culture familiarity. And also since the player can use their own gesture to fight with the imp in the game, they are naturally absorbed in the game. And the users can choose the multiple endings of the contents in the end of the scenario. In the part of the gesture recognition, KINECT is used to obtain the three-dimensional coordinates of each joint of the limb to capture the static pose of the actions. The vision-based 3D human pose recognition technology is used to method for convey human gesture in HCI(Human-Computer Interaction). 2D pose model based recognition method recognizes simple 2D human pose in particular environment On the other hand, 3D pose model which describes 3D human body skeletal structure can recognize more complex 3D pose than 2D pose model in because it can use joint angle and shape information of body part Because gestures can be presented through sequential static poses, we recognize the gestures which are configured poses by using HMM In this paper, we describe the interactive content which is used as input interface by using gesture recognition result. So, we can control the contents using only user's gestures naturally. And we intended to improve the immersion and the interest by using the imp who is used real-time interaction with user.

• Journal of Internet Computing and Services
• /
• v.21 no.3
• /
• pp.113-121
• /
• 2020

Human Pose Estimation (HPE) which localizes the human body joints becomes a high potential for high-level applications in the field of computer vision. The main challenges of HPE in real-time are occlusion, illumination change and diversity of pose appearance. The single RGB image is fed into HPE framework in order to reduce the computation cost by using depth-independent device such as a common camera, webcam, or phone cam. However, HPE based on the single RGB is not able to solve the above challenges due to inherent characteristics of color or texture. On the other hand, depth information which is fed into HPE framework and detects the human body parts in 3D coordinates can be usefully used to solve the above challenges. However, the depth information-based HPE requires the depth-dependent device which has space constraint and is cost consuming. Especially, the result of depth information-based HPE is less reliable due to the requirement of pose initialization and less stabilization of frame tracking. Therefore, this paper proposes a new method of HPE which is robust in estimating self-occlusion. There are many human parts which can be occluded by other body parts. However, this paper focuses only on head self-occlusion. The new method is a combination of the RGB image-based HPE framework and the depth information-based HPE framework. We evaluated the performance of the proposed method by COCO Object Keypoint Similarity library. By taking an advantage of RGB image-based HPE method and depth information-based HPE method, our HPE method based on RGB-D achieved the mAP of 0.903 and mAR of 0.938. It proved that our method outperforms the RGB-based HPE and the depth-based HPE.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.16 no.5
• /
• pp.171-177
• /
• 2021

It is difficult to accurately extract features of pedestrian because the pedestrian is photographed at a crosswalk using a camera positioned higher than the pedestrian. In addition, it is more difficult to extract features when a part of the pedestrian's body is covered by an umbrella or parasol or when the pedestrian is holding an object. Representative methods to solve this problem include Object Detection, Instance Segmentation, and Pose Estimation. Among them, this study intends to use the Pose Estimation method. In particular, we intend to increase the recognition rate of pedestrians in crosswalks by maintaining the image resolution through HigherHRNet and applying the foot estimation technique. Finally, we show the superiority of the proposed method by applying and analyzing several data sets covered by body parts to the existing method and the proposed method.

• Journal of the Institute of Convergence Signal Processing
• /
• v.24 no.1
• /
• pp.69-75
• /
• 2023

In recent years, there has been a growing interest in deep learning-based animal pose estimation, especially in the areas of animal behavior analysis and healthcare. However, existing animal pose estimation techniques do not perform well when body parts are occluded or not present. In particular, the occlusion of dog tail or ear might lead to a significant degradation of performance in pet behavior and emotion recognition. In this paper, to solve this intractable problem, we propose a simple yet novel framework for pet pose estimation where pet pose is predicted on an outpainted image where some body parts hidden outside the input image are reconstructed by the image inpainting network preceding the pose estimation network, and we performed a preliminary study to test the feasibility of the proposed approach. We assessed CE-GAN and BAT-Fill for image outpainting, and evaluated SimpleBaseline for pet pose estimation. Our experimental results show that pet pose estimation on outpainted images generated using BAT-Fill outperforms the existing methods of pose estimation on outpainting-less input image.

• Journal of the Korea Computer Graphics Society
• /
• v.29 no.3
• /
• pp.59-67
• /
• 2023

This paper proposes a 3D animated body profile using 3D body scanning and motion capture devices. Users can create their own personalized body profiles with animation by performing 3D scans for a predetermined set of poses. To achieve this, a template animation was obtained through motion capture for a series of poses, and the acquired 3D scan data from users was mapped to the key poses of the animation using Pose-space deformer. The resulting 3D animated body profiles provide users with greater satisfaction compared to traditional static 2D images or 3D scan data.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.6
• /
• pp.3182-3198
• /
• 2019

Vision-based Human Pose Estimation has been considered as one of challenging research subjects due to problems including confounding background clutter, diversity of human appearances and illumination changes in scenes. To tackle these problems, we propose to use a new multi-stage convolution machine for estimating human pose. To provide better heatmap prediction of body joints, the proposed machine repeatedly produces multiple predictions according to stages with receptive field large enough for learning the long-range spatial relationship. And stages are composed of various modules according to their strategic purposes. Pyramid stacking module and dilation module are used to handle problem of human pose at multiple scales. Their multi-scale information from different receptive fields are fused with concatenation, which can catch more contextual information from different features. And spatial and channel information of a given input are converted to gating factors by squeezing the feature maps to a single numeric value based on its importance in order to give each of the network channels different weights. Compared with other ConvNet-based architectures, we demonstrated that our proposed architecture achieved higher accuracy on experiments using standard benchmarks of LSP and MPII pose datasets.

• Journal of the Society of Disaster Information
• /
• v.20 no.2
• /
• pp.302-314
• /
• 2024

Purpose: This study proposes a fall detection model based on a top-down deep learning pose estimation model to automatically determine falls of multiple workers in an underground utility tunnel, and evaluates the performance of the proposed model. Method: A model is presented that combines fall discrimination rules with the results inferred from YOLOv8-pose, one of the top-down pose estimation models, and metrics of the model are evaluated for images of standing and falling two or fewer workers in the tunnel. The same process is also conducted for a bottom-up type of pose estimation model (OpenPose). In addition, due to dependency of the falling interference of the models on worker detection by YOLOv8-pose and OpenPose, metrics of the models for fall was not only investigated, but also for person. Result: For worker detection, both YOLOv8-pose and OpenPose models have F1-score of 0.88 and 0.71, respectively. However, for fall detection, the metrics were deteriorated to 0.71 and 0.23. The results of the OpenPose based model were due to partially detected worker body, and detected workers but fail to part them correctly. Conclusion: Use of top-down type of pose estimation models would be more effective way to detect fall of workers in the underground utility tunnel, with respect to joint recognition and partition between workers.

• Journal of information and communication convergence engineering
• /
• v.21 no.4
• /
• pp.337-345
• /
• 2023

In this study, we formulated a method that evaluates Taekwondo Poomsae performance using a series of choreographed training movements. Despite recent achievements in 3D human pose estimation (HPE) performance, the analysis of human actions remains challenging. In particular, Taekwondo Poomsae action analysis is challenging owing to the absence of time synchronization data and necessity to compare postures, rather than directly relying on joint locations owing to differences in human shapes. To address these challenges, we first decomposed human joint representation into joint rotation (posture) and limb length (body shape), then synchronized a comparison between test and reference pose sequences using DTW (dynamic time warping), and finally compared pose angles for each joint. Experimental results demonstrate that our method successfully synchronizes test action sequences with the reference sequence and reflects a considerable gap in performance between practitioners and professionals. Thus, our method can detect incorrect poses and help practitioners improve accuracy, balance, and speed of movement.

• Smart Media Journal
• /
• v.12 no.7
• /
• pp.59-67
• /
• 2023

Recently, research on behavior analysis tracking human posture and movement has been actively conducted. In particular, OpenPose, an open-source software developed by CMU in 2017, is a representative method for estimating human appearance and behavior. OpenPose can detect and estimate various body parts of a person, such as height, face, and hands in real-time, making it applicable to various fields such as smart healthcare, exercise training, security systems, and medical fields. In this paper, we propose a method for classifying four exercise movements - Squat, Walk, Wave, and Fall-down - which are most commonly performed by users in the gym, using OpenPose-based deep learning models, DNN and CNN. The training data is collected by capturing the user's movements through recorded videos and real-time camera captures. The collected dataset undergoes preprocessing using OpenPose. The preprocessed dataset is then used to train the proposed DNN and CNN models for exercise movement classification. The performance errors of the proposed models are evaluated using MSE, RMSE, and MAE. The performance evaluation results showed that the proposed DNN model outperformed the proposed CNN model.