• ETRI Journal
• /
• v.44 no.2
• /
• pp.327-338
• /
• 2022

Classification models for human action recognition require robust features and large training sets for good generalization. However, data augmentation methods are employed for imbalanced training sets to achieve higher accuracy. These samples generated using data augmentation only reflect existing samples within the training set, their feature representations are less diverse and hence, contribute to less precise classification. This paper presents new data augmentation and action representation approaches to grow training sets. The proposed approach is based on two fundamental concepts: virtual video generation for augmentation and representation of the action videos through robust features. Virtual videos are generated from the motion history templates of action videos, which are convolved using a convolutional neural network, to generate deep features. Furthermore, by observing an objective function of the genetic algorithm, the spatiotemporal features of different samples are combined, to generate the representations of the virtual videos and then classified through an extreme learning machine classifier on MuHAVi-Uncut, iXMAS, and IAVID-1 datasets.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.5
• /
• pp.217-228
• /
• 2023

Video instance segmentation is an intelligent visual task with high complexity because it not only requires object instance segmentation for each image frame constituting a video, but also requires accurate tracking of instances throughout the frame sequence of the video. In special, human instance segmentation in drama videos has an unique characteristic that requires accurate tracking of several main characters interacting in various places and times. Also, it is also characterized by a kind of the class imbalance problem because there is a significant difference between the frequency of main characters and that of supporting or auxiliary characters in drama videos. In this paper, we introduce a new human instance datatset called MHIS, which is built upon drama videos, Miseang, and then propose a novel video data augmentation method, CDVA, in order to overcome the data imbalance problem between character classes. Different from the previous video data augmentation methods, the proposed CDVA generates more realistic augmented videos by deciding the optimal location within the background clip for a target human instance to be inserted with taking rich spatio-temporal context embedded in videos into account. Therefore, the proposed augmentation method, CDVA, can improve the performance of a deep neural network model for video instance segmentation. Conducting both quantitative and qualitative experiments using the MHIS dataset, we prove the usefulness and effectiveness of the proposed video data augmentation method.

• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.13-19
• /
• 2022

Video Object Segmentation (VOS) task requires an amount of labeled sequence data, which limits the performance of the current VOS methods trained with public datasets. In this paper, we propose two effective data augmentation schemes for VOS. The first augmentation method is to swap the background segment to the background from another image, and the other method is to play the sequence in reverse. The two augmentation schemes for VOS enable the current VOS methods to robustly predict the segmentation labels and improve the performance of VOS.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.521-523
• /
• 2021

Data augmentation are techniques used to increase the amount of data by using small amount of existing data. With the spread of the Internet, we can easily obtain data. However, there are still certain industries, like medicine, where it is difficult to obtain data. The same is true for image data in which a black screen is displayed on video wall controller. Because it is rare that a black screen is displayed during operation, it is not easy to obtain an image with a black screen. We propose a DCGAN based architecture that generate dataset using a small amount of black screen image.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.1
• /
• pp.49-54
• /
• 2024

Data imbalance is a common issue encountered in classification problems, stemming from a significant disparity in the number of samples between classes within the dataset. Such data imbalance typically leads to problems in classification models, including overfitting, underfitting, and misinterpretation of performance metrics. Methods to address this issue include resampling, augmentation, regularization techniques, and adjustment of loss functions. In this paper, we focus on loss function adjustment, particularly comparing the performance of various configurations of loss functions (Cross Entropy, Balanced Cross Entropy, two settings of Focal Loss: 𝛼 = 1 and 𝛼 = Balanced, Asymmetric Loss) on Multi-Class black-box video data with imbalance issues. The comparison is conducted using the I3D, and R3D_18 models.

• Journal of Korea Multimedia Society
• /
• v.14 no.2
• /
• pp.329-338
• /
• 2011

Recent augmentation reality demands more realistic multimedia data with the mixture of various media. High-technology for multimedia data which combines existing media data with various media such as audio and video dominates entire media industries. In particular, there is a growing need to serve augmentation reality, 3-dimensional contents and realtime interaction system development which are communication method and visualization tool in Internet. The existing services do not correspond to generate depth value for 3-dimensional space structure recovery which is to form solidity in existing contents. Therefore, it requires research for effective depth-map generation using 2-dimensional video. Complementing shortcomings of existing depth-map generation method using 2-dimensional video, this paper proposes an enhanced depth-map generation method that defines the depth direction in regard to loss location in a video in which none of existing algorithms has defined.

• Journal of Sensor Science and Technology
• /
• v.33 no.1
• /
• pp.18-23
• /
• 2024

Humanoid robots, designed to interact in human environments, require stable mobility to ensure safety. When a humanoid robot falls, it causes damage, breakdown, and potential harm to the robot. Therefore, fall detection is critical to preventing the robot from falling. Prevention of falling of a humanoid robot requires an operator controlling a crane. For efficient and safe walking control experiments, a system that can replace a crane operator is needed. To replace such a crane operator, it is essential to detect the falling conditions of humanoid robots. In this study, we propose falling detection methods using Convolution Neural Network (CNN) model. The image data of a humanoid robot are collected from various angles and environments. A large amount of data is collected by dividing video data into frames per second, and data augmentation techniques are used. The effectiveness of the proposed CNN model is verified by the experiments with the humanoid robot MAX-E1.

• ETRI Journal
• /
• v.42 no.3
• /
• pp.411-419
• /
• 2020

In real-world intelligent transportation systems, accuracy in vehicle license plate detection and recognition is considered quite critical. Many algorithms have been proposed for still images, but their accuracy on actual videos is not satisfactory. This stems from several problematic conditions in videos, such as vehicle motion blur, variety in viewpoints, outliers, and the lack of publicly available video datasets. In this study, we focus on these challenges and propose a license plate detection and recognition scheme for videos based on a temporal matching prior network. Specifically, to improve the robustness of detection and recognition accuracy in the presence of motion blur and outliers, forward and bidirectional matching priors between consecutive frames are properly combined with layer structures specifically designed for plate detection. We also built our own video dataset for the deep training of the proposed network. During network training, we perform data augmentation based on image rotation to increase robustness regarding the various viewpoints in videos.

• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.44-55
• /
• 2022

In order to automatically provide information on fruits in agricultural related broadcasting contents, instance image segmentation of target fruits is required. In addition, the information on the 3D pose of the corresponding fruit may be meaningfully used. This paper represents research that provides information about tomatoes in video content. A large amount of data is required to learn the instance segmentation, but it is difficult to obtain sufficient training data. Therefore, the training data is generated through a data augmentation technique based on a small amount of real images. Compared to the result using only the real images, it is shown that the detection performance is improved as a result of learning through the synthesized image created by separating the foreground and background. As a result of learning augmented images using images created using conventional image pre-processing techniques, it was shown that higher performance was obtained than synthetic images in which foreground and background were separated. To estimate the pose from the result of object detection, a point cloud was obtained using an RGB-D camera. Then, cylinder fitting based on least square minimization is performed, and the tomato pose is estimated through the axial direction of the cylinder. We show that the results of detection, instance image segmentation, and cylinder fitting of a target object effectively through various experiments.

• Journal of Aerospace System Engineering
• /
• v.16 no.5
• /
• pp.70-77
• /
• 2022

This paper proposes database generation and management system for small-pixelized airborne target recognition. The proposed system has five main features: 1) image extraction from in-flight test video frames, 2) automatic image archiving, 3) image data labeling and Meta data annotation, 4) virtual image data generation based on color channel convert conversion and seamless cloning and 5) HOG/LBP-based tiny-pixelized target augmented image data. The proposed framework is Python-based PyQt5 and has an interface that includes OpenCV. Using video files collected from flight tests, an image dataset for airborne target recognition on generates by using the proposed system and system input.