• Journal of Korea Multimedia Society
• /
• v.19 no.8
• /
• pp.1310-1319
• /
• 2016

Face detection is the first step in a wide range of face applications. However, detecting faces in the wild is still a challenging task due to the wide range of variations in pose, scale, and occlusions. Recently, many deep learning methods have been proposed for face detection. However, further improvements are required in the wild. Another important issue to be considered in the face detection is the computational complexity. Current state-of-the-art deep learning methods require a large number of patches to deal with varying scales and the arbitrary image sizes, which result in an increased computational complexity. To reduce the complexity while achieving better detection accuracy, we propose a fully convolutional network-based face detection that can take arbitrarily-sized input and produce feature maps (heat maps) corresponding to the input image size. To deal with the various face scales, a multi-scale network architecture that utilizes the facial components when learning the feature maps is proposed. On top of it, we design multi-task learning technique to improve detection performance. Extensive experiments have been conducted on the FDDB dataset. The experimental results show that the proposed method outperforms state-of-the-art methods with the accuracy of 82.33% at 517 false alarms, while improving computational efficiency significantly.

• Journal of Korea Multimedia Society
• /
• v.24 no.10
• /
• pp.1380-1390
• /
• 2021

The video face recognition (FR) is one of the most popular researches in the field of computer vision due to a variety of applications. In particular, research using the attention mechanism is being actively conducted. In video face recognition, attention represents where to focus on by using the input value of the whole or a specific region, or which frame to focus on when there are many frames. In this paper, we propose a novel attention based deep learning method. Main novelties of our method are (1) the use of combining two loss functions, namely weighted Softmax loss function and a Triplet loss function and (2) the feasibility of end-to-end learning which includes the feature embedding network and attention weight computation. The feature embedding network has a positive effect on the attention weight computation by using combined loss function and end-to-end learning. To demonstrate the effectiveness of our proposed method, extensive and comparative experiments have been carried out to evaluate our method on IJB-A dataset with their standard evaluation protocols. Our proposed method represented better or comparable recognition rate compared to other state-of-the-art video FR methods.

• Journal of the Korean Society for Aeronautical & Space Sciences
• /
• v.49 no.10
• /
• pp.883-892
• /
• 2021

This paper provides a process for developing deep learning-based aerial object detection models that can run in realtime on onboard. To improve object detection performance, we pre-process and augment the training data in the training stage. In addition, we perform transfer learning and apply a weighted cross-entropy method to reduce the variations of detection performance for each class. To improve the inference speed, we have generated inference acceleration engines with quantization. Then, we analyze the real-time performance and detection performance on custom aerial image dataset to verify generalization.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.27 no.2
• /
• pp.167-176
• /
• 2024

Lip-reading is the task of inferring the speaker's utterance from silent video based on learning of lip movements. It is very challenging due to the inherent ambiguities present in the lip movement such as different characters that produce the same lip appearances. Recent advances in deep learning models such as Transformer and Temporal Convolutional Network have led to improve the performance of lip-reading. However, most previous works deal with English lip-reading which has limitations in directly applying to Korean lip-reading, and moreover, there is no a large scale Korean lip-reading dataset. In this paper, we introduce the first large-scale Korean lip-reading dataset with more than 120 k utterances collected from TV broadcasts containing news, documentary and drama. We also present a preprocessing method which uniformly extracts a facial region of interest and propose a transformer-based model based on grapheme unit for sentence-level Korean lip-reading. We demonstrate that our dataset and model are appropriate for Korean lip-reading through statistics of the dataset and experimental results.

• Smart Media Journal
• /
• v.12 no.10
• /
• pp.19-28
• /
• 2023

In the fruit harvesting field, interest in automatic robot harvesting is increasing due to various seasonality and rising harvesting costs. Accurate apple detection is a difficult problem in complex orchard environments with changes in light, vibrations caused by wind, and occlusion of leaves and branches. In this paper, we introduce a dataset and an adaptive heatmap regression model that are advantageous for robot automatic apple harvesting. The apple dataset was labeled with not only the apple location but also the visibility. We propose a method to detect the center point of an apple using an adaptive heatmap regression model that adjusts the Gaussian shape according to visibility. The experimental results showed that the performance of the proposed method was applicable to apple harvesting robots, with MAP@K of 0.9809 and 0.9801 when K=5 and K=10, respectively.

• ETRI Journal
• /
• v.46 no.1
• /
• pp.71-81
• /
• 2024

Recent advances in deep learning for speech and visual recognition have accelerated the development of multimodal speech recognition, yielding many innovative results. We introduce a Korean audiovisual speech recognition corpus. This dataset comprises approximately 150 h of manually transcribed and annotated audiovisual data supplemented with additional 2000 h of untranscribed videos collected from YouTube under the Creative Commons License. The dataset is intended to be freely accessible for unrestricted research purposes. Along with the corpus, we propose an open-source framework for automatic speech recognition (ASR) and audiovisual speech recognition (AVSR). We validate the effectiveness of the corpus with evaluations using state-of-the-art ASR and AVSR techniques, capitalizing on both pretrained models and fine-tuning processes. After fine-tuning, ASR and AVSR achieve character error rates of 11.1% and 18.9%, respectively. This error difference highlights the need for improvement in AVSR techniques. We expect that our corpus will be an instrumental resource to support improvements in AVSR.

• KIPS Transactions on Software and Data Engineering
• /
• v.8 no.7
• /
• pp.311-316
• /
• 2019

Classification problem is to predict the class to which an input data belongs. One of the most popular methods to do this is training a machine learning algorithm using the given dataset. In this case, the dataset should have a well-balanced class distribution for the best performance. However, when the dataset has an imbalanced class distribution, its classification performance could be very poor. To overcome this problem, we propose an over-sampling scheme that balances the number of data by using Conditional Generative Adversarial Networks (CGAN). CGAN is a generative model developed from Generative Adversarial Networks (GAN), which can learn data characteristics and generate data that is similar to real data. Therefore, CGAN can generate data of a class which has a small number of data so that the problem induced by imbalanced class distribution can be mitigated, and classification performance can be improved. Experiments using actual collected data show that the over-sampling technique using CGAN is effective and that it is superior to existing over-sampling techniques.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.1
• /
• pp.36-43
• /
• 2021

In this research, the accuracy of YOLO v3 algorithm in object detection during AGV (Automated Guided Vehicle) operation was investigated. First of all, AGV with 2D LiDAR and stereo camera was prepared. AGV was driven along the route scanned with SLAM (Simultaneous Localization and Mapping) using 2D LiDAR while front objects were detected through stereo camera. In order to evaluate the accuracy of YOLO v3 algorithm, recall, AP (Average Precision), and mAP (mean Average Precision) of the algorithm were measured with a degree of machine learning. Experimental results show that mAP, precision, and recall are improved by 10%, 6.8%, and 16.4%, respectively, when YOLO v3 is fitted with 4000 training dataset and 500 testing dataset which were collected through online search and is trained additionally with 1200 dataset collected from the stereo camera on AGV.

• Journal of IKEEE
• /
• v.26 no.3
• /
• pp.488-493
• /
• 2022

In manufacturing sites, bearing fault in eletrically driven motors cause the entire system to shut down. Stopping the operation of this environment causes huge losses in time and money. The reason of this bearing defects can be various factors such as wear due to continuous contact of rotating elements, excessive load addition, and operating environment. In this paper, a motor driving environment is created which is similar to the domestic manufacturing sites. In addition, based on the established environment, we propose a dataset for bearing fault detection by collecting changes in vibration characteristics that vary depending on normal and defective conditions. The sensor used to collect the vibration characteristics is Microphone G.R.A.S. 40PH-10. We used various machine learning models to build a prototype bearing fault detection system trained on the proposed dataset. As the result, based on the deep neural network model, it shows high accuracy performance of 92.3% in the time domain and 98.3% in the frequency domain.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.19 no.6
• /
• pp.227-233
• /
• 2019

Recently, along with the recent development of deep learning technique, neural networks are achieving success in computer vision filed. Convolutional neural network have shown outstanding performance in not only for a simple image classification task, but also for tasks with high difficulty such as object segmentation and detection. However many such deep learning models are based on supervised-learning, which requires more annotation labels than image-level label. Especially image semantic segmentation model requires pixel-level annotations for training, which is very. To solve these problems, this paper proposes a weakly-supervised semantic segmentation method which requires only image level label to train network. Existing weakly-supervised learning methods have limitations in detecting only specific area of object. In this paper, on the other hand, we use multi-classifier deep learning architecture so that our model recognizes more different parts of objects. The proposed method is evaluated using VOC 2012 validation dataset.