• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.10
• /
• pp.3390-3405
• /
• 2022

Long Short-Term Memory (LSTM) combined with attention mechanism is extensively used to generate semantic sentences of images in image captioning models. However, features of salient regions and spatial information are not utilized sufficiently in most related works. Meanwhile, the LSTM also suffers from the problem of underutilized information in a single time step. In the paper, two innovative approaches are proposed to solve these problems. First, the Synergy-Gated Attention (SGA) method is proposed, which can process the spatial features and the salient region features of given images simultaneously. SGA establishes a gated mechanism through the global features to guide the interaction of information between these two features. Then, the Recurrent Fusion LSTM (RF-LSTM) mechanism is proposed, which can predict the next hidden vectors in one time step and improve linguistic coherence by fusing future information. Experimental results on the benchmark dataset of MSCOCO show that compared with the state-of-the-art methods, the proposed method can improve the performance of image captioning model, and achieve competitive performance on multiple evaluation indicators.

• Journal of Broadcast Engineering
• /
• v.27 no.6
• /
• pp.885-896
• /
• 2022

Motor failure in manufacturing plays an important role in future A/S and reliability. Motor failure is detected by measuring sound, current, and vibration. For the data used in this paper, the sound of the car's side mirror motor gear box was used. Motor sound consists of three classes. Sound data is input to the network model through a conversion process through MelSpectrogram. In this paper, various methods were applied, such as data augmentation to improve the performance of classifying fault motors and various methods according to class imbalance were applied resampling, reweighting adjustment, change of loss function and representation learning and classification into two stages. In addition, the curriculum learning method and self-space learning method were compared through a total of five network models such as Bidirectional LSTM Attention, Convolutional Recurrent Neural Network, Multi-Head Attention, Bidirectional Temporal Convolution Network, and Convolution Neural Network, and the optimal configuration was found for motor sound classification.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.39 no.5
• /
• pp.289-295
• /
• 2021

Recently, owing to the development of autonomous vehicles, research on precisely determining the position of a moving object has been actively conducted. Previous research mainly used the fusion of GNSS/IMU (Global Positioning System / Inertial Navigation System) and sensors attached to the vehicle through a Kalman filter. However, in recent years, new technologies have been used to determine the location of a moving object owing to the improvement in computing power and the advent of deep learning. Various techniques using RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and NARX (Nonlinear Auto-Regressive eXogenous model) exist for such learning-based positioning methods. The purpose of this study is to compare the precision of existing filter-based sensor fusion technology and the NARX-based method in case of GNSS signal blockages using simulation data. When the filter-based sensor integration technology was used, an average horizontal position error of 112.8 m occurred during 60 seconds of GNSS signal outages. The same experiment was performed 100 times using the NARX. Among them, an improvement in precision was confirmed in approximately 20% of the experimental results. The horizontal position accuracy was 22.65 m, which was confirmed to be better than that of the filter-based fusion technique.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.7
• /
• pp.2304-2320
• /
• 2021

Intelligently detecting anomalies in health sensor data streams (e.g., Electrocardiogram, ECG) can improve the development of E-health industry. The physiological signals of patients are collected through sensors. Timely diagnosis and treatment save medical resources, promote physical health, and reduce complications. However, it is difficult to automatically classify the ECG data, as the features of ECGs are difficult to extract. And the volume of labeled ECG data is limited, which affects the classification performance. In this paper, we propose a Generative Adversarial Network (GAN)-based deep learning framework (called CAB) for heart arrhythmia classification. CAB focuses on improving the detection accuracy based on a small number of labeled samples. It is trained based on the class-imbalance ECG data. Augmenting ECG data by a GAN model eliminates the impact of data scarcity. After data augmentation, CAB classifies the ECG data by using a Bidirectional Long Short Term Memory Recurrent Neural Network (Bi-LSTM). Experiment results show a better performance of CAB compared with state-of-the-art methods. The overall classification accuracy of CAB is 99.71%. The F1-scores of classifying Normal beats (N), Supraventricular ectopic beats (S), Ventricular ectopic beats (V), Fusion beats (F) and Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively. Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively.