• Journal of Korean Society of Coastal and Ocean Engineers
• /
• v.34 no.4
• /
• pp.109-118
• /
• 2022

The data of the missing section among the vertex surface sea temperature observation data was imputed using the Bidirectional Recurrent Neural Network(BiRNN). Among artificial intelligence techniques, Recurrent Neural Networks (RNNs), which are commonly used for time series data, only estimate in the direction of time flow or in the reverse direction to the missing estimation position, so the estimation performance is poor in the long-term missing section. On the other hand, in this study, estimation performance can be improved even for long-term missing data by estimating in both directions before and after the missing section. Also, by using all available data around the observation point (sea surface temperature, temperature, wind field, atmospheric pressure, humidity), the imputation performance was further improved by estimating the imputation data from these correlations together. For performance verification, a statistical model, Multivariate Imputation by Chained Equations (MICE), a machine learning-based Random Forest model, and an RNN model using Long Short-Term Memory (LSTM) were compared. For imputation of long-term missing for 7 days, the average accuracy of the BiRNN/statistical models is 70.8%/61.2%, respectively, and the average error is 0.28 degrees/0.44 degrees, respectively, so the BiRNN model performs better than other models. By applying a temporal decay factor representing the missing pattern, it is judged that the BiRNN technique has better imputation performance than the existing method as the missing section becomes longer.

• Wind and Structures
• /
• v.38 no.1
• /
• pp.75-91
• /
• 2024

Reliable wind signal reconstruction can be beneficial to the operational safety of long-span bridges. Non-Gaussian characteristics of wind signals make the reconstruction process challenging. In this paper, non-Gaussian wind signals are converted into a combined prediction of two kinds of features, actual wind speeds and wind angles of attack. First, two decomposition techniques, empirical mode decomposition (EMD) and variational mode decomposition (VMD), are introduced to decompose wind signals into intrinsic mode functions (IMFs) to reduce the randomness of wind signals. Their principles and applicability are also discussed. Then, four artificial intelligence (AI) algorithms are utilized for wind signal reconstruction by combining the particle swarm optimization (PSO) algorithm with back propagation neural network (BPNN), support vector regression (SVR), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM), respectively. Measured wind signals from a bridge site in a deep-cutting gorge are taken as experimental subjects. The results showed that the reconstruction error of high-frequency components of EMD is too large. On the contrary, VMD fully extracts the multiscale rules of the signal, reduces the component complexity. The combination of VMD-PSO-Bi-LSTM is demonstrated to be the most effective among all hybrid models.

• Annual Conference on Human and Language Technology
• /
• 2017.10a
• /
• pp.159-162
• /
• 2017

Long Short-Term Memory Recurrent Neural Network(LSTM RNN)는 순차 데이터 모델링에 적합한 딥러닝 모델이다. Bidirectional LSTM RNN(BiLSTM RNN)은 RNN의 그래디언트 소멸 문제(vanishing gradient problem)를 해결한 LSTM RNN을 입력 데이터의 양 방향에 적용시킨 것으로 입력 열의 모든 정보를 볼 수 있는 장점이 있어 자연어처리를 비롯한 다양한 분야에서 많이 사용되고 있다. Highway Network는 비선형 변환을 거치지 않은 입력 정보를 히든레이어에서 직접 사용할 수 있게 LSTM 유닛에 게이트를 추가한 딥러닝 모델이다. 본 논문에서는 Highway Network를 한국어 의미역 결정에 적용하여 기존 연구 보다 더 높은 성능을 얻을 수 있음을 보인다.

• 한국어정보학회:학술대회논문집
• /
• 2017.10a
• /
• pp.159-162
• /
• 2017

Long Short-Term Memory Recurrent Neural Network(LSTM RNN)는 순차 데이터 모델링에 적합한 딥러닝 모델이다. Bidirectional LSTM RNN(BiLSTM RNN)은 RNN의 그래디언트 소멸 문제(vanishing gradient problem)를 해결한 LSTM RNN을 입력 데이터의 양 방향에 적용시킨 것으로 입력 열의 모든 정보를 볼 수 있는 장점이 있어 자연어처리를 비롯한 다양한 분야에서 많이 사용되고 있다. Highway Network는 비선형 변환을 거치지 않은 입력 정보를 히든레이어에서 직접 사용할 수 있게 LSTM 유닛에 게이트를 추가한 딥러닝 모델이다. 본 논문에서는 Highway Network를 한국어 의미역 결정에 적용하여 기존 연구 보다 더 높은 성능을 얻을 수 있음을 보인다.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.527-529
• /
• 2018

초기 자연언어처리에 FNN(Feedforward Neural Network)을 적용한 연구들에 비해 LSTM(Long Short-Term Memory)은 현재 시점의 정보뿐만 아니라 이전 시점의 정보를 담고 있어 문장을 이루는 어절들, 어절을 이루는 형태소 등 순차적인(sequential) 데이터를 처리하는데 좋은 성능을 보인다. 본 논문에서는 스택과 버퍼에 있는 어절을 양방향 LSTM encoding을 이용한 representation으로 표현하여 전이기반 의존구문분석에 적용하여 현재 UAS 89.4%의 정확도를 보였고, 자질 추가 및 정제작업을 통해 성능이 개선될 것으로 보인다.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2021.06a
• /
• pp.131-131
• /
• 2021

Deep learning models, especially those based on long short-term memory (LSTM), have presented their superiority in addressing time series data issues recently. This study aims to comprehensively evaluate the performance of deep learning models that belong to the supervised learning category in streamflow prediction. Therefore, six deep learning models-standard LSTM, standard gated recurrent unit (GRU), stacked LSTM, bidirectional LSTM (BiLSTM), feed-forward neural network (FFNN), and convolutional neural network (CNN) models-were of interest in this study. The Red River system, one of the largest river basins in Vietnam, was adopted as a case study. In addition, deep learning models were designed to forecast flowrate for one- and two-day ahead at Son Tay hydrological station on the Red River using a series of observed flowrate data at seven hydrological stations on three major river branches of the Red River system-Thao River, Da River, and Lo River-as the input data for training, validation, and testing. The comparison results have indicated that the four LSTM-based models exhibit significantly better performance and maintain stability than the FFNN and CNN models. Moreover, LSTM-based models may reach impressive predictions even in the presence of upstream reservoirs and dams. In the case of the stacked LSTM and BiLSTM models, the complexity of these models is not accompanied by performance improvement because their respective performance is not higher than the two standard models (LSTM and GRU). As a result, we realized that in the context of hydrological forecasting problems, simple architectural models such as LSTM and GRU (with one hidden layer) are sufficient to produce highly reliable forecasts while minimizing computation time because of the sequential data nature.

• International Journal of Contents
• /
• v.17 no.4
• /
• pp.1-15
• /
• 2021

Named entity recognition (NER) is a crucial task for NLP, which aims to extract information from texts. To build NER systems, deep learning (DL) models are learned with dictionary features by mapping each word in the dataset to dictionary features and generating a unique index. However, this technique might generate noisy labels, which pose significant challenges for the NER task. In this paper, we proposed DL-dictionary features, and evaluated them on two datasets, including the OntoNotes 5.0 dataset and our new infectious disease outbreak dataset named GFID. We used (1) a Bidirectional Long Short-Term Memory (BiLSTM) character and (2) pre-trained embedding to concatenate with (3) our proposed features, named the Convolutional Neural Network (CNN), BiLSTM, and self-attention dictionaries, respectively. The combined features (1-3) were fed through BiLSTM - Conditional Random Field (CRF) to predict named entity classes as outputs. We compared these outputs with other predictions of the BiLSTM character, pre-trained embedding, and dictionary features from previous research, which used the exact matching and partial matching dictionary technique. The findings showed that the model employing our dictionary features outperformed other models that used existing dictionary features. We also computed the F1 score with the GFID dataset to apply this technique to extract medical or healthcare information.

• International Journal of Contents
• /
• v.18 no.2
• /
• pp.68-80
• /
• 2022

The health of the human heart is commonly measured using ECG (Electrocardiography) signals. To identify any anomaly in the human heart, the time-sequence of ECG signals is examined manually by a cardiologist or cardiac electrophysiologist. Lightweight anomaly detection on ECG signals in an embedded system is expected to be popular in the near future, because of the increasing number of heart disease symptoms. Some previous research uses deep learning networks such as LSTM and BiLSTM to detect anomaly signals without any handcrafted feature. Unfortunately, lightweight LSTMs show low precision and heavy LSTMs require heavy computing powers and volumes of labeled dataset for symptom classification. This paper proposes an ECG anomaly detection system based on two level BiLSTM for acceptable precision with lightweight networks, which is lightweight and usable at home. Also, this paper presents a new threshold technique which considers statistics of the current ECG pattern. This paper's proposed model with BiLSTM detects ECG signal anomaly in 0.467 ~ 1.0 F₁ score, compared to 0.426 ~ 0.978 F₁ score of the similar model with LSTM except one highly noisy dataset.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.189-192
• /
• 2018

자동 띄어쓰기란 컴퓨터를 사용하여 띄어쓰기가 수행되어 있지 않은 문장에 대해 띄어쓰기를 수행하는 것이다. 이는 자연언어처리 분야에서 형태소 분석 전에 수행되는 과정으로, 띄어쓰기에 오류가 발생할 경우, 형태소 분석이나 구문 분석 등에 영향을 주어 그 결과의 모호성을 높이기 때문에 매우 중요한 전처리 과정 중 하나이다. 본 논문에서는 기계학습의 방법 중 하나인 CRFs(Conditional Random Fields)를 이용하여 자동 띄어쓰기를 수행하고 심층 학습의 방법 중 하나인 양방향 LSTM/CRFs (Bidirectional Long Short Term Memory/CRFs)를 이용하여 자동 띄어쓰기를 수행한 뒤 각 모델의 성능을 비교하고 분석한다. CRFs 모델이 양방향 LSTM/CRFs모델보다 성능이 약간 더 높은 모습을 보였다. 따라서 소형 기기와 같은 환경에서는 CRF와 같은 모델을 적용하여 모델의 경량화 및 시간복잡도를 개선하는 것이 훨씬 더 효과적인 것으로 생각된다.

• Journal of KIISE
• /
• v.44 no.3
• /
• pp.306-313
• /
• 2017

Named entity recognition (NER) seeks to locate and classify named entities in text into pre-defined categories such as names of persons, organizations, locations, expressions of times, etc. Recently, many state-of-the-art NER systems have been implemented with bidirectional LSTM CRFs. Deep learning models based on long short-term memory (LSTM) generally depend on word representations as input. In this paper, we propose an approach to expand word representation by using pre-trained word embedding, part of speech (POS) tag embedding, syllable embedding and named entity dictionary feature vectors. Our experiments show that the proposed approach creates useful word representations as an input of bidirectional LSTM CRFs. Our final presentation shows its efficacy to be 8.05%p higher than baseline NERs with only the pre-trained word embedding vector.