• Title/Summary/Keyword: LSTM with Sequence-to-Sequence learning

Search Result 37, Processing Time 0.023 seconds

Application of sequence to sequence learning based LSTM model (LSTM-s2s) for forecasting dam inflow (Sequence to Sequence based LSTM (LSTM-s2s)모형을 이용한 댐유입량 예측에 대한 연구)

  • Han, Heechan;Choi, Changhyun;Jung, Jaewon;Kim, Hung Soo
    • Journal of Korea Water Resources Association
    • /
    • v.54 no.3
    • /
    • pp.157-166
    • /
    • 2021
  • Forecasting dam inflow based on high reliability is required for efficient dam operation. In this study, deep learning technique, which is one of the data-driven methods and has been used in many fields of research, was manipulated to predict the dam inflow. The Long Short-Term Memory deep learning with Sequence-to-Sequence model (LSTM-s2s), which provides high performance in predicting time-series data, was applied for forecasting inflow of Soyang River dam. Various statistical metrics or evaluation indicators, including correlation coefficient (CC), Nash-Sutcliffe efficiency coefficient (NSE), percent bias (PBIAS), and error in peak value (PE), were used to evaluate the predictive performance of the model. The result of this study presented that the LSTM-s2s model showed high accuracy in the prediction of dam inflow and also provided good performance for runoff event based runoff prediction. It was found that the deep learning based approach could be used for efficient dam operation for water resource management during wet and dry seasons.

LSTM based sequence-to-sequence Model for Korean Automatic Word-spacing (LSTM 기반의 sequence-to-sequence 모델을 이용한 한글 자동 띄어쓰기)

  • Lee, Tae Seok;Kang, Seung Shik
    • Smart Media Journal
    • /
    • v.7 no.4
    • /
    • pp.17-23
    • /
    • 2018
  • We proposed a LSTM-based RNN model that can effectively perform the automatic spacing characteristics. For those long or noisy sentences which are known to be difficult to handle within Neural Network Learning, we defined a proper input data format and decoding data format, and added dropout, bidirectional multi-layer LSTM, layer normalization, and attention mechanism to improve the performance. Despite of the fact that Sejong corpus contains some spacing errors, a noise-robust learning model developed in this study with no overfitting through a dropout method helped training and returned meaningful results of Korean word spacing and its patterns. The experimental results showed that the performance of LSTM sequence-to-sequence model is 0.94 in F1-measure, which is better than the rule-based deep-learning method of GRU-CRF.

Feature Selection with Ensemble Learning for Prostate Cancer Prediction from Gene Expression

  • Abass, Yusuf Aleshinloye;Adeshina, Steve A.
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.12spc
    • /
    • pp.526-538
    • /
    • 2021
  • Machine and deep learning-based models are emerging techniques that are being used to address prediction problems in biomedical data analysis. DNA sequence prediction is a critical problem that has attracted a great deal of attention in the biomedical domain. Machine and deep learning-based models have been shown to provide more accurate results when compared to conventional regression-based models. The prediction of the gene sequence that leads to cancerous diseases, such as prostate cancer, is crucial. Identifying the most important features in a gene sequence is a challenging task. Extracting the components of the gene sequence that can provide an insight into the types of mutation in the gene is of great importance as it will lead to effective drug design and the promotion of the new concept of personalised medicine. In this work, we extracted the exons in the prostate gene sequences that were used in the experiment. We built a Deep Neural Network (DNN) and Bi-directional Long-Short Term Memory (Bi-LSTM) model using a k-mer encoding for the DNA sequence and one-hot encoding for the class label. The models were evaluated using different classification metrics. Our experimental results show that DNN model prediction offers a training accuracy of 99 percent and validation accuracy of 96 percent. The bi-LSTM model also has a training accuracy of 95 percent and validation accuracy of 91 percent.

Prediction of Highy Pathogenic Avian Influenza(HPAI) Diffusion Path Using LSTM (LSTM을 활용한 고위험성 조류인플루엔자(HPAI) 확산 경로 예측)

  • Choi, Dae-Woo;Lee, Won-Been;Song, Yu-Han;Kang, Tae-Hun;Han, Ye-Ji
    • The Journal of Bigdata
    • /
    • v.5 no.1
    • /
    • pp.1-9
    • /
    • 2020
  • The study was conducted with funding from the government (Ministry of Agriculture, Food and Rural Affairs) in 2018 with support from the Agricultural, Food, and Rural Affairs Agency, 318069-03-HD040, and in based on artificial intelligence-based HPAI spread analysis and patterning. The model that is actively used in time series and text mining recently is LSTM (Long Short-Term Memory Models) model utilizing deep learning model structure. The LSTM model is a model that emerged to resolve the Long-Term Dependency Problem that occurs during the Backpropagation Through Time (BPTT) process of RNN. LSTM models have resolved the problem of forecasting very well using variable sequence data, and are still widely used.In this paper study, we used the data of the Call Detailed Record (CDR) provided by KT to identify the migration path of people who are expected to be closely related to the virus. Introduce the results of predicting the path of movement by learning the LSTM model using the path of the person concerned. The results of this study could be used to predict the route of HPAI propagation and to select routes or areas to focus on quarantine and to reduce HPAI spread.

Evaluation of LSTM Model for Inflow Prediction of Lake Sapgye (삽교호 유입량 예측을 위한 LSTM 모형의 적용성 평가)

  • Hwang, Byung-Gi
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.22 no.4
    • /
    • pp.287-294
    • /
    • 2021
  • A Python-based LSTM model was constructed using a Tensorflow backend to estimate the amount of outflow during floods in the Gokgyo-cheon basin flowing into the Sapgyo Lake. To understand the effects of the length of input data used for learning, i.e., the sequence length, on the performance of the model, the model was implemented by increasing the sequence length to three, five, and seven hours. Consequently, when the sequence length was three hours, the prediction performance was excellent over the entire period. As a result of predicting three extreme rainfall events in the model verification, it was confirmed that an average NSE of 0.96 or higher was obtained for one hour in the leading time, and the accuracy decreased gradually for more than two hours in the leading time. In conclusion, the flood level at the Gangcheong station of Gokgyo-cheon can be predicted with high accuracy if the prediction is performed for one hour of leading time with a sequence length of three hours.

Anomaly Detection for User Action with Generative Adversarial Networks (적대적 생성 모델을 활용한 사용자 행위 이상 탐지 방법)

  • Choi, Nam woong;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.3
    • /
    • pp.43-62
    • /
    • 2019
  • At one time, the anomaly detection sector dominated the method of determining whether there was an abnormality based on the statistics derived from specific data. This methodology was possible because the dimension of the data was simple in the past, so the classical statistical method could work effectively. However, as the characteristics of data have changed complexly in the era of big data, it has become more difficult to accurately analyze and predict the data that occurs throughout the industry in the conventional way. Therefore, SVM and Decision Tree based supervised learning algorithms were used. However, there is peculiarity that supervised learning based model can only accurately predict the test data, when the number of classes is equal to the number of normal classes and most of the data generated in the industry has unbalanced data class. Therefore, the predicted results are not always valid when supervised learning model is applied. In order to overcome these drawbacks, many studies now use the unsupervised learning-based model that is not influenced by class distribution, such as autoencoder or generative adversarial networks. In this paper, we propose a method to detect anomalies using generative adversarial networks. AnoGAN, introduced in the study of Thomas et al (2017), is a classification model that performs abnormal detection of medical images. It was composed of a Convolution Neural Net and was used in the field of detection. On the other hand, sequencing data abnormality detection using generative adversarial network is a lack of research papers compared to image data. Of course, in Li et al (2018), a study by Li et al (LSTM), a type of recurrent neural network, has proposed a model to classify the abnormities of numerical sequence data, but it has not been used for categorical sequence data, as well as feature matching method applied by salans et al.(2016). So it suggests that there are a number of studies to be tried on in the ideal classification of sequence data through a generative adversarial Network. In order to learn the sequence data, the structure of the generative adversarial networks is composed of LSTM, and the 2 stacked-LSTM of the generator is composed of 32-dim hidden unit layers and 64-dim hidden unit layers. The LSTM of the discriminator consists of 64-dim hidden unit layer were used. In the process of deriving abnormal scores from existing paper of Anomaly Detection for Sequence data, entropy values of probability of actual data are used in the process of deriving abnormal scores. but in this paper, as mentioned earlier, abnormal scores have been derived by using feature matching techniques. In addition, the process of optimizing latent variables was designed with LSTM to improve model performance. The modified form of generative adversarial model was more accurate in all experiments than the autoencoder in terms of precision and was approximately 7% higher in accuracy. In terms of Robustness, Generative adversarial networks also performed better than autoencoder. Because generative adversarial networks can learn data distribution from real categorical sequence data, Unaffected by a single normal data. But autoencoder is not. Result of Robustness test showed that he accuracy of the autocoder was 92%, the accuracy of the hostile neural network was 96%, and in terms of sensitivity, the autocoder was 40% and the hostile neural network was 51%. In this paper, experiments have also been conducted to show how much performance changes due to differences in the optimization structure of potential variables. As a result, the level of 1% was improved in terms of sensitivity. These results suggest that it presented a new perspective on optimizing latent variable that were relatively insignificant.

LSTM Network with Tracking Association for Multi-Object Tracking

  • Farhodov, Xurshedjon;Moon, Kwang-Seok;Lee, Suk-Hwan;Kwon, Ki-Ryong
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.10
    • /
    • pp.1236-1249
    • /
    • 2020
  • In a most recent object tracking research work, applying Convolutional Neural Network and Recurrent Neural Network-based strategies become relevant for resolving the noticeable challenges in it, like, occlusion, motion, object, and camera viewpoint variations, changing several targets, lighting variations. In this paper, the LSTM Network-based Tracking association method has proposed where the technique capable of real-time multi-object tracking by creating one of the useful LSTM networks that associated with tracking, which supports the long term tracking along with solving challenges. The LSTM network is a different neural network defined in Keras as a sequence of layers, where the Sequential classes would be a container for these layers. This purposing network structure builds with the integration of tracking association on Keras neural-network library. The tracking process has been associated with the LSTM Network feature learning output and obtained outstanding real-time detection and tracking performance. In this work, the main focus was learning trackable objects locations, appearance, and motion details, then predicting the feature location of objects on boxes according to their initial position. The performance of the joint object tracking system has shown that the LSTM network is more powerful and capable of working on a real-time multi-object tracking process.

Prediction of water level in a tidal river using a deep-learning based LSTM model (딥러닝 기반 LSTM 모형을 이용한 감조하천 수위 예측)

  • Jung, Sungho;Cho, Hyoseob;Kim, Jeongyup;Lee, Giha
    • Journal of Korea Water Resources Association
    • /
    • v.51 no.12
    • /
    • pp.1207-1216
    • /
    • 2018
  • Discharge or water level predictions at tidally affected river reaches are currently still a great challenge in hydrological practices. This research aims to predict water level of the tide dominated site, Jamsu bridge in the Han River downstream. Physics-based hydrodynamic approaches are sometimes not applicable for water level prediction in such a tidal river due to uncertainty sources like rainfall forecasting data. In this study, TensorFlow deep learning framework was used to build a deep neural network based LSTM model and its applications. The LSTM model was trained based on 3 data sets having 10-min temporal resolution: Paldang dam release, Jamsu bridge water level, predicted tidal level for 6 years (2011~2016) and then predict the water level time series given the six lead times: 1, 3, 6, 9, 12, 24 hours. The optimal hyper-parameters of LSTM model were set up as follows: 6 hidden layers number, 0.01 learning rate, 3000 iterations. In addition, we changed the key parameter of LSTM model, sequence length, ranging from 1 to 6 hours to test its affect to prediction results. The LSTM model with the 1 hr sequence length led to the best performing prediction results for the all cases. In particular, it resulted in very accurate prediction: RMSE (0.065 cm) and NSE (0.99) for the 1 hr lead time prediction case. However, as the lead time became longer, the RMSE increased from 0.08 m (1 hr lead time) to 0.28 m (24 hrs lead time) and the NSE decreased from 0.99 (1 hr lead time) to 0.74 (24 hrs lead time), respectively.

ViStoryNet: Neural Networks with Successive Event Order Embedding and BiLSTMs for Video Story Regeneration (ViStoryNet: 비디오 스토리 재현을 위한 연속 이벤트 임베딩 및 BiLSTM 기반 신경망)

  • Heo, Min-Oh;Kim, Kyung-Min;Zhang, Byoung-Tak
    • KIISE Transactions on Computing Practices
    • /
    • v.24 no.3
    • /
    • pp.138-144
    • /
    • 2018
  • A video is a vivid medium similar to human's visual-linguistic experiences, since it can inculcate a sequence of situations, actions or dialogues that can be told as a story. In this study, we propose story learning/regeneration frameworks from videos with successive event order supervision for contextual coherence. The supervision induces each episode to have a form of trajectory in the latent space, which constructs a composite representation of ordering and semantics. In this study, we incorporated the use of kids videos as a training data. Some of the advantages associated with the kids videos include omnibus style, simple/explicit storyline in short, chronological narrative order, and relatively limited number of characters and spatial environments. We build the encoder-decoder structure with successive event order embedding, and train bi-directional LSTMs as sequence models considering multi-step sequence prediction. Using a series of approximately 200 episodes of kids videos named 'Pororo the Little Penguin', we give empirical results for story regeneration tasks and SEOE. In addition, each episode shows a trajectory-like shape on the latent space of the model, which gives the geometric information for the sequence models.

Machine Learning Assisted Information Search in Streaming Video (기계학습을 이용한 동영상 서비스의 검색 편의성 향상)

  • Lim, Yeon-sup
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.3
    • /
    • pp.361-367
    • /
    • 2021
  • Information search in video streaming services such as YouTube is replacing traditional information search services. To find desired detailed information in such a video, users should repeatedly navigate several points in the video, resulting in a waste of time and network traffic. In this paper, we propose a method to assist users in searching for information in a video by using DBSCAN clustering and LSTM. Our LSTM model is trained with a dataset that consists of user search sequences and their final target points categorized by DBSCAN clustering algorithm. Then, our proposed method utilizes the trained model to suggest an expected category for the user's desired target point based on a partial search sequence that can be collected at the beginning of the search. Our experiment results show that the proposed method successfully finds user destination points with 98% accuracy and 7s of the time difference by average.