• Title, Summary, Keyword: CNN-LSTM

Search Result 70, Processing Time 0.033 seconds

Constructing for Korean Traditional culture Corpus and Development of Named Entity Recognition Model using Bi-LSTM-CNN-CRFs (한국 전통문화 말뭉치구축 및 Bi-LSTM-CNN-CRF를 활용한 전통문화 개체명 인식 모델 개발)

  • Kim, GyeongMin;Kim, Kuekyeng;Jo, Jaechoon;Lim, HeuiSeok
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.12
    • /
    • pp.47-52
    • /
    • 2018
  • Named Entity Recognition is a system that extracts entity names such as Persons(PS), Locations(LC), and Organizations(OG) that can have a unique meaning from a document and determines the categories of extracted entity names. Recently, Bi-LSTM-CRF, which is a combination of CRF using the transition probability between output data from LSTM-based Bi-LSTM model considering forward and backward directions of input data, showed excellent performance in the study of object name recognition using deep-learning, and it has a good performance on the efficient embedding vector creation by character and word unit and the model using CNN and LSTM. In this research, we describe the Bi-LSTM-CNN-CRF model that enhances the features of the Korean named entity recognition system and propose a method for constructing the traditional culture corpus. We also present the results of learning the constructed corpus with the feature augmentation model for the recognition of Korean object names.

CNN-LSTM Combination Method for Improving Particular Matter Contamination (PM2.5) Prediction Accuracy (미세먼지 예측 성능 개선을 위한 CNN-LSTM 결합 방법)

  • Hwang, Chul-Hyun;Shin, Kwang-Wook
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.1
    • /
    • pp.57-64
    • /
    • 2020
  • Recently, due to the proliferation of IoT sensors, the development of big data and artificial intelligence, time series prediction research on fine dust pollution is actively conducted. However, because the data representing fine dust contamination changes rapidly, traditional time series prediction methods do not provide a level of accuracy that can be used in the field. In this paper, we propose a method that reflects the classification results of environmental conditions through CNN when predicting micro dust contamination using LSTM. Although LSTM and CNN are independent, they are integrated into one network through the interface, so this method is easier to understand than the application LSTM. In the verification experiments of the proposed method using Beijing PM2.5 data, the prediction accuracy and predictive power for the timing of change were consistently improved in various experimental cases.

Sentiment Analysis of Movie Review Using Integrated CNN-LSTM Mode (CNN-LSTM 조합모델을 이용한 영화리뷰 감성분석)

  • Park, Ho-yeon;Kim, Kyoung-jae
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.4
    • /
    • pp.141-154
    • /
    • 2019
  • Rapid growth of internet technology and social media is progressing. Data mining technology has evolved to enable unstructured document representations in a variety of applications. Sentiment analysis is an important technology that can distinguish poor or high-quality content through text data of products, and it has proliferated during text mining. Sentiment analysis mainly analyzes people's opinions in text data by assigning predefined data categories as positive and negative. This has been studied in various directions in terms of accuracy from simple rule-based to dictionary-based approaches using predefined labels. In fact, sentiment analysis is one of the most active researches in natural language processing and is widely studied in text mining. When real online reviews aren't available for others, it's not only easy to openly collect information, but it also affects your business. In marketing, real-world information from customers is gathered on websites, not surveys. Depending on whether the website's posts are positive or negative, the customer response is reflected in the sales and tries to identify the information. However, many reviews on a website are not always good, and difficult to identify. The earlier studies in this research area used the reviews data of the Amazon.com shopping mal, but the research data used in the recent studies uses the data for stock market trends, blogs, news articles, weather forecasts, IMDB, and facebook etc. However, the lack of accuracy is recognized because sentiment calculations are changed according to the subject, paragraph, sentiment lexicon direction, and sentence strength. This study aims to classify the polarity analysis of sentiment analysis into positive and negative categories and increase the prediction accuracy of the polarity analysis using the pretrained IMDB review data set. First, the text classification algorithm related to sentiment analysis adopts the popular machine learning algorithms such as NB (naive bayes), SVM (support vector machines), XGboost, RF (random forests), and Gradient Boost as comparative models. Second, deep learning has demonstrated discriminative features that can extract complex features of data. Representative algorithms are CNN (convolution neural networks), RNN (recurrent neural networks), LSTM (long-short term memory). CNN can be used similarly to BoW when processing a sentence in vector format, but does not consider sequential data attributes. RNN can handle well in order because it takes into account the time information of the data, but there is a long-term dependency on memory. To solve the problem of long-term dependence, LSTM is used. For the comparison, CNN and LSTM were chosen as simple deep learning models. In addition to classical machine learning algorithms, CNN, LSTM, and the integrated models were analyzed. Although there are many parameters for the algorithms, we examined the relationship between numerical value and precision to find the optimal combination. And, we tried to figure out how the models work well for sentiment analysis and how these models work. This study proposes integrated CNN and LSTM algorithms to extract the positive and negative features of text analysis. The reasons for mixing these two algorithms are as follows. CNN can extract features for the classification automatically by applying convolution layer and massively parallel processing. LSTM is not capable of highly parallel processing. Like faucets, the LSTM has input, output, and forget gates that can be moved and controlled at a desired time. These gates have the advantage of placing memory blocks on hidden nodes. The memory block of the LSTM may not store all the data, but it can solve the CNN's long-term dependency problem. Furthermore, when LSTM is used in CNN's pooling layer, it has an end-to-end structure, so that spatial and temporal features can be designed simultaneously. In combination with CNN-LSTM, 90.33% accuracy was measured. This is slower than CNN, but faster than LSTM. The presented model was more accurate than other models. In addition, each word embedding layer can be improved when training the kernel step by step. CNN-LSTM can improve the weakness of each model, and there is an advantage of improving the learning by layer using the end-to-end structure of LSTM. Based on these reasons, this study tries to enhance the classification accuracy of movie reviews using the integrated CNN-LSTM model.

Using CNN-LSTM for Effective Application of Dialogue Context to Emotion Classification (CNN-LSTM을 이용한 대화 문맥 반영과 감정 분류)

  • Shin, Dong-Won;Lee, Yeon-Soo;Jang, Jung-Sun;Rim, Hae-Chang
    • 한국어정보학회:학술대회논문집
    • /
    • /
    • pp.141-146
    • /
    • 2016
  • 대화 시스템에서 사용자가 나타내는 발화에 내재된 감정을 분류하는 것은, 시스템이 적절한 응답과 서비스를 제공하는데 있어 매우 중요하다. 본 연구에서는 대화 내 감정 분류를 하는데 있어 직접적, 간접적으로 드러나는 감정 자질을 자동으로 학습하고 감정이 지속되는 대화 문맥을 효과적으로 반영하기 위해 CNN-LSTM 방식의 딥 뉴럴 네트워크 구조를 제안한다. 그리고 대량의 구어체 코퍼스를 이용한 사전 학습으로 데이터 부족 문제를 완화하였다. 실험 결과 제안하는 방법이 기존의 SVM이나, 단순한 RNN, CNN 네트워크 구조에 비해 전반전인 성능 향상을 보였고, 특히 감정이 있는 경우 더 잘 분류하는 것을 확인할 수 있었다.

  • PDF

Using CNN-LSTM for Effective Application of Dialogue Context to Emotion Classification (CNN-LSTM을 이용한 대화 문맥 반영과 감정 분류)

  • Shin, Dong-Won;Lee, Yeon-Soo;Jang, Jung-Sun;Rim, Hae-Chang
    • Annual Conference on Human and Language Technology
    • /
    • /
    • pp.141-146
    • /
    • 2016
  • 대화 시스템에서 사용자가 나타내는 발화에 내재된 감정을 분류하는 것은, 시스템이 적절한 응답과 서비스를 제공하는데 있어 매우 중요하다. 본 연구에서는 대화 내 감정 분류를 하는데 있어 직접적, 간접적으로 드러나는 감정 자질을 자동으로 학습하고 감정이 지속되는 대화 문맥을 효과적으로 반영하기 위해 CNN-LSTM 방식의 딥 뉴럴 네트워크 구조를 제안한다. 그리고 대량의 구어체 코퍼스를 이용한 사전 학습으로 데이터 부족 문제를 완화하였다. 실험 결과 제안하는 방법이 기존의 SVM이나, 단순한 RNN, CNN 네트워크 구조에 비해 전반전인 성능 향상을 보였고, 특히 감정이 있는 경우 더 잘 분류하는 것을 확인할 수 있었다.

  • PDF

MALICIOUS URL RECOGNITION AND DETECTION USING ATTENTION-BASED CNN-LSTM

  • Peng, Yongfang;Tian, Shengwei;Yu, Long;Lv, Yalong;Wang, Ruijin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.11
    • /
    • pp.5580-5593
    • /
    • 2019
  • A malicious Uniform Resource Locator (URL) recognition and detection method based on the combination of Attention mechanism with Convolutional Neural Network and Long Short-Term Memory Network (Attention-Based CNN-LSTM), is proposed. Firstly, the WHOIS check method is used to extract and filter features, including the URL texture information, the URL string statistical information of attributes and the WHOIS information, and the features are subsequently encoded and pre-processed followed by inputting them to the constructed Convolutional Neural Network (CNN) convolution layer to extract local features. Secondly, in accordance with the weights from the Attention mechanism, the generated local features are input into the Long-Short Term Memory (LSTM) model, and subsequently pooled to calculate the global features of the URLs. Finally, the URLs are detected and classified by the SoftMax function using global features. The results demonstrate that compared with the existing methods, the Attention-based CNN-LSTM mechanism has higher accuracy for malicious URL detection.

A Deeping Learning-based Article- and Paragraph-level Classification

  • Kim, Euhee
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.11
    • /
    • pp.31-41
    • /
    • 2018
  • Text classification has been studied for a long time in the Natural Language Processing field. In this paper, we propose an article- and paragraph-level genre classification system using Word2Vec-based LSTM, GRU, and CNN models for large-scale English corpora. Both article- and paragraph-level classification performed best in accuracy with LSTM, which was followed by GRU and CNN in accuracy performance. Thus, it is to be confirmed that in evaluating the classification performance of LSTM, GRU, and CNN, the word sequential information for articles is better than the word feature extraction for paragraphs when the pre-trained Word2Vec-based word embeddings are used in both deep learning-based article- and paragraph-level classification tasks.

Speed Prediction of Urban Freeway Using LSTM and CNN-LSTM Neural Network (LSTM 및 CNN-LSTM 신경망을 활용한 도시부 간선도로 속도 예측)

  • Park, Boogi;Bae, Sang hoon;Jung, Bokyung
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.20 no.1
    • /
    • pp.86-99
    • /
    • 2021
  • One of the methods to alleviate traffic congestion is to increase the efficiency of the roads by providing traffic condition information on road user and distributing the traffic. For this, reliability must be guaranteed, and quantitative real-time traffic speed prediction is essential. In this study, and based on analysis of traffic speed related to traffic conditions, historical data correlated with traffic flow were used as input. We developed an LSTM model that predicts speed in response to normal traffic conditions, along with a CNN-LSTM model that predicts speed in response to incidents. Through these models, we try to predict traffic speeds during the hour in five-minute intervals. As a result, predictions had an average error rate of 7.43km/h for normal traffic flows, and an error rate of 7.66km/h for traffic incident flows when there was an incident.

Generate Korean image captions using LSTM (LSTM을 이용한 한국어 이미지 캡션 생성)

  • Park, Seong-Jae;Cha, Jeong-Won
    • Annual Conference on Human and Language Technology
    • /
    • /
    • pp.82-84
    • /
    • 2017
  • 본 논문에서는 한국어 이미지 캡션을 학습하기 위한 데이터를 작성하고 딥러닝을 통해 예측하는 모델을 제안한다. 한국어 데이터 생성을 위해 MS COCO 영어 캡션을 번역하여 한국어로 변환하고 수정하였다. 이미지 캡션 생성을 위한 모델은 CNN을 이용하여 이미지를 512차원의 자질로 인코딩한다. 인코딩된 자질을 LSTM의 입력으로 사용하여 캡션을 생성하였다. 생성된 한국어 MS COCO 데이터에 대해 어절 단위, 형태소 단위, 의미형태소 단위 실험을 진행하였고 그 중 가장 높은 성능을 보인 형태소 단위 모델을 영어 모델과 비교하여 영어 모델과 비슷한 성능을 얻음을 증명하였다.

  • PDF