• Title/Summary/Keyword: Text data

Search Result 2,953, Processing Time 0.029 seconds

Korean Text Classification Using Randomforest and XGBoost Focusing on Seoul Metropolitan Civil Complaint Data (RandomForest와 XGBoost를 활용한 한국어 텍스트 분류: 서울특별시 응답소 민원 데이터를 중심으로)

  • Ha, Ji-Eun;Shin, Hyun-Chul;Lee, Zoon-Ky
    • The Journal of Bigdata
    • /
    • v.2 no.2
    • /
    • pp.95-104
    • /
    • 2017
  • In 2014, Seoul Metropolitan Government launched a response service aimed at responding promptly to civil complaints. The complaints received are categorized based on their content and sent to the department in charge. If this part can be automated, the time and labor costs will be reduced. In this study, we collected 17,700 cases of complaints for 7 years from June 1, 2010 to May 31, 2017. We compared the XGBoost with RandomForest and confirmed the suitability of Korean text classification. As a result, the accuracy of XGBoost compared to RandomForest is generally high. The accuracy of RandomForest was unstable after upsampling and downsampling using the same sample, while XGBoost showed stable overall accuracy.

  • PDF

Emerging Gender Issues in Korean Online Media: A Temporal Semantic Network Analysis Approach

  • Lee, Young-Joo;Park, Ji-Young
    • Journal of Contemporary Eastern Asia
    • /
    • v.18 no.2
    • /
    • pp.118-141
    • /
    • 2019
  • In South Korea, as awareness of gender equality increased since the 1990s, policies for gender equality and social awareness of equality have been established. Until recently, however, the gap between men and women in social and economic activities has not reached the globally desired level and led to social conflict throughout the country. In this study, we analyze the content of online news comments to understand the public perception of gender equality and the details of gender conflict and to grasp the emergence and diffusion process of emerging issues on gender equality. We collected text data from the online news that included the word 'gender equality' posted from January 2012 to June 2017 and also collected comments on each selected news item. Through text mining and the temporal semantic network analysis, we tracked the changes in discourse on gender equality and conflict. Results revealed that gender conflicts are increasing in the online media, and the focus of conflict is shifting from 'position and role inequality' to 'opportunity inequality'.

Hybrid Approach to Sentiment Analysis based on Syntactic Analysis and Machine Learning (구문분석과 기계학습 기반 하이브리드 텍스트 논조 자동분석)

  • Hong, Mun-Pyo;Shin, Mi-Young;Park, Shin-Hye;Lee, Hyung-Min
    • Language and Information
    • /
    • v.14 no.2
    • /
    • pp.159-181
    • /
    • 2010
  • This paper presents a hybrid approach to the sentiment analysis of online texts. The sentiment of a text refers to the feelings that the author of a text has towards a certain topic. Many existing approaches employ either a pattern-based approach or a machine learning based approach. The former shows relatively high precision in classifying the sentiments, but suffers from the data sparseness problem, i.e. the lack of patterns. The latter approach shows relatively lower precision, but 100% recall. The approach presented in the current work adopts the merits of both approaches. It combines the pattern-based approach with the machine learning based approach, so that the relatively high precision and high recall can be maintained. Our experiment shows that the hybrid approach improves the F-measure score for more than 50% in comparison with the pattern-based approach and for around 1% comparing with the machine learning based approach. The numerical improvement from the machine learning based approach might not seem to be quite encouraging, but the fact that in the current approach not only the sentiment or the polarity information of sentences but also the additional information such as target of sentiments can be classified makes the current approach promising.

  • PDF

Design and Implementation of Mobile Communication System for Hearing- impaired Person (청각 장애인을 위한 모바일 통화 시스템 설계 및 구현)

  • Yun, Dong-Hee;Kim, Young-Ung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.5
    • /
    • pp.111-116
    • /
    • 2016
  • According to the Ministry of Science, ICT and Future Planning's survey of information gap, smartphone retention rate of disabled people stayed in one-third of non-disabled people, the situation is significantly less access to information for people with disabilities than non-disabled people. In this paper, we develop an application, CallHelper, that helps to be more convenient to use mobile voice calls to the auditory disabled people. CallHelper runs automatically when a call comes in, translates caller's voice to text output on the mobile screen, and displays the emotion reasoning from the caller's voice to visualize emoticons. It also saves voice, translated text, and emotion data that can be played back.

Investigating the Value of Information in Mobile Commerce: A Text Mining Approach

  • Wang, Ying;Aguirre-Urreta, Miguel;Song, Jaeki
    • Asia pacific journal of information systems
    • /
    • v.26 no.4
    • /
    • pp.577-592
    • /
    • 2016
  • The proliferation of mobile applications and the unique characteristics of the mobile environment have attracted significant research interest in understanding customers' purchasing behaviors in mobile commerce. In this study, we extend customer value theory by combining the predictors of product performance with customer value framework to investigate how in-store information creates value for customers and influences mobile application downloads. Using a data set collected from the Google Application Store, we find that customers value both text and non-text information when they make downloading decisions. We apply latent semantic analysis techniques to analyze customer reviews and product descriptions in the mobile application store and determine the embedded valuable information. Results show that, for mobile applications, price, number of raters, and helpful information in customer reviews and product descriptions significantly affect the number of downloads. Conversely, average rating does not work in the mobile environment. This study contributes to the literature by revealing the role of in-store information in mobile application downloads and by providing application developers with useful guidance about increasing application downloads by improving in-store information management.

CNN Architecture Predicting Movie Rating from Audience's Reviews Written in Korean (한국어 관객 평가기반 영화 평점 예측 CNN 구조)

  • Kim, Hyungchan;Oh, Heung-Seon;Kim, Duksu
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.1
    • /
    • pp.17-24
    • /
    • 2020
  • In this paper, we present a movie rating prediction architecture based on a convolutional neural network (CNN). Our prediction architecture extends TextCNN, a popular CNN-based architecture for sentence classification, in three aspects. First, character embeddings are utilized to cover many variants of words since reviews are short and not well-written linguistically. Second, the attention mechanism (i.e., squeeze-and-excitation) is adopted to focus on important features. Third, a scoring function is proposed to convert the output of an activation function to a review score in a certain range (1-10). We evaluated our prediction architecture on a movie review dataset and achieved a low MSE (e.g., 3.3841) compared with an existing method. It showed the superiority of our movie rating prediction architecture.

Predicting Missing Ratings of Each Evaluation Criteria for Hotel by Analyzing User Reviews (사용자 리뷰 분석을 통한 호텔 평가 항목별 누락 평점 예측 방법론)

  • Lee, Donghoon;Boo, Hyunkyung;Kim, Namgyu
    • Journal of Information Technology Services
    • /
    • v.16 no.4
    • /
    • pp.161-176
    • /
    • 2017
  • Recently, most of the users can easily get access to a variety of information sources about companies, products, and services through online channels. Therefore, the online user evaluations are becoming the most powerful tool to generate word of mouth. The user's evaluation is provided in two forms, quantitative rating and review text. The rating is then divided into an overall rating and a detailed rating according to various evaluation criteria. However, since it is a burden for the reviewer to complete all required ratings for each evaluation criteria, so most of the sites requested only mandatory inputs for overall rating and optional inputs for other evaluation criteria. In fact, many users input only the ratings for some of the evaluation criteria and the percentage of missed ratings for each criteria is about 40%. As these missed ratings are the missing values in each criteria, the simple average calculation by ignoring the average 40% of the missed ratings can sufficiently distort the actual phenomenon. Therefore, in this study, we propose a methodology to predict the rating for the missed values of each criteria by analyzing user's evaluation information included the overall rating and text review for each criteria. The experiments were conducted on 207,968 evaluations collected from the actual hotel evaluation site. As a result, it was confirmed that the prediction accuracy of the detailed criteria ratings by the proposed methodology was much higher than the existing average-based method.

Application of Text Mining for Legal Information System: Focusing on Defamation Precedent (법률정보시스템을 위한 텍스트 마이닝 적용 방안 - 명예 훼손 판례를 대상으로 -)

  • Kim, Yong Hwan
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.54 no.1
    • /
    • pp.387-409
    • /
    • 2020
  • Precedents are data containing various types of information. In this study, I proposed a method to be utilized as legal information system for the public using automatic text analysis performed on precedents. It is carried out to analyze the defamation precedent using reference provision, judgment issues, major points of judgment, and reference precedents. As a result of the analysis, legal provisions used in defamation, key issues covered by defamation, and key cases are extracted. Although only applied to the Supreme Court case regarding defamation, the proposed methodology could be applied to various legal topics.

A Study on the Database Structure for Utilizing Classical Literature Knowledge (고문헌 지식활용을 위한 DB구조에 관한 고찰)

  • Woo, Dong-Hyun;Kim, Ki-Wook;Lee, Byung-Wook
    • The Journal of Korean Medical History
    • /
    • v.33 no.2
    • /
    • pp.89-104
    • /
    • 2020
  • The purpose of this research is to build a database structure that can be useful for evidence-based medical practices by constructing the knowledge related to oriental medicine in the classical literature knowledge in a form that can utilize new forms of information technology. As a method, "database" is used as a keyword to search published studies in the field of oriental medicine, research is conducted on classic literature knowledge, and studies describing the contents of the data structure are found and analyzed. In conclusion, the original text DB for the preservation of the original texts and the presentation of the supporting texts should include 'Contents Text', 'Tree Structure', 'Herbal Structure', 'Medicine Manufacture', and 'Disease Structure' tables. In order to search, calculate, and automatically extract expressions written in the original text of the old literature, the tool DB should include 'Unit List', 'Capacity Notation List', 'CUI', 'LUI', and 'SUI' tables. In addition, In order to manage integrated knowledge such as herbal, medicine, acupuncture, disease, and literature, and to implement a search function such as comparison of similarity of control composition, the knowledge DB must contain 'dose-controlled medicine name', 'dose-controlled medicine composition', 'relational knowledge', 'knowledge structure', and 'computational knowledge' tables.

Keyword Reorganization Techniques for Improving the Identifiability of Topics (토픽 식별성 향상을 위한 키워드 재구성 기법)

  • Yun, Yeoil;Kim, Namgyu
    • Journal of Information Technology Services
    • /
    • v.18 no.4
    • /
    • pp.135-149
    • /
    • 2019
  • Recently, there are many researches for extracting meaningful information from large amount of text data. Among various applications to extract information from text, topic modeling which express latent topics as a group of keywords is mainly used. Topic modeling presents several topic keywords by term/topic weight and the quality of those keywords are usually evaluated through coherence which implies the similarity of those keywords. However, the topic quality evaluation method based only on the similarity of keywords has its limitations because it is difficult to describe the content of a topic accurately enough with just a set of similar words. In this research, therefore, we propose topic keywords reorganizing method to improve the identifiability of topics. To reorganize topic keywords, each document first needs to be labeled with one representative topic which can be extracted from traditional topic modeling. After that, classification rules for classifying each document into a corresponding label are generated, and new topic keywords are extracted based on the classification rules. To evaluated the performance our method, we performed an experiment on 1,000 news articles. From the experiment, we confirmed that the keywords extracted from our proposed method have better identifiability than traditional topic keywords.