• Title/Summary/Keyword: LDA (Latent Dirichlet allocation)

Search Result 183, Processing Time 0.025 seconds

Construction of Event Networks from Large News Data Using Text Mining Techniques (텍스트 마이닝 기법을 적용한 뉴스 데이터에서의 사건 네트워크 구축)

  • Lee, Minchul;Kim, Hea-Jin
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.183-203
    • /
    • 2018
  • News articles are the most suitable medium for examining the events occurring at home and abroad. Especially, as the development of information and communication technology has brought various kinds of online news media, the news about the events occurring in society has increased greatly. So automatically summarizing key events from massive amounts of news data will help users to look at many of the events at a glance. In addition, if we build and provide an event network based on the relevance of events, it will be able to greatly help the reader in understanding the current events. In this study, we propose a method for extracting event networks from large news text data. To this end, we first collected Korean political and social articles from March 2016 to March 2017, and integrated the synonyms by leaving only meaningful words through preprocessing using NPMI and Word2Vec. Latent Dirichlet allocation (LDA) topic modeling was used to calculate the subject distribution by date and to find the peak of the subject distribution and to detect the event. A total of 32 topics were extracted from the topic modeling, and the point of occurrence of the event was deduced by looking at the point at which each subject distribution surged. As a result, a total of 85 events were detected, but the final 16 events were filtered and presented using the Gaussian smoothing technique. We also calculated the relevance score between events detected to construct the event network. Using the cosine coefficient between the co-occurred events, we calculated the relevance between the events and connected the events to construct the event network. Finally, we set up the event network by setting each event to each vertex and the relevance score between events to the vertices connecting the vertices. The event network constructed in our methods helped us to sort out major events in the political and social fields in Korea that occurred in the last one year in chronological order and at the same time identify which events are related to certain events. Our approach differs from existing event detection methods in that LDA topic modeling makes it possible to easily analyze large amounts of data and to identify the relevance of events that were difficult to detect in existing event detection. We applied various text mining techniques and Word2vec technique in the text preprocessing to improve the accuracy of the extraction of proper nouns and synthetic nouns, which have been difficult in analyzing existing Korean texts, can be found. In this study, the detection and network configuration techniques of the event have the following advantages in practical application. First, LDA topic modeling, which is unsupervised learning, can easily analyze subject and topic words and distribution from huge amount of data. Also, by using the date information of the collected news articles, it is possible to express the distribution by topic in a time series. Second, we can find out the connection of events in the form of present and summarized form by calculating relevance score and constructing event network by using simultaneous occurrence of topics that are difficult to grasp in existing event detection. It can be seen from the fact that the inter-event relevance-based event network proposed in this study was actually constructed in order of occurrence time. It is also possible to identify what happened as a starting point for a series of events through the event network. The limitation of this study is that the characteristics of LDA topic modeling have different results according to the initial parameters and the number of subjects, and the subject and event name of the analysis result should be given by the subjective judgment of the researcher. Also, since each topic is assumed to be exclusive and independent, it does not take into account the relevance between themes. Subsequent studies need to calculate the relevance between events that are not covered in this study or those that belong to the same subject.

A Study on the Research Trends for Smart City using Topic Modeling (토픽 모델링을 활용한 스마트시티 연구동향 분석)

  • Park, Keon Chul;Lee, Chi Hyung
    • Journal of Internet Computing and Services
    • /
    • v.20 no.3
    • /
    • pp.119-128
    • /
    • 2019
  • This study aims to analyze the research trends on Smart City and to present implications to policy maker, industry professional, and researcher. Cities around globe have undergone the rapid progress in urbanization and the consequent dramatic increase in urban dwellings over the past few decades, and faced many urban problems in such areas as transportation, environment and housing. Cities around the globe are in a hurry to introduce Smart City to pursue a common goal of solving these urban problems and improving the quality of their lives. However, various conceptual approaches to smart city are causing uncertainty in setting policy goals and establishing direction for implementation. The study collected 11,527 papers titled "Smart City(cities)" from the Scopus DB and Springer DB, and then analyze research status, topic, trends based on abstracts and publication date(year) information using the LDA based Topic Modeling approaches. Research topics are classified into three categories(Services, Technologies, and User Perspective) and eight regarding topics. Out of eight topics, citizen-driven innovation is the most frequently referred. Additional topic network analysis reveals that data and privacy/security are the most prevailing topics affecting others. This study is expected to helps understand the trends of Smart City researches and predict the future researches.

Policy agenda proposals from text mining analysis of patents and news articles (특허 및 뉴스 기사 텍스트 마이닝을 활용한 정책의제 제안)

  • Lee, Sae-Mi;Hong, Soon-Goo
    • Journal of Digital Convergence
    • /
    • v.18 no.3
    • /
    • pp.1-12
    • /
    • 2020
  • The purpose of this study is to explore the trend of blockchain technology through analysis of patents and news articles using text mining, and to suggest the blockchain policy agenda by grasping social interests. For this purpose, 327 blockchain-related patent abstracts in Korea and 5,941 full-text online news articles were collected and preprocessed. 12 patent topics and 19 news topics were extracted with latent dirichlet allocation topic modeling. Analysis of patents showed that topics related to authentication and transaction accounted were largely predominant. Analysis of news articles showed that social interests are mainly concerned with cryptocurrency. Policy agendas were then derived for blockchain development. This study demonstrates the efficient and objective use of an automated technique for the analysis of large text documents. Additionally, specific policy agendas are proposed in this study which can inform future policy-making processes.

AI speakers!, Speak with feelings - Focusing on Analysis of SNS Comments (AI 스피커!, 감정을 담아 말해봐 - SNS 댓글 분석을 중심으로)

  • Kim, Joon-Hwan;Lee, Namyeon
    • Journal of Digital Convergence
    • /
    • v.18 no.7
    • /
    • pp.101-110
    • /
    • 2020
  • Devices that add emotion-specific services or various functions are appearing in AI speakers and related devices. To this end, this study performed topic modeling analysis on the topics of post-purchase texts written by AI speaker users, and compared them with the data collected via survey questionnaires. Furthermore, data on the emotional intelligence of AI speakers and relationship quality were collected from 600 users and analyzed using structural equation modeling. The findings of the study are as follows: First, the analysis results of topic modeling showed that most of the articles mainly mention the functional aspects of AI speakers. Second, emotional intelligence of AI speaker perceived by consumer affected relationship quality, and relationship quality had a positive effect on customer satisfaction. Therefore, this study expands the area of AI research by integrating the concept of emotional intelligence and relationship quality to provide new theoretical and practical implications.

Exploration of Constituent Factors for Corporate Reputation and Development of Index Using Online News : Sentiment Analysis and AHP Application (온라인 뉴스를 이용한 기업평판 구성요인 탐색 및 지수 개발 연구 : 감성분석과 AHP적용)

  • Lee, Byung Hyun;Choi, Il Young;Lee, Jung Jae;Kim, Jae Kyeong;Kang, Hyun Mo
    • Journal of Information Technology Services
    • /
    • v.19 no.6
    • /
    • pp.145-159
    • /
    • 2020
  • Because of the recent development of information and communication technology, companies are exposed to various media such as blogs, social media, and YouTube. In particular, exposed news affects the company's reputation. So, while positive news can improve corporate value, negative news can lead to financial losses for the company. In this study, we redefine corporate reputation as social responsibility, vision and leadership, financial performance, products and services through existing literature, and conducted an AHP survey with a total of four components to calculate the weight of each factor. As a result of the calculation, the proportion of financial performance was the highest at 0.41, and products and services, vision and leadership, and social responsibility were the lowest. In addition, in order to measure the reputation of a company, it is classified as a component that defines online news using the LDA technique. In addition, through sentiment analysis, an index for each corporate reputation factor was derived, and the reputation index was calculated by combining it with the AHP analysis result, and Spearman ranking correlation analysis was performed to secure the validity of the research results. Therefore, the significance of this study is that the definition and importance of the constituent factors can contribute to the future planning and development direction of the company, and also contribute to the derivation of the corporate reputation index. This study is significant in that a new analysis methodology that applied AHP analysis results to sentiment analysis was suggested.

Detection of Complaints of Non-Face-to-Face Work before and during COVID-19 by Using Topic Modeling and Sentiment Analysis (동적 토픽 모델링과 감성 분석을 이용한 COVID-19 구간별 비대면 근무 부정요인 검출에 관한 연구)

  • Lee, Sun Min;Chun, Se Jin;Park, Sang Un;Lee, Tae Wook;Kim, Woo Ju
    • The Journal of Information Systems
    • /
    • v.30 no.4
    • /
    • pp.277-301
    • /
    • 2021
  • Purpose The purpose of this study is to analyze the sentiment responses of the general public to non-face-to-face work using text mining methodology. As the number of non-face-to-face complaints is increasing over time, it is difficult to review and analyze in traditional methods such as surveys, and there is a limit to reflect real-time issues. Approach This study has proposed a method of the research model, first by collecting and cleansing the data related to non-face-to-face work among tweets posted on Twitter. Second, topics and keywords are extracted from tweets using LDA(Latent Dirichlet Allocation), a topic modeling technique, and changes for each section are analyzed through DTM(Dynamic Topic Modeling). Third, the complaints of non-face-to-face work are analyzed through the classification of positive and negative polarity in the COVID-19 section. Findings As a result of analyzing 1.54 million tweets related to non-face-to-face work, the number of IDs using non-face-to-face work-related words increased 7.2 times and the number of tweets increased 4.8 times after COVID-19. The top frequently used words related to non-face-to-face work appeared in the order of remote jobs, cybersecurity, technical jobs, productivity, and software. The words that have increased after the COVID-19 were concerned about lockdown and dismissal, and business transformation and also mentioned as to secure business continuity and virtual workplace. New Normal was newly mentioned as a new standard. Negative opinions found to be increased in the early stages of COVID-19 from 34% to 43%, and then stabilized again to 36% through non-face-to-face work sentiment analysis. The complaints were, policies such as strengthening cybersecurity, activating communication to improve work productivity, and diversifying work spaces.

A Trend Analysis of Radiological Research in Korea using Topic Modeling (토픽모델링을 이용한 국내 방사선 학술연구 트렌드 분석)

  • Hong, Dong-Hee
    • Journal of the Korean Society of Radiology
    • /
    • v.16 no.3
    • /
    • pp.343-349
    • /
    • 2022
  • We intend to use topic modeling to identify radiation-themed papers published from 1989 to 2022 and analyze the relevance and weight between topics. This study analyzed topics derived from national subjects for 717 papers published until recently in 2022 to contribute to the revitalization of research in the field of radiation. Through text mining, overall research trends on the subject distribution of the study were analyzed, and five topics were derived through topic modeling. First, among the papers to be analyzed, a total of 1,675 words were frequency-analyzed through the preprocessing process of key words in a total of 717 papers centered on keywords. Second, as a result of analyzing topics based on the association of constituent words for five topics, it was found that studies focused on minimizing dose in the range that does not degrade image quality in the fields of radiation, image, CT clinical. In addition, it was found that various studies were mainly conducted in the MRI, and the study of ultrasound in various areas of disease analysis was actively attempted.

A Topic Analysis of Fine Particle Matter by Using Newspaper Articles (신문기사를 이용한 미세먼지 이슈의 토픽 분석)

  • Yang, Ji-Yeon
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.6
    • /
    • pp.1-14
    • /
    • 2022
  • This study aims to identify topics in newspaper articles related to fine particle matter and to investigate the characteristics and time series trend of each topic. Related national newspaper articles during 1990 and 2021 were collected from Bigkinds. A total of 18 topics have been discovered using LDA, and 11 clusters deduced from clustering. Hot topics include related products/residence, overseas cause(China), power plant as a domestic cause, nationwide emergency reduction measures, international cooperation, political issues, current situation & countermeasure in other countries, and consumption patterns. Cold topics include the concentration standard and indoor air quality improvement. These findings would be useful in inferring the political direction and strategies. In particular, the consumer protection policy should be expanded as the related market is growing. It will also be necessary to pursue policies that will promote public safety and health, and that will enhance public consensus and international cooperation.

A Convergence Study on the Topic and Sentiment of COVID19 Research in Korea Using Text Analysis (텍스트 분석을 이용한 코로나19 관련 국내 논문의 주제 및 감성에 관한 융합 연구)

  • Heo, Seong-Min;Yang, Ji-Yeon
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.4
    • /
    • pp.31-42
    • /
    • 2021
  • The purpose of this study was to explore research topics and examine the trend in COVID19 related research papers. We identified eight topics using latent Dirichlet allocation and found acceptable validity in comparison with the structural topic model. The subtopics have been extracted using k-means clustering and plotted in PCA space. Additionally, we discovered the topics bearing negative tones and warning signs by sentiment analysis. The results flagged up the issues of the topics, Biomedical Related, International Dynamics and Psychological Impact. The findings could serve as a guideline for researchers who explore new research directions and policymakers who need to make decisions about which research projects to support.

Text Data Analysis Model Based on Web Application (웹 애플리케이션 기반의 텍스트 데이터 분석 모델)

  • Jin, Go-Whan
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.11
    • /
    • pp.785-792
    • /
    • 2021
  • Since the Fourth Industrial Revolution, various changes have occurred in society as a whole due to advance in technologies such as artificial intelligence and big data. The amount of data that can be collect in the process of applying important technologies tends to increase rapidly. Especially in academia, existing generated literature data is analyzed in order to grasp research trends, and analysis of these literature organizes the research flow and organizes some research methodologies and themes, or by grasping the subjects that are currently being talked about in academia, we are making a lot of contributions to setting the direction of future research. However, it is difficult to access whether data collection is necessary for the analysis of document data without the expertise of ordinary programs. In this paper, propose a text mining-based topic modeling Web application model. Even if you lack specialized knowledge about data analysis methods through the proposed model, you can perform various tasks such as collecting, storing, and text-analyzing research papers, and researchers can analyze previous research and research trends. It is expect that the time and effort required for data analysis can be reduce order to understand.