• Title/Summary/Keyword: topic model

Search Result 856, Processing Time 0.031 seconds

Extension and Case Analysis of Topic Modeling for Inductive Social Science Research Methodology (귀납적 사회과학연구 방법론을 위한 토픽모델링의 확장 및 사례분석)

  • Kim, Keun Hyung
    • The Journal of Information Systems
    • /
    • v.31 no.4
    • /
    • pp.25-45
    • /
    • 2022
  • Purpose In this paper, we propose the method to extend topic modeling techniques in order to derive data-based research hypotheses when establishing research hypotheses for social sciences, As a concept in contrast to the existing deductive hypothesis establishment methodology for the social science research, the topic modeling technique was expanded to enable the so-called inductive hypothesis establishment methodology, and an analysis case of the Seongsan Ilchulbong online review based on the proposed methodology was presented. Design/methodology/approach In this paper, an extension architecture and extension algorithm in the form of extending the existing topic modeling were proposed. The extended architecture and algorithm include data processing method based on topic ratio in document, correlation analysis and regression analysis of processed data for topics derived by existing topic modeling. In addition, in this paper, an analysis case of the online review of Seongsan Ilchulbong Peak was presented by applying the extended topic modeling algorithm. An exploratory analysis was performed on the Seongsan Ilchulbong online reviews through the basic text analysis. The data was transformed into 5-point scale to enable correlation and regression analysis based on the topic ratio in each online review. A regression analysis was performed using the derived topics as the independent variable and the review rating as the dependent variable, and hypotheses could be derived based on this, which enable the so-called inductive hypothesis establishment. Findings This paper is meaningful in that it confirmed the possibility of deriving a causal model and setting an inductive hypothesis through an extended analysis of topic modeling.

A Design on Informal Big Data Topic Extraction System Based on Spark Framework (Spark 프레임워크 기반 비정형 빅데이터 토픽 추출 시스템 설계)

  • Park, Kiejin
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.11
    • /
    • pp.521-526
    • /
    • 2016
  • As on-line informal text data have massive in its volume and have unstructured characteristics in nature, there are limitations in applying traditional relational data model technologies for data storage and data analysis jobs. Moreover, using dynamically generating massive social data, social user's real-time reaction analysis tasks is hard to accomplish. In the paper, to capture easily the semantics of massive and informal on-line documents with unsupervised learning mechanism, we design and implement automatic topic extraction systems according to the mass of the words that consists a document. The input data set to the proposed system are generated first, using N-gram algorithm to build multiple words to capture the meaning of the sentences precisely, and Hadoop and Spark (In-memory distributed computing framework) are adopted to run topic model. In the experiment phases, TB level input data are processed for data preprocessing and proposed topic extraction steps are applied. We conclude that the proposed system shows good performance in extracting meaningful topics in time as the intermediate results come from main memories directly instead of an HDD reading.

Analyzing Students' Non-face-to-face Course Evaluation by Topic Modeling and Developing Deep Learning-based Classification Model (토픽 모델링 기반 비대면 강의평 분석 및 딥러닝 분류 모델 개발)

  • Han, Ji Yeong;Heo, Go Eun
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.55 no.4
    • /
    • pp.267-291
    • /
    • 2021
  • Due to the global pandemic caused by COVID-19 in 2020, there have been major changes in the education sites. Universities have fully introduced remote learning, which was considered as an auxiliary education, and non-face-to-face classes have become commonplace, and professors and students are making great efforts to adapt to the new educational environment. In order to improve the quality of non-face-to-face lectures amid these changes, it is necessary to study the factors affecting lecture satisfaction. Therefore, This paper presents a new methodology using big data to identify the factors affecting university lecture satisfaction changed before and after COVID-19. We use Topic Modeling method to analyze lecture reviews before and after COVID-19, and identify factors affecting lecture satisfaction. Through this, we suggest the direction for university education to move forward. In addition, we can identify the factors of satisfaction and dissatisfaction of lectures from multiangle by establishing a topic classification model with an F1-score of 0.84 based on KoBERT, a deep learning language model, and further contribute to continuous qualitative improvement of lecture satisfaction.

A Study on Customer Satisfaction of Mobile Shopping Apps Using Topic Analysis of User Reviews (사용자 리뷰 토픽분석을 활용한 모바일 쇼핑 앱 고객만족도에 관한 연구)

  • Kim, Kwang-Kook;Kim, Yong-Hwan;Kim, Ja-Hee
    • The Journal of Society for e-Business Studies
    • /
    • v.23 no.4
    • /
    • pp.41-62
    • /
    • 2018
  • Despite the rapid growth of the mobile shopping market, major market participants are continuing to suffer operating losses due to severe competition. To solve this problem, the mobile shopping market requires research to improve customer satisfaction and customer loyalty rather than excessive competition. However, the existing studies have limits to reflect the direct needs of customers because they extract the factors on the basis of the Technology Acceptance Model and the literature study. In this study, to reflect the direct requirements of users of mobile shopping Apps, we derived concretely and various factors influencing customer satisfaction through a topic analysis using user reviews. And then we assessed the importance of derived factors to customer satisfaction and analyzed the effects of customer satisfaction on customer complaints and customer loyalty on a structural equation model based on the American customer satisfaction index. We expect that our framework linking a topic analysis and a structural equation model is to be applicable to studies on the customer satisfaction of other mobile services.

Collaborative Filtering Recommendation Algorithm Based on LDA2Vec Topic Model (LDA2Vec 항목 모델을 기반으로 한 협업 필터링 권장 알고리즘)

  • Xin, Zhang;Lee, Scott Uk-Jin
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2020.07a
    • /
    • pp.385-386
    • /
    • 2020
  • In this paper, we propose a collaborative filtering recommendation algorithm based on the LDA2Vec topic model. By extracting and analyzing the article's content, calculate their semantic similarity then combine the traditional collaborative filtering algorithm to recommend. This approach may promote the system's recommend accuracy.

  • PDF

Language Model Adaptation Based on Topic Probability of Latent Dirichlet Allocation

  • Jeon, Hyung-Bae;Lee, Soo-Young
    • ETRI Journal
    • /
    • v.38 no.3
    • /
    • pp.487-493
    • /
    • 2016
  • Two new methods are proposed for an unsupervised adaptation of a language model (LM) with a single sentence for automatic transcription tasks. At the training phase, training documents are clustered by a method known as Latent Dirichlet allocation (LDA), and then a domain-specific LM is trained for each cluster. At the test phase, an adapted LM is presented as a linear mixture of the now trained domain-specific LMs. Unlike previous adaptation methods, the proposed methods fully utilize a trained LDA model for the estimation of weight values, which are then to be assigned to the now trained domain-specific LMs; therefore, the clustering and weight-estimation algorithms of the trained LDA model are reliable. For the continuous speech recognition benchmark tests, the proposed methods outperform other unsupervised LM adaptation methods based on latent semantic analysis, non-negative matrix factorization, and LDA with n-gram counting.

The MeSH-Term Query Expansion Models using LDA Topic Models in Health Information Retrieval (MeSH 기반의 LDA 토픽 모델을 이용한 검색어 확장)

  • You, Sukjin
    • Journal of Korean Library and Information Science Society
    • /
    • v.52 no.1
    • /
    • pp.79-108
    • /
    • 2021
  • Information retrieval in the health field has several challenges. Health information terminology is difficult for consumers (laypeople) to understand. Formulating a query with professional terms is not easy for consumers because health-related terms are more familiar to health professionals. If health terms related to a query are automatically added, it would help consumers to find relevant information. The proposed query expansion (QE) models show how to expand a query using MeSH terms. The documents were represented by MeSH terms (i.e. Bag-of-MeSH), found in the full-text articles. And then the MeSH terms were used to generate LDA (Latent Dirichlet Analysis) topic models. A query and the top k retrieved documents were used to find MeSH terms as topic words related to the query. LDA topic words were filtered by threshold values of topic probability (TP) and word probability (WP). Threshold values were effective in an LDA model with a specific number of topics to increase IR performance in terms of infAP (inferred Average Precision) and infNDCG (inferred Normalized Discounted Cumulative Gain), which are common IR metrics for large data collections with incomplete judgments. The top k words were chosen by the word score based on (TP *WP) and retrieved document ranking in an LDA model with specific thresholds. The QE model with specific thresholds for TP and WP showed improved mean infAP and infNDCG scores in an LDA model, comparing with the baseline result.

Aviation Safety Mandatory Report Topic Prediction Model using Latent Dirichlet Allocation (LDA) (잠재 디리클레 할당(LDA)을 이용한 항공안전 의무보고 토픽 예측 모형)

  • Jun Hwan Kim;Hyunjin Paek;Sungjin Jeon;Young Jae Choi
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.31 no.3
    • /
    • pp.42-49
    • /
    • 2023
  • Not only in aviation industry but also in other industries, safety data plays a key role to improve the level of safety performance. By analyzing safety data such as aviation safety report (text data), hazard can be identified and removed before it leads to a tragic accident. However, pre-processing of raw data (or natural language data) collected from each site should be carried out first to utilize proactive or predictive safety management system. As air traffic volume increases, the amount of data accumulated is also on the rise. Accordingly, there are clear limitation in analyzing data directly by manpower. In this paper, a topic prediction model for aviation safety mandatory report is proposed. In addition, the prediction accuracy of the proposed model was also verified using actual aviation safety mandatory report data. This research model is meaningful in that it not only effectively supports the current aviation safety mandatory report analysis work, but also can be applied to various data produced in the aviation safety field in the future.

Customer Service Evaluation based on Online Text Analytics: Sentiment Analysis and Structural Topic Modeling

  • Park, KyungBae;Ha, Sung Ho
    • The Journal of Information Systems
    • /
    • v.26 no.4
    • /
    • pp.327-353
    • /
    • 2017
  • Purpose Social media such as social network services, online forums, and customer reviews have produced a plethora amount of information online. Yet, the information deluge has created both opportunities and challenges at the same time. This research particularly focuses on the challenges in order to discover and track the service defects over time derived by mining publicly available online customer reviews. Design/methodology/approach Synthesizing the streams of research from text analytics, we apply two stages of methods of sentiment analysis and structural topic model incorporating meta-information buried in review texts into the topics. Findings As a result, our study reveals that the research framework effectively leverages textual information to detect, prioritize, and categorize service defects by considering the moving trend over time. Our approach also highlights several implications theoretically and practically of how methods in computational linguistics can offer enriched insights by leveraging the online medium.

Discovering Community Interests Approach to Topic Model with Time Factor and Clustering Methods

  • Ho, Thanh;Thanh, Tran Duy
    • Journal of Information Processing Systems
    • /
    • v.17 no.1
    • /
    • pp.163-177
    • /
    • 2021
  • Many methods of discovering social networking communities or clustering of features are based on the network structure or the content network. This paper proposes a community discovery method based on topic models using a time factor and an unsupervised clustering method. Online community discovery enables organizations and businesses to thoroughly understand the trend in users' interests in their products and services. In addition, an insight into customer experience on social networks is a tremendous competitive advantage in this era of ecommerce and Internet development. The objective of this work is to find clusters (communities) such that each cluster's nodes contain topics and individuals having similarities in the attribute space. In terms of social media analytics, the method seeks communities whose members have similar features. The method is experimented with and evaluated using a Vietnamese corpus of comments and messages collected on social networks and ecommerce sites in various sectors from 2016 to 2019. The experimental results demonstrate the effectiveness of the proposed method over other methods.