• Title/Summary/Keyword: Sentiment Evaluation

Search Result 98, Processing Time 0.026 seconds

Customer Service Evaluation based on Online Text Analytics: Sentiment Analysis and Structural Topic Modeling

  • Park, KyungBae;Ha, Sung Ho
    • The Journal of Information Systems
    • /
    • v.26 no.4
    • /
    • pp.327-353
    • /
    • 2017
  • Purpose Social media such as social network services, online forums, and customer reviews have produced a plethora amount of information online. Yet, the information deluge has created both opportunities and challenges at the same time. This research particularly focuses on the challenges in order to discover and track the service defects over time derived by mining publicly available online customer reviews. Design/methodology/approach Synthesizing the streams of research from text analytics, we apply two stages of methods of sentiment analysis and structural topic model incorporating meta-information buried in review texts into the topics. Findings As a result, our study reveals that the research framework effectively leverages textual information to detect, prioritize, and categorize service defects by considering the moving trend over time. Our approach also highlights several implications theoretically and practically of how methods in computational linguistics can offer enriched insights by leveraging the online medium.

Comparative Study of Tokenizer Based on Learning for Sentiment Analysis (고객 감성 분석을 위한 학습 기반 토크나이저 비교 연구)

  • Kim, Wonjoon
    • Journal of Korean Society for Quality Management
    • /
    • v.48 no.3
    • /
    • pp.421-431
    • /
    • 2020
  • Purpose: The purpose of this study is to compare and analyze the tokenizer in natural language processing for customer satisfaction in sentiment analysis. Methods: In this study, a supervised learning-based tokenizer Mecab-Ko and an unsupervised learning-based tokenizer SentencePiece were used for comparison. Three algorithms: Naïve Bayes, k-Nearest Neighbor, and Decision Tree were selected to compare the performance of each tokenizer. For performance comparison, three metrics: accuracy, precision, and recall were used in the study. Results: The results of this study are as follows; Through performance evaluation and verification, it was confirmed that SentencePiece shows better classification performance than Mecab-Ko. In order to confirm the robustness of the derived results, independent t-tests were conducted on the evaluation results for the two types of the tokenizer. As a result of the study, it was confirmed that the classification performance of the SentencePiece tokenizer was high in the k-Nearest Neighbor and Decision Tree algorithms. In addition, the Decision Tree showed slightly higher accuracy among the three classification algorithms. Conclusion: The SentencePiece tokenizer can be used to classify and interpret customer sentiment based on online reviews in Korean more accurately. In addition, it seems that it is possible to give a specific meaning to a short word or a jargon, which is often used by users when evaluating products but is not defined in advance.

Aspect-Based Sentiment Analysis with Position Embedding Interactive Attention Network

  • Xiang, Yan;Zhang, Jiqun;Zhang, Zhoubin;Yu, Zhengtao;Xian, Yantuan
    • Journal of Information Processing Systems
    • /
    • v.18 no.5
    • /
    • pp.614-627
    • /
    • 2022
  • Aspect-based sentiment analysis is to discover the sentiment polarity towards an aspect from user-generated natural language. So far, most of the methods only use the implicit position information of the aspect in the context, instead of directly utilizing the position relationship between the aspect and the sentiment terms. In fact, neighboring words of the aspect terms should be given more attention than other words in the context. This paper studies the influence of different position embedding methods on the sentimental polarities of given aspects, and proposes a position embedding interactive attention network based on a long short-term memory network. Firstly, it uses the position information of the context simultaneously in the input layer and the attention layer. Secondly, it mines the importance of different context words for the aspect with the interactive attention mechanism. Finally, it generates a valid representation of the aspect and the context for sentiment classification. The model which has been posed was evaluated on the datasets of the Semantic Evaluation 2014. Compared with other baseline models, the accuracy of our model increases by about 2% on the restaurant dataset and 1% on the laptop dataset.

Airline Customer Satisfaction Analysis using Social Media Sentiment Evaluation: Full Service Carriers vs. Low Cost Carriers (소셜 미디어 감성평가를 활용한 항공사 고객만족도 분석 - 대형항공사와 저비용항공사 비교연구)

  • Lee, Ju-Yang;Jang, Phil-Sik
    • Journal of Digital Convergence
    • /
    • v.15 no.6
    • /
    • pp.189-196
    • /
    • 2017
  • This study investigates customer satisfaction with full service carriers (FSC) and low cost carriers (LCC) using social media sentiment evaluation. From 2008 to 2016, a total of 77,591 tweets about two FSC and six LCC were aggregated and classified as per airline choice factors. Sentiment evaluation was employed to assess customer satisfaction by three appraisers. The results showed that customer satisfaction with LCC was significantly higher (p<0.001) compared to FSC. Furthermore, overall customer satisfaction with both FSC and LCC has been facing a consistent downward trend since the last seven years. The results also highlighted low customer satisfaction with respect to booking and flight operation factors, and a steep decline in customer satisfaction across booking, onboard services, and marketing factors for FSC. The results of this study have practical implications for the airline industry, which can use this quantitative data to improve customer satisfaction with FSC and LCC.

Hotel Service Quality Evaluation Based on LQI using Sentiment Analysis of Online Reviews (온라인 후기에 내재된 고객의 감성분석과 LQI 차원별 호텔 서비스 품질 평가)

  • Sakong, Won;Ha, Sung Ho;Park, KyungBae
    • The Journal of Information Systems
    • /
    • v.25 no.3
    • /
    • pp.217-245
    • /
    • 2016
  • Purpose With the increasing number of foreign travelers visiting Korea, it is a heavy question to evaluate service quality of typical domestic hotel companies. Our research aims to evaluate service quality of domestic hotels in Korea from the perspective of foreign travelers in order to provide the quality improvements that call attention for the hotel management. Design/Methodology/Approach In this paper, topics of sentiment followed Lodging Quality Index(LQI) dimensions classifying lodging service quality appropriately. Also, we employed word2vec algorithm which calculates similarity and affinity among the vocabularies accurately. To calculate sentiment of each dimension, we adopted scores from SentiWordNet. Findings From the result, we found the number of foreign travelers particularly satisfied with cleanliness, politeness, and problem solving skills. In contrast, it has also been found out that both promptness of services and efficiency of communication do not fulfill the requirements of travelers.

How to improve the accuracy of recommendation systems: Combining ratings and review texts sentiment scores (평점과 리뷰 텍스트 감성분석을 결합한 추천시스템 향상 방안 연구)

  • Hyun, Jiyeon;Ryu, Sangyi;Lee, Sang-Yong Tom
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.1
    • /
    • pp.219-239
    • /
    • 2019
  • As the importance of providing customized services to individuals becomes important, researches on personalized recommendation systems are constantly being carried out. Collaborative filtering is one of the most popular systems in academia and industry. However, there exists limitation in a sense that recommendations were mostly based on quantitative information such as users' ratings, which made the accuracy be lowered. To solve these problems, many studies have been actively attempted to improve the performance of the recommendation system by using other information besides the quantitative information. Good examples are the usages of the sentiment analysis on customer review text data. Nevertheless, the existing research has not directly combined the results of the sentiment analysis and quantitative rating scores in the recommendation system. Therefore, this study aims to reflect the sentiments shown in the reviews into the rating scores. In other words, we propose a new algorithm that can directly convert the user 's own review into the empirically quantitative information and reflect it directly to the recommendation system. To do this, we needed to quantify users' reviews, which were originally qualitative information. In this study, sentiment score was calculated through sentiment analysis technique of text mining. The data was targeted for movie review. Based on the data, a domain specific sentiment dictionary is constructed for the movie reviews. Regression analysis was used as a method to construct sentiment dictionary. Each positive / negative dictionary was constructed using Lasso regression, Ridge regression, and ElasticNet methods. Based on this constructed sentiment dictionary, the accuracy was verified through confusion matrix. The accuracy of the Lasso based dictionary was 70%, the accuracy of the Ridge based dictionary was 79%, and that of the ElasticNet (${\alpha}=0.3$) was 83%. Therefore, in this study, the sentiment score of the review is calculated based on the dictionary of the ElasticNet method. It was combined with a rating to create a new rating. In this paper, we show that the collaborative filtering that reflects sentiment scores of user review is superior to the traditional method that only considers the existing rating. In order to show that the proposed algorithm is based on memory-based user collaboration filtering, item-based collaborative filtering and model based matrix factorization SVD, and SVD ++. Based on the above algorithm, the mean absolute error (MAE) and the root mean square error (RMSE) are calculated to evaluate the recommendation system with a score that combines sentiment scores with a system that only considers scores. When the evaluation index was MAE, it was improved by 0.059 for UBCF, 0.0862 for IBCF, 0.1012 for SVD and 0.188 for SVD ++. When the evaluation index is RMSE, UBCF is 0.0431, IBCF is 0.0882, SVD is 0.1103, and SVD ++ is 0.1756. As a result, it can be seen that the prediction performance of the evaluation point reflecting the sentiment score proposed in this paper is superior to that of the conventional evaluation method. In other words, in this paper, it is confirmed that the collaborative filtering that reflects the sentiment score of the user review shows superior accuracy as compared with the conventional type of collaborative filtering that only considers the quantitative score. We then attempted paired t-test validation to ensure that the proposed model was a better approach and concluded that the proposed model is better. In this study, to overcome limitations of previous researches that judge user's sentiment only by quantitative rating score, the review was numerically calculated and a user's opinion was more refined and considered into the recommendation system to improve the accuracy. The findings of this study have managerial implications to recommendation system developers who need to consider both quantitative information and qualitative information it is expect. The way of constructing the combined system in this paper might be directly used by the developers.

Cloud Service Evaluation Techniques Using User Feedback based on Sentiment Analysis (감정 분석 기반의 사용자 피드백을 이용한 클라우드 서비스 평가 기법)

  • Yun, Donggyu;Kim, Ungsoo;Park, Joonseok;Yeom, Keunhyuk
    • Journal of Software Engineering Society
    • /
    • v.27 no.1
    • /
    • pp.8-14
    • /
    • 2018
  • As cloud computing has emerged as a hot trend in the IT industry, various types of cloud services have emerged. In addition, cloud service broker (CSB) technology has emerged to alleviate the complexity of the process of selecting the desired service that user wants among the various cloud services. One of the key features of the CSB is to recommend the best cloud services to users. In general, CSB can use a method to evaluate a service by receiving feedback about a service from users in order to recommend a cloud service. However, since each user has different criteria for giving a rating, there is a problem that reliability of service evaluation can be low when the rating is only used. In this paper, a method is proposed to supplement evaluation of rating based service by applying machine learning based sentiment analysis to cloud service user's review. In addition, the CSB prototype is implemented based on proposed method. Further, the results of comparing the performance of various learning algorithms is proposed that can be used for sentiment analysis through experiments using actual cloud service review as learning data. The proposed service evaluation method complements the disadvantages of the existing rating-based service evaluation and can reflect the service quality in terms of user experience.

  • PDF

A Study on Sentiment Evaluation and Satisfaction of the Vertical Rope-type Platform Safety Door(RPSD) (로프타입 상하개폐 스크린도어의 감성평가 및 만족도에 관한 연구)

  • Park, Jungsik;Jung, Byungdoo
    • Journal of Korean Society of Transportation
    • /
    • v.32 no.5
    • /
    • pp.462-472
    • /
    • 2014
  • As the Rope Type Platform Safety Door (RPSD) is now commercially available, the technology of RSPD and the public sentiment towards RPSD are being scrutinized. During the period of RPSD development and trial installation, there has been a need to examine its technical reliability and safety, and its users' emotional attitudes. Though often dichotomized in practice, technological innovation of, and the public sentiment towards RPSD are directly related to continuing and collaborated efforts to enhance public satisfaction with the service. Therefore, based on the analyses of public sentiment towards the RPSD system and the log files of operation, this study evaluates public satisfaction with RPSD during its trial phase at Munyang Station in the Daegu Subway System.

System Design for Analysis and Evaluation of E-commerce Products Using Review Sentiment Word Analysis (리뷰 감정 분석을 통한 전자상거래 상품 분석 및 평가 시스템 설계)

  • Choi, Jieun;Ryu, Hyejin;Yu, Dabeen;Kim, Nara;Kim, Yoonhee
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.5
    • /
    • pp.209-217
    • /
    • 2016
  • As smartphone usage increases, the number of consumers who refer to review data of e-commercial products using web sites and SNS is also explosively multiplying. However, reading review data using traditional websites and SNS is time consuming. Also, it is impossible for consumers to read all the reviews. Therefore, a system that collects review data of products and conducts sentiment word analysis of the review is required to provide useful information. The majority of systems that provide such information inadequately reflect the properties of the product. In this study, we described a system that provides analysis and evaluation of e-commerce products through review sentiment words as reflected properties of the product. Furthermore, the system enables consumers to access processed information about reviews quickly and in visual format.

Multilayer Knowledge Representation of Customer's Opinion in Reviews (리뷰에서의 고객의견의 다층적 지식표현)

  • Vo, Anh-Dung;Nguyen, Quang-Phuoc;Ock, Cheol-Young
    • Annual Conference on Human and Language Technology
    • /
    • 2018.10a
    • /
    • pp.652-657
    • /
    • 2018
  • With the rapid development of e-commerce, many customers can now express their opinion on various kinds of product at discussion groups, merchant sites, social networks, etc. Discerning a consensus opinion about a product sold online is difficult due to more and more reviews become available on the internet. Opinion Mining, also known as Sentiment analysis, is the task of automatically detecting and understanding the sentimental expressions about a product from customer textual reviews. Recently, researchers have proposed various approaches for evaluation in sentiment mining by applying several techniques for document, sentence and aspect level. Aspect-based sentiment analysis is getting widely interesting of researchers; however, more complex algorithms are needed to address this issue precisely with larger corpora. This paper introduces an approach of knowledge representation for the task of analyzing product aspect rating. We focus on how to form the nature of sentiment representation from textual opinion by utilizing the representation learning methods which include word embedding and compositional vector models. Our experiment is performed on a dataset of reviews from electronic domain and the obtained result show that the proposed system achieved outstanding methods in previous studies.

  • PDF