Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지
DOI QR코드

DOI QR Code

An Analysis of IT Proposal Evaluation Results using Big Data-based Opinion Mining

빅데이터 분석 기반의 오피니언 마이닝을 이용한 정보화 사업 평가 분석

  • Kim, Hong Sam (Department of Industrial & Management Engineering, Hannam University) ;
  • Kim, Chong Su (Department of Industrial & Management Engineering, Hannam University)
  • 김홍삼 (한남대학교 산업경영공학과) ;
  • 김종수 (한남대학교 산업경영공학과)
  • Received : 2017.11.28
  • Accepted : 2018.01.09
  • Published : 2018.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Current evaluation practices for IT projects suffer from several problems, which include the difficulty of self-explanation for the evaluation results and the improperly scaled scoring system. This study aims to develop a methodology of opinion mining to extract key factors for the causal relationship analysis and to assess the feasibility of quantifying evaluation scores from text comments using opinion mining based on big data analysis. The research has been performed on the domain of publicly procured IT proposal evaluations, which are managed by the National Procurement Service. Around 10,000 sets of comments and evaluation scores have been gathered, most of which are in the form of digital data but some in paper documents. Thus, more refined form of text has been prepared using various tools. From them, keywords for factors and polarity indicators have been extracted, and experts on this domain have selected some of them as the key factors and indicators. Also, those keywords have been grouped into into dimensions. Causal relationship between keyword or dimension factors and evaluation scores were analyzed based on the two research models-a keyword-based model and a dimension-based model, using the correlation analysis and the regression analysis. The results show that keyword factors such as planning, strategy, technology and PM mostly affects the evaluation result and that the keywords are more appropriate forms of factors for causal relationship analysis than the dimensions. Also, it can be asserted from the analysis that evaluation scores can be composed or calculated from the unstructured text comments using opinion mining, when a comprehensive dictionary of polarity for Korean language can be provided. This study may contribute to the area of big data-based evaluation methodology and opinion mining for IT proposal evaluation, leading to a more reliable and effective IT proposal evaluation method.

Keywords

References

  1. Ahmed, K., El Tazi, N., and Hossny, A., Sentiment Analysis over Social Networks : An Overview, IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, 2015.
  2. Asur, S. and Huberman, B., Predicting the future with social media., Arxiv preprint arXiv:1003.5699, 2010.
  3. Cheon, M. and Baek, D., An Assessment System for Evaluating Big Data Capability Based on a Reference Model, Journal of Society of Korea Industrial and Systems Engineering, 2016, Vol. 39, No. 2, pp. 54-63. https://doi.org/10.11627/jkise.2016.39.2.054
  4. Choi, S. and Kwon, O., The Study of Developing Korean SentiWordNet for Big Data Analytic : Focusing on Anger Emotion, The Journal of Society for e-Business Studies, 2014, Vol. 19, No. 4, pp. 1-19. https://doi.org/10.7838/JSEBS.2014.19.4.001
  5. Dave, K., Lawrence, S., and Pennock, D., Mining the Peanut Gallery : Opinion Extraction and Semantic Classification of Product Reviews, WWW '03 Proceedings of the 12th international conference on World Wide Web, 2003, pp. 519-528.
  6. Duan, W., CaQ, Q., Yu, Y., and Levy, S., Mining Online User-Generated Content-Using Sentiment Analysis Technique to Study Hotel Service Quality Hawaii International Conference On System Sciences, 2013, pp. 3119-3128.
  7. Feldman, R., Ronenfeld, B., Bar-Haim, R., and Fresko, M., The Stock Sonar-Sentiment Analysis of Stocks Based on a Hybrid Approach., Proceedings of 23rd IAAI Conference on Artificial Intelligence(IAAI-2011), 2011.
  8. Goeuriot, L., Kelly, L., Jones, G., Muller, M., and Zobel, J., Report on the SIGIR 2014 Workshop on Medical Information Retrieval (MedIR), ACM SIGIR Forum, 2014, Vol. 48, No. 2, pp. 78-82.
  9. Hong, Y. and Steven, S., The Wisdom of Bookies? Sentiment Analysis vs. the NFL Point Spread., Proceedings of the International Conference on Weblogs and Social Media (ICWSM-2010), 2010.
  10. Hu, H. and Liu, B., Mining and summarizing customer reviews, KDD '04 Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 168-177.
  11. Hwang, H.-S., Clustering Corporate Brands based on Opinion Mining : A Case Study of the Automobile Industry, Journal of the Korea Academia-Industrial Cooperation Society, 2016, Vol. 17, No. 11, pp 453-462. https://doi.org/10.5762/KAIS.2016.17.11.453
  12. Jeon, S., Lee, D., and Bae, M., A Study on the Application Method of Munition's Quality Information based on Big Data, Journal of the Korea Academia-Industrial Cooperation Society, 2016, Vol. 17, No. 6, pp. 315-325. https://doi.org/10.5762/KAIS.2016.17.6.315
  13. Kamps, J., Marx, M., Mokken, R.J., and Rijke, M.D., Using WordNet to Measure Semantic Orientation of Adjectives, In Proceedings of LREC-04, 4th International Conference on Language Resources and Evaluation, 2004, Vol. 4, pp. 1115-1118.
  14. Kim, S. and Kim, N., A Study on the Effect of Using Sentiment Lexicon in Opinion Classification, Journal of Intelligence and Information Systems, 2014, Vol. 20, No. 1, pp. 133-148. https://doi.org/10.13088/JIIS.2014.20.1.133
  15. Lee, J. and Baek, D., The Effect of Smartphone Purchasing Determinants on Repurchase Intention, Journal of Society of Korea Industrial and Systems Engineering, 2017, Vol. 40, No. 2, pp. 1-12. https://doi.org/10.11627/jkise.2017.40.2.001
  16. Liu, B., Sentiment Analysis and Opinion Mining, Computational Linguistics, 2014, Vol. 40, No. 2, pp 511-513. https://doi.org/10.1162/COLI_r_00186
  17. Liu, B., Sentiment Analysis and Opinion Mining, Morgan and Claypool Publishers, 2012.
  18. Liu, B., Web Data Mining : Exploring Hyperlinks, Contents, and Usage Data, Springer, 2011.
  19. Medhat, W., Hassan, A., and Korashy, H., Sentiment analysis algorithms and applications : A survey, Ain Shams Engineering Journal, 2014, Vol. 5, No. 4, pp. 1093-1113. https://doi.org/10.1016/j.asej.2014.04.011
  20. Mohammad, S. and Yang, T., Tracking Sentiment in Mail : How Genders Differ on Emotional Axes, Proceedings of the ACL Workshop on ACL 2011 Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA-2011), 2011.
  21. Mohammad, S., From Once Upon a Time to Happily Ever After : Tracking Emotions in Novels and Fairy Tales., Proceedings of the ACL 2011 Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH), 2011.
  22. O'Connor, B., Balasubramanyan, R., Routledge, B., and Smith, N., From Tweets to Polls : Linking Text Sentiment to Public Opinion Time Series., Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM 2010), 2010.
  23. Rajput, Q., Haider, S., and Ghani, S., Lexicon-Based Sentiment Analysis of Teachers' Evaluation, Applied Computational Intelligence and Soft Computing, 2016.
  24. Roh, J., Kim, H., and Chang, J., Improving Hypertext Classification Systems through WordNet-based Feature Abstraction, Journal of Society for e-Business Studies, 2013, Vol. 18, No. 2, pp. 95-110. https://doi.org/10.7838/jsebs.2013.18.2.095
  25. Tumasjan, A., Sprenger, T., Sandner, P., and Welpe, I., Predicting elections with twitter : What 140 characters reveal about political sentiment, Proceedings of the International Conference on Weblogs and Social Media (ICWSM-2010), 2010.
  26. Turney, P. and Littman, M., Measuring Praise and Criticism : Inference of Semantic Orientation from Association, ACM Transactions on Information Systems, 2003, Vol. 21, No. 4, pp. 315-346. https://doi.org/10.1145/944012.944013
  27. Turney, P., Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, Proceedings of Annual Meeting of the Association for computational Linguistics, 2002, pp. 417- 424.
  28. Wang, X. and Park, B., Extraction of Feature and Opinion Words from Product Reviews Using the 12-Structure, Proceedings of Symposium of the Korean Institute of communications and Information Sciences, 2014, pp. 896-897.
  29. Yoon, D. and Kim, K., Prediction of Rating Score from Short Comments on Movies using Word-Rating Correlation Analysis, HCI Korea, 2011, No. 1, pp. 484-486.

Cited by

  1. 국방무기체계 연구개발 기반 경제적·기술적 파급효과 : 지상·해상·공중 무기체계를 중심으로 vol.41, pp.4, 2018, https://doi.org/10.11627/jkise.2018.41.4.111
  2. 머신러닝을 이용한 웹페이지 내의 특정 정보 추출 vol.41, pp.4, 2018, https://doi.org/10.11627/jkise.2018.41.4.189