Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Evaluating the Quality of Recommendation System by Using Serendipity Measure

세렌디피티 지표를 이용한 추천시스템의 품질 평가

  • Dorjmaa, Tserendulam (Wonju University Innovation Support Project Team Yonsei University MIRAE Campus) ;
  • Shin, Taeksoo (Division of Business Administration College of Government and Business Yonsei University)
  • 체렌돌람 (연세대학교 미래캠퍼스 원주대학혁신지원사업단) ;
  • 신택수 (연세대학교 정경대학 경영학부)
  • Received : 2019.06.11
  • Accepted : 2019.12.06
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, various approaches to recommendation systems have been studied in terms of the quality of recommendation system. A recommender system basically aims to provide personalized recommendations to users for specific items. Most of these systems always recommend the most relevant items of users or items. Traditionally, the evaluation of recommender system quality has focused on the various predictive accuracy metrics of these. However, recommender system must be not only accurate but also useful to users. User satisfaction with recommender systems as an evaluation criterion of recommender system is related not only to how accurately the system recommends but also to how much it supports the user's decision making. In particular, highly serendipitous recommendation would help a user to find a surprising and interesting item. Serendipity in this study is defined as a measure of the extent to which the recommended items are both attractive and surprising to the users. Therefore, this paper proposes an application of serendipity measure to recommender systems to evaluate the performance of recommender systems in terms of recommendation system quality. In this study we define relevant or attractive unexpectedness as serendipity measure for assessing recommendation systems. That is, serendipity measure is evaluated as the measure indicating how the recommender system can find unexpected and useful items for users. Our experimental results show that highly serendipitous recommendation such as item-based collaborative filtering method has better performance than the other recommendations, i.e. user-based collaborative filtering method in terms of recommendation system quality.

최근 추천시스템의 품질평가 관점에서 이에 대한 다양한 연구들이 진행되고 있다. 추천시스템은 기본적으로 사용자들에게 특정 아이템에 대한 개인화된 추천을 제공하는데 목적이 있으며, 대부분의 추천시스템은 항상 사용자 또는 아이템과 가장 관련 있는 아이템을 추천한다. 그리고 이러한 추천시스템의 성과는 전통적으로 다양한 예측정확도 등에 초점을 두어 왔다. 그러나, 추천시스템은 예측가능성 차원에서 정확해야 할 뿐만 아니라 사용자들에게 유용해야 한다. 특히 최근의 추천시스템에 대한 연구로서, 추천시스템의 평가기준에 속하는, 추천시스템에 대한 사용자 만족도(품질)는 추천시스템이 얼마나 정확하게 추천하느냐 뿐만 아니라 사용자의 의사결정에 얼마나 충분히 도움이 되는지와 관계가 깊다. 예를 들어, 특히 높은 수준의 세렌디티피한 추천은 사용자들이 뜻밖의 아이템이면서 흥미로운 아이템을 찾는데 도움이 된다. 여기서, 세렌디피티란 추천 아이템이 사용자에게 매력적인 동시에 뜻밖의(비기대성의) 아이템인 정도를 의미한다. 본 연구는 추천시스템의 성과를 나타내는 세렌디피티 지표를 추천시스템에 적용하여 추천시스템의 품질을 평가하는 것을 목표로 한다. 본 연구에서는 세렌디피티 지표는 관련성(매력)이 있는 동시에 뜻밖인(비기대성의) 아이템을 추천하는 정도로 정의하고, 이 세렌디피티 지표를 측정하기 위해, 추천시스템이 사용자들에게 예상치 못한 유용한 아이템을 찾을 수(또는 추천할 수) 있는 정도를 평가하였다. 본 연구의 주요 실증분석결과로는, 아이템기반 협력 필터링 기법이 사용자기반 협력 필터링 기법보다 더 높은 세렌디피티값을 가지며, 따라서, 추천시스템의 품질평가 차원에서 아이템기반 협력 필터링 기법은 사용자기반 협력 필터링 기법보다는 더 좋은 추천 품질을 갖고 있음을 보여 주었다.

Keywords

References

  1. Adamopoulos, P. and Tuzhilin, A., "On Unexpectedness in Recommender Systems: Or How to Better Expect the Unexpected," ACM Transactions on Intelligent Systems and Technology, Vol.1, No.1(2014), 1-51.
  2. Adomavicius, G. and Tuzhilin, A., "Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions," IEEE Transactions on Knowledge and Data Engineering, Vol.17, No.6(2005), 734-749. https://doi.org/10.1109/TKDE.2005.99
  3. Aggarwal, C. C., Procopiuc, C., and Yu, P. S., "Finding Localized Associations in Market Basket Data," IEEE Transactions on Knowledge and Data Engineering, Vol.14, No.1(2002), 51-62. https://doi.org/10.1109/69.979972
  4. Bernardi, L., Kamps, J., Kiseleva, J., and Muller, M., "The Continuous Cold Start Problem in e-Commerce Recommender Systems," 2nd Workshop on New Trends on Content-Based Recommender Systems, (2015), 30-33.
  5. Chen, Y., Wu, C., Xie, M. and Guo, X., "Solving the Sparsity Problem in Recommender Systems Using Association Retrieval," Journal of Computers, Vol.6, No.9(2011), 1896-1902.
  6. Chiu, Y. S., Lin, K. H., and Chen, J. S., "A Social Network-based Serendipity Recommender System," International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), (2011).
  7. Deshpande, M. and Karypis, G., "Item-based top-N Recommendation Algorithms," ACM Transactions on Information Systems, Vol.22, No.1(2004), 143-177. https://doi.org/10.1145/963770.963776
  8. Ge, M., Delgado-Battenfeld, C., and Hannach, D., "Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity," Proceeding of the fourth ACM conference on Recommender systems, (2010), 257-260.
  9. Gemmis, M. D., Lops, P., Semeraro, G., and Musto, C., "An Investigation on the Serendipity Problem in Recommendation System," Information Processing and Management, Vol.51(2015), 695-717. https://doi.org/10.1016/j.ipm.2015.06.008
  10. Ghazanfar. M. A. and Prugel-Bennet, A., "Leveraging Clustering Approaches to Solve the Gray-Sheep Users Problem in Recommender Systems," Expert Systems with Applications, Vol.41(2014), 3261-3275. https://doi.org/10.1016/j.eswa.2013.11.010
  11. Hahsler, M., "recommenderlab: A Framework for Developing and Testing Recommendation Algorithms," Comprehensive R Archive Network, 2014. Available at http://cran.r-project.org/web/packages/recommenderlab/vignettes/recommenderlab.pdf.
  12. Herlocker, J.L., Konstan, J.A., Terveen, L.G., and Riedl, J.T, "Evaluating Collaborative Filtering Recommender Systems," ACM Transactions on Information Systems, Vol.22, No.1(2004), 5-53. https://doi.org/10.1145/963770.963772
  13. Huang, Z., Zeng, D., and Chen, H., "A Link Analysis Approach to Recommendation under Sparse Data," Proceedings of Americas Conference on Information Systems, 2004.
  14. Kim, M. S. and Im, I., "Resolving the Gray Sheep Problem Using Social Network Analysis (SNA) in Collaborative Filtering (CF) Recommender Systems," Journal of Intelligent Information System, Vol.20(2014), 137-148. https://doi.org/10.13088/jiis.2014.20.2.137
  15. Linden, G., Smith, B., and York, J., "Amazon.com Recommendation," IEEE Internet Computing, Vol.7, No.1(2003), 76-80. https://doi.org/10.1109/MIC.2003.1167344
  16. McNee, S. N., Riedl. J., and Konstan, J. A., "Being Accurate is Not Enough: How Accuracy Metrics have hurt Recommender Systems," Extended Abstracts on Human factors in Computing Systems, CHI(06), (2006), 1097-1101.
  17. Murakami, T., Mori, K., and Orihara, R., "Metrics for Evaluating the Serendipity of Recommendation Lists," Proceedings of the Conference on New Frontiers in Artificial Intelligence, (2007), 40-46.
  18. Sarwar, B., Karyps, G., Konstan, J., and Reidl, J, "Analysis of Recommendation Algorithms for e-Commerce," Proceedings of the 2nd ACM conference on Electronic Commerce, (2000), 158-167.
  19. Sarwar, B., Karyps, G., Konstan, J., and Reidl, J, "Item-based Collaborative Filtering Recommendation Algorithms," Proceedings of the 10th international conference on World Wide Web. ACM, New York, NY, USA, (2001), 285-295.
  20. Schafer, J.B., Konstan, J.A., and Riedl, J., "E-Commerce Recommendation Applications," Data Mining and Knowledge Discovery, Vol.5(1/2)(2001), 115-153. https://doi.org/10.1023/A:1009804230409
  21. Smith, B. and Linden, G., "Two Decades of Recommender Systems at Amazon.com," IEEE Internet Computing, Vol.21, No.3(2017), 12-18. https://doi.org/10.1109/MIC.2017.72
  22. Sridharan, S., "Introducing Serendipity in Recommender Systems through Collaborative Methods," Master of Science Thesis, University of Rhode Island, 2014.
  23. Su, X. and Khoshgoftaar, T. M., "A Survey of Collaborative Filtering Techniques," Advances in Artificial Intelligence, Vol.2009(2009), 1-19.
  24. Surti, T, "Social Recommender Systems: Improving Recommendations through Personalization," Computer Science Department, Haverford College, 2011.
  25. Zhang, Y. C., Seaghdha, D. O., Quercia, D., and Jambor, T., "Auralist: Introducing Serendipity into Music Recommendation," Research Note, 2011.