Deep learning based image retrieval system for O2O shopping mall platform service design

O2O 쇼핑몰 플랫폼 서비스디자인을 위한 딥 러닝 기반의 이미지 검색 시스템

  • 성재경 (세종대학교 컴퓨터공학과) ;
  • 박상민 (세종대학교 컴퓨터공학과) ;
  • 신상윤 (세종대학교 컴퓨터공학과) ;
  • 김영복 (세종대학교 컴퓨터공학과) ;
  • 김용국 (세종대학교 컴퓨터공학과)
  • Received : 2017.05.23
  • Accepted : 2017.07.20
  • Published : 2017.07.28


This paper proposes a new service design which is deep learning-based image retrieval system for product search on O2O shopping mall platform. We have implemented deep learning technology that provides more convenient retrieval service for diverse images of many products that are sold in the internet shopping malls. In order to implement this retrieval system, real data used by shopping mall companies were used as experimental data. However, result from several experiments have confirmed deterioration of retrieval performance due to data components. In order to improve the performance, the learning data that interferes with the retrieval is revised several times, and then the values of experimental result are quantified with the verification data. Using the numerical values of these experiments, we have applied them to the new service design in this system.


Deep Learning;Service Design;O2O;Shopping Mall;Classification System


Supported by : 한국산업기술평가관리원


  1. Sung-Ho Cho, Youn-Joon Lee, "A Comparative Study on Design of User Interface for O2O (Online to Offline) Coupon Application Service", Master's Thesis, p.15-80, Hongik University, 2016.
  2. Yun Ji Moon, Hun Choi, Yuri Jang, Sumin Lee, "O2O Web Site Design Considering User's Shopping Values", The Korea Contents Society, 363-364 (2 pages), May. 2015.
  3. Bosnjak. M, Galesic. M, Tutem. T, "Personality determinants of online shopping: Explaining online purchase intentions using a hierarchical approach", Journal of Business Research, Vol. 60, pp. 597-605, 2007.
  4. DOI: Kyu-dong Kim, Yung-mok Yu, Jeong-lae Kim, "A Study on the Influence of Mobile Commerce Characteristics Perception on Mobile Shopping Intentions", JIIBC, Vol.13. No.6, pp.297-303, Jun. 2013.
  5. DOI: Won-Tae Lee, JangMook Kang, "A study on Model of Personal Information Protection based on Artificial Intelligence Technology or Service", JIIBC, Vol.16 No.4, pp.1-6, 2016.
  6. DOI: Yan Ha, "Design and Implementation of Shopping Mall System based on Image", Journal of the Korea Society of Computer and Information, Vol.17, No.11, 173-177 (5 pages), Nov. 2012.
  7. Chung S. H, Goswami A, Lee H, & Hu J, "The impact of images on user clicks in product search", In Proceedings of the Twelfth International Workshop on Multimedia Data Mining, pp. 25-33, ACM, Aug. 2012.
  8. D. Lowe, "Object Recognition from Local Scale Invariant Features", In International Conference on Computer Vision, pp. 1150-1157, 1999.
  9. D. Lowe, "Distinctive Image Features from Scale Invariant Keypoints," International Journal of Computer Vision, Vol.2, No.60, pp. 91-110, 2004.
  10. Dong-yub Lee, Jae-Choon Jo, Heui-Seok Lim, "User Sentiment Analysis on Amazon Fashion Product Review Using Word Embedding", Journal of the Korea Convergence Society, Vol. 8. No. 4, pp. 1-8, 2017.
  11. Gi-Ryong Choi, Hye-Wuk Jung and Jee-Hyoung Lee "Contents-based Image Retrieval System Design of Shopping," Proceedings of KIIS Spring Conference, Vol. 22, No. 1, 2012.
  12. Veit. A, Kovacs. B, Bell. S, McAuley. J, Bala. K, & Belongie. S, "Learning visual clothing style with heterogeneous dyadic co-occurrences", In Proceedings of the IEEE International Conference on Computer Vision, pp. 4642-4650, 2015.
  13. DOI: Qing-Bo Han, Eun-Ju Kim, Ji-Hyun Lee "Website Homepage Design and Service Design of Analysis", Journal of Digital Convergence, Vol. 14, No.10, pp. 423-430, Oct. 2016.
  14. DOI: Sang-shik Lee, "A Case Study of Bandi&Luni's Bookstore Using an Online to Offline(O2O) Service Design", Journal of the Korea Industrial Information Systems Research Vol. 22, No. 1, pp. 117-126 (10 pages), Feb. 2017.
  15. Dong-yub Lee, Jae-Choon Jo, Heui-Seok Lim, "User Sentiment Analysis on Amazon Fashion Product Review Using Word Embedding", Journal of the Korea Convergence Society, Vol. 8. No. 4, pp. 1-8, 2017.
  16. ml?idxno=27009
  17. DOI: Yeon-gyu Kim, Eui-young Cha, "Streamlined GoogLeNet Algorithm Based on CNN for Korean Character Recognition", J. Korea Inst. Inf. Commun. Eng, Vol. 20, No. 9, pp. 1657-1665, Sep. 2016.
  18. DOI: Seung-Cheol Baek, "Fast and All-Purpose Area-Based Imagery Registration Using ConvNets", Journal of KIISE, Vol. 43, No. 9, pp. 1034-1042, 2016.
  19. Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, Serge Belongie, "Deep Learning of Binary Hash Codes for Fast Image Retrieval", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 27-35, 2015.
  20. Myung-Kyu Kwon, Hyeong-Sik Yang, "A scene search method based on principal character identification using convolutional neural network", Journal of Convergence for Information Technology, Vol. 7, No. 2, pp. 31-36, 2017.
  21. A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks", Advances in Neural Information Processing Systems 25, pp. 1097-1105, 2012.
  22. Kim D, Kim K, Choe D, and Jung J, "Service Issues and Policy Directions for Promoting the O2O Industry in Korea,", The Journal of Society for e-Business Studies, Vol. 21, No. 4, pp. 137-150, 2016.
  23. DOI: Kyoo-Sung Noh, Joo-Yeoun Lee, "Convergence Study on Model of Job Design Support Platform Using Big data and AI", Journal of Digital Convergence, Vol.14, No.7 pp.167-174, 2016.
  24. Yoon-Su Jeong, "Business Process Model for Efficient SMB using Big Data", Journal of IT Convergence Society for SMB, Vol. 5, No. 4, pp. 11-16, 2015.