Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of web server monitoring system using crawling technology

  • Yu, Young-Geun (Dept of Computer Science & Engineering. Kongju National University) ;
  • Nam, Ki-Bok (Dept of Computer Science & Engineering. Kongju National University) ;
  • Park, Koo-Rack (Dept of Computer Science & Engineering. Kongju National University)
  • Received : 2018.09.04
  • Accepted : 2019.04.29
  • Published : 2019.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In modern society, there are WEB sites that provides information in various fields by many advances in IT technology. As the number of users accessing the Web increases, one server becomes unavailable, and multiple servers are deployed to provide the service. In addition, systems that control servers are required to manage multiple servers. However, server control systems in the existing market are mostly those that notify managers through SMS and apps when a server's down or a controlled port is closed. However, in case of servers that generate a lot of traffic, the Web server and the WAS are operated and managed each independently. The WEB and WAS provide service by connecting to each other. However, the connection between WEB and WAS may be disconnected due to various environmental factors. In this case, the existing control system can not determine whether the service is working properly. Even in the case of WEB and WAS of a server that is operated independently, there is a phenomenon that the existing control system does not know the problem even when the normal service is not provided due to environmental factors such as disconnection to DB. In this paper, we implemented a system to check the normal state of Web service using Web crawling to solve this problem.

Keywords

CPTSCQ_2019_v24n4_123_f0001.png 이미지

Fig. 1. System sequence diagram

CPTSCQ_2019_v24n4_123_f0002.png 이미지

Fig. 2. Excel file configuration

CPTSCQ_2019_v24n4_123_f0003.png 이미지

Fig. 3. Excel file load

CPTSCQ_2019_v24n4_123_f0004.png 이미지

Fig. 4. Web crawling source

CPTSCQ_2019_v24n4_123_f0005.png 이미지

Fig. 5. Controlled site

CPTSCQ_2019_v24n4_123_f0006.png 이미지

Fig. 6. Run the daemon program

CPTSCQ_2019_v24n4_123_f0007.png 이미지

Fig. 7. Title Tag check

CPTSCQ_2019_v24n4_123_f0008.png 이미지

Fig. 8. Fault notification screen

References

  1. Kyungryun C, Donghan S, SeongJung K, Sumin H, Soochan H. "Multi Visualizing Method for Datacenter Server Monitoring" Proceedings of 1st IITA Korea Information Science Society 2017 Korea Computer Engineering Conference, pp. 268-270, 2017.
  2. Yucheng,L., L. Yubin., "A Monitoring System Design Program Based on B/S Mode" 2010 International Conference on Intelligent Computation Technology and Automation, pp. 184-187, May 2010.
  3. RTMS Project, http://livemedia-soft.com/
  4. S. Ye, J. Lang, and F. Wu. "Crawling Online Social Graphs", In Proceedings of the 12th International Asia-Pacific Web Conference, pp. 236-242. IEEE, 2010.
  5. C Seung-ju, K Jongbae. "Examine the Relationships Between Portal Article of Naver and Real Time Search Word Using Web Crawling", In Proceedings of Society for Humanities and Social Sciences Convergence, pages 787-794. KCI, 2017.
  6. S. H. Jensen, M. Madsen, and A. Moller. "Modeling the HTML DOM and browser API in static analysis of JavaScript web applications", In Proceedings of the Symposium on the Foundations of Software Engineering, September 2011.
  7. Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March, 2014.
  8. S.-W. Lee, and S.-M. Moon, "Selective Just-in-Time Compilation for Client-side Mobile JavaScript Engine," International Conference on Compilers, Architectures and Synthesis of Embedded Systems, ACM, pp. 5-14, 2011.
  9. J. Oh, and S. -M. Moon, "Snapshot-based Loading-Time Acceleration for Web Applications," Proc. Of the 13th Annual IEEE/ACM International Symposium on Code Generation and Optimization, IEEE, pp. 179-189, 2015
  10. H. Park, M. Cha, and S. -M. Moon, "Concurrent JavaScript parsing for faster loading of Web apps," ACM Transactions on Architecture and Code Optimization, Vol. 13, Issue 4, No. 41, 2016
  11. W3C,"DocumentObjectModel(DOM)",http://www.w3.org/DOM/,2009
  12. W3C,"DocumentObjectModel(DOM)Level1Specification"http://www.w3.org/TR/REC-DOM-Level-1/,1998
  13. "interpreter and JavaScript Engine", http://huns.me/development/360, 2013.10.
  14. "Robots Darabase", http://www.robotstxt.org/db.html
  15. "Web crawler" Wikipedia, http://en.wikipedia.org/wiki/Web_crawler