Journal of Institute of Control, Robotics and Systems (제어로봇시스템학회논문지)

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Big Data Analysis of Software Performance Trend using SPC with Flexible Moving Window and Fuzzy Theory

가변 윈도우 기법을 적용한 통계적 공정 제어와 퍼지추론 기법을 이용한 소프트웨어 성능 변화의 빅 데이터 분석

  • Received : 2012.08.27
  • Accepted : 2012.09.25
  • Published : 2012.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

In enterprise software projects, performance issues have become more critical during recent decades. While developing software products, many performance tests are executed in the earlier development phase against the newly added code pieces to detect possible performance regressions. In our previous research, we introduced the framework to enable automated performance anomaly detection and reduce the analysis overhead for identifying the root causes, and showed Statistical Process Control (SPC) can be successfully applied to anomaly detection. In this paper, we explain the special performance trend in which the existing anomaly detection system can hardly detect the noticeable performance change especially when a performance regression is introduced and recovered again a while later. Within the fixed number of sampling period, the fluctuation gets aggravated and the lower and upper control limit get relaxed so that sometimes the existing system hardly detect the noticeable performance change. To resolve the issue, we apply dynamically tuned sampling window size based on the performance trend, and Fuzzy theory to find an appropriate size of the moving window.

Keywords

References

  1. D. H. Lee, S. K. Cha, and A. H. Lee, "A performance anomaly detection and analysis framework for DBMS development," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 8, pp. 1345-1360,Aug. 2012. https://doi.org/10.1109/TKDE.2011.88
  2. D. H. Lee, "Performance anomaly detection and management using statistical process control during software development" Journal of KIISE : Software and Applications (in Korean), vol. 39, no. 8, pp. 639-645, Aug. 2012.
  3. D. C. Montgomery, Introduction to Statistical Quality Control, 5th Edition. John Wiley & Sons, Inc., 2005.
  4. T. Terano, K. Asai, and M. Sugeno, Applied Fuzzy Systems, AP Professional, 1994.
  5. J. J. Park and G. S. Choi, "Fuzzy control system," Kyowoosa, 2001.
  6. S. Barber, Beyond Performance Testing, http://www-128.ibm. com/developerworks/rational/library/4169.html
  7. S. Barber, http://www.logigear.com/newsletter/explanation_ of_performance_testing_on_an_agile_team-part-1.asp
  8. M. Woodside, G. Franks, and D. C. Petriu, "The future of software performance engineering," Proc. of International Conference on Software Engineering, 2007 Future of Software Engineering, pp. 171-187, 2007.
  9. A. de Vries and B. J. Conlin, "Article: Design and performance of statistical process control charts applied to estrous detection efficiency," Journal of Dairy Science, vol. 86, pp. 1970-1984, 2003. https://doi.org/10.3168/jds.S0022-0302(03)73785-0
  10. V. S. Puranik, "CUSUM quality control chart for monitoring energy use performance," Proc. IEEE International Conference on Industrial Engineering and Engineering Management, pp. 1231-1235, Dec. 2007.
  11. M. Komuro, "Experiences of applying SPC techniques to software development processes," ICSE '06: Proc. of the 28th international conference on Software engineering, pp. 577-584, 2006.
  12. J. W. Cangussu, R. A. DeCarlo, and A. P. Mathur, "Monitoring the software test process using statistical process control: a logarithmic approach," ACM SIGSOFT Software Engineering Notes, vol. 28, no. 5, pp. 158-167, 2003. https://doi.org/10.1145/949952.940093