Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Evaluation of Static Analyzers for Weakness in C/C++ Programs using Juliet and STONESOUP Test Suites

  • Seo, Hyunji (Dept. of Computer Engineering, Hongik University) ;
  • Park, Young-gwan (Dept. of Computer Engineering, Hongik University) ;
  • Kim, Taehwan (Dept. of Computer Engineering, Hongik University) ;
  • Han, Kyungsook (Dept. of Computer Engineering, Korea Polytechnic University) ;
  • Pyo, Changwoo (Dept. of Computer Engineering, Hongik University)
  • Received : 2017.03.13
  • Accepted : 2017.03.23
  • Published : 2017.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we compared four analyzers Clang, CppCheck, Compass, and a commercial one from a domestic startup using the NIST's Juliet test suit and STONESOUP that is introduced recently. Tools showed detection efficacy in the order of Clang, CppCheck, the domestic one, and Compass under Juliet tests; and Clang, the domestic one, Compass, and CppCheck under STONESOUP tests. We expect it would be desirable to utilize symbolic execution for vulnerability analysis in the future. On the other hand, the results of tool evaluation also testifies that Juliet and STONESOUP as a benchmark for static analysis tools can reveal differences among tools. Finally, each analyzer has different CWEs that it can detect all given test programs. This result can be used for selection of proper tools with respect to specific CWEs.

Keywords

References

  1. C. Joo, and H. Na, "A Study of Research Trend about Internet of Things," NIA(National Information society Agency), Vol. 22, No. 3, pp.3-15, Autumn 2015
  2. CWE, Common Weakness Enumeration, http://cwe.mitre.org
  3. CVE, Common Vulnerabilities and Exposures, http://cve.mitre.org
  4. C. Cadar, and K. Sen, "Symbolic execution for software testing: three decades later," Communications of the ACM, 56.2, pp.82-90, July 2013. https://doi.org/10.1145/2408776.2408795
  5. P. Cousot, and R. Cousot, "Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints," Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, pp.238-252, ACM, January 1977.
  6. S. Hendrik, and S. Kowalewski, "Static analysis of Sequential Function Charts using abstract interpretation," Emerging Technologies and Factory Automation (ETFA), pp.1-4, 2016 IEEE 21st International Conference on, September 2016.
  7. Clang, http://Clang-analyzer.llvm.org
  8. CppCheck, http://CppCheck.sourceforge.net
  9. Compass User Manual, http://rosecompiler.org/Compass.pdf
  10. NIST, http://samate.nist.gov/SRD/testsuite.php
  11. IARPA, STONESOUP(Securely Taking On New Executable Software of Uncertain Provenance)
  12. SAMATE, Juliet Test Suite v1.2 for C/C++ User Guide, National Security Agency
  13. IARPA, http://www.iarpa.gov
  14. MINESTRONE, http://nsl.cs.columbia.edu/projects/m-inestrone
  15. PEASOUP, http://www.grammatech.com/software-hardening/research
  16. VIBRANCE, http://stonesoup.kestrel.edu
  17. NIST, Report on the Static Analysis Tool Exposition (SATE) IV
  18. LDRA Testbed, http://www.ldra.com/en/testbedtbvision
  19. INFER, http://fbinfer.com
  20. Parasoft C++ test, http://www.parasoft.com/product/static-analysis-cc
  21. Red Lizard Software Goanna, http://redlizards.com
  22. C. Lattner, and V. Adve, "LLVM: A compilation framework for lifelong program analysis & transformation," Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization, pp.75, IEEE Computer Society, March 2004.
  23. B. C. Lopes, and R. Auler, "Getting started with LLVM core libraries," Packt Publishing Ltd, pp.73-104, 2014.
  24. K. Cooper, and L. Torczon, "Engineering a compiler," Elsevier, pp.231-232, 2011.
  25. ROSE compiler infrastructure, http://rosecompiler.org