Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

A Dynamic Approach to Extract the Original Semantics and Structure of VM-based Obfuscated Binary Executables

가상 머신 기반으로 난독화된 실행파일의 구조 및 원본의미 추출 동적 방법

  • 이성호 (한국과학기술원 전산학과) ;
  • 한태숙 (한국과학기술원 전산학과)
  • Received : 2014.05.20
  • Accepted : 2014.08.14
  • Published : 2014.10.15
⟨ Previous Next ⟩

Abstract

In recent years, the obfuscation techniques are commonly exploited to protect malwares, so obfuscated malwares have become a big threat. Especially, it is extremely hard to analyze virtualization-obfuscated malwares based on unusual virtual machines, because the original program is hidden by the virtual machine as well as its semantics is mixed with the semantics of the virtual machine. To confront this threat, we suggest a framework to analyze virtualization-obfuscated programs based on the dynamic analysis. First, we extract the dynamic execution trace of the virtualization-obfuscated executables. Second, we analyze the traces by translating machine instruction sequences into the intermediate representation and extract the virtual machine architecture by constructing dynamic context flow graphs. Finally, we extract abstract semantics of the original program using the extracted virtual machine architecture. In this paper, we propose a method to extract the information of the original program from a virtualization-obfuscated program by some commercial obfuscation tools. We expect that our tool can be used to understand virtualization-obfuscated programs and integrate other program analysis techniques so that it can be applied to analysis of the semantics of original programs using the abstract semantics.

최근 몇 년 동안, 난독화 기술은 악성 코드를 보호하기 위해 악용되어 큰 위협이 되고 있다. 특히, 가상 머신 기반으로 난독화된 악성 코드의 경우, 원본 프로그램이 직접적으로 드러나지 않고 가상머신의 의미와 원본 프로그램의 의미가 함께 수행되므로 분석하기 어렵다. 이러한 위협에 대응하기 위하여, 가상 머신 기반으로 난독화된 프로그램을 분석하는 동적 분석 기반의 프레임워크를 제안한다. 첫째, 난독화된 실행파일의 동적 실행 트레이스를 추출한다. 둘째, 동적 실행 트레이스를 중간언어로 변환하고 동적 제어 흐름 그래프를 이용하여 가상 머신의 구조를 추출한다. 결과적으로, 추출된 가상 머신 구조를 이용하여 원본 프로그램의 의미를 추출한다. 본 논문은 최신 상용 난독화 도구로 난독화된 실행파일에서 원본 프로그램을 추측할 수 있는 방안을 제시한다. 개발된 도구는 가상 머신 기반으로 난독화된 프로그램을 이해하고 프로그램 분석 기법을 적용하는 데 활용될 수 있으며 추출된 원본 프로그램의 요약 의미를 이용하여 추가적인 분석을 적용할 수 있을 것으로 기대한다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. DigiCAP, Codejam service [Online]. Available: http://www.codejam.or.kr/.
  2. SafeNet, Sentinel envelope [Online]. Available: http://www.safenet-inc.com/software-momentization/sentinel-envelope/.
  3. Kindsight, Kindsight security labs malware report-Q3 2013 [Online]. Available: http://www.kindsight.net/sites/default/_les/Kindsight-Q3-2013-Malware-Report-_nal.pdf
  4. M. Dalla Preda, M. Christodorescu, S. Jha, S. Debray, "A Semantics-Based Approach to Malware Detection," Proc. of the 34th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL'07), pp. 377-388, Jan. 2007.
  5. E. Kirda, C. Kruegel, "Behavior-based Spyware Detection," Proc. of the 15th conference on USENIX Security Symposium, pp. 273-288, Jul. 2006.
  6. VMProtect software, VMProtect [Online]. Available: http://vmpsoft.com/.
  7. Oreans, Code Virtualizer [Online]. Available: http://www.oreans.com/codevirtualizer.php
  8. K. Coogan, G. Lu, S. Debray, "Deobfuscation of Virtualization-Obfuscated Software," Proc. of the 18th ACM Conference on Computer and Communications Security (CCS'11), Oct. 2011.
  9. J. Raber, "Virtual Deobfuscator - A DARPA Cyber Fast Track Funded Effort," Proc. of the 16th Black Hat USA, Jul. 2013.
  10. R. Rolles, "Unpacking Virtualization Obfuscators," Proc. of the 3rd USENIX conference on Offensive technologies (WOOT'09), Aug. 2009.
  11. M. Sharif, A. Lanzi, J. Giffin, W. Lee, "Automatic Reverse Engineering of Malware Emulators," Proc. of the 30th IEEE Symposium on Security & Privacy (S&P'09), May 2009.
  12. C. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallance, V. J. Reddi, K. Hazelwood, "Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation," Proc. of the 2005 ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI' 05), pp. 190-200, Jun. 2005.
  13. N. Nethercote, J. Seward, "Valgrind: a framework for heavyweight dynamic binary instrumentation," Proc. of the 2007 ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI' 07), pp. 89-100, Jun. 2007.
  14. D. Bruening, E. Duesterwald, S. Amarasinghe, "Design and Implementation of a Dynamic Optimization Framework for Windows," Proc. of the 4th ACM Workshop on Feedback-Directed and Dynamic Optimization (FDDO-4), Dec. 2001.
  15. C. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallance, V. J. Reddi, K. Hazelwood, "Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation," Proc. of the 2005 ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI' 05), pp. 190- 200, Jun. 2005.
  16. D. Brumley, I. Jager, T. Avgerinos, E. J. Schwartz, "BAP: a binary analysis platform," Proc. of the 23rd International Conference on Computer Aided Verification (CAV' 11), pp. 463-469, Jul. 2011.
  17. G. Dabah, Powerful Disassembler Library for AMD64 [Online]. Available: http://www.ragestorm.net/distorm/.
  18. Graphviz, Graphviz - Graph Visualization Software [Online]. Available: http://www.graphviz.org/.