• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.30 no.4
• /
• pp.605-616
• /
• 2020

Obfuscation technology delays the analysis of a program by modifying internal logic such as data structure and control flow while maintaining the program's functionality. However, the application of such obfuscation technology to malicious code frequently occurs to reduce the detection rate of malware in antivirus software. The obfuscation technology applied to protect software intellectual property is applied to the malicious code in reverse, which not only lowers the detection rate of the malicious code but also makes it difficult to analyze and thus makes it difficult to identify the functionality of the malicious code. The study of reverse obfuscation techniques that can be closely restored should also continue. This paper analyzes the characteristics of obfuscated code with the option of Pack the Output File and Import Protection among detailed obfuscation technologies provided by VMProtect 3.4.0, a popular tool among commercial obfuscation tools. We present a de-obfuscation algorithm.

• KIPS Transactions on Computer and Communication Systems
• /
• v.10 no.11
• /
• pp.297-304
• /
• 2021

Commercial obfuscation tools (protectors) aim to create difficulties in analyzing the operation process of software by applying obfuscation techniques and Anti-reversing techniques that delay and interrupt the analysis of programs in software reverse engineering process. In particular, in case of virtualization detection and anti-debugging functions, the analysis tool exits the normal execution flow and terminates the program. In this paper, we analyze Anti-reversing techniques of executables with Debugger Detection and Viralization Tools Detection options through VMProtect 3.5.0, one of the commercial obfuscation tools (protector), and address bypass methods using Pin. In addition, we predicted the location of the applied obfuscation technique by finding out a specific program termination routine through API analysis since there is a problem that the program is terminated by the Anti-VM technology and the Anti-DBI technology and drew up the algorithm flowchart for bypassing the Anti-reversing techniques. Considering compatibility problems and changes in techniques from differences in versions of the software used in experiment, it was confirmed that the bypass was successful by writing the pin automation bypass code in the latest version of the software (VMProtect, Windows, Pin) and conducting the experiment. By improving the proposed analysis method, it is possible to analyze the Anti-reversing method of the obfuscation tool for which the method is not presented so far and find a bypass method.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.32 no.2
• /
• pp.181-192
• /
• 2022

The emergence of new malware is incapacitating existing signature-based malware detection techniques., and applying various anti-analysis techniques makes it difficult to analyze. Recent studies related to signature-based malware detection have limitations in that malware creators can easily bypass them. Therefore, in this study, we try to build a machine learning model that can detect and classify the anti-analysis techniques of packers applied to malware, not using the characteristics of the malware itself. In this study, the n-gram opcodes are extracted from the malicious binary to which various anti-analysis techniques of the commercial packers are applied, and the features are extracted by using TF-IDF, and through this, each anti-analysis technique is detected and classified. In this study, real-world malware samples packed using The mida and VMProtect with multiple anti-analysis techniques were trained and tested with 6 machine learning models, and it constructed the optimal model showing 81.25% accuracy for The mida and 95.65% accuracy for VMProtect.