Journal of the Semiconductor & Display Technology (반도체디스플레이기술학회지)

The Korean Society Of Semiconductor & Display Technology (한국반도체디스플레이기술학회)
권호 표지

A Study of Reinforcement Learning-based Cyber Attack Prediction using Network Attack Simulator (NASim)

네트워크 공격 시뮬레이터를 이용한 강화학습 기반 사이버 공격 예측 연구

  • Bum-Sok Kim (Dept. of Electronic Information Systems Engineering, Graduate School, Sangmyung University) ;
  • Jung-Hyun Kim (Dept. of Electronic Information Systems Engineering, Graduate School, Sangmyung University) ;
  • Min-Suk Kim (Dept. of Human Intelligent Robotics Engineering, Sangmyung University)
  • 김범석 (상명대학교 전자정보시스템공학과) ;
  • 김정현 (상명대학교 전자정보시스템공학과) ;
  • 김민석 (상명대학교 휴먼지능로봇공학과)
  • Received : 2023.09.08
  • Accepted : 2023.09.18
  • Published : 2023.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

As technology advances, the need for enhanced preparedness against cyber-attacks becomes an increasingly critical problem. Therefore, it is imperative to consider various circumstances and to prepare for cyber-attack strategic technology. This paper proposes a method to solve network security problems by applying reinforcement learning to cyber-security. In general, traditional static cyber-security methods have difficulty effectively responding to modern dynamic attack patterns. To address this, we implement cyber-attack scenarios such as 'Tiny Alpha' and 'Small Alpha' and evaluate the performance of various reinforcement learning methods using Network Attack Simulator, which is a cyber-attack simulation environment based on the gymnasium (formerly Open AI gym) interface. In addition, we experimented with different RL algorithms such as value-based methods (Q-Learning, Deep-Q-Network, and Double Deep-Q-Network) and policy-based methods (Actor-Critic). As a result, we observed that value-based methods with discrete action spaces consistently outperformed policy-based methods with continuous action spaces, demonstrating a performance difference ranging from a minimum of 20.9% to a maximum of 53.2%. This result shows that the scheme not only suggests opportunities for enhancing cybersecurity strategies, but also indicates potential applications in cyber-security education and system validation across a large number of domains such as military, government, and corporate sectors.

Keywords

Acknowledgement

This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (MSIT) (No.2022-0-00961).

References

  1. Kim, D. W., Lee, M. S., Jeong, J. Y., Kim, and H. C., "COVID19 Related Keyword Analysis: Based on Topic Modeling and Semantic Network Analysis," Journal of the Semiconductor & Display Technology, Vol. 21, No. 2, pp.127-132, 2022.
  2. Kang, Y. G., Yoo, J. D., Park, E.J., Kim, D. H., and Kim, H.K., "Design and Implementation of Cyber Attack Simulator based on Attack Techniques Modeling," Journal of the Korea Society of Computer and Information, Vol.25, pp.65-72, 2020. https://doi.org/10.9708/JKSCI.2020.25.03.065
  3. Taherdoost, H., "Understanding cybersecurity frameworks and information security standards-a review and comprehensive overview," Electronics, Vol. 11, pp. 2181, 2022.
  4. Lee, J. Y., Moon, D. S. and Kim, I. K., "Technological Trends in Cyber Attack Simulations." Electronics and Telecommunications Trends, Vol. 35, pp. 34-48, 2020.
  5. Rabie, A. R., Bassam, W. A., Jalawi, S. A., Abdullah, J. A., Ayman, E.S., and Mohamed, M. D., "Cybersecurity and Countermeasures at the Time of Pandemic", Journal of Advanced Transportation, Vol. 2021, pp. 1-19, 2021.
  6. Al-Qahtani, A. F., and Cresci, S., "The COVID-19 scamdemic: A survey of phishing attacks and their countermeasures during COVID-19," IET Information Security, Vol.16, pp.324-345, 2022. https://doi.org/10.1049/ise2.12073
  7. Singh, S., Sharma, P. K., Moon, S. Y., Moon, D. S., and Park, J. H., "A comprehensive study on APT attacks and countermeasures for future networks and communications: challenges and solutions", The Journal of Supercomputing, Vol. 75, pp. 4543-4574, 2019. https://doi.org/10.1007/s11227-016-1850-4
  8. Choi,Y. K., Jang I. S., Whoang, I., Kim, T. G., Hong, S. J., Park, I. S., Yang, J. S., Kwon, Y. J., and Kang, J. M., "Design and Implementation of Cyber Range for Cyber Defense Exercise Based on Cyber Crisis Alert." Journal of the Korea Institute of Information Security & Cryptology, Vol. 30, pp. 805-821, 2020.
  9. Milian, E. Z., Spinola, M. D. M., and Carvalho, M. M., "Fintechs: A literature review and research agenda", Electronic Commerce Research and Applications, Vol. 34, 2019.
  10. Jiang, H., Choi, T., and Ko, R. K. L., "Pandora: A cyber range environment for the safe testing and deployment of autonomous cyber-attack tools," Security in Computing and Communications, Vol. 1364, pp. 1-20, 2021. https://doi.org/10.1007/978-981-16-0422-5_1
  11. Yamin, M. M., Katt, B., and Gkioulos, V., "Cyber ranges and security testbeds: Scenarios, functions, tools and architecture," Computers & Security, Vol. 88, pp. 101636, 2020.
  12. Yamin, M. M., and Katt, B., "Use of cyber attack and defense agents in cyber ranges: A case study," Computers & Security, Vol. 122, pp. 102892, 2022.
  13. Katina, P. F., and Keskin, O. F., "Complex system governance as a foundation for enhancing the cybersecurity of cyber-physical systems," International Journal of Cyber Warfare and Terrorism, Vol. 11, pp. 1-14, 2021. https://doi.org/10.4018/IJCWT.2021070101
  14. Kim, D. H., Kim, Y. H., Ahn, M. K., and Lee, H. J., "Automated cyber threat emulation based on ATT&CK for cyber security training," Journal of the Korea Society of Computer and Information, Vol. 25, pp. 71-80, 2020. https://doi.org/10.9708/JKSCI.2020.25.09.071
  15. Yoo, J. D., Park, E., Lee, G., Ahn, M. K., Kim, D., Seo, S., and Kim, H. K., "Cyber attack and defense emulation agents," Applied Sciences, Vol. 10, pp. 2140, 2020.
  16. Mohamed, N., "Study of bypassing Microsoft Windows Security using the MITRE CALDERA framework" F1000 Research, Vol. 11, pp. 16-25, 2022. https://doi.org/10.12688/f1000research.109148.3
  17. Sainadh, J., Navya, Y. S., Raja, P., Tagore, G., and Rao, G. R. K., "Dynamic Malware Analysis Using Cuckoo Sandbox", 2018 Second International Conference on Inventive Communication and Computational Technologies, pp. 1056-1060, 2018.
  18. Jjschwartz, "NetworkAttackSimulator (Version 0.9.1)." Retrieved August 12, 2021, from https://github.com/Jjschwartz/NetworkAttackSimulator (Publication Date: N/A).
  19. Microsoft, "CyberBattleSim (Version 2.4.0)." Retrieved August 26, 2021, from https://github.com/microsoft/CyberBattleSim.
  20. Walter, E., Ferguson-Walter, K., and Ridley, A., "Incorporating Deception into CyberBattleSim for Autonomous Defense." Retrieved May 18, 2023, from https://arxiv.org/abs/2108.13980, 2021.
  21. Xiang, X., Foo, S., and Zang, H., "Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems Part 2 - Applications in Transportation, Industries, Communications and Networking and More Topics", Machine Learning and Knowledge Extraction, Vol. 3, pp. 863-878, 2021. https://doi.org/10.3390/make3040043