• Title/Summary/Keyword: Anomaly detect

Search Result 257, Processing Time 0.023 seconds

Enhancing Smart Grid Anomaly Detection Through Deep Learning (딥 러닝을 통해 스마트 그리드 이상 탐지 기능 강화)

  • Razzokov Otabek;Gwang-Jun Kim
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.6
    • /
    • pp.1215-1224
    • /
    • 2024
  • Modernizing the power grid to a smart grid offers many benefits in terms of efficiency, reliability, and sustainability. However, complex and interconnected smart grid systems present new challenges in detecting anomalies and maintaining grid resilience. Existing anomaly detection methods have difficulty adapting to the dynamic and heterogeneous characteristics of smart grid data, resulting in inefficiency in anomaly detection and mitigation. To solve these problems, this study proposes a new approach to improve smart grid anomaly detection and resilience through deep learning technology using the TensorFlow framework. The goals of the research are divided into two. First, to develop an advanced deep learning model that can accurately detect anomalies within smart grid data, and second, to strengthen grid resilience by proactively responding to and mitigating detected anomalies.

Anomaly Detection Methodology Based on Multimodal Deep Learning (멀티모달 딥 러닝 기반 이상 상황 탐지 방법론)

  • Lee, DongHoon;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.2
    • /
    • pp.101-125
    • /
    • 2022
  • Recently, with the development of computing technology and the improvement of the cloud environment, deep learning technology has developed, and attempts to apply deep learning to various fields are increasing. A typical example is anomaly detection, which is a technique for identifying values or patterns that deviate from normal data. Among the representative types of anomaly detection, it is very difficult to detect a contextual anomaly that requires understanding of the overall situation. In general, detection of anomalies in image data is performed using a pre-trained model trained on large data. However, since this pre-trained model was created by focusing on object classification of images, there is a limit to be applied to anomaly detection that needs to understand complex situations created by various objects. Therefore, in this study, we newly propose a two-step pre-trained model for detecting abnormal situation. Our methodology performs additional learning from image captioning to understand not only mere objects but also the complicated situation created by them. Specifically, the proposed methodology transfers knowledge of the pre-trained model that has learned object classification with ImageNet data to the image captioning model, and uses the caption that describes the situation represented by the image. Afterwards, the weight obtained by learning the situational characteristics through images and captions is extracted and fine-tuning is performed to generate an anomaly detection model. To evaluate the performance of the proposed methodology, an anomaly detection experiment was performed on 400 situational images and the experimental results showed that the proposed methodology was superior in terms of anomaly detection accuracy and F1-score compared to the existing traditional pre-trained model.

A machine learning framework for performance anomaly detection

  • Hasnain, Muhammad;Pasha, Muhammad Fermi;Ghani, Imran;Jeong, Seung Ryul;Ali, Aitizaz
    • Journal of Internet Computing and Services
    • /
    • v.23 no.2
    • /
    • pp.97-105
    • /
    • 2022
  • Web services show a rapid evolution and integration to meet the increased users' requirements. Thus, web services undergo updates and may have performance degradation due to undetected faults in the updated versions. Due to these faults, many performances and regression anomalies in web services may occur in real-world scenarios. This paper proposed applying the deep learning model and innovative explainable framework to detect performance and regression anomalies in web services. This study indicated that upper bound and lower bound values in performance metrics provide us with the simple means to detect the performance and regression anomalies in updated versions of web services. The explainable deep learning method enabled us to decide the precise use of deep learning to detect performance and anomalies in web services. The evaluation results of the proposed approach showed us the detection of unusual behavior of web service. The proposed approach is efficient and straightforward in detecting regression anomalies in web services compared with the existing approaches.

Flow-based Anomaly Detection Using Access Behavior Profiling and Time-sequenced Relation Mining

  • Liu, Weixin;Zheng, Kangfeng;Wu, Bin;Wu, Chunhua;Niu, Xinxin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.6
    • /
    • pp.2781-2800
    • /
    • 2016
  • Emerging attacks aim to access proprietary assets and steal data for business or political motives, such as Operation Aurora and Operation Shady RAT. Skilled Intruders would likely remove their traces on targeted hosts, but their network movements, which are continuously recorded by network devices, cannot be easily eliminated by themselves. However, without complete knowledge about both inbound/outbound and internal traffic, it is difficult for security team to unveil hidden traces of intruders. In this paper, we propose an autonomous anomaly detection system based on behavior profiling and relation mining. The single-hop access profiling model employ a novel linear grouping algorithm PSOLGA to create behavior profiles for each individual server application discovered automatically in historical flow analysis. Besides that, the double-hop access relation model utilizes in-memory graph to mine time-sequenced access relations between different server applications. Using the behavior profiles and relation rules, this approach is able to detect possible anomalies and violations in real-time detection. Finally, the experimental results demonstrate that the designed models are promising in terms of accuracy and computational efficiency.

Anomaly Detection Scheme of Web-based attacks by applying HMM to HTTP Outbound Traffic (HTTP Outbound Traffic에 HMM을 적용한 웹 공격의 비정상 행위 탐지 기법)

  • Choi, Byung-Ha;Choi, Sung-Kyo;Cho, Kyung-San
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.5
    • /
    • pp.33-40
    • /
    • 2012
  • In this paper we propose an anomaly detection scheme to detect new attack paths or new attack methods without false positives by monitoring HTTP Outbound Traffic after efficient training. Our proposed scheme detects web-based attacks by comparing tags or javascripts of HTTP Outbound Traffic with normal behavioral models which apply HMM(Hidden Markov Model). Through the verification analysis under the real-attacked environment, we show that our scheme has superior detection capability of 0.0001% false positive and 96% detection rate.

The Design and Implementation of Anomaly Traffic Analysis System using Data Mining

  • Lee, Se-Yul;Cho, Sang-Yeop;Kim, Yong-Soo
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.8 no.4
    • /
    • pp.316-321
    • /
    • 2008
  • Advanced computer network technology enables computers to be connected in an open network environment. Despite the growing numbers of security threats to networks, most intrusion detection identifies security attacks mainly by detecting misuse using a set of rules based on past hacking patterns. This pattern matching has a high rate of false positives and can not detect new hacking patterns, which makes it vulnerable to previously unidentified attack patterns and variations in attack and increases false negatives. Intrusion detection and analysis technologies are thus required. This paper investigates the asymmetric costs of false errors to enhance the performances the detection systems. The proposed method utilizes the network model to consider the cost ratio of false errors. By comparing false positive errors with false negative errors, this scheme achieved better performance on the view point of both security and system performance objectives. The results of our empirical experiment show that the network model provides high accuracy in detection. In addition, the simulation results show that effectiveness of anomaly traffic detection is enhanced by considering the costs of false errors.

Anomaly Detection Method Using Entropy of Network Traffic Distributions (네트워크 트래픽 분포 엔트로피를 이용한 비정상행위 탐지 방법)

  • Kang Koo-Hong;Oh Jin-Tae;Jang Jong-Soo
    • The KIPS Transactions:PartC
    • /
    • v.13C no.3 s.106
    • /
    • pp.283-294
    • /
    • 2006
  • Hostile network traffic is often different from normal traffic in ways that can be distinguished without knowing the exact nature of the attack. In this paper, we propose a new anomaly detection method using inbound network traffic distributions. For this purpose, we first characterize the traffic of a real campus network by the distributions of IP protocols, packet length, destination IP/port addresses, TTL value, TCP SYN packet, and fragment packet. And then we introduce the concept of entropy to transform the obtained baseline traffic distributions into manageable values. Finally, we can detect the anomalies by the difference of entropies between the current and baseline distributions. In particular, we apply the well-known denial-of-service attacks to a real campus network and show the experimental results.

Intelligent Intrusion Detection System based on Computer Immune System (컴퓨터 면역 시스템을 기반으로 한 지능형 침입탐지시스템)

  • Lee, Jong-Sung;Chae, Soo-Hoan
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.12
    • /
    • pp.3622-3633
    • /
    • 1999
  • Computer security is considered important due to tile side effect generated from the expansion of computer network and rapid increase of the use of computers. Intrusion Detection System(IDS) has been an active research area to reduce the risk from intruders. This paper discusses IDS of detecting anomaly behaviors and proposes a new intelligent IDS model, which consists of several computers with intelligent IDS, based on computer immune system. The intelligent IDSs are distributed and if any of distributed IDSs detect anomaly system call among system call sequences generated by a privilege process, the anomaly system call can be dynamically shared with other IDSs. This makes the intelligent IDSs improve the ability of immunity for new intruders.

  • PDF

A Study on Anomaly Detection Model using Worker Access Log in Manufacturing Terminal PC (제조공정 단말PC 작업자 접속 로그를 통한 이상 징후 탐지 모델 연구)

  • Ahn, Jong-seong;Lee, Kyung-ho
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.29 no.2
    • /
    • pp.321-330
    • /
    • 2019
  • Prevention of corporate confidentiality leakage by insiders in enterprises is an essential task for the survival of enterprises. In order to prevent information leakage by insiders, companies have adopted security solutions, but there is a limit to effectively detect abnormal behavior of insiders with access privileges. In this study, we use the Unsupervised Learning algorithm of the machine learning technique to effectively and efficiently cluster the normal and abnormal access logs of the worker's work screen in the manufacturing information system, which includes the company's product manufacturing history and quality information. We propose an optimal feature selection model for anomaly detection by studying clustering methods.

A New Distributed Log Anomaly Detection Method based on Message Middleware and ATT-GRU

  • Wei Fang;Xuelei Jia;Wen Zhang;Victor S. Sheng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.2
    • /
    • pp.486-503
    • /
    • 2023
  • Logs play an important role in mastering the health of the system, experienced operation and maintenance engineer can judge which part of the system has a problem by checking the logs. In recent years, many system architectures have changed from single application to distributed application, which leads to a very huge number of logs in the system and manually check the logs to find system errors impractically. To solve the above problems, we propose a method based on Message Middleware and ATT-GRU (Attention Gate Recurrent Unit) to detect the logs anomaly of distributed systems. The works of this paper mainly include two aspects: (1) We design a high-performance distributed logs collection architecture to complete the logs collection of the distributed system. (2)We improve the existing GRU by introducing the attention mechanism to weight the key parts of the logs sequence, which can improve the training efficiency and recognition accuracy of the model to a certain extent. The results of experiments show that our method has better superiority and reliability.