• Title/Summary/Keyword: anomaly detection algorithm

Search Result 164, Processing Time 0.024 seconds

Clustering Normal User Behavior for Anomaly Intrusion Detection (비정상행위 탐지를 위한 사용자 정상행위 클러스터링 기법)

  • Oh, Sang-Hyun;Lee, Won-Suk
    • The KIPS Transactions:PartC
    • /
    • v.10C no.7
    • /
    • pp.857-866
    • /
    • 2003
  • For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques in order to analyze an audit data set. However. since they mainly analyze the average behavior of a user's activities, some anomalies can be detected inaccurately. In this paper, a new clustering algorithm for modeling the normal pattern of a user's activities is proposed. Since clustering can identify an arbitrary number of dense ranges in an analysis domain, it can eliminate the inaccuracy caused by statistical analysis. Also, clustering can be used to model common knowledge occurring frequently in a set of transactions. Consequently, the common activities of a user can be found more accurately. The common knowledge is represented by the occurrence frequency of similar data objects by the unit of a transaction as veil as the common repetitive ratio of similar data objects in each transaction. Furthermore, the proposed method also addresses how to maintain identified common knowledge as a concise profile. As a result, the profile can be used to detect any anomalous behavior In an online transaction.

Preliminary Study on Detection of Marine Heat Waves using Satellite-based Sea Surface Temperature Anomaly in 2017-2018 (인공위성 해수면온도 편차 이용 한반도 연안 해역 고수온 탐지 : 2017-2018년도)

  • Kim, Tae-Ho;Yang, Chan-Su
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.25 no.6
    • /
    • pp.678-686
    • /
    • 2019
  • In this study, marine heat waves on coastal waters of Republic of Korea were detected using satellite-based Sea Surface Temperature Anomaly (SSTA). The detected results were compared with the warm water issues reported by the National Institute of Fisheries Science (NIFS). Marine heat waves detection algorithm using SSTA based on a threshold has proposed. The threshold value was defined as 2℃ for caution and 3℃ for warning issues, respectively. Daily averaged SST data from July to September of 2017-2018 were used to generate SSTA. The satellite-based detection results were classified into nine areas according to the place names used in the NIFS warm water issues. In the comparison of frequency of marine heat waves occurrence to each area with the warm water issue, most areas in the southern coast showed a similar pattern, that is probably NIFS uses spatially well distributed buoys. On the other hand, other sea areas had about two times more satellite detection results. This result seems to be because NIFS only considers the water temperature data measured at limited points. The results of this study are expected to contribute to the development of a satellite-based warm/cold water monitoring system in coastal waters.

A Security Nonce Generation Algorithm Scheme Research for Improving Data Reliability and Anomaly Pattern Detection of Smart City Platform Data Management (스마트시티 플랫폼 데이터 운영의 이상패턴 탐지 및 데이터 신뢰성 향상을 위한 보안 난수 생성 알고리즘 방안 연구)

  • Lee, Jaekwan;Shin, Jinho;Joo, Yongjae;Noh, Jaekoo;Kim, Jae Do;Kim, Yongjoon;Jung, Namjoon
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.4 no.2
    • /
    • pp.75-80
    • /
    • 2018
  • The smart city is developing an energy system efficiently through a common management of the city resource for the growth and a low carbon social. However, the smart city doesn't counter a verification effectively about a anomaly pattern detection when existing security technology (authentication, integrity, confidentiality) is used by fixed security key and key deodorization according to generated big data. This paper is proposed the "security nonce generation based on security nonce generation" for anomaly pattern detection of the adversary and a safety of the key is high through the key generation of the KDC (Key Distribution Center; KDC) for improvement. The proposed scheme distributes the generated security nonce and authentication keys to each facilities system by the KDC. This proposed scheme can be enhanced to the security by doing the external pattern detection and changed new security key through distributed security nonce with keys. Therefore, this paper can do improving the security and a responsibility of the smart city platform management data through the anomaly pattern detection and the safety of the keys.

The application of machine learning for the prognostics and health management of control element drive system

  • Oluwasegun, Adebena;Jung, Jae-Cheon
    • Nuclear Engineering and Technology
    • /
    • v.52 no.10
    • /
    • pp.2262-2273
    • /
    • 2020
  • Digital twin technology can provide significant value for the prognostics and health management (PHM) of critical plant components by improving insight into system design and operating conditions. Digital twinning of systems can be utilized for anomaly detection, diagnosis and the estimation of the system's remaining useful life in order to optimize operations and maintenance processes in a nuclear plant. In this regard, a conceptual framework for the application of digital twin technology for the prognosis of Control Element Drive Mechanism (CEDM), and a data-driven approach to anomaly detection using coil current profile are presented in this study. Health management of plant components can capitalize on the data and signals that are already recorded as part of the monitored parameters of the plant's instrumentation and control systems. This work is focused on the development of machine learning algorithm and workflow for the analysis of the CEDM using the recorded coil current data. The workflow involves features extraction from the coil-current profile and consequently performing both clustering and classification algorithms. This approach provides an opportunity for health monitoring in support of condition-based predictive maintenance optimization and in the development of the CEDM digital twin model for improved plant safety and availability.

A semi-supervised interpretable machine learning framework for sensor fault detection

  • Martakis, Panagiotis;Movsessian, Artur;Reuland, Yves;Pai, Sai G.S.;Quqa, Said;Cava, David Garcia;Tcherniak, Dmitri;Chatzi, Eleni
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.251-266
    • /
    • 2022
  • Structural Health Monitoring (SHM) of critical infrastructure comprises a major pillar of maintenance management, shielding public safety and economic sustainability. Although SHM is usually associated with data-driven metrics and thresholds, expert judgement is essential, especially in cases where erroneous predictions can bear casualties or substantial economic loss. Considering that visual inspections are time consuming and potentially subjective, artificial-intelligence tools may be leveraged in order to minimize the inspection effort and provide objective outcomes. In this context, timely detection of sensor malfunctioning is crucial in preventing inaccurate assessment and false alarms. The present work introduces a sensor-fault detection and interpretation framework, based on the well-established support-vector machine scheme for anomaly detection, combined with a coalitional game-theory approach. The proposed framework is implemented in two datasets, provided along the 1st International Project Competition for Structural Health Monitoring (IPC-SHM 2020), comprising acceleration and cable-load measurements from two real cable-stayed bridges. The results demonstrate good predictive performance and highlight the potential for seamless adaption of the algorithm to intrinsically different data domains. For the first time, the term "decision trajectories", originating from the field of cognitive sciences, is introduced and applied in the context of SHM. This provides an intuitive and comprehensive illustration of the impact of individual features, along with an elaboration on feature dependencies that drive individual model predictions. Overall, the proposed framework provides an easy-to-train, application-agnostic and interpretable anomaly detector, which can be integrated into the preprocessing part of various SHM and condition-monitoring applications, offering a first screening of the sensor health prior to further analysis.

An Empirical Comparison Study on Attack Detection Mechanisms Using Data Mining (데이터 마이닝을 이용한 공격 탐지 메커니즘의 실험적 비교 연구)

  • Kim, Mi-Hui;Oh, Ha-Young;Chae, Ki-Joon
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.31 no.2C
    • /
    • pp.208-218
    • /
    • 2006
  • In this paper, we introduce the creation methods of attack detection model using data mining technologies that can classify the latest attack types, and can detect the modification of existing attacks as well as the novel attacks. Also, we evaluate comparatively these attack detection models in the view of detection accuracy and detection time. As the important factors for creating detection models, there are data, attribute, and detection algorithm. Thus, we used NetFlow data gathered at the real network, and KDD Cup 1999 data for the experiment in large quantities. And for attribute selection, we used a heuristic method and a theoretical method using decision tree algorithm. We evaluate comparatively detection models using a single supervised/unsupervised data mining approach and a combined supervised data mining approach. As a result, although a combined supervised data mining approach required more modeling time, it had better detection rate. All models using data mining techniques could detect the attacks within 1 second, thus these approaches could prove the real-time detection. Also, our experimental results for anomaly detection showed that our approaches provided the detection possibility for novel attack, and especially SOM model provided the additional information about existing attack that is similar to novel attack.

State Transition Algorithm for Penetration Scenarios Detection using Association Mining Technique (연관마이닝 기법을 이용한 침입 시나리오 탐지를 위한 상태전이 알고리즘)

  • 김창수;황현숙
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2001.05a
    • /
    • pp.720-723
    • /
    • 2001
  • 현재 인터넷 환경에서 크래킹은 보편화되어 있다. 이러한 크래킹을 탐지하거나 방어하기 위한 기법들은 대부분 기존의 불법 침입 유형을 분석하여 대응 알고리즘을 개발하는 것이 대부분이다. 현재 알려진 침입 탐지 기법은 비정상 탐지(Anomaly Detection)와 오용 탐지(Misuse Detection)로 분류할 수 있는데, 전자는 통계적 방법, 특징 추출 등을 이용하며, 후자는 조건부 화률, 전문가 시스템, 상태 전이 분석, 패턴 매칭 둥을 적용한다. 본 연구에서는 상태전이 기반의 연관 마이닝 기법을 이용한 침입 시나리오 탐지 알고리즘을 제안한다. 이를 위해 본 연구에서는 의사결정지원시스템에서 많이 적용한 연관 마이닝 기법을 여러 가지 불법 침입과 연관된 상태 정보를 분석할 수 있는 수정된 상태전이 알고리즘을 제시한다.

  • PDF

Structural novelty detection based on sparse autoencoders and control charts

  • Finotti, Rafaelle P.;Gentile, Carmelo;Barbosa, Flavio;Cury, Alexandre
    • Structural Engineering and Mechanics
    • /
    • v.81 no.5
    • /
    • pp.647-664
    • /
    • 2022
  • The powerful data mapping capability of computational deep learning methods has been recently explored in academic works to develop strategies for structural health monitoring through appropriate characterization of dynamic responses. In many cases, these studies concern laboratory prototypes and finite element models to validate the proposed methodologies. Therefore, the present work aims to investigate the capability of a deep learning algorithm called Sparse Autoencoder (SAE) specifically focused on detecting structural alterations in real-case studies. The idea is to characterize the dynamic responses via SAE models and, subsequently, to detect the onset of abnormal behavior through the Shewhart T control chart, calculated with SAE extracted features. The anomaly detection approach is exemplified using data from the Z24 bridge, a classical benchmark, and data from the continuous monitoring of the San Vittore bell-tower, Italy. In both cases, the influence of temperature is also evaluated. The proposed approach achieved good performance, detecting structural changes even under temperature variations.

An Analysis of Intrusion Pattern Based on Backpropagation Algorithm (역전파 알고리즘 기반의 침입 패턴 분석)

  • Woo Chong-Woo;Kim Sang-Young
    • Journal of Internet Computing and Services
    • /
    • v.5 no.5
    • /
    • pp.93-103
    • /
    • 2004
  • The main function of the intrusion Detection System (IDS) usee to be more or less passive detection of the intrusion evidences, but recently it is developed with more diverse types and methodologies. Especially, it is required that the IDS should process large system audit data fast enough. Therefore the data mining or neural net algorithm is being focused on, since they could satisfy those situations. In this study, we first surveyed and analyzed the several recent intrusion trends and types. And then we designed and implemented an IDS using back-propagation algorithm of the neural net, which could provide more effective solution. The distinctive feature of our study could be stated as follows. First, we designed the system that allows both the Anomaly dection and the Misuse detection. Second, we carried out the intrusion analysis experiment by using the reliable KDD Cup ‘99 data, which would provide us similar results compared to the real data. Finally, we designed the system based on the object-oriented concept, which could adapt to the other algorithms easily.

  • PDF

A Pre-processing Study to Solve the Problem of Rare Class Classification of Network Traffic Data (네트워크 트래픽 데이터의 희소 클래스 분류 문제 해결을 위한 전처리 연구)

  • Ryu, Kyung Joon;Shin, DongIl;Shin, DongKyoo;Park, JeongChan;Kim, JinGoog
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.12
    • /
    • pp.411-418
    • /
    • 2020
  • In the field of information security, IDS(Intrusion Detection System) is normally classified in two different categories: signature-based IDS and anomaly-based IDS. Many studies in anomaly-based IDS have been conducted that analyze network traffic data generated in cyberspace by machine learning algorithms. In this paper, we studied pre-processing methods to overcome performance degradation problems cashed by rare classes. We experimented classification performance of a Machine Learning algorithm by reconstructing data set based on rare classes and semi rare classes. After reconstructing data into three different sets, wrapper and filter feature selection methods are applied continuously. Each data set is regularized by a quantile scaler. Depp neural network model is used for learning and validation. The evaluation results are compared by true positive values and false negative values. We acquired improved classification performances on all of three data sets.