• Title/Summary/Keyword: unsupervised detection

Search Result 173, Processing Time 0.026 seconds

Extraction of Changed Pixels for Hyperion Hyperspectral Images Using Range Average Based Buffer Zone Concept (구간평균 그래프 기반의 버퍼존 개념을 적용한 Hyperion 초분광영상의 변화화소 추출)

  • Kim, Dae-Sung;Pyen, Mu-Wook
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.29 no.5
    • /
    • pp.487-496
    • /
    • 2011
  • This study is aimed to perform more reliable unsupervised change detection through the re-extraction of the changed pixels which were extracted with global thresholding by applying buffer zone concept. First, three buffer zone was divided on the basis of the thresholding value which was determined using range average and the maximum distance point from a straight line. We re-extracted the changed pixels by performing unsupervised classification for buffer zone II which consists of changed pixels and unchanged pixels. The proposed method was implemented in Hyperion hyperspectral images and evaluated comparing to the existing global thresholding method. The experimental results demonstrated that the proposed method performed more accuracy change detection for vegetation area even if extracted slightly more changed pixels.

Autoencoder-Based Automotive Intrusion Detection System Using Gaussian Kernel Density Estimation Function (가우시안 커널 밀도 추정 함수를 이용한 오토인코더 기반 차량용 침입 탐지 시스템)

  • Donghyeon Kim;Hyungchul Im;Seongsoo Lee
    • Journal of IKEEE
    • /
    • v.28 no.1
    • /
    • pp.6-13
    • /
    • 2024
  • This paper proposes an approach to detect abnormal data in automotive controller area network (CAN) using an unsupervised learning model, i.e. autoencoder and Gaussian kernel density estimation function. The proposed autoencoder model is trained with only message ID of CAN data frames. Afterwards, by employing the Gaussian kernel density estimation function, it effectively detects abnormal data based on the trained model characterized by the optimally determined number of frames and a loss threshold. It was verified and evaluated using four types of attack data, i.e. DoS attacks, gear spoofing attacks, RPM spoofing attacks, and fuzzy attacks. Compared with conventional unsupervised learning-based models, it has achieved over 99% detection performance across all evaluation metrics.

Dynamic Asset Allocation by Applying Regime Detection Analysis (Regime 탐지 분석을 이용한 동적 자산 배분 기법)

  • Kim, Woo Chang
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.38 no.4
    • /
    • pp.258-261
    • /
    • 2012
  • In this paper, I propose a new asset allocation framework to cope with the dynamic nature of the financial market. The investment performance can be much improved by protecting the capital from the market crashes, and such crashes can be pre-identified with high probabilities by regime detection analysis via a specialized unsupervised machine learning technique.

Cloudy Area Detection in Satellite Image using K-Means & GHA (K-Means 와 GHA를 이용한 위성영상 구름영역 검출)

  • 서석배;김종우;최해진
    • Proceedings of the IEEK Conference
    • /
    • 2003.11a
    • /
    • pp.405-408
    • /
    • 2003
  • This paper proposes a new algorithm for cloudy area detection using K-Means and GHA (Generalized Hebbian Algorithm). K-Means is one of simple classification algorithm, and GHA is unsupervised neural network for data compression and pattern classification. Proposed algorithm is based on block based image processing that size is l6$\times$l6. Experimental results shows good performance of cloudy area detection except blur cloudy areas.

  • PDF

Improvement of Network Intrusion Detection Rate by Using LBG Algorithm Based Data Mining (LBG 알고리즘 기반 데이터마이닝을 이용한 네트워크 침입 탐지율 향상)

  • Park, Seong-Chul;Kim, Jun-Tae
    • Journal of Intelligence and Information Systems
    • /
    • v.15 no.4
    • /
    • pp.23-36
    • /
    • 2009
  • Network intrusion detection have been continuously improved by using data mining techniques. There are two kinds of methods in intrusion detection using data mining-supervised learning with class label and unsupervised learning without class label. In this paper we have studied the way of improving network intrusion detection accuracy by using LBG clustering algorithm which is one of unsupervised learning methods. The K-means method, that starts with random initial centroids and performs clustering based on the Euclidean distance, is vulnerable to noisy data and outliers. The nonuniform binary split algorithm uses binary decomposition without assigning initial values, and it is relatively fast. In this paper we applied the EM(Expectation Maximization) based LBG algorithm that incorporates the strength of two algorithms to intrusion detection. The experimental results using the KDD cup dataset showed that the accuracy of detection can be improved by using the LBG algorithm.

  • PDF

A Study on Automatic Vehicle Extraction within Drone Image Bounding Box Using Unsupervised SVM Classification Technique (무감독 SVM 분류 기법을 통한 드론 영상 경계 박스 내 차량 자동 추출 연구)

  • Junho Yeom
    • Land and Housing Review
    • /
    • v.14 no.4
    • /
    • pp.95-102
    • /
    • 2023
  • Numerous investigations have explored the integration of machine leaning algorithms with high-resolution drone image for object detection in urban settings. However, a prevalent limitation in vehicle extraction studies involves the reliance on bounding boxes rather than instance segmentation. This limitation hinders the precise determination of vehicle direction and exact boundaries. Instance segmentation, while providing detailed object boundaries, necessitates labour intensive labelling for individual objects, prompting the need for research on automating unsupervised instance segmentation in vehicle extraction. In this study, a novel approach was proposed for vehicle extraction utilizing unsupervised SVM classification applied to vehicle bounding boxes in drone images. The method aims to address the challenges associated with bounding box-based approaches and provide a more accurate representation of vehicle boundaries. The study showed promising results, demonstrating an 89% accuracy in vehicle extraction. Notably, the proposed technique proved effective even when dealing with significant variations in spectral characteristics within the vehicles. This research contributes to advancing the field by offering a viable solution for automatic and unsupervised instance segmentation in the context of vehicle extraction from image.

A Study on the Design of Supervised and Unsupervised Learning Models for Fault and Anomaly Detection in Manufacturing Facilities (제조 설비 이상탐지를 위한 지도학습 및 비지도학습 모델 설계에 관한 연구)

  • Oh, Min-Ji;Choi, Eun-Seon;Roh, Kyung-Woo;Kim, Jae-Sung;Cho, Wan-Sup
    • The Journal of Bigdata
    • /
    • v.6 no.1
    • /
    • pp.23-35
    • /
    • 2021
  • In the era of the 4th industrial revolution, smart factories have received great attention, where production and manufacturing technology and ICT converge. With the development of IoT technology and big data, automation of production systems has become possible. In the advanced manufacturing industry, production systems are subject to unscheduled performance degradation and downtime, and there is a demand to reduce safety risks by detecting and reparing potential errors as soon as possible. This study designs a model based on supervised and unsupervised learning for detecting anomalies. The accuracy of XGBoost, LightGBM, and CNN models was compared as a supervised learning analysis method. Through the evaluation index based on the confusion matrix, it was confirmed that LightGBM is most predictive (97%). In addition, as an unsupervised learning analysis method, MD, AE, and LSTM-AE models were constructed. Comparing three unsupervised learning analysis methods, the LSTM-AE model detected 75% of anomalies and showed the best performance. This study aims to contribute to the advancement of the smart factory by combining supervised and unsupervised learning techniques to accurately diagnose equipment failures and predict when abnormal situations occur, thereby laying the foundation for preemptive responses to abnormal situations. do.

Network Intrusion Detection System Using Feature Extraction Based on AutoEncoder in IOT environment (IOT 환경에서의 오토인코더 기반 특징 추출을 이용한 네트워크 침입탐지 시스템)

  • Lee, Joohwa;Park, Keehyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.8 no.12
    • /
    • pp.483-490
    • /
    • 2019
  • In the Network Intrusion Detection System (NIDS), the function of classification is very important, and detection performance depends on various features. Recently, a lot of research has been carried out on deep learning, but network intrusion detection system experience slowing down problems due to the large volume of traffic and a high dimensional features. Therefore, we do not use deep learning as a classification, but as a preprocessing process for feature extraction and propose a research method from which classifications can be made based on extracted features. A stacked AutoEncoder, which is a representative unsupervised learning of deep learning, is used to extract features and classifications using the Random Forest classification algorithm. Using the data collected in the IOT environment, the performance was more than 99% when normal and attack traffic are classified into multiclass, and the performance and detection rate were superior even when compared with other models such as AE-RF and Single-RF.

Anomaly Detection of Generative Adversarial Networks considering Quality and Distortion of Images (이미지의 질과 왜곡을 고려한 적대적 생성 신경망과 이를 이용한 비정상 검출)

  • Seo, Tae-Moon;Kang, Min-Guk;Kang, Dong-Joong
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.3
    • /
    • pp.171-179
    • /
    • 2020
  • Recently, studies have shown that convolution neural networks are achieving the best performance in image classification, object detection, and image generation. Vision based defect inspection which is more economical than other defect inspection, is a very important for a factory automation. Although supervised anomaly detection algorithm has far exceeded the performance of traditional machine learning based method, it is inefficient for real industrial field due to its tedious annotation work, In this paper, we propose ADGAN, a unsupervised anomaly detection architecture using the variational autoencoder and the generative adversarial network which give great results in image generation task, and demonstrate whether the proposed network architecture identifies anomalous images well on MNIST benchmark dataset as well as our own welding defect dataset.

A study in fault detection and diagnosis of induction motor by clustering and fuzzy fault tree (클러스터링과 fuzzy fault tree를 이용한 유도전동기 고장 검출과 진단에 관한 연구)

  • Lee, Seong-Hwan;Shin, Hyeon-Ik;Kang, Sin-Jun;Woo, Cheon-Hui;Woo, Gwang-Bang
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.4 no.1
    • /
    • pp.123-133
    • /
    • 1998
  • In this paper, an algorithm of fault detection and diagnosis during operation of induction motors under the condition of various loads and rates is investigated. For this purpose, the spectrum pattern of input currents is used in monitoring the state of induction motors, and by clustering the spectrum pattern of input currents, the newly occurrence of spectrum patterns caused by faults are detected. For the diagnosis of the fault detected, a fuzzy fault tree is designed, and the fuzzy relation equation representing the relation between an induction motor fault and each fault type, is solved. The solution of the fuzzy relation equation shows the possibility of occurence of each fault. The results obtained are summarized as follows : (1) Using clustering algorithm by unsupervised learning, an on-line fault detection method unaffected by the characteristics of loads and rates is implemented, and the degree of dependency for experts during fault detection is reduced. (2) With the fuzzy fault tree, the fault diagnosis process become systematic and expandable to the whole system, and the diagnosis for sub-systems can be made as an object-oriented module.

  • PDF