• Title/Summary/Keyword: Unsupervised

Search Result 814, Processing Time 0.027 seconds

Unsupervised learning with hierarchical feature selection for DDoS mitigation within the ISP domain

  • Ko, Ili;Chambers, Desmond;Barrett, Enda
    • ETRI Journal
    • /
    • v.41 no.5
    • /
    • pp.574-584
    • /
    • 2019
  • A new Mirai variant found recently was equipped with a dynamic update ability, which increases the level of difficulty for DDoS mitigation. Continuous development of 5G technology and an increasing number of Internet of Things (IoT) devices connected to the network pose serious threats to cyber security. Therefore, researchers have tried to develop better DDoS mitigation systems. However, the majority of the existing models provide centralized solutions either by deploying the system with additional servers at the host site, on the cloud, or at third party locations, which may cause latency. Since Internet service providers (ISP) are links between the internet and users, deploying the defense system within the ISP domain is the panacea for delivering an efficient solution. To cope with the dynamic nature of the new DDoS attacks, we utilized an unsupervised artificial neural network to develop a hierarchical two-layered self-organizing map equipped with a twofold feature selection for DDoS mitigation within the ISP domain.

Detecting Anomalies in Time-Series Data using Unsupervised Learning and Analysis on Infrequent Signatures

  • Bian, Xingchao
    • Journal of IKEEE
    • /
    • v.24 no.4
    • /
    • pp.1011-1016
    • /
    • 2020
  • We propose a framework called Stacked Gated Recurrent Unit - Infrequent Residual Analysis (SG-IRA) that detects anomalies in time-series data that can be trained on streams of raw sensor data without any pre-labeled dataset. To enable such unsupervised learning, SG-IRA includes an estimation model that uses a stacked Gated Recurrent Unit (GRU) structure and an analysis method that detects anomalies based on the difference between the estimated value and the actual measurement (residual). SG-IRA's residual analysis method dynamically adapts the detection threshold from the population using frequency analysis, unlike the baseline model that relies on a constant threshold. In this paper, SG-IRA is evaluated using the industrial control systems (ICS) datasets. SG-IRA improves the detection performance (F1 score) by 5.9% compared to the baseline model.

Evaluating Unsupervised Deep Learning Models for Network Intrusion Detection Using Real Security Event Data

  • Jang, Jiho;Lim, Dongjun;Seong, Changmin;Lee, JongHun;Park, Jong-Geun;Cheong, Yun-Gyung
    • International journal of advanced smart convergence
    • /
    • v.11 no.4
    • /
    • pp.10-19
    • /
    • 2022
  • AI-based Network Intrusion Detection Systems (AI-NIDS) detect network attacks using machine learning and deep learning models. Recently, unsupervised AI-NIDS methods are getting more attention since there is no need for labeling, which is crucial for building practical NIDS systems. This paper aims to test the impact of designing autoencoder models that can be applied to unsupervised an AI-NIDS in real network systems. We collected security events of legacy network security system and carried out an experiment. We report the results and discuss the findings.

Deep Learning-Based Inverse Design for Engineering Systems: A Study on Supervised and Unsupervised Learning Models

  • Seong-Sin Kim
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.16 no.2
    • /
    • pp.127-135
    • /
    • 2024
  • Recent studies have shown that inverse design using deep learning has the potential to rapidly generate the optimal design that satisfies the target performance without the need for iterative optimization processes. Unlike traditional methods, deep learning allows the network to rapidly generate a large number of solution candidates for the same objective after a single training, and enables the generation of diverse designs tailored to the objectives of inverse design. These inverse design techniques are expected to significantly enhance the efficiency and innovation of design processes in various fields such as aerospace, biology, medical, and engineering. We analyzes inverse design models that are mainly utilized in the nano and chemical fields, and proposes inverse design models based on supervised and unsupervised learning that can be applied to the engineering system. It is expected to present the possibility of effectively applying inverse design methodologies to the design optimization problem in the field of engineering according to each specific objective.

Implementation of Unsupervised Nonlinear Classifier with Binary Harmony Search Algorithm (Binary Harmony Search 알고리즘을 이용한 Unsupervised Nonlinear Classifier 구현)

  • Lee, Tae-Ju;Park, Seung-Min;Ko, Kwang-Eun;Sung, Won-Ki;Sim, Kwee-Bo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.23 no.4
    • /
    • pp.354-359
    • /
    • 2013
  • In this paper, we suggested the method for implementation of unsupervised nonlinear classification using Binary Harmony Search (BHS) algorithm, which is known as a optimization algorithm. Various algorithms have been suggested for classification of feature vectors from the process of machine learning for pattern recognition or EEG signal analysis processing. Supervised learning based support vector machine or fuzzy c-mean (FCM) based on unsupervised learning have been used for classification in the field. However, conventional methods were hard to apply nonlinear dataset classification or required prior information for supervised learning. We solved this problems with proposed classification method using heuristic approach which took the minimal Euclidean distance between vectors, then we assumed them as same class and the others were another class. For the comparison, we used FCM, self-organizing map (SOM) based on artificial neural network (ANN). KEEL machine learning datset was used for simulation. We concluded that proposed method was superior than other algorithms.

근접 문맥정보와 대규모 웹 데이터를 이용한 단어 의미 중의성 해소

  • Kang, Sin-Jae;Kang, In-Su
    • Proceedings of the Korea Society for Industrial Systems Conference
    • /
    • 2009.05a
    • /
    • pp.208-211
    • /
    • 2009
  • 본 논문은 구글(Google), 워드넷(WordNet)과 같이 공개된 웹 자원과 리소스를 이용한 비교사학습(Unsupervised learning) 방법을 제안하여 단어 의미의 중의성 문제를 해결하고자 한다. 구글 검색 API를 이용하여 단어의 확장된 근접 문맥정보를 추출하고, 워드넷의 계층체계와 synset을 이용하여 단어 의미 구분정보를 자동 추출한 후, 추출된 정보 간 유사도 계산을 통해 중의성을 갖는 단어의 의미를 결정한다.

  • PDF

Development of Brain-Style Intelligent Information Processing Algorithm Through the Merge of Supervised and Unsupervised Learning I: Generation of Exemplar Patterns for Training (교사학습과 비교사 학습의 접목에 의한 두뇌방식의 지능 정보 처리 알고리즘I: 학습패턴의 생성)

  • 오상훈
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2004.05a
    • /
    • pp.56-62
    • /
    • 2004
  • In the case that we do not have enough number of training patterns because of limitation such as time consuming, economic problem, and so on, we geneterate a new patterns using the brain-style Information processing algorithm, that is, supervised and unsupervised learning methods.

  • PDF

An unsupervised learning of dependency grammar Using inside-outside probability (내부 및 외부 확률을 이용한 의존문법의 비통제 학습)

  • 장두성;최기선
    • Proceedings of the Korean Society for Cognitive Science Conference
    • /
    • 2000.06a
    • /
    • pp.133-137
    • /
    • 2000
  • 구문태그가 부착되지 않은 코퍼스를 사용하여 문법규칙의 확률을 훈련하는 비통제 학습(unsupervised learning) 방법의 대표적인 것이 CNF(Chomsky Normal Form)의 CFG(Context Free Grammar)를 입력으로 하는 inside-outside 알고리즘이다. 본 연구에서는 의존문법을 CNF로 변환하는 기법에 대해 논하고 의존문법을 위해 변형된 inside-outside 알고리즘을 논한다. 또한 이 알고리즘을 사용하여 실제 훈련한 결과를 보이고, 의존규칙과 구문구조 확률을 같이 사용하는 hybrid방식 구문분석기에 적용한 결과를 보인다.

  • PDF

Unsupervised segmentation of Multi -Source Remotely Sensed images using Binary Decision Trees and Canonical Transform

  • Mohammad, Rahmati;Kim, Jung-Ha
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2001.10a
    • /
    • pp.23.4-23
    • /
    • 2001
  • This paper proposes a new approach to unsupervised classification of remotely sensed images. Fusion of optic images (Landsat TM) and radar data (SAR) has beer used to increase the accuracy of classification. Number of clusters is estimated using generalized Dunns measure. Performance of the proposed method is best observed comparing the classified images with classified aerial images.

  • PDF

Unsupervised Change Detection of Hyperspectral images Using Range Average and Maximum Distance Methods (구간평균 기법과 직선으로부터의 최대거리를 이용한 초분광영상의 무감독변화탐지)

  • Kim, Dae-Sung;Kim, Yong-Il;Pyeon, Mu-Wook
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.29 no.1
    • /
    • pp.71-80
    • /
    • 2011
  • Thresholding is important step for detecting binary change/non-change information in the unsupervised change detection. This study proposes new unsupervised change detection method using Hyperion hyperspectral images, which are expected with data increased demand. A graph is drawn with applying the range average method for the result value through pixel-based similarity measurement, and thresholding value is decided at the maximum distance point from a straight line. The proposed method is assessed in comparison with expectation-maximization algorithm, coner method, Otsu's method using synthetic images and Hyperion hyperspectral images. Throughout the results, we validated that the proposed method can be applied simply and had similar or better performance than the other methods.