• Title/Summary/Keyword: outlier detecting

Search Result 48, Processing Time 0.018 seconds

Regression diagnostics for response transformations in a partial linear model (부분선형모형에서 반응변수변환을 위한 회귀진단)

  • Seo, Han Son;Yoon, Min
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.1
    • /
    • pp.33-39
    • /
    • 2013
  • In the transformation of response variable in partial linear models outliers can cause a bad effect on estimating the transformation parameter, just as in the linear models. To solve this problem the processes of estimating transformation parameter and detecting outliers are needed, but have difficulties to be performed due to the arbitrariness of the nonparametric function included in the partial linear model. In this study, through the estimation of nonparametric function and outlier detection methods such as a sequential test and a maximum trimmed likelihood estimation, processes for transforming response variable robust to outliers in partial linear models are suggested. The proposed methods are verified and compared their effectiveness by simulation study and examples.

Data anomaly detection for structural health monitoring using a combination network of GANomaly and CNN

  • Liu, Gaoyang;Niu, Yanbo;Zhao, Weijian;Duan, Yuanfeng;Shu, Jiangpeng
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.53-62
    • /
    • 2022
  • The deployment of advanced structural health monitoring (SHM) systems in large-scale civil structures collects large amounts of data. Note that these data may contain multiple types of anomalies (e.g., missing, minor, outlier, etc.) caused by harsh environment, sensor faults, transfer omission and other factors. These anomalies seriously affect the evaluation of structural performance. Therefore, the effective analysis and mining of SHM data is an extremely important task. Inspired by the deep learning paradigm, this study develops a novel generative adversarial network (GAN) and convolutional neural network (CNN)-based data anomaly detection approach for SHM. The framework of the proposed approach includes three modules : (a) A three-channel input is established based on fast Fourier transform (FFT) and Gramian angular field (GAF) method; (b) A GANomaly is introduced and trained to extract features from normal samples alone for class-imbalanced problems; (c) Based on the output of GANomaly, a CNN is employed to distinguish the types of anomalies. In addition, a dataset-oriented method (i.e., multistage sampling) is adopted to obtain the optimal sampling ratios between all different samples. The proposed approach is tested with acceleration data from an SHM system of a long-span bridge. The results show that the proposed approach has a higher accuracy in detecting the multi-pattern anomalies of SHM data.

Marker Detection by Using Affine-SIFT Matching Points for Marker Occlusion of Augmented Reality (증강현실에서 가려진 마커를 위한 Affine-SIFT 정합 점들을 이용한 마커 검출 기법)

  • Kim, Yong-Min;Park, Chan-Woo;Park, Ki-Tae;Moon, Young-Shik
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.48 no.2
    • /
    • pp.55-65
    • /
    • 2011
  • In this paper, a novel method of marker detection robust against marker occlusion in augmented reality is proposed. the proposed method consists of four steps. In the first step, in order to effectively detect an occluded marker, we first utilize the Affine-SIFT (ASIFT, Affine-Scale Invariant Features Transform) for detecting matching points between an enrolled marker and an input images with an occluded marker. In the second step, we apply the Principal Component Analysis (PCA) for eliminating outlier of the matching points in the enrolled marker. And then matching points are projected to the first and second axis for longest value and the shortest value of an ellipse are determined by average distance between the projected points and a center of the points. In the third step, Convex-hull vertices including matching points are considered as polygon vertices for estimating a geometric affine transformation. In the final step, by estimating the geometric affine transformation of the points, a marker robust against a marker occlusion is detected. Experimental results have shown that the proposed method effectively detects occlude markers.

Optimal National Coordinate System Transform Model using National Control Point Network Adjustment Results (국가지준점 망조정 성과를 활용한 최적 국가 좌표계 변환 모델 결정)

  • Song, Dong-Seob;Jang, Eun-Seok;Kim, Tae-Woo;Yun, Hong-Sic
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.25 no.6_2
    • /
    • pp.613-623
    • /
    • 2007
  • The main purpose of this study is to investigate the coordinate transformation based on two different systems between local geodetic datum(tokyo datum) and international geocentric datum(new Korea geodetic datum). For this purpose, three methods were used to determine seven parameters as follows: Bursa-Wolf model, Molodensky-Badekas model, and Veis model. Also, we adopted multiple regression equation method to convert from Tokyo datum to KTRF. We used 935 control points as a common points and applied gross error analysis for detecting the outlier among those control points. The coordinate transformation was carried out using similarity transformation applied the obtained seven parameters and the precision of transformed coordinate was evaluated about 9,917 third or forth order control points. From these results, it was found that Bursa-Wolf model and Molodensky-Badekas model are more suitable than other for the determination of transformation parameters in Korea. And, transforming accuracy using MRE is lower than other similarity transformation model.

Expression Analysis System of Game Player based on Multi-modal Interface (멀티 모달 인터페이스 기반 플레이어 얼굴 표정 분석 시스템 개발)

  • Jung, Jang-Young;Kim, Young-Bin;Lee, Sang-Hyeok;Kang, Shin-Jin
    • Journal of Korea Game Society
    • /
    • v.16 no.2
    • /
    • pp.7-16
    • /
    • 2016
  • In this paper, we propose a method for effectively detecting specific behavior. The proposed method detects outlying behavior based on the game players' characteristics. These characteristics are captured non-invasively in a general game environment and add keystroke based on repeated pattern. In this paper, cameras were used to analyze observed data such as facial expressions and player movements. Moreover, multimodal data from the game players was used to analyze high-dimensional game-player data for a detection effect of repeated behaviour pattern. A support vector machine was used to efficiently detect outlying behaviors. We verified the effectiveness of the proposed method using games from several genres. The recall rate of the outlying behavior pre-identified by industry experts was approximately 70%. In addition, Repeated behaviour pattern can be analysed possible. The proposed method can also be used for feedback and quantification about analysis of various interactive content provided in PC environments.

Time Series Modeling Pipeline for Urban Behavioral Demand Prediction under Uncertainty (COVID-19 사례를 통한 도시 내 비정상적 수요 예측을 위한 시계열 모형 파이프라인 개발 연구)

  • Minsoo Jin;Dongwoo Lee;Youngrok Kim;Hyunsoo Lee
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.22 no.2
    • /
    • pp.80-92
    • /
    • 2023
  • As cities are becoming densely populated, previously unexpected events such as crimes, accidents, and infectious diseases are bound to affect user demands. With a time-series prediction of demand using information with uncertainty, it is impossible to derive reliable results. In particular, the COVID-19 outbreak in early 2020 caused changes in abnormal travel patterns and made it difficult to predict demand for time series. A methodology that accurately predicts demand by detecting and reflecting these changes is, therefore, required. The current study suggests a time series modeling pipeline that automatically detects and predicts abnormal events caused by COVID-19. We expect its wide application in various situations where there is a change in demand due to irregular and abnormal events.

Performance Comparison of Machine Learning Algorithms for Network Traffic Security in Medical Equipment (의료기기 네트워크 트래픽 보안 관련 머신러닝 알고리즘 성능 비교)

  • Seung Hyoung Ko;Joon Ho Park;Da Woon Wang;Eun Seok Kang;Hyun Wook Han
    • Journal of Information Technology Services
    • /
    • v.22 no.5
    • /
    • pp.99-108
    • /
    • 2023
  • As the computerization of hospitals becomes more advanced, security issues regarding data generated from various medical devices within hospitals are gradually increasing. For example, because hospital data contains a variety of personal information, attempts to attack it have been continuously made. In order to safely protect data from external attacks, each hospital has formed an internal team to continuously monitor whether the computer network is safely protected. However, there are limits to how humans can monitor attacks that occur on networks within hospitals in real time. Recently, artificial intelligence models have shown excellent performance in detecting outliers. In this paper, an experiment was conducted to verify how well an artificial intelligence model classifies normal and abnormal data in network traffic data generated from medical devices. There are several models used for outlier detection, but among them, Random Forest and Tabnet were used. Tabnet is a deep learning algorithm related to receive and classify structured data. Two algorithms were trained using open traffic network data, and the classification accuracy of the model was measured using test data. As a result, the random forest algorithm showed a classification accuracy of 93%, and Tapnet showed a classification accuracy of 99%. Therefore, it is expected that most outliers that may occur in a hospital network can be detected using an excellent algorithm such as Tabnet.

Reduced Order Modeling of Marine Engine Status by Principal Component Analysis (주성분 분석을 통한 선박 기관 상태의 차수 축소 모델링)

  • Seungbeom Lee;Jeonghwa Seo;Dong-Hwan Kim;Sangmin Han;Kwanwoo Kim;Sungwook Chung;Byeongwoo Yoo
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.61 no.1
    • /
    • pp.8-18
    • /
    • 2024
  • The present study concerns reduced order modeling of a marine diesel engine, which can be used for outlier detection in status monitoring and carbon intensity index calculation. Principal Component Analysis (PCA) is introduced for the reduced order modeling, focusing on the feasibility of detecting and treating nonlinear variables. By cross-correlation, it is found that there are seven non-linear data channels among 23 data channels, i.e., fuel mode, exhaust gas temperature after the turbocharger, and cylinder coolant temperatures. The dataset is handled so that the mean is located at the nominal continuous rating. Polynomial presentation of the dataset is also applied to reflect the linearity between the engine speed and other channels. The first principal mode shows strong effects of linearity of the most data channels to show the linearity of the system. The non-linear variables are effectively explained by other modes. second mode concerns the temperature of the cylinder cooling water, which shows small correlation with other variables. The third and fourth modes correlates the fuel mode and turbocharger exhaust gas temperature, which have inferior linearity to other channels. PCA is proven to be applicable to data given in binary type of fuel mode selection, as well as numerical type data.