• Title/Summary/Keyword: unsupervised model

Search Result 239, Processing Time 0.022 seconds

Outlier detection of main engine data of a ship using ensemble method (앙상블 기법을 이용한 선박 메인엔진 빅데이터의 이상치 탐지)

  • KIM, Dong-Hyun;LEE, Ji-Hwan;LEE, Sang-Bong;JUNG, Bong-Kyu
    • Journal of the Korean Society of Fisheries and Ocean Technology
    • /
    • v.56 no.4
    • /
    • pp.384-394
    • /
    • 2020
  • This paper proposes an outlier detection model based on machine learning that can diagnose the presence or absence of major engine parts through unsupervised learning analysis of main engine big data of a ship. Engine big data of the ship was collected for more than seven months, and expert knowledge and correlation analysis were performed to select features that are closely related to the operation of the main engine. For unsupervised learning analysis, ensemble model wherein many predictive models are strategically combined to increase the model performance, is used for anomaly detection. As a result, the proposed model successfully detected the anomalous engine status from the normal status. To validate our approach, clustering analysis was conducted to find out the different patterns of anomalies the anomalous point. By examining distribution of each cluster, we could successfully find the patterns of anomalies.

Blind Drift Calibration using Deep Learning Approach to Conventional Sensors on Structural Model

  • Kutchi, Jacob;Robbins, Kendall;De Leon, David;Seek, Michael;Jung, Younghan;Qian, Lei;Mu, Richard;Hong, Liang;Li, Yaohang
    • International conference on construction engineering and project management
    • /
    • 2022.06a
    • /
    • pp.814-822
    • /
    • 2022
  • The deployment of sensors for Structural Health Monitoring requires a complicated network arrangement, ground truthing, and calibration for validating sensor performance periodically. Any conventional sensor on a structural element is also subjected to static and dynamic vertical loadings in conjunction with other environmental factors, such as brightness, noise, temperature, and humidity. A structural model with strain gauges was built and tested to get realistic sensory information. This paper investigates different deep learning architectures and algorithms, including unsupervised, autoencoder, and supervised methods, to benchmark blind drift calibration methods using deep learning. It involves a fully connected neural network (FCNN), a long short-term memory (LSTM), and a gated recurrent unit (GRU) to address the blind drift calibration problem (i.e., performing calibrations of installed sensors when ground truth is not available). The results show that the supervised methods perform much better than unsupervised methods, such as an autoencoder, when ground truths are available. Furthermore, taking advantage of time-series information, the GRU model generates the most precise predictions to remove the drift overall.

  • PDF

A Study on the Design of Supervised and Unsupervised Learning Models for Fault and Anomaly Detection in Manufacturing Facilities (제조 설비 이상탐지를 위한 지도학습 및 비지도학습 모델 설계에 관한 연구)

  • Oh, Min-Ji;Choi, Eun-Seon;Roh, Kyung-Woo;Kim, Jae-Sung;Cho, Wan-Sup
    • The Journal of Bigdata
    • /
    • v.6 no.1
    • /
    • pp.23-35
    • /
    • 2021
  • In the era of the 4th industrial revolution, smart factories have received great attention, where production and manufacturing technology and ICT converge. With the development of IoT technology and big data, automation of production systems has become possible. In the advanced manufacturing industry, production systems are subject to unscheduled performance degradation and downtime, and there is a demand to reduce safety risks by detecting and reparing potential errors as soon as possible. This study designs a model based on supervised and unsupervised learning for detecting anomalies. The accuracy of XGBoost, LightGBM, and CNN models was compared as a supervised learning analysis method. Through the evaluation index based on the confusion matrix, it was confirmed that LightGBM is most predictive (97%). In addition, as an unsupervised learning analysis method, MD, AE, and LSTM-AE models were constructed. Comparing three unsupervised learning analysis methods, the LSTM-AE model detected 75% of anomalies and showed the best performance. This study aims to contribute to the advancement of the smart factory by combining supervised and unsupervised learning techniques to accurately diagnose equipment failures and predict when abnormal situations occur, thereby laying the foundation for preemptive responses to abnormal situations. do.

Bagging deep convolutional autoencoders trained with a mixture of real data and GAN-generated data

  • Hu, Cong;Wu, Xiao-Jun;Shu, Zhen-Qiu
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.11
    • /
    • pp.5427-5445
    • /
    • 2019
  • While deep neural networks have achieved remarkable performance in representation learning, a huge amount of labeled training data are usually required by supervised deep models such as convolutional neural networks. In this paper, we propose a new representation learning method, namely generative adversarial networks (GAN) based bagging deep convolutional autoencoders (GAN-BDCAE), which can map data to diverse hierarchical representations in an unsupervised fashion. To boost the size of training data, to train deep model and to aggregate diverse learning machines are the three principal avenues towards increasing the capabilities of representation learning of neural networks. We focus on combining those three techniques. To this aim, we adopt GAN for realistic unlabeled sample generation and bagging deep convolutional autoencoders (BDCAE) for robust feature learning. The proposed method improves the discriminative ability of learned feature embedding for solving subsequent pattern recognition problems. We evaluate our approach on three standard benchmarks and demonstrate the superiority of the proposed method compared to traditional unsupervised learning methods.

Bathymetric mapping in Dong-Sha Atoll using SPOT data

  • Huang, Shih-Jen;Wen, Yao-Chung
    • Proceedings of the KSRS Conference
    • /
    • v.2
    • /
    • pp.525-528
    • /
    • 2006
  • The remote sensing data can be used to calculate the water depth especially in the clear and shallow water area. In this study, the SPOT data was used for bathymetric mapping in Dong-Sha atoll, located in northern South China Sea. The in situ sea depth was collected by echo sounder as well. A global positioning system was employed to locate the accurate sampling points for sea depth. An empirical model between measurement sea depth and band digital count was determined and based on least squares regression analysis. Both non-classification and unsupervised classification were used in this study. The results show that the standard error is less than 0.9m for non-classification. Besides, the 10% error related to the measurement water depth can be satisfied for more than 85% in situ data points. Otherwise, the 10% relative error can reach more than 97%, 69%, and 51% data points at class 4, 5, and 6 respectively if supervised classification is applied. Meanwhile, we also find that the unsupervised classification can get more accuracy to estimate water depth with standard error less than 0.63, 0.93, and 0.68m at class 4, 5, and 6 respectively.

  • PDF

Development of Brain-Style Intelligent Information Processing Algorithm Through the Merge of Supervised and Unsupervised Learning: Generation of Exemplar Patterns for Training (교사학습과 비교사학습의 접목에 의한 두뇌방식의 지능 정보 처리 알고리즘 개발: 학습패턴의 생성)

  • 오상훈
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.41 no.6
    • /
    • pp.61-67
    • /
    • 2004
  • We propose a new algorithm to generate additional training patterns using the brain-style information processing algorithm, that is, supervised and unsupervised learning models. This will be useful in the case that we do not have enough number of training patterns because of limitation such as time consuming, economic problem, and so on. We adopt the independent component analysis as an unsupervised model for generating exempalr patterns and multilayer perceptions as supervised models for verifying usefulness of the generated patterns. After statistical analysis of the proposed pattern generation algorithm, we verify successful operations of our algorithm through simulation of handwritten digit recognition with various numbers of training patterns.

Adolescent Drinking Behaviors in Pusan City : An Analysis on the Sociopsychological Model (부산시(釜山市) 청소년(責少年)의 반주행위(飯酒行爲)에 관한 연구(硏究) - 사회심리적(社會心理的) Model에 의한 분석(分析) -)

  • Ko, Jung Ja
    • Korean Journal of Child Studies
    • /
    • v.7 no.2
    • /
    • pp.55-73
    • /
    • 1986
  • This study analyzed the socio-psychological process of adolescent drinking behaviors. A total 1,732 high school students in Pusan city were studied by the questionnaire from May to July, 1985. A structural model based on review of the literature was examined in order to test the following three hypotheses: (1) sociocultural and environmental impact on the adolescent belief system for drinking, on drinking situations, and on experiences of deviation, (2) relationships among adolescent belief system, drinking situations, and experiences of deviation, and (3) impact of antecedent variables on adolescent drinking levels. All hypotheses were supported by the data. The important outcomes were discussed as follows: 1. Because interpersonal factors were influential for the adolescent belief system concerning drinking, public drinking education through mass communication or drinking education in the curriculum were recommended. In addition to sex variables, friends' drinking and sibling's drinking were shown to have a positive impacts on drinking situations. Also, adolescent self-reported parents' views on drinking had significant effects. Because adolescent deviant experiences were generally affected by environmental factors, it is recommended that positive extra-curricular activities at both home and school should be investigated. 2. There were significant relationships among adolescent belief systems, drinking situations, and deviant experiences. However, adolescent drinking behaviors in supervised situations had weak correlations with their belif systems and deviant behaviors. 3. Adolescent drinking levels were remarkably influenced by drinking behaviors in unsupervised situations. Because it is difficult to control actual adolescent drinking behaviors in unsupervised situations, it is important to fortify their belief system with continuous education programs.

  • PDF

A Study on Classification Evaluation Prediction Model by Cluster for Accuracy Measurement of Unsupervised Learning Data (비지도학습 데이터의 정확성 측정을 위한 클러스터별 분류 평가 예측 모델에 대한 연구)

  • Jung, Se Hoon;Kim, Jong Chan;Kim, Cheeyong;You, Kang Soo;Sim, Chun Bo
    • Journal of Korea Multimedia Society
    • /
    • v.21 no.7
    • /
    • pp.779-786
    • /
    • 2018
  • In this paper, we are applied a nerve network to allow for the reflection of data learning methods in their overall forms by using cluster data rather than data learning by the stages and then selected a nerve network model and analyzed its variables through learning by the cluster. The CkLR algorithm was proposed to analyze the reaction variables of clustering outcomes through an approach to the initialization of K-means clustering and build a model to assess the prediction rate of clustering and the accuracy rate of prediction in case of new data inputs. The performance evaluation results show that the accuracy rate of test data by the class was over 92%, which was the mean accuracy rate of the entire test data, thus confirming the advantages of a specialized structure found in the proposed learning nerve network by the class.

Topic Masks for Image Segmentation

  • Jeong, Young-Seob;Lim, Chae-Gyun;Jeong, Byeong-Soo;Choi, Ho-Jin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.7 no.12
    • /
    • pp.3274-3292
    • /
    • 2013
  • Unsupervised methods for image segmentation are recently drawing attention because most images do not have labels or tags. A topic model is such an unsupervised probabilistic method that captures latent aspects of data, where each latent aspect, or a topic, is associated with one homogeneous region. The results of topic models, however, usually have noises, which decreases the overall segmentation performance. In this paper, to improve the performance of image segmentation using topic models, we propose two topic masks applicable to topic assignments of homogeneous regions obtained from topic models. The topic masks capture the noises among the assigned topic assignments or topic labels, and remove the noises by replacements, just like image masks for pixels. However, as the nature of topic assignments is different from image pixels, the topic masks have properties that are different from the existing image masks for pixels. There are two contributions of this paper. First, the topic masks can be used to reduce the noises of topic assignments obtained from topic models for image segmentation tasks. Second, we test the effectiveness of the topic masks by applying them to segmented images obtained from the Latent Dirichlet Allocation model and the Spatial Latent Dirichlet Allocation model upon the MSRC image dataset. The empirical results show that one of the masks successfully reduces the topic noises.

PERFORMANCE EVALUATION OF INFORMATION CRITERIA FOR THE NAIVE-BAYES MODEL IN THE CASE OF LATENT CLASS ANALYSIS: A MONTE CARLO STUDY

  • Dias, Jose G.
    • Journal of the Korean Statistical Society
    • /
    • v.36 no.3
    • /
    • pp.435-445
    • /
    • 2007
  • This paper addresses for the first time the use of complete data information criteria in unsupervised learning of the Naive-Bayes model. A Monte Carlo study sets a large experimental design to assess these criteria, unusual in the Bayesian network literature. The simulation results show that complete data information criteria underperforms the Bayesian information criterion (BIC) for these Bayesian networks.