• International Journal of Internet, Broadcasting and Communication
• /
• v.14 no.2
• /
• pp.192-198
• /
• 2022

Companies are building and utilizing their own data analysis systems according to business characteristics in the distributed cloud. However, as businesses and data types become more complex and diverse, the demand for more efficient analytics has increased. In response to these demands, in this paper, we propose an unsupervised learning-based data analysis agent to which reinforcement learning is applied for effective data analysis. The proposal agent consists of reinforcement learning processing manager and unsupervised learning manager modules. These two modules configure an agent with k-means clustering on multiple nodes and then perform distributed training on multiple data sets. This enables data analysis in a relatively short time compared to conventional systems that perform analysis of large-scale data in one batch.

• Proceedings of the Korean Environmental Sciences Society Conference
• /
• 2003.11a
• /
• pp.33-36
• /
• 2003

Principal component analysis (PCA) was used to improve image classification by the unsupervised classification techniques, the K-means. To do this, I selected a Landsat TM scene of Jeju Island, Korea and proposed two methods for PCA： unstandardized PCA (UPCA) and standardized PCA (SPCA). The estimated accuracy of the image classification of Jeju area was computed by error matrix. The error matrix was derived from three unsupervised classification methods. Error matrices indicated that classifications done on the first three principal components for UPCA and SPCA of the scene were more accurate than those done on the seven bands of TM data and that also the results of UPCA and SPCA were better than those of the raw Landsat TM data. The classification of TM data by the K-means algorithm was particularly poor at distinguishing different land covers on the island. From the classification results, we also found that the principal component based classifications had characteristics independent of the unsupervised techniques (numerical algorithms) while the TM data based classifications were very dependent upon the techniques. This means that PCA data has uniform characteristics for image classification that are less affected by choice of classification scheme. In the results, we also found that UPCA results are better than SPCA since UPCA has wider range of digital number of an image.

• International Journal of Control, Automation, and Systems
• /
• v.4 no.6
• /
• pp.705-713
• /
• 2006

For the analysis of multiunit extracellular neural signals as multiple spike trains, neural spike sorting is essential. Existing algorithms for the spike sorting have been unsatisfactory when the signal-to-noise ratio(SNR) is low, especially for implementation of fully-automated systems. We present a novel method that shows satisfactory performance even under low SNR, and compare its performance with a recent method based on principal component analysis(PCA) and fuzzy c-means(FCM) clustering algorithm. Our system consists of a spike detector that shows high performance under low SNR, a feature extractor that utilizes projection pursuit based on negentropy maximization, and an unsupervised classifier based on Gaussian mixture model. It is shown that the proposed feature extractor gives better performance compared to the PCA, and the proposed combination of spike detector, feature extraction, and unsupervised classification yields much better performance than the PCA-FCM, in that the realization of fully-automated unsupervised spike sorting becomes more feasible.

• Asia-Pacific Journal of Atmospheric Sciences
• /
• v.54 no.4
• /
• pp.527-544
• /
• 2018

This paper presents a nighttime sea fog detection algorithm incorporating unsupervised learning technique. The algorithm is based on data sets that combine brightness temperatures from the $3.7{\mu}m$ and $10.8{\mu}m$ channels of the meteorological imager (MI) onboard the Communication, Ocean and Meteorological Satellite (COMS), with sea surface temperature from the Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA). Previous algorithms generally employed threshold values including the brightness temperature difference between the near infrared and infrared. The threshold values were previously determined from climatological analysis or model simulation. Although this method using predetermined thresholds is very simple and effective in detecting low cloud, it has difficulty in distinguishing fog from stratus because they share similar characteristics of particle size and altitude. In order to improve this, the unsupervised learning approach, which allows a more effective interpretation from the insufficient information, has been utilized. The unsupervised learning method employed in this paper is the expectation-maximization (EM) algorithm that is widely used in incomplete data problems. It identifies distinguishing features of the data by organizing and optimizing the data. This allows for the application of optimal threshold values for fog detection by considering the characteristics of a specific domain. The algorithm has been evaluated using the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) vertical profile products, which showed promising results within a local domain with probability of detection (POD) of 0.753 and critical success index (CSI) of 0.477, respectively.

• Journal of IKEEE
• /
• v.24 no.4
• /
• pp.1011-1016
• /
• 2020

We propose a framework called Stacked Gated Recurrent Unit - Infrequent Residual Analysis (SG-IRA) that detects anomalies in time-series data that can be trained on streams of raw sensor data without any pre-labeled dataset. To enable such unsupervised learning, SG-IRA includes an estimation model that uses a stacked Gated Recurrent Unit (GRU) structure and an analysis method that detects anomalies based on the difference between the estimated value and the actual measurement (residual). SG-IRA's residual analysis method dynamically adapts the detection threshold from the population using frequency analysis, unlike the baseline model that relies on a constant threshold. In this paper, SG-IRA is evaluated using the industrial control systems (ICS) datasets. SG-IRA improves the detection performance (F1 score) by 5.9% compared to the baseline model.

• Journal of Information Processing Systems
• /
• v.13 no.2
• /
• pp.370-384
• /
• 2017

The image segmentation is the most important operation in an image processing system. It is located at the joint between the processing and analysis of the images. Unsupervised segmentation aims to automatically separate the image into natural clusters. However, because of its complexity several methods have been proposed, specifically methods of optimization. In our work we are interested to the technique SFLA (Shuffled Frog-Leaping Algorithm). It's a memetic meta-heuristic algorithm that is based on frog populations in nature searching for food. This paper proposes a new approach of unsupervised image segmentation based on SFLA method. It is implemented and applied to different types of images. To validate the performances of our approach, we performed experiments which were compared to the method of K-means.

• ETRI Journal
• /
• v.44 no.2
• /
• pp.232-243
• /
• 2022

Changes in household composition and the residential environment have had a considerable impact on the features of postal delivery regions in recent years, resulting in a large increase in the overall workload of domestic postal delivery services. In this paper, we provide complex analysis results for postal delivery areas using various unsupervised learning approaches. First, we extract highly influential features using several feature-engineering methods. Then, using quantitative and qualitative cluster analyses, we find the distinctive traits and semantics of postal delivery zones. Unsupervised learning approaches are useful for successfully grouping postal service zones, according to our findings. Furthermore, by comparing a postal delivery region to other areas in the same group, workload balancing was achieved.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.47 no.1
• /
• pp.9-19
• /
• 2024

Deep learning-based computer vision anomaly detection algorithms are widely utilized in various fields. Especially in the manufacturing industry, the difficulty in collecting abnormal data compared to normal data, and the challenge of defining all potential abnormalities in advance, have led to an increasing demand for unsupervised learning methods that rely on normal data. In this study, we conducted a comparative analysis of deep learning-based unsupervised learning algorithms that define and detect abnormalities that can occur when transparent contact lenses are immersed in liquid solution. We validated and applied the unsupervised learning algorithms used in this study to the existing anomaly detection benchmark dataset, MvTecAD. The existing anomaly detection benchmark dataset primarily consists of solid objects, whereas in our study, we compared unsupervised learning-based algorithms in experiments judging the shape and presence of lenses submerged in liquid. Among the algorithms analyzed, EfficientAD showed an AUROC and F1-score of 0.97 in image-level tests. However, the F1-score decreased to 0.18 in pixel-level tests, making it challenging to determine the locations where abnormalities occurred. Despite this, EfficientAD demonstrated excellent performance in image-level tests classifying normal and abnormal instances, suggesting that with the collection and training of large-scale data in real industrial settings, it is expected to exhibit even better performance.

• Korean Journal of Remote Sensing
• /
• v.22 no.2
• /
• pp.111-121
• /
• 2006

The classification using spectral angle is a new approach based on the fact that the spectra of the same type of surface objects in RS data are approximately linearly scaled variations of one another due to atmospheric and topographic effects. There are many researches on the unsupervised classification using spectral angle recently. Nevertheless, there are only a few which consider the characteristics of Hyperspectral data. On this study, we propose the ISOMUSAC(Iterative Self-Organizing Modified Unsupervised Spectral Angle Classification) which can supplement the defects of previous unsupervised spectral angle classification. ISOMUSAC uses the Angle Division for the selection of seed points and calculates the center of clusters using spectral angle. In addition, ISOMUSAC perform the iterative merging and splitting clusters. As a result, the proposed algorithm can reduce the time of processing and generate better classification result than previous unsupervised classification algorithms by visual and quantitative analysis. For the comparison with previous unsupervised spectral angle classification by quantitative analysis, we propose Validity Index using spectral angle.

• Journal of Korean Institute of Industrial Engineers
• /
• v.40 no.3
• /
• pp.275-282
• /
• 2014

One of the most widely used methods for dimensionality reduction is principal component analysis (PCA). However, the reduced dimensions from PCA do not provide a clear interpretation with respect to the original features because they are linear combinations of a large number of original features. This interpretation problem can be overcome by feature selection approaches that identifying the best subset of given features. In this study, we propose an unsupervised feature selection method based on the geometrical information of PCA loading vectors. Experimental results from a simulation study demonstrated the efficiency and usefulness of the proposed method.