• Title/Summary/Keyword: unsupervised model

Search Result 240, Processing Time 0.022 seconds

A Case Study: Unsupervised Approach for Tourist Profile Analysis by K-means Clustering in Turkey

  • Yildirim, Mustafa Eren;Kaya, Murat;FurkanInce, Ibrahim
    • Journal of Internet Computing and Services
    • /
    • v.23 no.1
    • /
    • pp.11-17
    • /
    • 2022
  • Data mining is the task of accessing useful information from a large capacity of data. It can also be referred to as searching for correlations that can provide clues about the future in large data warehouses by using computer algorithms. It has been used in the tourism field for marketing, analysis, and business improvement purposes. This study aims to analyze the tourist profile in Turkey through data mining methods. The reason relies behind the selection of Turkey is the fact that Turkey welcomes millions of tourist every year which can be a role model for other touristic countries. In this study, an anonymous and large-scale data set was used under the law on the protection of personal data. The dataset was taken from a leading tourism company that is still active in Turkey. By using the k-means clustering algorithm on this data, key parameters of profiles were obtained and people were clustered into groups according to their characteristics. According to the outcomes, distinguishing characteristics are gathered under three main titles. These are the age of the tourists, the frequency of their vacations and the period between the reservation and the vacation itself. The results obtained show that the frequency of tourist vacations, the time between bookings and vacations, and age are the most important and characteristic parameters for a tourist's profile. Finally, planning future investments, events and campaign packages can make tourism companies more competitive and improve quality of service. For both businesses and tourists, it is advantageous to prepare individual events and offers for the three major groups of tourists.

Feature selection for text data via topic modeling (토픽 모형을 이용한 텍스트 데이터의 단어 선택)

  • Woosol, Jang;Ye Eun, Kim;Won, Son
    • The Korean Journal of Applied Statistics
    • /
    • v.35 no.6
    • /
    • pp.739-754
    • /
    • 2022
  • Usually, text data consists of many variables, and some of them are closely correlated. Such multi-collinearity often results in inefficient or inaccurate statistical analysis. For supervised learning, one can select features by examining the relationship between target variables and explanatory variables. On the other hand, for unsupervised learning, since target variables are absent, one cannot use such a feature selection procedure as in supervised learning. In this study, we propose a word selection procedure that employs topic models to find latent topics. We substitute topics for the target variables and select terms which show high relevance for each topic. Applying the procedure to real data, we found that the proposed word selection procedure can give clear topic interpretation by removing high-frequency words prevalent in various topics. In addition, we observed that, by applying the selected variables to the classifiers such as naïve Bayes classifiers and support vector machines, the proposed feature selection procedure gives results comparable to those obtained by using class label information.

Procedure for monitoring autocorrelated processes using LSTM Autoencoder (LSTM Autoencoder를 이용한 자기상관 공정의 모니터링 절차)

  • Pyoungjin Ji;Jaeheon Lee
    • The Korean Journal of Applied Statistics
    • /
    • v.37 no.2
    • /
    • pp.191-207
    • /
    • 2024
  • Many studies have been conducted to quickly detect out-of-control situations in autocorrelated processes. The most traditionally used method is a residual control chart, which uses residuals calculated from a fitted time series model. However, many procedures for monitoring autocorrelated processes using statistical learning methods have recently been proposed. In this paper, we propose a monitoring procedure using the latent vector of LSTM Autoencoder, a deep learning-based unsupervised learning method. We compare the performance of this procedure with the LSTM Autoencoder procedure based on the reconstruction error, the RNN classification procedure, and the residual charting procedure through simulation studies. Simulation results show that the performance of the proposed procedure and the RNN classification procedure are similar, but the proposed procedure has the advantage of being useful in processes where sufficient out-of-control data cannot be obtained, because it does not require out-of-control data for training.

Document Image Binarization by GAN with Unpaired Data Training

  • Dang, Quang-Vinh;Lee, Guee-Sang
    • International Journal of Contents
    • /
    • v.16 no.2
    • /
    • pp.8-18
    • /
    • 2020
  • Data is critical in deep learning but the scarcity of data often occurs in research, especially in the preparation of the paired training data. In this paper, document image binarization with unpaired data is studied by introducing adversarial learning, excluding the need for supervised or labeled datasets. However, the simple extension of the previous unpaired training to binarization inevitably leads to poor performance compared to paired data training. Thus, a new deep learning approach is proposed by introducing a multi-diversity of higher quality generated images. In this paper, a two-stage model is proposed that comprises the generative adversarial network (GAN) followed by the U-net network. In the first stage, the GAN uses the unpaired image data to create paired image data. With the second stage, the generated paired image data are passed through the U-net network for binarization. Thus, the trained U-net becomes the binarization model during the testing. The proposed model has been evaluated over the publicly available DIBCO dataset and it outperforms other techniques on unpaired training data. The paper shows the potential of using unpaired data for binarization, for the first time in the literature, which can be further improved to replace paired data training for binarization in the future.

Active Contours Level Set Based Still Human Body Segmentation from Depth Images For Video-based Activity Recognition

  • Siddiqi, Muhammad Hameed;Khan, Adil Mehmood;Lee, Seok-Won
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.7 no.11
    • /
    • pp.2839-2852
    • /
    • 2013
  • Context-awareness is an essential part of ubiquitous computing, and over the past decade video based activity recognition (VAR) has emerged as an important component to identify user's context for automatic service delivery in context-aware applications. The accuracy of VAR significantly depends on the performance of the employed human body segmentation algorithm. Previous human body segmentation algorithms often engage modeling of the human body that normally requires bulky amount of training data and cannot competently handle changes over time. Recently, active contours have emerged as a successful segmentation technique in still images. In this paper, an active contour model with the integration of Chan Vese (CV) energy and Bhattacharya distance functions are adapted for automatic human body segmentation using depth cameras for VAR. The proposed technique not only outperforms existing segmentation methods in normal scenarios but it is also more robust to noise. Moreover, it is unsupervised, i.e., no prior human body model is needed. The performance of the proposed segmentation technique is compared against conventional CV Active Contour (AC) model using a depth-camera and obtained much better performance over it.

An adaptive method of multi-scale edge detection for underwater image

  • Bo, Liu
    • Ocean Systems Engineering
    • /
    • v.6 no.3
    • /
    • pp.217-231
    • /
    • 2016
  • This paper presents a new approach for underwater image analysis using the bi-dimensional empirical mode decomposition (BEMD) technique and the phase congruency information. The BEMD algorithm, fully unsupervised, it is mainly applied to texture extraction and image filtering, which are widely recognized as a difficult and challenging machine vision problem. The phase information is the very stability feature of image. Recent developments in analysis methods on the phase congruency information have received large attention by the image researchers. In this paper, the proposed method is called the EP model that inherits the advantages of the first two algorithms, so this model is suitable for processing underwater image. Moreover, the receiver operating characteristic (ROC) curve is presented in this paper to solve the problem that the threshold is greatly affected by personal experience when underwater image edge detection is performed using the EP model. The EP images are computed using combinations of the Canny detector parameters, and the binaryzation image results are generated accordingly. The ideal EP edge feature extractive maps are estimated using correspondence threshold which is optimized by ROC analysis. The experimental results show that the proposed algorithm is able to avoid the operation error caused by manual setting of the detection threshold, and to adaptively set the image feature detection threshold. The proposed method has been proved to be accuracy and effectiveness by the underwater image processing examples.

An Ensemble Model for Machine Failure Prediction (앙상블 모델 기반의 기계 고장 예측 방법)

  • Cheon, Kang Min;Yang, Jaekyung
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.43 no.1
    • /
    • pp.123-131
    • /
    • 2020
  • There have been a lot of studies in the past for the method of predicting the failure of a machine, and recently, a lot of researches and applications have been generated to diagnose the physical condition of the machine and the parts and to calculate the remaining life through various methods. Survival models are also used to predict plant failures based on past anomaly cycles. In particular, special machine that reflect the fluid flow and process characteristics of chemical plants are connected to hundreds or thousands of sensors, so there are not many factors that need to be considered, such as process and material data as well as application of derivative variables. In this paper, the data were preprocessed through time series anomaly detection based on unsupervised learning to predict the abnormalities of these special machine. Next, clustering results reflecting clustering-based data characteristics were applied to produce additional variables, and a learning data set was created based on the history of past facility abnormalities. Finally, the prediction methodology based on the supervised learning algorithm was applied, and the model update was confirmed to improve the accuracy of the prediction of facility failure. Through this, it is expected to improve the efficiency of facility operation by flexibly replacing the maintenance time and parts supply and demand by predicting abnormalities of machine and extracting key factors.

AI photo storyteller based on deep encoder-decoder architecture (딥인코더-디코더 기반의 인공지능 포토 스토리텔러)

  • Min, Kyungbok;Dang, L. Minh;Lee, Sujin;Moon, Hyeonjoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2019.10a
    • /
    • pp.931-934
    • /
    • 2019
  • Research using artificial intelligence to generate captions for an image has been studied extensively. However, these systems are unable to create creative stories that include more than one sentence based on image content. A story is a better way that humans use to foster social cooperation and develop social norms. This paper proposes a framework that can generate a relatively short story to describe based on the context of an image. The main contributions of this paper are (1) An unsupervised framework which uses recurrent neural network structure and encoder-decoder model to construct a short story for an image. (2) A huge English novel dataset, including horror and romantic themes that are manually collected and validated. By investigating the short stories, the proposed model proves that it can generate more creative contents compared to existing intelligent systems which can produce only one concise sentence. Therefore, the framework demonstrated in this work will trigger the research of a more robust AI story writer and encourages the application of the proposed model in helping story writer find a new idea.

The Clustering Application of Spectral Characteristics of Rock Samples from Ulsan (울산 지역 암석 시료의 스펙트럼 특성과 이의 Clustering 응용)

  • 박종남;김지훈
    • Korean Journal of Remote Sensing
    • /
    • v.6 no.2
    • /
    • pp.115-133
    • /
    • 1990
  • Study was made on the spectral characteristics of rock samples including bentonites collected from the northern Ulsan area. The geology of the area consists mainly of sediments of the Kyongsang Series and Bulguksa granite, the Tertiary volcanics, andesites and tuffs. Relative reflectances of meshed samples(2.5~10mm) to BaSO$_4$ are measured at 6 Landsat TM spectral windows (excluding the thermal band) with HHRR, and their reflection charactristics were analysed. In addition, three different data selection schemes including the Eulidean distance, multiple regression, and PCA weight methods were applied to the 30 TM ratio channels, derived from the above 6 bands. The selected data sets were subject to two unsupervised classification techniques(FA and ISODATA) in order to compare the effectiveness for classification of particularly bentonite from others. As a result, in ISODATA analysis the multiple regression model shows the best, followed by the Euliean distances one. The PCA weight model seems to show some confusion. In FA, though difficult for quantitative analysis, the best still seems to be the regression model. Among ratio bands, rations of band 7 or 5 against other bands represent the best contribution in classification of bentonites from others.

A Novel Text Sample Selection Model for Scene Text Detection via Bootstrap Learning

  • Kong, Jun;Sun, Jinhua;Jiang, Min;Hou, Jian
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.2
    • /
    • pp.771-789
    • /
    • 2019
  • Text detection has been a popular research topic in the field of computer vision. It is difficult for prevalent text detection algorithms to avoid the dependence on datasets. To overcome this problem, we proposed a novel unsupervised text detection algorithm inspired by bootstrap learning. Firstly, the text candidate in a novel form of superpixel is proposed to improve the text recall rate by image segmentation. Secondly, we propose a unique text sample selection model (TSSM) to extract text samples from the current image and eliminate database dependency. Specifically, to improve the precision of samples, we combine maximally stable extremal regions (MSERs) and the saliency map to generate sample reference maps with a double threshold scheme. Finally, a multiple kernel boosting method is developed to generate a strong text classifier by combining multiple single kernel SVMs based on the samples selected from TSSM. Experimental results on standard datasets demonstrate that our text detection method is robust to complex backgrounds and multilingual text and shows stable performance on different standard datasets.