• Journal of KIISE:Software and Applications
• /
• v.30 no.3_4
• /
• pp.251-258
• /
• 2003

A support vector machine (SVM) is supposed to provide a good generalization performance, but the actual performance of a actually implemented SVM is often far from the theoretically expected level. This is largely because the implementation is based on an approximated algorithm, due to the high complexity of time and space. To improve this limitation, we propose ensemble of SVMs by using Bagging (bootstrap aggregating) and Boosting. By a Bagging stage each individual SVM is trained independently using randomly chosen training samples via a bootstrap technique. By a Boosting stage an individual SVM is trained by choosing training samples according to their probability distribution. The probability distribution is updated by the error of independent classifiers, and the process is iterated. After the training stage, they are aggregated to make a collective decision in several ways, such ai majority voting, the LSE(least squares estimation) -based weighting, and double layer hierarchical combining. The simulation results for IRIS data classification, the hand-written digit recognition and Face detection show that the proposed SVM ensembles greatly outperforms a single SVM in terms of classification accuracy.

• Korean Journal of Remote Sensing
• /
• v.35 no.1
• /
• pp.93-115
• /
• 2019

The availability of high-resolution laser scanning data and advanced machine learning algorithms has enabled an accurate potential rockfall source identification. However, the presence of other mass movements, such as landslides within the same region of interest, poses additional challenges to this task. Thus, this research presents a method based on an integration of Gaussian mixture model (GMM) and ensemble artificial neural network (bagging ANN [BANN]) for automatic detection of potential rockfall sources at Kinta Valley area, Malaysia. The GMM was utilised to determine slope angle thresholds of various geomorphological units. Different algorithms(ANN, support vector machine [SVM] and k nearest neighbour [kNN]) were individually tested with various ensemble models (bagging, voting and boosting). Grid search method was adopted to optimise the hyperparameters of the investigated base models. The proposed model achieves excellent results with success and prediction accuracies at 95% and 94%, respectively. In addition, this technique has achieved excellent accuracies (ROC = 95%) over other methods used. Moreover, the proposed model has achieved the optimal prediction accuracies (92%) on the basis of testing data, thereby indicating that the model can be generalised and replicated in different regions, and the proposed method can be applied to various landslide studies.

• Journal of Information Technology Services
• /
• v.14 no.4
• /
• pp.121-135
• /
• 2015

Ensemble classification is to utilize multiple classifiers instead of using a single classifier. Recently ensemble classifiers have attracted much attention in data mining community. Ensemble learning techniques has been proved to be very useful for improving the prediction accuracy. Bagging, boosting and random subspace are the most popular ensemble methods. In random subspace, each base classifier is trained on a randomly chosen feature subspace of the original feature space. The outputs of different base classifiers are aggregated together usually by a simple majority vote. In this study, we applied the random subspace method to the bankruptcy problem. Moreover, we proposed a method for optimizing the random subspace ensemble. The genetic algorithm was used to optimize classifier subset of random subspace ensemble for bankruptcy prediction. This paper applied the proposed genetic algorithm based random subspace ensemble model to the bankruptcy prediction problem using a real data set and compared it with other models. Experimental results showed the proposed model outperformed the other models.

• The Journal of Society for e-Business Studies
• /
• v.25 no.3
• /
• pp.1-13
• /
• 2020

Machine learning refers to a model generation technique that can solve specific problems from the generalization process for given data. In order to generate a high performance model, high quality training data and learning algorithms for generalization process should be prepared. As one way of improving the performance of model to be learned, the Ensemble technique generates multiple models rather than a single model, which includes bagging, boosting, and stacking learning techniques. This paper proposes a new Ensemble technique with multiple stacking that outperforms the conventional stacking technique. The learning structure of multiple stacking ensemble technique is similar to the structure of deep learning, in which each layer is composed of a combination of stacking models, and the number of layers get increased so as to minimize the misclassification rate of each layer. Through experiments using four types of datasets, we have showed that the proposed method outperforms the exiting ones.

• The Korean Journal of Applied Statistics
• /
• v.27 no.3
• /
• pp.357-371
• /
• 2014

There are many studies related to imbalanced data in which the class distribution is highly skewed. To address the problem of imbalanced data, previous studies deal with resampling techniques which correct the skewness of the class distribution in each sampled subset by using under-sampling, over-sampling or hybrid-sampling such as SMOTE. Ensemble methods have also alleviated the problem of class imbalanced data. In this paper, we compare around a dozen algorithms that combine the ensemble methods and resampling techniques based on simulated data sets generated by the Backbone model, which can handle the imbalance rate. The results on various real imbalanced data sets are also presented to compare the effectiveness of algorithms. As a result, we highly recommend the resampling technique combining ensemble methods for imbalanced data in which the proportion of the minority class is less than 10%. We also find that each ensemble method has a well-matched sampling technique. The algorithms which combine bagging or random forest ensembles with random undersampling tend to perform well; however, the boosting ensemble appears to perform better with over-sampling. All ensemble methods combined with SMOTE outperform in most situations.

• Journal of KIISE
• /
• v.43 no.2
• /
• pp.269-274
• /
• 2016

Named entity recognition is a process which extracts named entities in sentences and determines categories of the named entities. Previous studies on named entity recognition have primarily been used for supervised learning. For supervised learning, a large training corpus manually annotated with named entity categories is needed, and it is a time-consuming and labor-intensive job to manually construct a large training corpus. We propose a semi-supervised learning method to minimize the cost needed for training corpus construction and to rapidly enhance the performance of named entity recognition. The proposed method uses distance supervision for the construction of the initial training corpus. It can then effectively remove noise sentences in the initial training corpus through the use of an active bagging method, an ensemble method of bagging and active learning. In the experiments, the proposed method improved the F1-score of named entity recognition from 67.36% to 76.42% after active bagging for 15 times.

• Journal of the Korean Society for Nondestructive Testing
• /
• v.30 no.4
• /
• pp.302-310
• /
• 2010

In this paper, we studied Bagging neural network for predicting defect size of steam generator(SG) tube in nuclear power plant. Bagging is a method for creating an ensemble of estimator based on bootstrap sampling. For predicting defect size of SG tube, we first generated eddy current testing signals for 4 defect patterns of SG tube with various widths and depths. Then, we constructed single neural network(SNN) and Bagging neural network(BNN) to estimate width and depth of each defect. The estimation performance of SNN and BNN were measured by means of peak error. According to our experiment result, average peak error of SNN and BNN for estimating defect depth were 0.117 and 0.089mm, respectively. Also, in the case of estimating defect width, average peak error of SNN and BNN were 0.494 and 0.306mm, respectively. This shows that the estimation performance of BNN is superior to that of SNN.

• Journal of the Korean Data and Information Science Society
• /
• v.21 no.5
• /
• pp.917-926
• /
• 2010

It is known that bagging and boosting techniques improve the performance in classification problem. A number of researchers have proved the high performance of bagging and boosting through experiments for categorical response but not for continuous response. We study whether bagging and boosting improve data mining methods for continuous responses such as linear regression, decision tree, neural network through bagging and boosting. The analysis of eight real data sets prove the high performance of bagging and boosting empirically.

• International Journal of Advanced Culture Technology
• /
• v.9 no.4
• /
• pp.268-273
• /
• 2021

In this paper, we propose a sentiment analysis model that improves performance on small-scale data. A sentiment analysis model for small-scale data is proposed and verified through experiments. To this end, we propose Bagging-Bi-GRU, which combines Bi-GRU, which learns GRU, which is a variant of LSTM (Long Short-Term Memory) with excellent performance on sequential data, in both directions and the bagging technique, which is one of the ensembles learning methods. In order to verify the performance of the proposed model, it is applied to small-scale data and large-scale data. And by comparing and analyzing it with the existing machine learning algorithm, Bi-GRU, it shows that the performance of the proposed model is improved not only for small data but also for large data.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2006.05a
• /
• pp.1455-1459
• /
• 2006

우리나라 수자원 관리에서 여름 유량은 이수 및 치수 측면에서 매우 중요한 역할을 한다. 이러한 점에서 여름유량의 예측 가능성을 검토하는 것은 수자원 관리에 유연성을 주는 동시에 상대적으로 위험도를 저감시킬 수 있는 역할을 할 수 있다. 따라서 본 연구의 목적은 여름 계절 유량을 대상으로 기상인자와의 상관성 분석을 통해 유량 예측을 위한 수문기상정보(hydroclimatics)를 전 지구적으로 검토하고 최종적으로 불확실성을 고려할 수 있는 Ensemble예측을 실시하고자 한다. Ensemble예측은 설정 가능한 입력 자료를 통하여 다수의 출력자료를 얻는 방법론으로서 불확실성이 큰 기상 및 수문기상자료 분석에 주로 이용되고 있다. 본 연구에서는 해수면온도(sea surface temperature), 해수면기압(sea level pressure)과 방출장파복사에너지(outgoing longwave radiation)를 주요 기상인자로 고려하였으며 예측모형으로서는 Cross Ensemble(out of bagging)방법에 근거한 Support Vector Machine 모형을 이용하였다. 분석결과 주요 기상인자와 50%이상의 상관관계를 보이고 있으며 다소 합리적인 예측 결과를 제시하여 주고 있어 수자원관리를 위한 보조수단으로 이용이 가능할 것으로 사료된다.