• 응용통계연구
• /
• 제18권2호
• /
• pp.343-354
• /
• 2005

데이터 마이닝에서 데이터를 효율적으로 분류하고자 할 때 많이 사용하고 있는 알고리즘을 실제 자료에 적용시켜 분류성능을 비교하였다. 분류자 생성기법으로는 의사결정나무기법 중의 하나인 CART, 배깅과 부스팅 알고리즘을 CART 모형에 결합한 분류자, 그리고 SVM 분류자를 비교하였다. CART는 결과 해석이 쉬운 장점을 가지고 있지만 데이터에 따라 생성된 분류자가 다양하여 불안정하다는 단점을 가지고 있다. 따라서 이러한 CART의 단점을 보완한 배깅 또는 부스팅 알고리즘과의 결합을 통해 분류자를 생성하고 그 성능에 대해 평가하였다. 또한 최근 들어 분류성능을 인정받고 있는 SVM의 분류성능과도 비교?평가하였다. 각 기법에 의한 분류 결과를 가지고 의사결정나무를 형성하여 자료가 가지는 데이터의 특성에 따른 분류 성능을 알아보았다. 그 결과 데이터의 결측치가 없고 관측값의 수가 적은 경우는 SVM의 분류성능이 뛰어남을 알 수 있었고, 관측값의 수가 많을 때에는 부스팅 알고리즘의 분류성능이 뛰어났으며, 데이터의 결측치가 존재하는 경우는 배깅의 분류성능이 뛰어남을 알 수 있었다.

• Communications for Statistical Applications and Methods
• /
• 제13권2호
• /
• pp.449-466
• /
• 2006

We propose to use variable selection methods based on penalized regression for pruning decision tree ensembles. Pruning methods based on LASSO and SCAD are compared with the cluster pruning method. Comparative studies are performed on some artificial datasets and real datasets. According to the results of comparative studies, the proposed methods based on penalized regression reduce the size of boosting ensembles without decreasing accuracy significantly and have better performance than the cluster pruning method. In terms of classification noise, the proposed pruning methods can mitigate the weakness of AdaBoost to some degree.

• Korean Journal of Radiology
• /
• 제23권12호
• /
• pp.1281-1289
• /
• 2022

Objective: Radiomic modeling using multiple regions of interest in MRI of the brain to diagnose juvenile myoclonic epilepsy (JME) has not yet been investigated. This study aimed to develop and validate radiomics prediction models to distinguish patients with JME from healthy controls (HCs), and to evaluate the feasibility of a radiomics approach using MRI for diagnosing JME. Materials and Methods: A total of 97 JME patients (25.6 ± 8.5 years; female, 45.5%) and 32 HCs (28.9 ± 11.4 years; female, 50.0%) were randomly split (7:3 ratio) into a training (n = 90) and a test set (n = 39) group. Radiomic features were extracted from 22 regions of interest in the brain using the T1-weighted MRI based on clinical evidence. Predictive models were trained using seven modeling methods, including a light gradient boosting machine, support vector classifier, random forest, logistic regression, extreme gradient boosting, gradient boosting machine, and decision tree, with radiomics features in the training set. The performance of the models was validated and compared to the test set. The model with the highest area under the receiver operating curve (AUROC) was chosen, and important features in the model were identified. Results: The seven tested radiomics models, including light gradient boosting machine, support vector classifier, random forest, logistic regression, extreme gradient boosting, gradient boosting machine, and decision tree, showed AUROC values of 0.817, 0.807, 0.783, 0.779, 0.767, 0.762, and 0.672, respectively. The light gradient boosting machine with the highest AUROC, albeit without statistically significant differences from the other models in pairwise comparisons, had accuracy, precision, recall, and F1 scores of 0.795, 0.818, 0.931, and 0.871, respectively. Radiomic features, including the putamen and ventral diencephalon, were ranked as the most important for suggesting JME. Conclusion: Radiomic models using MRI were able to differentiate JME from HCs.

• IMMUNE NETWORK
• /
• 제3권2호
• /
• pp.110-117
• /
• 2003

Background: The usefulness of DNA vaccine at priming step of heterologous prime-boost vaccination led to DNA vaccine closer to practical reality. DNA vaccine priming followed by recombinant viral vector boosting via systemic route induces optimal systemic immunity but no mucosal immunity. Mucosal vaccination of the reversed protocol (recombinant viral vector priming-DNA vaccine boosting), however, can induce both maximal mucosal and systemic immunity. Here, we tried to address the reason why the mucosal protocol of prime-boost vaccination differs from that of systemic vaccination. Methods: To address the importance of primary immunity induced at priming step, mice were primed with different doses of DNA vaccine or coadministration of DNA vaccine plus mucosal adjuvant, and immunity including serum IgG and mucosal IgA was then determined following boosting with recombinant viral vector. Next, to assess influence of humoral pre-existing immunity on boosting $CD8^+$ T cell-mediated immunity, $CD8^+$ T cell-mediated immunity in B cell-deficient (${\mu}K/O$) mice immunized with prime-boost regimens was evaluated by CTL assay and $IFN-{\gamma}$-producing cells. Results: Immunity primed with recombinant viral vector was effectively boosted with DNA vaccine even 60 days later. In particular, animals primed by increasing doses of DNA vaccine or incorporating an adjuvant at priming step and boosted by recombinant viral vector elicited comparable responses to recombinant viral vector primed-DNA vaccine boosted group. Humoral pre-existing immunity was also unlikely to interfere the boosting effect of $CD8^+$ T cell-mediated immunity by recombinant viral vector. Conclusion: This report provides the important point that optimally primed responses should be considered in mucosal immunization of heterologous prime-boost regimens for inducing the effective boosting at both mucosal and systemic sites.

• Journal of Information Processing Systems
• /
• 제19권5호
• /
• pp.652-662
• /
• 2023

Identifying highly discriminating genes is a critical step in tumor recognition tasks based on microarray gene expression profile data and machine learning. Gene selection based on tree models has been the subject of several studies. However, these methods are based on a single-tree model, often not robust to ultra-highdimensional microarray datasets, resulting in the loss of useful information and unsatisfactory classification accuracy. Motivated by the limitations of single-tree-based gene selection, in this study, ensemble gene selection methods based on multiple-tree models were studied to improve the classification performance of tumor identification. Specifically, we selected the three most representative tree models: ID3, random forest, and gradient boosting decision tree. Each tree model selects top-n genes from the microarray dataset based on its intrinsic mechanism. Subsequently, three ensemble gene selection methods were investigated, namely multipletree model intersection, multiple-tree module union, and multiple-tree module cross-union, were investigated. Experimental results on five benchmark public microarray gene expression datasets proved that the multiple tree module union is significantly superior to gene selection based on a single tree model and other competitive gene selection methods in classification accuracy.

• Advances in Energy Research
• /
• 제6권2호
• /
• pp.91-101
• /
• 2019

This paper proposes an efficient data-driven approach to build models for predicting energy consumption in buildings. Data used in this research is collected by installing humidity and temperature sensors at different locations in a building. In addition to this, weather data from nearby weather station is also included in the dataset to study the impact of weather conditions on energy consumption. One of the main emphasize of this research is to make feature selection independent of domain knowledge. Therefore, to extract useful features from data, two different approaches are tested: one is feature selection through principal component analysis and second is relative importance-based feature selection in original domain. The regression model used in this research is gradient boosting regression and its optimal parameters are chosen through a two staged coarse-fine search approach. In order to evaluate the performance of model, different performance evaluation metrics like r2-score and root mean squared error are used. Results have shown that best performance is achieved, when relative importance-based feature selection is used with gradient boosting regressor. Results of proposed technique has also outperformed the results of support vector machines and neural network-based approaches tested on the same dataset.

• Geomechanics and Engineering
• /
• 제37권6호
• /
• pp.539-554
• /
• 2024

This study explores development of prediction model for seismic site classification through the integration of machine learning techniques with horizontal-to-vertical spectral ratio (HVSR) methodologies. To improve model accuracy, the research employs outlier detection methods and, synthetic minority over-sampling technique (SMOTE) for data balance, and evaluates using seven machine learning models using seismic data from KiK-net. Notably, light gradient boosting method (LGBM), gradient boosting, and decision tree models exhibit improved performance when coupled with SMOTE, while Multiple linear regression (MLR) and Support vector machine (SVM) models show reduced efficacy. Outlier detection techniques significantly enhance accuracy, particularly for LGBM, gradient boosting, and voting boosting. The ensemble of LGBM with the isolation forest and SMOTE achieves the highest accuracy of 0.91, with LGBM and local outlier factor yielding the highest F1-score of 0.79. Consistently outperforming other models, LGBM proves most efficient for seismic site classification when supported by appropriate preprocessing procedures. These findings show the significance of outlier detection and data balancing for precise seismic soil classification prediction, offering insights and highlighting the potential of machine learning in optimizing site classification accuracy.

• Journal of the Korean Data and Information Science Society
• /
• 제27권1호
• /
• pp.255-264
• /
• 2016

Classification is a predictive modeling for a categorical target variable. Various classification ensemble methods, which predict with better accuracy by combining multiple classifiers, became a powerful machine learning and data mining paradigm. Well-known methodologies of classification ensemble are boosting, bagging and random forest. In this article, we assume that decision trees are used as classifiers in the ensemble. Further, we hypothesized that tree size affects classification accuracy. To study how the tree size in uences accuracy, we performed experiments using twenty-eight data sets. Then we compare the performances of ensemble algorithms; bagging, double-bagging, boosting and random forest, with different tree sizes in the experiment.

• Communications for Statistical Applications and Methods
• /
• 제20권4호
• /
• pp.301-309
• /
• 2013

This paper predicts gross box office revenue for domestic films using the Korean film data from 2008-2011. We use three regression methods, Linear Regression, Random Forest and Gradient Boosting to predict the gross box office revenue. We only consider domestic films with a revenue size of at least KRW 500 million; relevant explanatory variables are chosen by data visualization and variable selection techniques. The key idea of analyzing this data is to construct the meaningful explanatory variables from the data sources available to the public. Some variables must be categorized to conduct more effective analysis and clustering methods are applied to achieve this task. We choose the best model based on performance in the test set and important explanatory variables are discussed.

• Journal of the Korean Data and Information Science Society
• /
• 제23권2호
• /
• pp.375-383
• /
• 2012

The purpose of the ensemble methods is to increase the accuracy of prediction through combining many classifiers. According to recent studies, it is proved that random forests and forward stagewise regression have good accuracies in classification problems. However they have great prediction error in separation boundary points because they used decision tree as a base learner. In this study, we use the kernel ridge regression instead of the decision trees in random forests and boosting. The usefulness of our proposed ensemble methods was shown by the simulation results of the prostate cancer and the Boston housing data.