• Title/Summary/Keyword: one class classifiers

Search Result 44, Processing Time 0.023 seconds

Solving Multi-class Problem using Support Vector Machines (Support Vector Machines을 이용한 다중 클래스 문제 해결)

  • Ko, Jae-Pil
    • Journal of KIISE:Software and Applications
    • /
    • v.32 no.12
    • /
    • pp.1260-1270
    • /
    • 2005
  • Support Vector Machines (SVM) is well known for a representative learner as one of the kernel methods. SVM which is based on the statistical learning theory shows good generalization performance and has been applied to various pattern recognition problems. However, SVM is basically to deal with a two-class classification problem, so we cannot solve directly a multi-class problem with a binary SVM. One-Per-Class (OPC) and All-Pairs have been applied to solve the face recognition problem, which is one of the multi-class problems, with SVM. The two methods above are ones of the output coding methods, a general approach for solving multi-class problem with multiple binary classifiers, which decomposes a complex multi-class problem into a set of binary problems and then reconstructs the outputs of binary classifiers for each binary problem. In this paper, we introduce the output coding methods as an approach for extending binary SVM to multi-class SVM and propose new output coding schemes based on the Error-Correcting Output Codes (ECOC) which is a dominant theoretical foundation of the output coding methods. From the experiment on the face recognition, we give empirical results on the properties of output coding methods including our proposed ones.

Integrating Multiple Classifiers in a GA-based Inductive Learning Environment (유전 알고리즘 기반 귀납적 학습 환경에서 분류기의 통합)

  • Kim, Yeong-Joon
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.10 no.3
    • /
    • pp.614-621
    • /
    • 2006
  • We have implemented a multiclassifier learning approach in a GA-based inductive learning environment that learns classification rules that are similar to rules used in PROSPECTOR. In the multiclassifier learning approach, a classification system is constructed with several classifiers that are obtained by running a GA-based learning system several times to improve the overall performance of a classification system. To implement the multiclassifier learning approach, we need a decision-making scheme that can draw a decision using multiple classifiers. In this paper, we introduce two decision-making schemes: one is based on combining posterior odds given by classifiers to each class and the other one is a voting scheme based on ranking assigned to each class by classifiers. We also present empirical results that evaluate the effect of the multiclassifier learning approach on the GA-based inductive teaming environment.

On Adaptive Learning HMM Classifiers Using Splitting-Merging Techniques (분할-합병기법을 이용한 HMM 분류기의 적응학습)

  • 오수환;김상운
    • Proceedings of the IEEK Conference
    • /
    • 2003.11b
    • /
    • pp.99-102
    • /
    • 2003
  • In this paper we propose an adaptive learning method for HMM classifiers by using splitting and merging techniques to overcome the problem of the conventional teaming, where one HMM classifier per class has been trained, individually. The experimental results demonstrate a possibility that the proposed mechanism could be applied for applications of having multiple clusters in a class.

  • PDF

Recognition of Handwritten Numerals using SVM Classifiers (SVM 분류기를 이용한 필기체 숫자인식)

  • Park, Joong-Jo;Kim, Kyoung-Min
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.8 no.3
    • /
    • pp.136-142
    • /
    • 2007
  • Recent researches in the recognition system have shown that SVM (Support Vector Machine) classifiers often have superior recognition rates in comparison to other classifiers. In this paper, we present the handwritten numeral recognition algorithm using SVM classifiers. The numeral features used in our algorithm are mesh features, directional features by Kirsch operators and concavity features, where first two features represent the foreground information of numerals and the last feature represents the background information of numerals. These features are complements each of the other. Since SVM is basically a binary classifier, it is required to construct and combine several binary SVMs to get the multi-class classifiers. We use two strategies for implementing multi-class SVM classifiers: "one against one" and "one against the rest", and examine their performances on the features used. The efficiency of our method is tested by the CENPARMI handwritten numeral database, and the recognition rate of 98.45% is achieved.

  • PDF

Effective Fingerprint Classification using Subsumed One-Vs-All Support Vector Machines and Naive Bayes Classifiers (포섭구조 일대다 지지벡터기계와 Naive Bayes 분류기를 이용한 효과적인 지문분류)

  • Hong, Jin-Hyuk;Min, Jun-Ki;Cho, Ung-Keun;Cho, Sung-Bae
    • Journal of KIISE:Software and Applications
    • /
    • v.33 no.10
    • /
    • pp.886-895
    • /
    • 2006
  • Fingerprint classification reduces the number of matches required in automated fingerprint identification systems by categorizing fingerprints into a predefined class. Support vector machines (SVMs), widely used in pattern classification, have produced a high accuracy rate when performing fingerprint classification. In order to effectively apply SVMs to multi-class fingerprint classification systems, we propose a novel method in which SVMs are generated with the one-vs-all (OVA) scheme and dynamically ordered with $na{\ddot{i}}ve$ Bayes classifiers. More specifically, it uses representative fingerprint features such as the FingerCode, singularities and pseudo ridges to train the OVA SVMs and $na{\ddot{i}}ve$ Bayes classifiers. The proposed method has been validated on the NIST-4 database and produced a classification accuracy of 90.8% for 5-class classification. Especially, it has effectively managed tie problems usually occurred in applying OVA SVMs to multi-class classification.

Fuzzy Behavior Knowledge Space for Integration of Multiple Classifiers (다중 분류기 통합을 위한 퍼지 행위지식 공간)

  • 김봉근;최형일
    • Korean Journal of Cognitive Science
    • /
    • v.6 no.2
    • /
    • pp.27-45
    • /
    • 1995
  • In this paper, we suggest the "Fuzzy Behavior Knowledge Space(FBKS)" and explain how to utilize the FBKS when aggregating decisions of individual classifiers. The concept of "Behavior Knowledge Space(BKS)" is known to be the best method in the context that each classifier offers only one class label as its decision. However. the BKS does not considers measurement value of class label. Furthermore, it does not allow the heuristic knowledge of human experts to be embedded when combining multiple decisions. The FBKS eliminates such drawbacks of the BKS by adapting the fwzy concepts. Our method applies to the classification results that contain both class labels and associated measurement values. Experimental results confirm that the FBKS could be a very promising tool in pattern recognition areas.

  • PDF

Fuzzy SVM for Multi-Class Classification

  • Na, Eun-Young;Hong, Dug-Hun;Hwang, Chang-Ha
    • 한국데이터정보과학회:학술대회논문집
    • /
    • 2003.10a
    • /
    • pp.123-123
    • /
    • 2003
  • More elaborated methods allowing the usage of binary classifiers for the resolution of multi-class classification problems are briefly presented. This way of using FSVC to learn a K-class classification problem consists in choosing the maximum applied to the outputs of K FSVC solving a one-per-class decomposition of the general problem.

  • PDF

Ensemble Learning with Support Vector Machines for Bond Rating (회사채 신용등급 예측을 위한 SVM 앙상블학습)

  • Kim, Myoung-Jong
    • Journal of Intelligence and Information Systems
    • /
    • v.18 no.2
    • /
    • pp.29-45
    • /
    • 2012
  • Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Multi-target Classification Method Based on Adaboost and Radial Basis Function (아이다부스트(Adaboost)와 원형기반함수를 이용한 다중표적 분류 기법)

  • Kim, Jae-Hyup;Jang, Kyung-Hyun;Lee, Jun-Haeng;Moon, Young-Shik
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.47 no.3
    • /
    • pp.22-28
    • /
    • 2010
  • Adaboost is well known for a representative learner as one of the kernel methods. Adaboost which is based on the statistical learning theory shows good generalization performance and has been applied to various pattern recognition problems. However, Adaboost is basically to deal with a two-class classification problem, so we cannot solve directly a multi-class problem with Adaboost. One-Vs-All and Pair-Wise have been applied to solve the multi-class classification problem, which is one of the multi-class problems. The two methods above are ones of the output coding methods, a general approach for solving multi-class problem with multiple binary classifiers, which decomposes a complex multi-class problem into a set of binary problems and then reconstructs the outputs of binary classifiers for each binary problem. However, two methods cannot show good performance. In this paper, we propose the method to solve a multi-target classification problem by using radial basis function of Adaboost weak classifier.

Comparison Study of Multi-class Classification Methods

  • Bae, Wha-Soo;Jeon, Gab-Dong;Seok, Kyung-Ha
    • Communications for Statistical Applications and Methods
    • /
    • v.14 no.2
    • /
    • pp.377-388
    • /
    • 2007
  • As one of multi-class classification methods, ECOC (Error Correcting Output Coding) method is known to have low classification error rate. This paper aims at suggesting effective multi-class classification method (1) by comparing various encoding methods and decoding methods in ECOC method and (2) by comparing ECOC method and direct classification method. Both SVM (Support Vector Machine) and logistic regression model were used as binary classifiers in comparison.