Journal of Institute of Control, Robotics and Systems (제어로봇시스템학회논문지)

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지

Improvement of learning method in pattern classification

패턴분류에서 학습방법 개선

  • 김명찬 (삼성 데이터 시스템 (주)) ;
  • 최종호 (서울대학교 전기공학부)
  • Published : 1997.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

A new algorithm is proposed for training the multilayer perceptrion(MLP) in pattern classification problems to accelerate the learning speed. It is shown that the sigmoid activation function of the output node can have deterimental effect on the performance of learning. To overcome this detrimental effect and to use the information fully in supervised learning, an objective function for binary modes is proposed. This objective function is composed with two new output activation functions which are selectively used depending on desired values of training patterns. The effect of the objective function is analyzed and a training algorithm is proposed based on this. Its performance is tested in several examples. Simulation results show that the performance of the proposed method is better than that of the conventional error back propagation (EBP) method.

Keywords

References

  1. IEEE ASSP Magazine An introduction to computing with neural nets R. P. Lippmann
  2. Parallel Distributed Processing v.1 no.2 D. E. Rumelhart;J. L. McClelland
  3. Introduction to the theory of neural computation J. Hertz;A. Krogh;R. G. Palmer
  4. Neural Networks v.1 Increased rates of convergence through learning rates adaptation R. A. Jacobs
  5. Proc. IJCNN v.1 Acceleration of backpropagation through learning rate and momentum adaptation A. A. minai;R. D. Williams
  6. Proc. IJCNN v.1 Back-propagation heuristics: A study of the extended delta-bar-delta algorithm A. A. Minai;R. D. Williams
  7. Proc. 1988 Connectionist Models, Summer School Improving the convergence of backpropagation learning with second order methods S. Becker;Y. Le Cun
  8. Proc. IJCNN v.3 MEKA - A fast, local algorithm for training feedforward neural networks S. Shah;F. Palmieri
  9. IEEE Trans. Signal Processing v.40 no.1 A fast new algorithm for training feedforward neural networks R. S. Scaleo;N. Tepedelenlioglu
  10. Proc. IJCNN v.2 Parallelism, hierarchy, scaling in time-delay neural networks for spotting Japanese phonemes /CV-syllabes H. Sawai;A. Waivel;P. affiner;M. Miyatake;K. Shikano
  11. Proc. 1988 Connectionist Models, Summer School Learning to tell two spirals apart K. Lang;M. Witbrok
  12. Proc. IJCNN v.1 A direct adaptive method for faster backpropagation learning the RPROP algorithm M. Riedmiller;H. Braun
  13. Proc. IJCNN Lambda learning rule for feedforward neural networks J. M. Zurada
  14. Proc. IJCNN v.3 Accelerated learning in multilayer network A. Atiya;A. Parlos;J. Muthusami;B. Fernandez;W. Tsai
  15. IEEE Trans. Neural Networks v.5 An accelerated learning algoirthm for multilayer perceptron networks A. G. Parlos;B. Fernandez;A. F. Atiya;J. Muthusami;W. K. Tsai
  16. Proc. ICONIP'94 v.2 Square root scale and momentum control for BP learning in classification problems Myung-Chan Kim;Chong-Ho Choi
  17. IEEE Trans. Neural Networks v.6 no.1 Efficient classification for multiclass problems using modular neural networks R. Anand;K. Mehrotra;C. K. Mohan;S. Ranka
  18. IEEE Trans. Neural Networks v.6 no.1 An accelerated learning algorithm for multilayer perceptron: optimization layer by layer S. Ergezinger;E. Thomsen
  19. 전자공학회논문집 v.32-B no.3 신경회로망에서 일괄 학습 김명찬;최종호
  20. 제 1회 항법유도제어 학술대회 배치모드 학습시 다층인식자의 가중치 초기화 방법 김명찬;최종호
  21. Proc. ICNN Square root learning in batch mode BP for classification problems Myung-Chan Kim;Chong-Ho Choi
  22. 제 1회 항법유도제어 학술대회 배치모드 학습시 다층인식자의 가중치 초기화 방법 김명찬;최종호
  23. IEEE Control System Magazine Learning algorithms for perceptrons using back-propagation with selective updates S. Huang;Y. Huang
  24. Proc. IJCNN v.I The effect of the slope of the activation function on the back propagation algorithm A. Rezgui;N. Tepedelenlioglu
  25. Neural Networks v.5 Improving the convergence of the back-propagation algorithm A. Van Ooyen;B. Nienhuis
  26. IEEE Trans. Circuit and System v.30 no.5 Back propagation fails to separate where perceptron succeed M. L. Brady;R. Raghaven;J. Slawny
  27. 박사학위 논문, 서울대학교 패턴 분류용 다층인식자의 오차 역전파 학습 알고리즘의 개선 김명찬