Performance Improvement of Multilayer Perceptrons with Increased Output Nodes

다층퍼셉트론의 출력 노드 수 증가에 의한 성능 향상

  • Published : 2009.01.28


When we apply MLPs(multilayer perceptrons) to pattern classification problems, we generally allocate one output node for each class and the index of output node denotes a class. On the contrary, in this paper, we propose to increase the number of output nodes per each class for performance improvement of MLPs. For theoretical backgrounds, we derive the misclassification probability in two class problems with additional outputs under the assumption that the two classes have equal probability and outputs are uniformly distributed in each class. Also, simulations of 50 isolated-word recognition show the effectiveness of our method.


Multilayer Perceptrons;Performance Improvement;Output Nodes


  1. Y. Ito, "Approximation of continuous functions on rdby linear combinations of shifted rotations of a sigmoid function with and without scaling," Neural Networks, Vol.5, pp.105-115, 1992.
  2. H. White, "Learning in artificial neural networks: a statistical perspective," Neural Computation, Vol.1, pp.425-464, 1989.
  3. 오상훈, "다층퍼셉트론의 잡음강건성 분석 및 향상 방법," 한국콘텐츠학회 논문지, 제9권, 제1호.
  4. Y. Lee, S.-H. Oh, and M. W. Kim, "An analysis of premature saturation in back-propagation learning," Neural networks, Vol.6, pp.719-728, 1993.
  5. A. van Ooyen and B. Nienhuis, "Improving the convgence of the back-propagation algorithm," Neural Networks, Vol.5, pp.465-471, 1992.
  6. S.-H. Oh, "Improving the error back-propagation algorithm with a modified error function," IEEE Trans. Neural Networks, Vol.8, pp.799-803, 1997.
  7. S.-H. Oh and S.-Y. Lee, "An adaptive learning rate with limited error signals for training of multilayer perceptrons," ETRI Journal, Vol.22, No.3, pp.10-18, 2000(9).
  8. J. B. Hampshire II and A. H. Waibel, "A novel objective function for improved phoneme recognirtion using time-delay neural networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990(6).
  9. B. B. Chaudhuri and U. Bhattacharya, "Efficient training and improved performance of multilayer perceptron in pattern classification," Neurocomputing, Vol.34, pp.11-27, 2000.
  10. K.-Y. Park and S.-Y. Lee, "Out-of vocabulary rejection based on selective attention model," Neural Processing Letters, Vol.12, pp.41-48, 2000.
  11. F. J. Owens, G. H. Zheng, and D. A. Irvine, "A multi-output-layer perceptron," Neural Computation & Applications, Vol.4, pp.10-20, 1996.
  12. T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alkon, "Accelerating the convergence of the back-propagation method," Biol. Cybern., Vol.59, pp.257-263, 1988.
  13. J. Y. F. Yam and W. S. Chow, "Extended least squares based algorithm for training feedfoward networks," IEEE Trans. Neural Networks, Vol.8, pp.806-810, 1997.
  14. R. Paris, E. D. Di Claudio, G. Orlandi, "A generalized learning paradigm exploiting the structure of feedforward neural networks," IEEE Trans. Neural Networks, Vol.7, pp.1450-1459, 1996.
  15. C. Yu, M. T. Manry, J. Li, and P. L. Narasimha, "An efficient hidden layer training method for multilayer perceptron," Neurocomputing, Vol.70, pp.525-535, 2006.
  16. S.-H. Oh and S.-Y. Lee, "A new error function at hidden layers for fast training of multilayer perceptrons," IEEE Trans. Neural Networks, Vol.10, pp.960-964, 1999.
  17. J.-H. Jeong, H. Kim, D.-S. Kim, and S.-Y. Lee, "Speaker adaptation based on judge neural networks for real world implementations of voice-command systems," Information Science, Vol.123, pp.13-24, 2000.
  18. D.-S. Kim, S.-Y. Lee, and R.-M. Kil, "Auditory processig of speech signals for robust speech recognition in real-worldnoisy environments," IEEE Trans. Speech and Audio Processing, Vol.7, pp.55-69, 1999.
  19. J. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pat. Ana. Mach. Int., Vol.16, pp.550-554, 1994.