DOI QR코드

DOI QR Code

Performance Improvement of Multilayer Perceptrons with Increased Output Nodes

다층퍼셉트론의 출력 노드 수 증가에 의한 성능 향상

  • Published : 2009.01.28

Abstract

When we apply MLPs(multilayer perceptrons) to pattern classification problems, we generally allocate one output node for each class and the index of output node denotes a class. On the contrary, in this paper, we propose to increase the number of output nodes per each class for performance improvement of MLPs. For theoretical backgrounds, we derive the misclassification probability in two class problems with additional outputs under the assumption that the two classes have equal probability and outputs are uniformly distributed in each class. Also, simulations of 50 isolated-word recognition show the effectiveness of our method.

Keywords

Multilayer Perceptrons;Performance Improvement;Output Nodes

References

  1. Y. Ito, "Approximation of continuous functions on rdby linear combinations of shifted rotations of a sigmoid function with and without scaling," Neural Networks, Vol.5, pp.105-115, 1992. https://doi.org/10.1016/S0893-6080(05)80009-7
  2. H. White, "Learning in artificial neural networks: a statistical perspective," Neural Computation, Vol.1, pp.425-464, 1989. https://doi.org/10.1162/neco.1989.1.4.425
  3. 오상훈, "다층퍼셉트론의 잡음강건성 분석 및 향상 방법," 한국콘텐츠학회 논문지, 제9권, 제1호.
  4. Y. Lee, S.-H. Oh, and M. W. Kim, "An analysis of premature saturation in back-propagation learning," Neural networks, Vol.6, pp.719-728, 1993. https://doi.org/10.1016/S0893-6080(05)80116-9
  5. A. van Ooyen and B. Nienhuis, "Improving the convgence of the back-propagation algorithm," Neural Networks, Vol.5, pp.465-471, 1992. https://doi.org/10.1016/0893-6080(92)90008-7
  6. S.-H. Oh, "Improving the error back-propagation algorithm with a modified error function," IEEE Trans. Neural Networks, Vol.8, pp.799-803, 1997. https://doi.org/10.1109/72.572117
  7. S.-H. Oh and S.-Y. Lee, "An adaptive learning rate with limited error signals for training of multilayer perceptrons," ETRI Journal, Vol.22, No.3, pp.10-18, 2000(9). https://doi.org/10.4218/etrij.00.0100.0302
  8. J. B. Hampshire II and A. H. Waibel, "A novel objective function for improved phoneme recognirtion using time-delay neural networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990(6). https://doi.org/10.1109/72.80233
  9. B. B. Chaudhuri and U. Bhattacharya, "Efficient training and improved performance of multilayer perceptron in pattern classification," Neurocomputing, Vol.34, pp.11-27, 2000. https://doi.org/10.1016/S0925-2312(00)00305-2
  10. K.-Y. Park and S.-Y. Lee, "Out-of vocabulary rejection based on selective attention model," Neural Processing Letters, Vol.12, pp.41-48, 2000. https://doi.org/10.1023/A:1009617830276
  11. F. J. Owens, G. H. Zheng, and D. A. Irvine, "A multi-output-layer perceptron," Neural Computation & Applications, Vol.4, pp.10-20, 1996. https://doi.org/10.1007/BF01413865
  12. T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alkon, "Accelerating the convergence of the back-propagation method," Biol. Cybern., Vol.59, pp.257-263, 1988. https://doi.org/10.1007/BF00332914
  13. J. Y. F. Yam and W. S. Chow, "Extended least squares based algorithm for training feedfoward networks," IEEE Trans. Neural Networks, Vol.8, pp.806-810, 1997. https://doi.org/10.1109/72.572119
  14. R. Paris, E. D. Di Claudio, G. Orlandi, "A generalized learning paradigm exploiting the structure of feedforward neural networks," IEEE Trans. Neural Networks, Vol.7, pp.1450-1459, 1996. https://doi.org/10.1109/72.548172
  15. C. Yu, M. T. Manry, J. Li, and P. L. Narasimha, "An efficient hidden layer training method for multilayer perceptron," Neurocomputing, Vol.70, pp.525-535, 2006. https://doi.org/10.1016/j.neucom.2005.11.008
  16. S.-H. Oh and S.-Y. Lee, "A new error function at hidden layers for fast training of multilayer perceptrons," IEEE Trans. Neural Networks, Vol.10, pp.960-964, 1999. https://doi.org/10.1109/72.774272
  17. J.-H. Jeong, H. Kim, D.-S. Kim, and S.-Y. Lee, "Speaker adaptation based on judge neural networks for real world implementations of voice-command systems," Information Science, Vol.123, pp.13-24, 2000. https://doi.org/10.1016/S0020-0255(99)00107-3
  18. D.-S. Kim, S.-Y. Lee, and R.-M. Kil, "Auditory processig of speech signals for robust speech recognition in real-worldnoisy environments," IEEE Trans. Speech and Audio Processing, Vol.7, pp.55-69, 1999. https://doi.org/10.1109/89.736331
  19. J. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pat. Ana. Mach. Int., Vol.16, pp.550-554, 1994. https://doi.org/10.1109/34.291440