The Journal of the Korea Contents Association (한국콘텐츠학회논문지)

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Improvement of Multilayer Perceptrons with Increased Output Nodes

다층퍼셉트론의 출력 노드 수 증가에 의한 성능 향상

  • 오상훈 (목원대학교 정보통신공학과)
  • Published : 2009.01.28
Download PDF
⟨ Previous Next ⟩

Abstract

When we apply MLPs(multilayer perceptrons) to pattern classification problems, we generally allocate one output node for each class and the index of output node denotes a class. On the contrary, in this paper, we propose to increase the number of output nodes per each class for performance improvement of MLPs. For theoretical backgrounds, we derive the misclassification probability in two class problems with additional outputs under the assumption that the two classes have equal probability and outputs are uniformly distributed in each class. Also, simulations of 50 isolated-word recognition show the effectiveness of our method.

일반적으로 다층퍼셉트론을 패턴인식 문제에 적용할 경우 클래스 당 하나의 출력 노드를 배정하고, 이 출력 노드의 인덱스가 입력 패턴의 클래스를 뜻하도록 한다. 이 논문에서는 이와 달리 다층퍼셉트론의 성능 향상을 위하여 클래스 당 출력노드 수를 증가시키는 방법을 제안한다. 두 개의 클래스 문제를 대상으로 클래스 발생확률이 동일하고 각 클래스 내에서 출력노드가 균일분포를 지닌다는 가정 하에, 이 방법의 효용성을 확률론적인 유도를 통하여 증명하였다. 그리고, 50개의 고립단어 인식의 시뮬레이션으로 출력노드를 증가 시킬 경우 성능이 향상됨을 확인하였다.

Keywords

References

  1. Y. Ito, "Approximation of continuous functions on rdby linear combinations of shifted rotations of a sigmoid function with and without scaling," Neural Networks, Vol.5, pp.105-115, 1992. https://doi.org/10.1016/S0893-6080(05)80009-7
  2. H. White, "Learning in artificial neural networks: a statistical perspective," Neural Computation, Vol.1, pp.425-464, 1989. https://doi.org/10.1162/neco.1989.1.4.425
  3. 오상훈, "다층퍼셉트론의 잡음강건성 분석 및 향상 방법," 한국콘텐츠학회 논문지, 제9권, 제1호.
  4. Y. Lee, S.-H. Oh, and M. W. Kim, "An analysis of premature saturation in back-propagation learning," Neural networks, Vol.6, pp.719-728, 1993. https://doi.org/10.1016/S0893-6080(05)80116-9
  5. A. van Ooyen and B. Nienhuis, "Improving the convgence of the back-propagation algorithm," Neural Networks, Vol.5, pp.465-471, 1992. https://doi.org/10.1016/0893-6080(92)90008-7
  6. S.-H. Oh, "Improving the error back-propagation algorithm with a modified error function," IEEE Trans. Neural Networks, Vol.8, pp.799-803, 1997. https://doi.org/10.1109/72.572117
  7. S.-H. Oh and S.-Y. Lee, "An adaptive learning rate with limited error signals for training of multilayer perceptrons," ETRI Journal, Vol.22, No.3, pp.10-18, 2000(9). https://doi.org/10.4218/etrij.00.0100.0302
  8. J. B. Hampshire II and A. H. Waibel, "A novel objective function for improved phoneme recognirtion using time-delay neural networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990(6). https://doi.org/10.1109/72.80233
  9. B. B. Chaudhuri and U. Bhattacharya, "Efficient training and improved performance of multilayer perceptron in pattern classification," Neurocomputing, Vol.34, pp.11-27, 2000. https://doi.org/10.1016/S0925-2312(00)00305-2
  10. K.-Y. Park and S.-Y. Lee, "Out-of vocabulary rejection based on selective attention model," Neural Processing Letters, Vol.12, pp.41-48, 2000. https://doi.org/10.1023/A:1009617830276
  11. F. J. Owens, G. H. Zheng, and D. A. Irvine, "A multi-output-layer perceptron," Neural Computation & Applications, Vol.4, pp.10-20, 1996. https://doi.org/10.1007/BF01413865
  12. T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alkon, "Accelerating the convergence of the back-propagation method," Biol. Cybern., Vol.59, pp.257-263, 1988. https://doi.org/10.1007/BF00332914
  13. J. Y. F. Yam and W. S. Chow, "Extended least squares based algorithm for training feedfoward networks," IEEE Trans. Neural Networks, Vol.8, pp.806-810, 1997. https://doi.org/10.1109/72.572119
  14. R. Paris, E. D. Di Claudio, G. Orlandi, "A generalized learning paradigm exploiting the structure of feedforward neural networks," IEEE Trans. Neural Networks, Vol.7, pp.1450-1459, 1996. https://doi.org/10.1109/72.548172
  15. C. Yu, M. T. Manry, J. Li, and P. L. Narasimha, "An efficient hidden layer training method for multilayer perceptron," Neurocomputing, Vol.70, pp.525-535, 2006. https://doi.org/10.1016/j.neucom.2005.11.008
  16. S.-H. Oh and S.-Y. Lee, "A new error function at hidden layers for fast training of multilayer perceptrons," IEEE Trans. Neural Networks, Vol.10, pp.960-964, 1999. https://doi.org/10.1109/72.774272
  17. J.-H. Jeong, H. Kim, D.-S. Kim, and S.-Y. Lee, "Speaker adaptation based on judge neural networks for real world implementations of voice-command systems," Information Science, Vol.123, pp.13-24, 2000. https://doi.org/10.1016/S0020-0255(99)00107-3
  18. D.-S. Kim, S.-Y. Lee, and R.-M. Kil, "Auditory processig of speech signals for robust speech recognition in real-worldnoisy environments," IEEE Trans. Speech and Audio Processing, Vol.7, pp.55-69, 1999. https://doi.org/10.1109/89.736331
  19. J. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pat. Ana. Mach. Int., Vol.16, pp.550-554, 1994. https://doi.org/10.1109/34.291440