한국정보과학회논문지:소프트웨어및응용 (Journal of KIISE:Software and Applications)

한국정보과학회 (Korean Institute of Information Scientists and Engineers)
권호 표지

다층 퍼셉트론에서 구조인자 제어 영향의 비교

Comparison of Factors for Controlling Effects in MLP Networks

  • 윤여창 (우석대학교 전산통계학과)
  • 발행 : 2004.05.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

다층 퍼셉트론(Multi-Layer Perceptron, MLP) 구조는 그의 비선형 적합능력으로 인하여 매우 다양한 실제 문제에 적용되고 있다. 그러나 일반화된 MLP 구조의 적합능력은 은닉노드의 개수. 초기 가중 값 그리고 학습 회수 또는 학습 오차와 같은 구조인자(factor)들에 크게 영향을 받는다. 만약 이들 구조인자가 부적절하게 선택되면 일반화된 MLP 구조의 적합능력이 매우 왜곡될 수 있다. 따라서 MLP구조에 영향을 주는 인자들의 결합 영향을 살펴보는 것은 중요한 문제이다. 이 논문에서는 제어상자(controller box)를 통한 학습결과와 더불어 MLP구조를 일반화할 때 영향을 줄 수 있는 신경망의 일반적인 구조인자 들을 실증적으로 살펴보고 이들의 상대효과를 비교한다.

Multi-Layer Perceptron network has been mainly applied to many practical problems because of its nonlinear mapping ability. However the generalization ability of MLP networks may be affected by the number of hidden nodes, the initial values of weights and the training errors. These factors, if improperly chosen, may result in poor generalization ability of MLP networks. It is important to identify these factors and their interaction in order to control effectively the generalization ability of MLP networks. In this paper, we have empirically identified the factors that affect the generalization ability of MLP networks, and compared their relative effects on the generalization performance for the conventional and visualized weight selecting methods using the controller box.

키워드

참고문헌

  1. Cherkassky, V. and Mulier, F., 'Learning from data-Concepts, Theory and Methods,' Wiley, New York, 1998
  2. Vapnik, V., 'The Nature of Statistical Learning Theory,' Wiley, New York, 1995
  3. Zhong, S., and Cherkassky, V., 'Factors Controlling Generalization Ability od MLP Networks,' Proceedings of International Joint Conference On Neural Networks, 1999 https://doi.org/10.1109/IJCNN.1999.831571
  4. 윤여창, '제어상자를 이용한 단순 신경망의 개선된 학습과정', 정보과학회논문지 : 소프트웨어 및 응용, 제28권, 제4호, pp.338-345, 2001
  5. Easton, G.S., 'A Simple Dynamic Graphical Diagnostic Method for Almost Any Model,' Journal of the American Statistical Association, 89, pp.201-207, 1994 https://doi.org/10.2307/2291216
  6. Kim, Y.K. and Ra, J.B., 'Weight value initialization for improving training speed in the back propagation network,' Proceedings of International Joint Conference On Neural Networks, Vol.3, pp.2396-2401, 1991
  7. Espinosa, C.H. and Redondo, M.F., 'Multilayer feedforward weight initialization,' Proceedings of International Joint Conference On Neural Networks, pp.166-170, 2001 https://doi.org/10.1109/IJCNN.2001.939011
  8. Thimm, G. AND Fiesler, E., 'High-order and multilayer perceptron initialization,' IEEE Transactions on Neural Networks, Vol.8, pp.349-359, 1997 https://doi.org/10.1109/72.557673
  9. Smith, M., 'Neural Networks for Statistical Modeling,' Van Nostrand Reinhold, New York, 1993
  10. Hagan, M.T., Demuth, H.B. and Beale, M., 'Neural Network Design,' PWS, Boston, 1995
  11. Atiya, A. and Ji, C., 'How Initial Conditions Affect Generalization Performance in Large Networks,' IEEE Transaction on Neural Networks, Vol.8, No.2, pp.448-451, 1997 https://doi.org/10.1109/72.557701
  12. Cherkassky, V. and Shepherd, R. 'Regularization Effect of Weight Initialization in Back Propagation Networks,' Proceedings of International Joint Conference On Neural Networks, pp.2258-2261, 1998 https://doi.org/10.1109/IJCNN.1998.687212