DOI QR코드

DOI QR Code

An Analysis of Noise Robustness for Multilayer Perceptrons and Its Improvements

다층퍼셉트론의 잡음 강건성 분석 및 향상 방법

  • 오상훈 (목원대학교 정보통신공학과)
  • Published : 2009.01.28

Abstract

In this paper, we analyse the noise robustness of MLPs(Multilayer perceptrons) through deriving the probability density function(p.d.f.) of output nodes with additive input noises and the misclassification ratio with the integral form of the p.d.f. functions. Also, we propose linear preprocessing methods to improve the noise robustness. As a preprocessing stage of MLPs, we consider ICA(independent component analysis) and PCA(principle component analysis). After analyzing the noise reduction effect using PCA or ICA in the viewpoints of SNR(Singal-to-Noise Ratio), we verify the preprocessing effects through the simulations of handwritten-digit recognition problems.

이 논문에서는 다층퍼셉트론(MLP:Multilayer Perceptron)에서 입력에 잡음이 섞인 경우 출력노드의 확률밀도 함수를 유도하고, 이의 적분으로 잡음에 의하여 패턴이 오인식될 확률을 유도하였다. 그리고, 이를 향상시키는 선형적 방법을 제안하였다. 즉, 독립성분분석(ICA: independent component analysis)과 주성분분석(PCA: principle component analysis)를 적용하여, 이들이 지닌 잡음 처리 효과를 SNR(Signal-to-Noise Ratio) 관점에서 분석하였다. 그리고 이들이 잡음을 처리한 후 MLP에 입력 시 나타나는 잡음 강건성을 필기체 숫자 인식의 시뮬레이션으로 확인하였다.

Keywords

References

  1. K. Hornik, M. Stincombe, and H. White, "Multilayer Feedforward Networks Are Universal Approximators," Neural Networks, Vol.2, pp.359-366, 1989. https://doi.org/10.1016/0893-6080(89)90020-8
  2. R. P. Lippmann, "Pattern Classification Using Neural Networks," IEEE Communication Magazine, pp.47-64, 1989(11). https://doi.org/10.1109/35.41401
  3. J. B. Hampshire II and A. H. Waibel, "A Novel Objective Function for Improved Phoneme Recognition Using Time-Delayed Neural Networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990(6). https://doi.org/10.1109/72.80233
  4. A. S. Weigend and N. A. Gershenfeld, Time Sereies Prediction: Forecastimg the Future and Understanding the Past, ASddison-Wesley Publishing Co., 1994.
  5. K. S. Narenda and K. Parthasarathy, "Identification and Control of Dynamic System Using Neural Networks," IEEE Trans. Neural Networks, Vol.1, pp.4-27, 1990. https://doi.org/10.1109/72.80202
  6. B. Aazhang, B.-P. Paris, and G. C. Orsak, "Neural Networks for Multiuser Detection in Code-Division Multiple-Access Communications," IEEE Trans. Communications, Vol.40, pp.1212-1222, 1992. https://doi.org/10.1109/26.153366
  7. I. W. Habib, A. A. Tarraf, and T. N. Saadawi, "Intelligent Traffic Control for ATM Broadband Networks," IEEE Communication Magazine, Vol.33, pp.76-85, 1995. https://doi.org/10.1109/35.466223
  8. M. Stevenson, R. Winter, and B. Widrow, "Sensitivity of Feedforward Neural Networks to Weight Errors," IEEE Trans. Neural Networks, Vol.1, pp.71-90, 1990(3). https://doi.org/10.1109/72.80206
  9. S.-H. Oh and Y. Lee, "Sensitivity Analysis of Single Hidden-Layer Neural Networks with Threshold Functions," IEEE Trans. Neural Networks, Vol.6, pp.1005-1007, 1995(7). https://doi.org/10.1109/72.392264
  10. J. Y. Choi and C.-H. Choi, "Sensitivity Analysis of Multilayer Perceptron with Differentiable Activation Transformations," IEEE Trans. Neural Networks, Vol.3, pp.101-107, 1992(1). https://doi.org/10.1109/72.105422
  11. W. H. Delashmit and M. T. Manry, "Enhanced Robustness of Multilayer Perceptron Training," Proc. Thirty-Sixth Asilomar Conf. Signals,Systems and Computers, Vol.2, pp.1029-1033, 2002(11). https://doi.org/10.1109/ACSSC.2002.1196940
  12. F.-L. Chung, "CATSMLP: Toward a Robust and Interpretable Multilayer Perceptron with Sigmoid Activation Functions," IEEE Trans. Sys. Man. and Cyber. Part B, Vol.36, pp.1319-1331, 2006(12). https://doi.org/10.1109/TSMCB.2006.875871
  13. V. Sanchez-Poblador et al., "ICA As A Preprocessing Technique for Classification," LNCS, Vol.3195, pp.1165-1172, 2004(10).
  14. U.-M. Bae, H.-M. Park, and S.-Y. Lee, "Top-Down Attention to Complement Independent Component Analysis for Blind Signal Separation," Neurocomputing, Vol.49, pp.315-327, 2002. https://doi.org/10.1016/S0925-2312(02)00529-5
  15. Y. R. Park, T. J. Murray, and C. Chen, "Predicting Sun Spots Using A Layered Perceptron Neural Networks," IEEE Trans. Neural Networks, Vol.7, pp. 501-505, 1996(3). https://doi.org/10.1109/72.485683
  16. T.-W. Lee, "A Unifying information- theoretic framework for Independent Component Analysis," Computers & Mathematics with Applications, Vol.31, No.11, pp.1-21, 2000(3).
  17. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, Vol.290, No.22, pp.2319-2323, 2000(12). https://doi.org/10.1126/science.290.5500.2319
  18. S. T. Roweis and L. K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, Vol.290, No.22, pp.2323-2326, 2000(12). https://doi.org/10.1126/science.290.5500.2323