Journal of Korean Institute of Industrial Engineers (대한산업공학회지)

Korean Institute of Industrial Engineers (대한산업공학회)
권호 표지

Analysis of Novelty Detection Properties of Autoassociative MLP

자기연상 다층퍼셉트론의 이상 탐지 성질 분석

  • Lee, Hyoung-joo (Department of Industrial Engineering, Seoul National University) ;
  • Hwang, Byung-ho (Digital Media Research Lab, LG Electronics) ;
  • Cho, Sungzoon (Department of Industrial Engineering, Seoul National University)
  • Published : 2002.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In novelty detection, one attempts to discriminate abnormal patterns from normal ones. Novelty detection is quite difficult since, unlike usual two class classification problems, only normal patterns are available for training. Auto-Associative Multi-Layer Perceptron (AAMLP) has been shown to provide a good performance based upon the property that novel patterns usually have larger auto-associative errors. In this paper, we give a mathematical analysis of 2-layer AAMLP's output characteristics and empirical results of 2-layer and 4-layer AAMLPs. Various activation functions such as linear, saturated linear and sigmoid are compared. The 2-layer AAMLPs cannot identify non-linear boundaries while the 4-layer ones can. When the data distribution is multi-modal, then an ensemble of AAMLPs, each of which is trained with pre-clustered data is required. This paper contributes to understanding of AAMLP networks and leads to practical recommendations regarding its use.

Keywords

Acknowledgement

Supported by : 과학기술부, 서울대학교

References

  1. Baldi, P. and Hornik, K. (1989), Neural Networks and Principal Component Analysis: Learning from Examples without Local Minima, Neural Networks, 2, 53-58
  2. Bianchini, M., Frasconi, P. and Gori, M. (1995), Learning in Multi-layered Networks used as Autoassociators, IEEE Transactions on Neural Networks, 6(2),512-515 https://doi.org/10.1109/72.363492
  3. Bishop, C. M. (1994), Novelty Detection and Neural Network Validation, lEE Proceedings of Visual Image Signal Process, 141(4),217-222 https://doi.org/10.1049/ip-vis:19941330
  4. Cho, S., Han, C., Han, D. and Kim, H. (2000a), Web Based Keystroke Dynamics Identity Verification using Neural Networks, Journal of Organizational Computing and Electronic Commerce. 10(4),295-307
  5. Cho, S. and Han, D. (2000b), Apparatus for Authenticating an Individual Based on a Typing Pattern by Using a Neural Network System, Patent No. 6,151,593, US Patent and Trademark Office, Washington DC 20231, USA, Granted on Nov. 21, 2000
  6. Frosini, A., Gori, M. and Priami, P. (1996), A Neural Network based Model for Paper Currency Recognition and Verification, IEEE Transactions on Neural Networks, 7(6),1482-1490 https://doi.org/10.1109/72.548175
  7. Ikbal, M., Misra, H. and Yegnanarayana, B. (1999), Analysis of Autoassociative Mapping Neural Networks, Proceedings of International Joint Conference on Neural Networks, #854
  8. Japkowicz, N., Myers, C. and Gluck, M. (1995), A Novelty Detection Approach to Classification, Proceedings of the Fourteenth International Conference on Artificial Intelligence, 518-523
  9. Kaski, S. and Lagus, K. (1996), Comparing Self-organizing Maps, Proceedings of International Conference on Artificial Neural Networks,809-814
  10. Kramer, M. (1991), Nonlinear Principal Component Analysis using Autoassociative Neural Networks, AIChE Journal, 37(2), 233-243 https://doi.org/10.1002/aic.690370209
  11. Malthouse, E. C. (1998), Limitations of Non-linear PCA as Performed with Generic Neural Networks, IEEE Transactions on Neural Networks, 9(1), 165-173 https://doi.org/10.1109/72.655038
  12. Obaidat, M. S. and Sadoun, B. (1997), Verification of Computer Users using Keystroke Dynamics, IEEE Transactions on Systems, Man, and Cybernetics, 27(2),261-269
  13. Reddy, V., Riley, P. and Mavrovouniotis, M. (1996), Analysis of Plant Measurements through Input-training Neural Networks, Computers Chemical Engineering, 20., Suppl., 889-894
  14. Xu, L. and Jordan, M. I. (1993), Unsupervised Learning by EM Algorithm based on Finite Mixture of Gaussians, Proceedings of World Congress of Neural Networks, 431-434
  15. Yin, H. and Allinson, N. M. (2001), Self-organizing Mixture Networks for Probability Density Estimation, IEEE Transactions on Neural Networks, 12(2), 405-411 https://doi.org/10.1109/72.914534