Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Development of Brain-Style Intelligent Information Processing Algorithm Through the Merge of Supervised and Unsupervised Learning: Generation of Exemplar Patterns for Training

교사학습과 비교사학습의 접목에 의한 두뇌방식의 지능 정보 처리 알고리즘 개발: 학습패턴의 생성

  • 오상훈 (목원대학교 정보통신공학부)
  • Published : 2004.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a new algorithm to generate additional training patterns using the brain-style information processing algorithm, that is, supervised and unsupervised learning models. This will be useful in the case that we do not have enough number of training patterns because of limitation such as time consuming, economic problem, and so on. We adopt the independent component analysis as an unsupervised model for generating exempalr patterns and multilayer perceptions as supervised models for verifying usefulness of the generated patterns. After statistical analysis of the proposed pattern generation algorithm, we verify successful operations of our algorithm through simulation of handwritten digit recognition with various numbers of training patterns.

시간/경제적 문제 혹은 수집 대상의 제한으로 충분한 수의 학습패턴을 모을 수 없는 경우에 인간의 두뇌를 모방한 교사학습 및 비교사학습 모델을 이용하여 새로운 학습패턴을 생성하는 알고리즘을 제안하였다. 비교사학습은 독립성분분석을 사용하여 패턴의 특성을 분석 후 생성하며, 교사학습은 다층퍼셉트론 모델을 사용하여 생성된 패턴의 검증을 하는 단계로 적용되었다. 통계학적으로 이와 같은 형태의 패턴 생성을 분석하였으며, 필기체 숫자의 학습 패턴 수를 변동시키면서 패턴 생성의 효과를 시험패턴에 대한 오인식률로 확인한 결과 성능이 향상됨을 보였다.

Keywords

References

  1. M.Tatsuno, 'Computational Neuroscience-Methods in Neuronal modeling,' Educational Lecture, BSI Retreat, Oiso, Japan, Sept. 18-20, 2000
  2. J. Albus, 'A theory of cerebellar function,' Mathematical Biosciences, vol. 10, pp. 25-61, 1971 https://doi.org/10.1016/0025-5564(71)90051-4
  3. A. J. Bell and T. J. Sejnowski, 'The independent components of natural scenes are edge filters,' Vision Research, vol. 37, pp. 3327-3338, 1997 https://doi.org/10.1016/S0042-6989(97)00121-1
  4. A. Hyvarinen, P. O. Hoyer, and M. Inki, 'Topographic independent component analysis,' Neural Cmputation, vol.13, no.7, pp.1527-1558, 2001 https://doi.org/10.1162/089976601750264992
  5. A. Hyvarinen, P. O. Hoyer, and J. Hurri, 'Extensions of ICA as models of natural images and visual processing,' Proceedings of ICA, April 1-4, 2003, Nara, Japan
  6. J. H. Lee and S.-Y. Lee, 'On the efficient speech feature extraction based on independent component analysis,' Neural Processing Letters, vol. 15, no. 3, pp. 235-245, June 2002 https://doi.org/10.1023/A:1015777200976
  7. U.-M. Bae and S.-Y. Lee, 'A complementary approach to blind signal separation for rea-world speech recognition,' Advances in Neural Information Processing Systems 13, pp, 765-771, Nov. 2000
  8. I. V. Tetko and A. E. P. Villa, 'Efficient partition of learning data sets for neural network training,' Neural Networks, vol. 10, pp. 1361-1374, 1997 https://doi.org/10.1016/S0893-6080(97)00005-1
  9. C. Wang and J. C. Principe, 'Training neural networks with additive noise in the desired signal,' IEEE Trans. Neural Networks, vol. 10, pp. 1511-1517, 1999 https://doi.org/10.1109/72.809097
  10. L. Holmstrom and P. Koistinen, 'Using additive noise in back-propagation training,' IEEE Trans. Neural Networks, vol. 3, pp. 24-38, 1992 https://doi.org/10.1109/72.105415
  11. K. Matsuoka, 'Noise injection into inputs in back-propagation learning,' IEEE Trans. Systems, Man, and Cybernetics, vol. 22, no. 3, pp. 436-440, 1992 https://doi.org/10.1109/21.155944
  12. S.-Y. Yoon and S.-Y. Lee, 'Training algorithm with incomplete data for feed-forward neural networks,' Neural Processing Letters, vol. 10, pp. 171-179, 1999 https://doi.org/10.1023/A:1018772122605
  13. Z. Ghahramani and M. Jordan, 'Supervised learning from incomplete data via an EM approach,' Advances in Neural Information Processing Systems 6, pp. 120-127, Nov. 1994
  14. B. Kamgar-Parsi, B. Kamgar-Parsi, J. E. Dayhoff, and A. K. Jain, 'Improving classification boundaries by exemplar generation for visual pattern discrimination,' Proc. IJCNN2001, vol. 4, pp. 2969-2974 https://doi.org/10.1109/IJCNN.2001.938850
  15. B. A. Olshausen and D. J. Field, 'Emergence of simple-cell receptive field properties by learning a sparse code for natural images,' Nature, vol. 381, pp. 607-609, 13 June, 1996 https://doi.org/10.1038/381607a0
  16. T.-W. Lee, et al., 'A unifying information-theoretic framework for independent component analysis,' Computers & Mathematics with Applications, vol. 31, no. 11, pp. 1-21, March 2000 https://doi.org/10.1016/S0898-1221(00)00101-2
  17. S.-H. Oh, A. Cichocki, S. Choi, S.-I. Amari, and S.-Y. Lee., 'Comparison of ICA/BSS algorithms in noisy environment,' Proc. ICONIP, vol. 2, pp. 1192-1197, Nov. 2000
  18. M. Girolami, A. Cichocki, and S.-I. Amari, 'A common neural-network model for unsupervised exploratory data analysis and independent component analysis,' IEEE Trans. Neural Networks, vol. 9, no. 6, Nov. 1998 https://doi.org/10.1109/72.728398
  19. J. Karhunen, et al., 'A class of neural networks for independent component analysis,' IEEE Trans. Neural Networks, vol. 8, pp. 486-504, 1997 https://doi.org/10.1109/72.572090
  20. S.-H. Oh, 'Improving the error back-propagation algorithm with a modified error function,' IEEE Trans. Neural Networks, vol. 8, pp. 799-803, 1997 https://doi.org/10.1109/72.572117
  21. J. C. Principe, et al., 'Learning from examples with information theoretic criteria,' The Journal of VLSI Signal Proc., special issue on neural networks, vol. 26, pp. 61-77, Aug. 2000 https://doi.org/10.1023/A:1008143417156
  22. R. Kamimura and S. Nakanishi, 'Hidden information maximization for feature detection and rule discovery,' Network: Computation in Neural Systems, vol. 6, pp. 577-602, 1995 https://doi.org/10.1088/0954-898X/6/4/004
  23. J. J. Hull, 'A database for handwritten text recognition research,' IEEE Trans. Pat. Ana. Mach. Int., vol. 15, no. 5, pp. 550-554, May 1994 https://doi.org/10.1109/34.291440