On-line Vector Quantizer Design Using Stochastic Relaxation

Stochastic Relaxation 방법을 이용한 온라인 벡터 양자화기 설계

  • Song, Geun-Bae (Department of Electronics Engineering, Graduate School, Ajou University) ;
  • Lee, Haing-Sei (Department of Electronics Engineering, Ajou University)
  • 송근배 (아주대학교 인지공학실험실) ;
  • 이행세 (아주대학교 전자공학부)
  • Published : 2001.09.25

Abstract

This paper proposes new design algorithms based on stochastic relaxation (SR) for an on-line vector quantizer (VQ) design. These proposed SR methods solve the local entrapment problems of the conventional Kohonen learning algorithm (KLA). These SR methods cover two different types depending upon the use of simulated annealing (SA) : the one that uses SA is called the OLVQ SA and the other the OLVQ SR. These methods arc combined with the KLA and therefore preserve the its convergence properties. Experimental results for Gauss Markov sources, real speech and image demonstrate that the proposed algorithms can consistently provide better codebooks than the KLA.

본 논문은 온라인 벡터 양자화기 설계에 stochastic relaxation (SR) 개념을 응용함으로써 SR 방법에 기초한 새로운 온라인 학습 알고리즘을 제안한다. 이는 전통적인 Kohonen 학습법 (KLA)이 안고 있는 극소점(local minimum)으로의 수렴 문제를 개선시켜준다. SR 방법의 응용은 simulated annealing (SA) 개념을 사용하느냐 안 하느냐에 따라 둘로 나눌 수 있는데, 이를 구분하기 위해 SA 개념을 이용하는 SR 알고리즘을 LOVQ-SA로, SA 개념을 이용하지 않는 알고리즘을 OLVQ SR로 부르기로 한다. 제안된 방법들은 KLA와 결합되어 있으며 KLA의 특성을 보존하도록 설계되었다. 이는 제안된 방법들의 수렴의 속도 및 안정성을 향상시켜준다. 제안된 방법의 우수성을 입증하기 위하여 Gauss-Markov 신호원과 음성 및 영상 자료에 대한 벡터양자화 실험을 하였으며 실험결과를 통하여 제안된 방법이 KLA 보다 일관되게 우수한 코드북을 생성함을 보인다.

Keywords

References

  1. T. Kohonen, Self Organizing Maps, Springer-Verlag, 1995
  2. B. Widrow and S. D. Sterns, Adaptive Signal Processing, Englewood Cliffs, NJ : Prentice Hall, 1985
  3. E. Erwin, K. Obermayer and K. Schulten, 'Self organizing maps: ordering, convergence properties and energy functions,' Biol. Cybern., vol. 67, pp. 47-55, 1992 https://doi.org/10.1007/BF00201801
  4. N. M. Nasrabadi and Y. Feng, 'Vector quantization of images based upon the Kohonen self organization feature maps,' in Proc. 2nd ICNN Conf., vol. 1, pp. 101-105, 1988 https://doi.org/10.1109/ICNN.1988.23837
  5. Jun Wang, Ce Zhu, Chenwu Wu and Zhenya He, 'Neural network approaches to fast and low rate vector quantization,' in Proc, IEEE ISCAS'95 vol. 1, USA, 1995 https://doi.org/10.1109/ISCAS.1995.521556
  6. SriGouri Kamarsu and H. C. Card, 'Vector quantization of speech using artificial neural learning,' in Proc. IEEE Pacific RIM Conf. Communications, Computers & Signal Processing, Canada, 1995 https://doi.org/10.1109/PACRIM.1995.519581
  7. Y. Linde, A. Buzo, and R. M Gray, 'An algorithm for vector quantizer design,' IEEE Trans. Commun., vol. COM 28, pp. 84-95, Jan. 1980
  8. P. -C. Chang and R. M. Gray, 'Gradient algorithms for designing predictive vector quantizers,' IEEE Trans. Acoust., Speech, Signal Processing, vol, ASSP 34, pp.679-690, Aug. 1986
  9. J. M. van Laarhoven and E. H. L. Aarts, Simmulated Annealing : Theory and Applications, Boston: D. Reidel Publishing, 1987
  10. V. Cerny, 'A thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm,' preprint, Inst. Phys. & Biophys., Comenius Univ., Bratislava, 1982
  11. S. Kirkpatrick, C. D. Gellatt, Jr., and M. P. Vecchi, 'Optimization by simulated annealing,' Science, vol. 220, pp. 671-680, May 1983 https://doi.org/10.1126/science.220.4598.671
  12. S. Geman and D. Gernan, 'Stochastic relaxation, Gibbs distributions, and the Baysien restoration of images,' IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, pp.721-741, Nov. 1984
  13. K. Zeger and A. Gersho, 'A stochastic relaxation algorithm for improved vector quantizer design,' Electron. Lett., vol. 25, no. 14, pp. 896-898, July. 1989
  14. K. Zeger, J. Vaisey and A. Gersho, 'Globally optimal vector quantizer design by stochastic relaxation,' IEEE Trans. Signal Proc., vol. 40, no. 2, Feb. 1992 https://doi.org/10.1109/78.124941
  15. P. D. Wasserman, Neural Computing, NY : Van Nostrand Reinhold, 1989
  16. D. DeSieno, 'Adding a conscience to competitive learning,' in Proc 2nd ICNN Conf., vol. 1, pp. 117-124, 1988 https://doi.org/10.1109/ICNN.1988.23839
  17. H. Ritter and K. Shulten, 'Kohonen's self organizing maps : Explorering their computational capabilities,' in Proc 2nd ICNN Conf., vol. 1, pp. 109-116, 1988 https://doi.org/10.1109/ICNN.1988.23838
  18. E. Yair, K. Zeger and A. Gersho, 'Competitive Learning and soft competition for vector quantizer design,' IEEE Trans. Signal Proc., vol. 40, no. 2, Feb. 1992 https://doi.org/10.1109/78.124940
  19. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, 'Equations of state calculations by fast computing machines,' J. Chem. Phys., vol.21, pp. 1087-1091, 1953 https://doi.org/10.1063/1.1699114
  20. L. Holmstrom and P. Koistinen, Using additive in backpropagation training,' IEEE Trans. Neural Networks, vol. 3, 1992 https://doi.org/10.1109/72.105415
  21. Y. Grandvalet and S. Canu, 'A comment on noise injection into inputs in backpropagation learning, IEEE Trans, Syst., Man, Cybern., vol. 25, 1995 https://doi.org/10.1109/21.370200