Information and Communications Magazine (정보와 통신)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지

기계학습 및 딥러닝 기술동향

  • Published : 2016.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

본 논문에서는 패턴 인식 및 회귀 문제를 풀기 위해 쓰이는 기계학습에 대한 전반적인 이론과 설계방법에 대해 알아본다. 대표적인 기계학습 방법인 신경회로망과 기저벡터머신 등에 대해 소개하고 이러한 기계학습 모델을 선택하고 구축하는 데에 있어 고려해야 하는 문제점들에 대해 이야기 한다. 그리고 특징 추출 과정이 기계학습 모델의 성능에 어떻게 영향을 미치는지, 일반적으로 특징 추출을 위해 어떤 방법들이 사용되는 지에 대해 알아본다. 또한, 최근 새로운 패러다임으로 대두되고 있는 딥러닝에 대해 소개한다. 자가인코더, 제한볼츠만기계, 컨볼루션신경회로망, 회귀신경회로망과 같이 딥러닝 기술이 적용된 대표적인 신경망 구조에 대해 설명하고 기존의 기계학습 모델과 비교하여 딥러닝이 가지고 있는 특장점을 알아본다.

Keywords

References

  1. McCulloch, W. A., and Pitts, W. "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115-133, 1943. https://doi.org/10.1007/BF02478259
  2. Rosenblatt, F. "The perceptron: a probabilistic model for information storage and organization in the brain," Psychological Review, vol. 65, no. 6, pp. 386-408, 1958. https://doi.org/10.1037/h0042519
  3. Minsky, M., and Papert, S., Perceptrons, MIT Press, 1968.
  4. Leshno, M., Ya, V., Pinkus, A., and Schocken, S. "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Networks, vol. 6, no. 6, pp. 861-867, 1993. https://doi.org/10.1016/S0893-6080(05)80131-5
  5. Charalambous, C. "Conjugate gradient algorithm for efficient training of artificial neural networks," in IEE Proceedings of Part G (Circuits, Devices and Systems), vol. 139, no. 3, pp. 301-310, 1992. https://doi.org/10.1049/ip-g-2.1992.0050
  6. Dennis, Jr, J. E., and More, J. J. "Quasi-Newton methods, motivation and theory," SIAM Review, vol. 19, no. 1, pp. 46-89, 1977. https://doi.org/10.1137/1019005
  7. More, J. J. "The Levenberg-Marquardt algorithm: implementation and theory," Numerical Analysis, Springer, 1978, pp. 105-116.
  8. Cortes, C., and Vapnik, V. "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995. https://doi.org/10.1007/BF00994018
  9. Blumer, A., Ehrenfeucht, A., Haussler, D., and Warmuth, M. K. "Occam's razor," Readings in Machine Learning, 1990, pp. 201-204.
  10. Domingos, P. "A few useful things to know about machine learning," Communications of the ACM, vol. 55, no. 10, pp. 78-87, 2012. https://doi.org/10.1145/2347736.2347755
  11. Hughes, G. "On the mean accuracy of statistical pattern recognizers," IEEE Transactions on Information Theory, vol. 14, no. 1, pp. 55-63, 1968. https://doi.org/10.1109/TIT.1968.1054102
  12. Jolliffe, I., Principal Component Analysis, John Wiley & Sons, 2002.
  13. Scholkopft, B., and Mullert, K. R. "Fisher discriminant analysis with kernels," Neural Networks for Signal Processing IX, vol. 1, no. 1, pp. 41-48, 1999.
  14. Imai, S. "Cepstral analysis synthesis on the mel frequency scale," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 8, 1983, pp. 93-96.
  15. Dalal, N., and Triggs, B. "Histograms of oriented gradients for human detection," in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 2005, pp. 886-893.
  16. Vincent, P., Larochelle, H., Lajoie, I., Bengio Y., and Manzagol. P. A. "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," Journal of Machine Learning Research, vol. 11, pp. 3371-3408, 2010.
  17. Hinton, G. E., Osindero, S., and Teh, Y. W. "A fast learning algorithm for deep belief nets," Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006. https://doi.org/10.1162/neco.2006.18.7.1527
  18. Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  19. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., and Khudanpur, S. "Extensions of recurrent neural network language model," in Proceedings of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011, pp. 5528-5531.
  20. Dong, C., Loy, C., He, K., and Tang, X. "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 2016. https://doi.org/10.1109/TPAMI.2015.2439281
  21. Long, J., Shelhamer, E., and Darrell, T. "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
  22. Krizhevsky, A., Sutskever, I., and Hinton, G. E. "Imagenet classification with deep convolutional neural networks," In Advances in Neural Information Processing Systems, 2012, pp. 1097-1105.
  23. Lin, M., Chen, Q., and Yan, S. "Network in network," arXiv preprint, arXiv: 1312.4400, 2013.
  24. Simonyan, K., and Zisserman, A. "Very deep convolutional networks for large-scale image recognition," arXiv preprint, arXiv: 1409.1556, 2014.
  25. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., and Rebinovich, A. "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. pp. 1-9.
  26. He, K., Zhang, X., Ren, S., and Sun, J. "Deep residual learning for image recognition," arXiv preprint, arXiv: 1512.03385, 2015.
  27. Gers, F. A., Schmidhuber, J., and Cummins, F. "Learning to forget: Continual prediction with LSTM," Neural Computation, vol. 12, no. 10, pp. 2451-2471, 2000. https://doi.org/10.1162/089976600300015015
  28. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. "Gated feedback recurrent neural networks," in Proceedings of the 32nd International Conference on Machine Learning, 2015, pp. 2067-2075.
  29. Graves, A., Jaitly, N., and Mohamed, A. R. "Hybrid speech recognition with deep bidirectional LSTM," in Automatic Speech Recognition and Understanding, 2013, pp. 273-278.
  30. Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., and Socher, R. "Ask me anything: Dynamic memory networks for natural language processing," arXiv preprint, arXiv: 1506.07285.
  31. Pan, S. J., and Yang, Q. "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345-1359, 2010. https://doi.org/10.1109/TKDE.2009.191
  32. Alemi, A.(2016, Aug 31). "Improving inception and image classification in Tensor Flow," Google Research Blog (https://research.googleblog.com/2016/08/improving-inception-and-image.html)
  33. Silver, D. et al. "Mastering the game of Go with deep neural network and tree search," Nature, vol. 529, no. 7287, pp. 484-489. https://doi.org/10.1038/nature16961
  34. Oord, A. et al. "Wavenet: A generative model for raw audio," arXiv preprint, arXiv: 1609.03499.
  35. Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. "Show and tell: A neural image caption generator," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3156-3164.