KIISE Transactions on Computing Practices (정보과학회 컴퓨팅의 실제 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Predictive Convolutional Networks for Learning Stream Data

스트림 데이터 학습을 위한 예측적 컨볼루션 신경망

  • Received : 2015.11.11
  • Accepted : 2016.07.27
  • Published : 2016.11.15
⟨ Previous Next ⟩

Abstract

As information on the internet and the data from smart devices are growing, the amount of stream data is also increasing in the real world. The stream data, which is a potentially large data, requires online learnable models and algorithms. In this paper, we propose a novel class of models: predictive convolutional neural networks to be able to perform online learning. These models are designed to deal with longer patterns as the layers become higher due to layering convolutional operations: detection and max-pooling on the time axis. As a preliminary check of the concept, we chose two-month gathered GPS data sequence as an observation sequence. On learning them with the proposed method, we compared the original sequence and the regenerated sequence from the abstract information of the models. The result shows that the models can encode long-range patterns, and can generate a raw observation sequence within a low error.

인터넷 상 데이터와 스마트 디바이스가 증가함에 따라 순차적으로 유입되는 스트림 형식의 데이터가 늘어나고 있다. 잠재적인 빅데이터인 스트림 데이터를 다루기 위해서는 온라인 학습이 가능해야 한다. 이에 본 고에서는 스트림 데이터 학습을 위한 새로운 모델인 예측적 컨볼루션 신경망과 온라인 학습방법을 제안한다. 이 모델은 탐지와 풀링을 반복하는 컨볼루션 연산을 통해 탐지 패턴을 계층화하여 상위 계층이 될수록 긴 길이의 패턴의 정보를 다루도록 한다. 본 모델의 기초적 검증을 위해 스마트폰으로 2달간 수집한 GPS 데이터를 이산화하여 관측데이터로 삼았다. 이를 제안모델을 통해 학습하여 계층을 따라 추상화된 정보로부터 복원한 데이터와 관측데이터를 비교하여, 긴 시간의 패턴을 다루면서도 관측 수준의 데이터를 복원하는 것을 확인하였다.

Keywords

Acknowledgement

Supported by : 정보통신기술진흥센터, 한국연구재단

References

  1. N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell, A Survey of Mobile Phone Sensing, IEEE Communications Magazine, Vol. 48, No. 9, pp. 140-150, 2010.
  2. Simon Haykin, Neural Networks and Learning Machine, Prentice Hall, 2009.
  3. D. Barber, T. Cemgil, and S. Chiappa, Bayesian Time Series Models, Cambridge University Press, 2011.
  4. S. Shalev-Shwartz, Online Learning and Online Convex Optimization, Foundations and $Trends^{(R)}$ in Machine Learning, Vol. 4, No. 2, pp. 107-194, 2012.
  5. G. Widmer and M. Kurat, Learning in the Presence of Concept Drift and Hidden Contexts, Machine Learning, Vol. 23, pp. 69-101, 1996.
  6. I. Zliobaite, Learning under Concept Drift: an Overview, Technical Report, Faculty of Mathematics and Informatics, Vilnius University, 2009.
  7. Y. LeCun, L. Bottou, Y. Bengio, & P. Haffner, Gradient-based learning applied to document recognition, Proc, of the IEEE, 86(11), 2278-2324, 1998. https://doi.org/10.1109/5.726791
  8. Lee, Honglak, et al. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, Proc. of the 26th Annual International Conference on Machine Learning (ICML), 2009.
  9. Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks, The Twenty-sixth Annual Conference on neural information processing systems (NIPS 2012), 2012.
  10. J. Masci, et al. Stacked convolutional auto-encoders for hierarchical feature extraction, Artificial Neural Networks and Machine Learning (ICANN 2011), 52-59, Springer Berlin Heidelberg, 2011.
  11. A. Makhzani and B. Frey, k-Sparse Autoencoders, Proc. of the 2nd International Conference on Learning Representations (ICLR 2014), 2014.
  12. A. Makhzani and B. Frey, Winner-Take-All Autoencoders, The Twenty-ninth Annual Conference on neural information processing systems (NIPS 2015), 2015.
  13. M.-O. Heo, M. Kang, B.-K. Lim, K.-B. Hwang, Y.-T., Park, and B.-T. Zhang, Real-time route inference and learning for smartphone users using probabilistic graphical models, Journal of the Korea Information Science Society: Software and Applications, 39(6):425-435, 2012.