Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

An Intelligent Video Streaming Mechanism based on a Deep Q-Network for QoE Enhancement

QoE 향상을 위한 Deep Q-Network 기반의 지능형 비디오 스트리밍 메커니즘

  • Kim, ISeul (School of Computer Science and Engineering, Kyungpook National University) ;
  • Hong, Seongjun (School of Computer Science and Engineering and Software Technology Research Center, Kyungpook National University) ;
  • Jung, Sungwook (School of Computer Science and Engineering, Kyungpook National University) ;
  • Lim, Kyungshik (School of Computer Science and Engineering and Software Technology Research Center, Kyungpook National University)
  • Received : 2018.01.12
  • Accepted : 2018.01.18
  • Published : 2018.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

With recent development of high-speed wide-area wireless networks and wide spread of highperformance wireless devices, the demand on seamless video streaming services in Long Term Evolution (LTE) network environments is ever increasing. To meet the demand and provide enhanced Quality of Experience (QoE) with mobile users, the Dynamic Adaptive Streaming over HTTP (DASH) has been actively studied to achieve QoE enhanced video streaming service in dynamic network environments. However, the existing DASH algorithm to select the quality of requesting video segments is based on a procedural algorithm so that it reveals a limitation to adapt its performance to dynamic network situations. To overcome this limitation this paper proposes a novel quality selection mechanism based on a Deep Q-Network (DQN) model, the DQN-based DASH ABR($DQN_{ABR}$) mechanism. The $DQN_{ABR}$ mechanism replaces the existing DASH ABR algorithm with an intelligent deep learning model which optimizes service quality to mobile users through reinforcement learning. Compared to the existing approaches, the experimental analysis shows that the proposed solution outperforms in terms of adapting to dynamic wireless network situations and improving QoE experience of end users.

Keywords

References

  1. X. Yin, A. Jindal, V. Sekar, and B. Sinopoli, "A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP," Proceedings of the Association for Computing Machinery Conference on Special Interest Group on Data Communication, Vol. 45, Issue 4, pp. 325-338, 2015.
  2. T. Stockhammer, "Dynamic Adaptive Streaming over HTTP-Standards and Design Principles," Proceedings of the Second Annual Association for Computing Machinery Conference on Multimedia Systems, pp. 133-144, 2011.
  3. M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hossfeld, and P. Tran-Gia, “A Survey on Quality of Experience of HTTP Adaptive Streaming,” IEEE Communications Surveys and Tutorials, Vol. 17, No. 1, pp. 469-492, 2015. https://doi.org/10.1109/COMST.2014.2360940
  4. D. Zegarra Rodriguez, R. Lopes Rosa, E. Costa Alfaia, J. Issy Abrahao, and G. Bressan, “Video Quality Metric for Streaming Service Using DASH Standard,” IEEE Transactions on Broadcasting, Vol. 62, No. 3, pp. 628-639, 2016. https://doi.org/10.1109/TBC.2016.2570012
  5. J. Kua, G. Armitage, and P. Branch, “A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP,” IEEE Communications Surveys and Tutorials, Vol. 19, No. 3, pp. 1842-1866, 2017. https://doi.org/10.1109/COMST.2017.2685630
  6. Dash Industry Forum Reference Player Documentation, https://github.com/Dash-Industry-Forum/dash.js/wiki, (accessed Jan., 11, 2018).
  7. M. Claeys, S. Latre, J. Famaey, and F. De Turck, “Design and Evaluation of a Self-Learning HTTP Adaptive Video Streaming Client,” IEEE Communications Letters, Vol. 18, No. 4, pp. 716-719, 2014. https://doi.org/10.1109/LCOMM.2014.020414.132649
  8. L.P. Kaelbling, M.L. Littman, and A.W. Moore, "Reinforcement Learning: A Survey," Journal of Artificial Intelligence Research, Vol. 4, pp. 237-285, 1996.
  9. C.J.C.H. Watkins and P. Dayan, “Q-learning,” Machine Learning, Vol. 8, No. 3-4, pp. 279-292, 1992. https://doi.org/10.1023/A:1022676722315
  10. V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, and et al., “Human-level Control through Deep Reinforcement Learning,” Nature, Vol. 518, No. 7540, pp. 529-533, 2015. https://doi.org/10.1038/nature14236
  11. A.C. Dalal, D.R. Musicant, J. Olson, B. McMenamy, S. Benzaid, and B. Kazez, and et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-72, 2007.
  12. P. Juluri, V. Tamarapalli, and D. Medhi, "QoE Management in DASH Systems Using the Segment Aware Rate Adaptation Algorithm," Proceeding of IEEE/IFIP Network Operations and Management Symposium, pp. 129-136, 2016.
  13. T. Hossfeld, S. Egger, R. Schatz, M. Fiedler, K. Masuch, and C. Lorentzen, "Initial Delay vs. Interruptions: Between the Devil and the Deep Blue Sea," Proceeding of 2012 Fourth International Workshop on Quality of Multimedia Experience, pp. 1-6, 2012.
  14. A. Dalal, D.R. Musicant, J. Olson, B. Mc Menamy, S. Benzaid, and B. Kazez, and et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-67, 2007.
  15. S. Adam, L. Busoniu, and R. Babuska, “Experience Replay for Real-Time Reinforcement Learning Control,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 2, pp. 201-212, 2012. https://doi.org/10.1109/TSMCC.2011.2106494
  16. T.R. Henderson, M. Lacage, and G.F. Riley, "Network Simulations with the ns-3 Simulator," Proceeding of Association for Computing Machinery Conference on Special Interest Group on Data Communication, pp. 527, 2008.
  17. P. Juluri, V. Tamarapalli, and D. Medhi, "QoE Management in DASH Systems Using the Segment Aware Rate Adaptation Algorithm," Proceeding of IEEE/IFIP Network Operations and Management Symposium, pp. 129-136, 2016.
  18. H.V. Hasselt, A. Guez, and D. Silver, "Deep Reinforcement Learning with Double Q-Learning," Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 2094-2100, 2016.
  19. C. Seong, S. Hong, and K. Lim, "A Context-based Adpative Multimedia Streaming Scheme in IoT Environments," Journal of Korea Multimedia Society, Vol. 19, No. 7, pp. 1166-1178, 2016. https://doi.org/10.9717/kmms.2016.19.7.1166