Journal of Information Processing Systems

  • 제18권6호
  • /
  • Pages.803-812
  • /
  • 2022
  • /
  • 1976-913X(pISSN)
  • /
  • 2092-805X(eISSN)
한국정보처리학회 (Korea Information Processing Society)
권호 표지
DOI QR코드

DOI QR Code

Deep Reinforcement Learning-Based Edge Caching in Heterogeneous Networks

  • Yoonjeong, Choi (Dept. of IT Engineering, Sookmyung Women's University) ;
  • Yujin, Lim (Dept. of IT Engineering, Sookmyung Women's University)
  • 투고 : 2022.08.08
  • 심사 : 2022.10.13
  • 발행 : 2022.12.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

With the increasing number of mobile device users worldwide, utilizing mobile edge computing (MEC) devices close to users for content caching can reduce transmission latency than receiving content from a server or cloud. However, because MEC has limited storage capacity, it is necessary to determine the content types and sizes to be cached. In this study, we investigate a caching strategy that increases the hit ratio from small base stations (SBSs) for mobile users in a heterogeneous network consisting of one macro base station (MBS) and multiple SBSs. If there are several SBSs that users can access, the hit ratio can be improved by reducing duplicate content and increasing the diversity of content in SBSs. We propose a Deep Q-Network (DQN)-based caching strategy that considers time-varying content popularity and content redundancy in multiple SBSs. Content is stored in the SBS in a divided form using maximum distance separable (MDS) codes to enhance the diversity of the content. Experiments in various environments show that the proposed caching strategy outperforms the other methods in terms of hit ratio.

키워드

과제정보

This paper is the extended version of paper published in the Annual Spring Conference in KIPS2022 on May 19-21, 2022; titled "Edge Caching Based on Reinforcement Learning Considering Edge Coverage Overlap in Vehicle Environment" by Y. Choi and Y. Lim. This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the ICT Challenge and Advanced Network of HRD (ICAN) program (No. IITP-2022-RS-2022-00156299) supervised by the Institute of Information & Communications Technology Planning & Evaluation (IITP), and also supported by a National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1F1A1047113).

참고문헌

  1. Cisco, "Cisco Annual Internet Report (2018-2023)," 2022 [Online]. Available: https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.pdf
  2. N. Abbas, Y. Zhang, A. Taherkordi, and T. Skeie, "Mobile edge computing: a survey," IEEE Internet of Things Journal, vol. 5, no. 1, pp. 450-465, 2018. https://doi.org/10.1109/JIOT.2017.2750180
  3. P. Lin, Q. Song, and A. Jamalipour, "Multidimensional cooperative caching in CoMP-integrated ultra-dense cellular networks," IEEE Transactions on Wireless Communications, vol. 19, no. 3, pp. 1977-1989, 2020. https://doi.org/10.1109/twc.2019.2960329
  4. J. Yao, T. Han, and N. Ansari, "On mobile edge caching," IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2525-2553, 2019. https://doi.org/10.1109/COMST.2019.2908280
  5. J. Chen, H. Wu, P. Yang, F. Lyu, and X. Shen, "Cooperative edge caching with location-based and popular contents for vehicular networks," IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 10291-10305, 2020. https://doi.org/10.1109/tvt.2020.3004720
  6. C. Song, W. Xu, T. Wu, S. Yu, P. Zeng, and N. Zhang, "QoE-driven edge caching in vehicle networks based on deep reinforcement learning," IEEE Transactions on Vehicular Technology, vol. 70, no. 6, pp. 5286-5295, 2021. https://doi.org/10.1109/TVT.2021.3077072
  7. R. Wang, Z. Kan, Y. Cui, D. Wu, and Y. Zhen, "Cooperative caching strategy with content request prediction in Internet of vehicles," IEEE Internet of Things Journal, vol. 8, no. 11, pp. 8964-8975, 2021. https://doi.org/10.1109/JIOT.2021.3056084
  8. G. Qiao, S. Leng, S. Maharjan, Y. Zhang, and N. Ansari, "Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks," IEEE Internet of Things Journal, vol. 7, no. 1, pp. 247-257, 2020. https://doi.org/10.1109/jiot.2019.2945640
  9. L. T. Tan and R. Q. Hu, "Mobility-aware edge caching and computing in vehicle networks: a deep reinforcement learning," IEEE Transactions on Vehicular Technology, vol. 67, no. 11, pp. 10190-10203, 2018. https://doi.org/10.1109/tvt.2018.2867191
  10. L. T. Tan, R. Q. Hu, and L. Hanzo, "Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks," IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3086-3099, 2019. https://doi.org/10.1109/tvt.2019.2893898
  11. F. Rezaei, B. H. Khalaj, M. Xiao, and M. Skoglund, "Performance analysis of heterogeneous cellular caching networks with overlapping small cells," IEEE Transactions on Vehicular Technology, vol. 71, no. 2, pp. 1941-1951, 2022. https://doi.org/10.1109/TVT.2021.3132648
  12. I. Keshavarzian, Z. Zeinalpour-Yazdi, and A. Tadaion, "Energy-efficient mobility-aware caching algorithms for clustered small cells in ultra-dense networks," IEEE Transactions on Vehicular Technology, vol. 68, no. 7, pp. 6833-6846, 2019. https://doi.org/10.1109/tvt.2019.2918712
  13. S. Gao, P. Dong, Z. Pan, and G. Y. Li, "Reinforcement learning based cooperative coded caching under dynamic popularities in ultra-dense networks," IEEE Transactions on Vehicular Technology, vol. 69, no. 5, pp. 5442-5456, 2020. https://doi.org/10.1109/tvt.2020.2979918
  14. L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker, "Web caching and Zipf-like distributions: evidence and implications," in Proceedings of IEEE Conference on Computer Communications (INFOCOM'99), New York, NY, 1999, pp. 126-134.
  15. K. Shanmugam, N. Golrezaei, A. G. Dimakis, A. F. Molisch, and G. Caire, "Femtocaching: wireless content delivery through distributed caching helpers," IEEE Transactions on Information Theory, vol. 59, no. 12, pp. 8402-8413, 2013. https://doi.org/10.1109/TIT.2013.2281606
  16. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing Atari with deep reinforcement learning," 2013 [Online]. Available: https://arxiv.org/abs/1312.5602.
  17. M. Kamel, W. Hamouda, and A. Youssef, "Ultra-dense networks: a survey," IEEE Communications Surveys & Tutorials, vol. 18, no. 4, pp. 2522-2545, 2016.  https://doi.org/10.1109/COMST.2016.2571730