Journal of Internet of Things and Convergence (사물인터넷융복합논문지)

The Korea Internet of Things Society (한국사물인터넷학회)
권호 표지
DOI QR코드

DOI QR Code

A Research on Low-power Buffer Management Algorithm based on Deep Q-Learning approach for IoT Networks

IoT 네트워크에서의 심층 강화학습 기반 저전력 버퍼 관리 기법에 관한 연구

  • Song, Taewon (Department of IoT, Soonchunhyang University)
  • 송태원 (순천향대학교 사물인터넷학과)
  • Received : 2022.07.05
  • Accepted : 2022.08.22
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the number of IoT devices increases, power management of the cluster head, which acts as a gateway between the cluster and sink nodes in the IoT network, becomes crucial. Particularly when the cluster head is a mobile wireless terminal, the power consumption of the IoT network must be minimized over its lifetime. In addition, the delay of information transmission in the IoT network is one of the primary metrics for rapid information collecting in the IoT network. In this paper, we propose a low-power buffer management algorithm that takes into account the information transmission delay in an IoT network. By forwarding or skipping received packets utilizing deep Q learning employed in deep reinforcement learning methods, the suggested method is able to reduce power consumption while decreasing transmission delay level. The proposed approach is demonstrated to reduce power consumption and to improve delay relative to the existing buffer management technique used as a comparison in slotted ALOHA protocol.

IoT 네트워크에서 클러스터와 싱크 노드 사이의 게이트웨이 역할을 하는 클러스터 헤드의 전력 관리는 IoT 단말의 수가 증가함에 따라 점점 더 중요해지고 있다. 특히 클러스터 헤드가 이동성을 가진 무선 단말인 경우, IoT 네트워크의 수명을 위하여 전력 소모를 최소화할 필요가 있다. 또한 IoT 네트워크에서의 전송 딜레이는 IoT 네트워크에서의 빠른 정보 수집을 위한 주요한 척도 중 하나이다. 본 논문에서는 IoT 네트워크에서 정보의 전송 딜레이를 고려한 저전력 버퍼 관리 기법을 제안한다. 제안하는 기법에서는 심층 강화학습 방법에서 사용되는 심층 Q 학습(Deep Q learning)를 사용하여 수신된 패킷을 포워딩하거나 폐기함으로써 전송 딜레이를 줄이면서도 소비 전력을 절약할 수 있다. 제안한 알고리즘은 비교에 사용된 기존 버퍼 관리 기법과 비교하여 Slotted ALOHA 프로토콜 기준 소모 전력 및 딜레이를 개선함을 보였다.

Keywords

Acknowledgement

본 논문은 2022년도 교육부의 재원으로 한국연구재단의 지원을 받아 수행된 지자체-대학 협력기반 지역혁신 사업의 결과입니다. (2021RIS-004)

References

  1. M. T. Lazarescu, "Design of a wsn platform for long-term environmental monitoring for iot applications," IEEE Journal on emerging and selected topics in circuits and systems, Vol.3, No.1, pp.45-54, 2013. https://doi.org/10.1109/JETCAS.2013.2243032
  2. W. R. Heinzelman, A. Chandrakasan, and H. Balakrishnan, "Energy-efficient communication protocol for wireless microsensor networks." IEEE Proceedings of the 33rd annual Hawaii international conference on system sciences, p.223, 2000.
  3. I. Gupta, D. Riordan, and S. Sampalli, "Cluster-head election using fuzzy logic for wireless sensor networks," IEEE 3rd Annual Communication Networks and Services Research Conference (CNSR'05), 2005.
  4. A. K. Dwivedi and A. K. Sharma, "EE-LEACH: energy enhancement in LEACH using fuzzy logic for homogeneous WSN," Wireless Personal Communications, Vol.120, pp.3035-3055, 2021. https://doi.org/10.1007/s11277-021-08598-7
  5. B. Manzoor, N. Javaid, O. Rehman, M. Akbar, Q. Nadeem, A. Iqbal, and M. Ishfaq, "Q-LEACH: A new routing protocol for WSNs," Vol.19. pp.926-931, 2013.
  6. M. E. Haque, T. Hossain, M. R. Sarker, M. Paul, M. S. Hoque, S. Uddin, A. A. Suman, M. H. M. Saad, and T. U. Huque, "A hybrid approach to enhance the lifespan of wsns in nuclear power plant monitoring system," Scientific Reports, Vol.12(1), pp.1-14, 2022. https://doi.org/10.1038/s41598-021-99269-x
  7. G. A. Senthil, A. Raaza, and N. Kumar, "Internet of things energy efficient cluster-based routing using hybrid particle swarm optimization for wireless sensor network," Wireless Personal Communications, Vol.122, pp.2603-2619, 2022. https://doi.org/10.1007/s11277-021-09015-9
  8. R. Maheswar, P. Jayarajan, S. Vimalraj, G. Sivagnanam, V. Sivasankaran, and I. S. Amiri, "Energy efficient real time environmental monitoring system using buffer management protocol; energy efficient real time environmental monitoring system using buffer management protocol," IEEE 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2018.
  9. D. Hosahalli and K. G. Srinivas, "Enhanced reinforcement learning assisted dynamic power management model for internet-of-things centric wireless sensor network," IET Communications, Vol.14, pp.3748-3760, 2020. https://doi.org/10.1049/iet-com.2020.0026
  10. H. A. Alwasef, "An energy-efficient buffer management scheme based on data integrity and multivariate data reduction for wireless sensor networks," Journal of Control Engineering and Applied Informatics, Vol.23, No.3, pp.53-61, 2021.
  11. S. Merlin, G. Barriac, H. Sampath, L. Cariou, T. Derham, J.-P. L. Rouzic, R. Stacey, M. Park, C. Ghosh, R. Porat, N. Jindal, Y. Inoue, Y. Asai, Y. Takatori, A. Kishida, A. Yamada, R. Hedayat, S. Choudhury, K. Doppler, J. Kneckt, E.-H. Rantala, D. X. Yang, Y. (Ross), Z. Lan, J. Zhang, Y. Li, Y. Li, J. Pang, H. Su, Y. Lin, W. Lee, H. Cho, S. Kim, H. Choi, J. Levy, F. L. Sita, J. Jiang, L. Chu, Y. Sun, F. Mestanov, G. Li, S. Marin, E. Sakai, W. Carney, B. Sun, K. Lv, Y. Ke, H. Zhiqiang, C.-C. Wang, R. Huang, C. Yu, J. Yee, E. Wong, J. Kim, and X. Wang, "TGax Simulation Scenarios." [Online]. Available: https://mentor.ieee.org/802.11/dcn/14/11-14-0980-16-00ax-simulation-scenarios.docx
  12. "Gym Documentation." [Online]. Available: https://www.gymlibrary.ml
  13. "NumPy." [Online]. Available: https://numpy.org
  14. "PyTorch." [Online]. Available: https://pytorch.org
  15. T. J. Ott, T. V. Lakshman, and L. H. Wong, "Sred: Stabilized red," IEEE INFOCOM'99, Vol.3, pp.1346-1355, 1999.
  16. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," 2014. [Online]. Available: https://arxiv.org/abs/1412.6980