Journal of Convergence for Information Technology (융합정보논문지)

Convergence Society for SMB (중소기업융합학회)
권호 표지
DOI QR코드

DOI QR Code

Q-NAV: NAV Setting Method based on Reinforcement Learning in Underwater Wireless Networks

Q-NAV: 수중 무선 네트워크에서 강화학습 기반의 NAV 설정 방법

  • Park, Seok-Hyeon (Department of Computer Science, Chungbuk National University) ;
  • Jo, Ohyun (Department of Computer Science, Chungbuk National University)
  • 박석현 (충북대학교 소프트웨어학과) ;
  • 조오현 (충북대학교 소프트웨어학과)
  • Received : 2020.03.26
  • Accepted : 2020.06.20
  • Published : 2020.06.28
Download PDF
⟨ Previous Next ⟩

Abstract

The demand on the underwater communications is extremely increasing in searching for underwater resources, marine expedition, or environmental researches, yet there are many problems with the wireless communications because of the characteristics of the underwater environments. Especially, with the underwater wireless networks, there happen inevitable delay time and spacial inequality due to the distances between the nodes. To solve these problems, this paper suggests a new solution based on ALOHA-Q. The suggested method use random NAV value. and Environments take reward through communications success or fail. After then, The environments setting NAV value from reward. This model minimizes usage of energy and computing resources under the underwater wireless networks, and learns and setting NAV values through intense learning. The results of the simulations show that NAV values can be environmentally adopted and select best value to the circumstances, so the problems which are unnecessary delay times and spacial inequality can be solved. Result of simulations, NAV time decreasing 17.5% compared with original NAV.

수중 자원 탐색 및 해양 탐사, 환경 조사 등 수중 통신에 대한 수요가 급격하게 증가하고 있다. 하지만 수중 무선 통신을 사용하기 앞서 많은 문제점을 가지고 있다. 특히 수중 무선 네트워크에서 환경적 요인으로 인해 불가피하게 발생하는 불필요한 지연 시간과 노드 거리에 따른 공간적 불평등 문제가 존재한다. 본 논문은 이러한 문제를 해결하기 위해 ALOHA-Q를 기반으로 한 새로운 NAV 설정 방법을 제안한다. 제안 방법은 NAV 값을 랜덤하게 사용하고 통신 성공, 실패 유무에 따라 보상을 측정한다. 이후 보상 값에 따라 NAV 값을 설정 한다. 수중 무선 네트워크에서 에너지와 컴퓨팅 자원을 최대한 낮게 사용하면서 NAV 값을 강화 학습을 통하여 학습하고 한다. 시뮬레이션 결과 NAV 값이 해당 환경에 적응하고 최선의 값을 선택하여 불필요한 지연 시간문제와 공간적 불평등 문제를 해결할 수 있음을 보여준다. 시뮬레이션 결과 설정한 환경 내에서 기존 NAV 설정 시간 대비 약 17.5%의 시간을 감소하는 것을 보여준다.

Keywords

References

  1. https://water.usgs.gov/edu/earthhowmuch.html.
  2. E. M. ozer, M. Stojanovic & J. G. Proakis. (2000). Underwater acoustic networks. IEEE journal of oceanic engineering, 25(1), 72-83. DOI : 10.1109/48.820738
  3. I. F. Akyildiz, D. Pompili & T. Melodia. (2004). Challenges for efficient communication in underwater acoustic sensor networks. ACM Sigbed Review, 1(2), 3-8. DOI : 10.1145/1121776.1121779
  4. F. Yunus, S. H. Ariffin & Y. Zahedi. (2010, May). A survey of existing medium access control (MAC) for underwater wireless sensor network (UWSN). In 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation (pp. 544-549). IEEE.
  5. X. Guo, M. R. Frater & M. J. Ryan. (2009). Design of a propagation-delay-tolerant MAC protocol for underwater acoustic sensor networks. IEEE Journal of Oceanic Engineering, 34(2), 170-180. DOI : 10.1109/JOE.2009.2015164
  6. D. Shin & D. Kim. (2008, October). A dynamic NAV determination protocol in 802.11 based underwater networks. In 2008 IEEE International Symposium on Wireless Communication Systems (pp. 401-405). IEEE.
  7. Y. D. Chen, C. C. Li, R. T. Dai & K. P. Shih. (2011, September). On enhancing four-way handshake with stair-like NAV setting for underwater acoustic networks. In OCEANS'11 MTS/IEEE KONA (pp. 1-6). IEEE.
  8. Y. Chu, S. Kosunalp, P. D. Mitchell, D. Grace & T. Clarke. (2015). Application of reinforcement learning to medium access control for wireless sensor networks. Engineering Applications of Artificial Intelligence, 46, 23-32. DOI : 10.1016/j.engappai.2015.08.004
  9. S. H. Park, P. D. Mitchell & D. Grace. (2019). Reinforcement Learning Based MAC Protocol (UW-ALOHA-Q) for Underwater Acoustic Sensor Networks. IEEE Access, 7, 165531-165542. https://doi.org/10.1109/access.2019.2953801
  10. T. Lee & O. J. Shin. (2020). CoRL: Collaborative Reinforcement Learning-Based MAC Protocol for IoT Networks. Electronics, 9(1), 143. DOI : 10.3390/electronics9010143
  11. IEEE Computer Society LAN MAN Standards Committee. (1999). Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. ANSI/IEEE Std. 802. 11-1999.