Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Rate Adaptation with Q-Learning in CSMA/CA Wireless Networks

  • Received : 2020.02.05
  • Accepted : 2020.05.03
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we propose a reinforcement learning agent to control the data transmission rates of nodes in carrier sensing multiple access with collision avoidance (CSMA/CA)-based wireless networks. We design a reinforcement learning (RL) agent, based on Q-learning. The agent learns the environment using the timeout events of packets, which are locally available in data sending nodes. The agent selects actions to control the data transmission rates of nodes that adjust the modulation and coding scheme (MCS) levels of the data packets to utilize the available bandwidth in dynamically changing channel conditions effectively. We use the ns3-gym framework to simulate RL and investigate the effects of the parameters of Q-learning on the performance of the RL agent. The simulation results indicate that the proposed RL agent adequately adjusts the MCS levels according to the changes in the network, and achieves a high throughput comparable to those of the existing data transmission rate adaptation schemes such as Minstrel.

Keywords

References

  1. H. Kim and S. Lee, "Document summarization model based on general context in RNN," Journal of Information Processing Systems, vol. 15, no. 6, pp. 1378-1391, 2019. https://doi.org/10.3745/JIPS.02.0123
  2. M. J. J. Ghrabat, G. Ma, I. Y. Maolood, S. S. Alresheedi, and Z. A. Abduljabbar, "An effective image retrieval based on optimized genetic algorithm utilized a novel SVM-based convolutional neural network classifier," Human-centric Computing and Information Sciences, vol. 9, article no. 31, 2019.
  3. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA: MIT Press, 2018.
  4. M. A. Rahman, Y. D. Lee, and I. Koo, "An efficient transmission mode selection based on reinforcement learning for cooperative cognitive radio networks," Human-centric Computing and Information Sciences, vol. 6, article no. 2, 2016.
  5. Y. Bengio, Learning Deep Architectures for AI. Hanover, MA: Now Publishers Inc., 2009.
  6. C. Zhang, P. Patras, and H. Haddadi, "Deep learning in mobile and wireless networking: a survey," IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2224-2287, 2019. https://doi.org/10.1109/COMST.2019.2904897
  7. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, et al., "Human-level control through deep reinforcement learning," Nature, vol. 518, pp. 529-533, 2015. https://doi.org/10.1038/nature14236
  8. M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, "The arcade learning environment: an evaluation platform for general agents," Journal of Artificial Intelligence Research, vol. 47, pp. 253-279, 2013. https://doi.org/10.1613/jair.3912
  9. C. J. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992. https://doi.org/10.1007/BF00992698
  10. Y. Sun and W. Tan, "A trust-aware task allocation method using deep q-learning for uncertain mobile crowdsourcing," Human-centric Computing and Information Sciences, vol. 9, article no. 25, 2019.
  11. G. A. Preethi and C. Chandrasekar, "Seamless mobility of heterogeneous networks based on Markov decision process," Journal of Information Processing Systems, vol. 11, no. 4, pp. 616-629, 2015. https://doi.org/10.3745/JIPS.03.0015
  12. OpenAI Gym [Online]. Available: https://gym.openai.com.
  13. P. Gawlowicz and A. Zubow, "Ns-3 meets OpenAI Gym: the playground for machine learning in networking research," in Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Miami Beach, FL, 2019, pp. 113-120).
  14. ns-3 is a discrete-event network simulator [Online]. Available: https://www.nsnam.org/.
  15. IEEE Standard for Telecommunications and Information Exchange Between Systems - LAN/MAN Specific Requirements - Part 11: Wireless Medium Access Control (MAC) and physical layer (PHY) specifications: High Speed Physical Layer in the 5 GHz band, IEEE Std 802.11a-1999(R2003), 2003.
  16. "Rate Adaptation for 802.11 Wireless Networks: Minstrel," 2010 [Online]. Available: http://blog.cerowrt.org/papers/minstrel-sigcomm-final.pdf.
  17. J. Kim, S. Kim, S. Choi, and D. Qiao, "CARA: collision-aware rate adaptation for IEEE 802.11 WLANs," in Proceedings of the 25th IEEE International Conference on Computer Communications (INFOCOM), Barcelona, Spain, 2006.
  18. IEEE Standard for Information technology - Telecommunications and information exchange between systems Local and metropolitan area networks - Specific requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, IEEE Std 802.11-2016, 2016.
  19. S. Cho, "SINR-based MCS level adaptation in CSMA/CA wireless networks to embrace IoT devices," Symmetry, vol. 9, article no. 236, 2017.
  20. D. Xia, J. Hart, and Q. Fu, "Evaluation of the Minstrel rate adaptation algorithm in IEEE 802.11g WLANs," in Proceedings of 2013 IEEE International Conference on Communications (ICC), Budapest, Hungary, 2013, pp. 2223-2228.
  21. A. Geron, Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA: O'Reilly Media, 2017.