KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-agent Q-learning based Admission Control Mechanism in Heterogeneous Wireless Networks for Multiple Services

  • Chen, Jiamei (Communication Research Center, Harbin Institute of Technology) ;
  • Xu, Yubin (Communication Research Center, Harbin Institute of Technology) ;
  • Ma, Lin (Communication Research Center, Harbin Institute of Technology) ;
  • Wang, Yao (Communication Research Center, Harbin Institute of Technology)
  • Received : 2013.04.03
  • Accepted : 2013.09.22
  • Published : 2013.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In order to ensure both of the whole system capacity and users QoS requirements in heterogeneous wireless networks, admission control mechanism should be well designed. In this paper, Multi-agent Q-learning based Admission Control Mechanism (MQACM) is proposed to handle new and handoff call access problems appropriately. MQACM obtains the optimal decision policy by using an improved form of single-agent Q-learning method, Multi-agent Q-learning (MQ) method. MQ method is creatively introduced to solve the admission control problem in heterogeneous wireless networks in this paper. In addition, different priorities are allocated to multiple services aiming to make MQACM perform even well in congested network scenarios. It can be observed from both analysis and simulation results that our proposed method not only outperforms existing schemes with enhanced call blocking probability and handoff dropping probability performance, but also has better network universality and stability than other schemes.

Keywords

References

  1. O.E. Falowo, "Joint call admission control algorithm for reducing call blocking/dropping probability in heterogeneous wireless networks supporting multihoming," in Proc. of GLOBECOM Workshops, pp. 611-615, 2010.
  2. Y.S. Zhao, X. Li and H. Ji, "Radio admission control scheme for high-speed railway communication with MIMO antennas," in Proc. of IEEE International Conference on Communications (ICC), pp. 5005-5009, 2012.
  3. S.A. AlQahtani, A.S. Mahmoud, "Performance analysis of two throughput-based call admission control schemes for 3G WCDMA wireless networks supporting multiservices," Computer Communications, vol. 31, pp. 49-57, 2008. https://doi.org/10.1016/j.comcom.2007.10.029
  4. S. Kim, Y.J. Cho, Y.K. Kim, "Computer networks admission control scheme based on priority access for wireless LANs," Computer Networks, vol. 54, pp. 3-12, 2010. https://doi.org/10.1016/j.comnet.2009.07.002
  5. D. Fooladivanda and C. Rosenberg, "Joint resource allocation and user association for heterogeneous," IEEE Transactions on Wireless Communications, vol. 12, pp. 248-257, 2012.
  6. D. Niyato and E. Hossain, "Dynamics of network selection in heterogeneous wireless networks: an evolutionary game approach," IEEE Transactions on Vehicular Technology, vol. 58, no. 4, pp. 2008-2017, May, 2009. https://doi.org/10.1109/TVT.2008.2004588
  7. C. Makaya, and S. Pierre, "An architecture for seamless mobility support in IP-based next-generation wireless networks," IEEE Transactions on Vehicular Technology, vol. 57, no. 2, March, 2008.
  8. H.T. Cheng and W.H. Zhuang, "QoS-driven MAC-layer resource allocation for wireless mesh networks with non-altruistic node cooperation and service differentiation," IEEE Transactions on Wireless Communications, vol. 8, no. 12, pp. 6089-6103, December, 2009. https://doi.org/10.1109/TWC.2009.12.090502
  9. Y. Choi, H. Kim, S.W. Han, et al, "Joint resource allocation for parallel multi-radio access in heterogeneous wireless networks," IEEE Transactions on Wireless Communications, vol. 9, no. 11, pp. 3324-3329, November, 2010. https://doi.org/10.1109/TWC.2010.11.100045
  10. S.K. Lee, K. Sriram, K. Kim, et al, "Vertical handoff decision algorithms for providing optimized performance in heterogeneous wireless networks," IEEE Transactions on Vehicular Technology, vol. 58, no. 2, pp. 865-881, 2009. https://doi.org/10.1109/TVT.2008.925301
  11. K. Samdanis, and A.H. Aghvami, "Scalable inter-area handovers for hierarchical wireless networks", IEEE Wireless Communications, vol.16, no. 6, pp. 62-68, 2009. https://doi.org/10.1109/MWC.2009.5361180
  12. W. Shen and Q.A. Zeng, "Cost-function-based network selection strategy in integrated wireless and mobile networks," IEEE Transactions Vehicular Technology, vol. 57, no. 6, pp. 3778-3788, November, 2008. https://doi.org/10.1109/TVT.2008.917257
  13. F. Yu and V. Krishnamurthy, "Optimal joint session admission control in integrated WLAN and CDMA cellular networks with vertical handoff," IEEE Transactions Mobile Computing, vol. 6, no. 1, pp. 126-139, April, 2007. https://doi.org/10.1109/TMC.2007.250676
  14. Y.H. Chen, C.J. Chang, and C.Y. Huang, "Fuzzy Q-learning admission control for WCDMA/WLAN heterogeneous networks with multimedia traffic," IEEE Transactions on Mobile Computing, vol. 8, no. 11, pp. 1469-1479, November, 2009. https://doi.org/10.1109/TMC.2009.65
  15. Y. Hosoya and M. Umano, "Australia dynamic fuzzy Q-learning with facility of tuning and removing fuzzy rules," in Proc. of IEEE World Congress on Computational Intelligence, pp. 1-8. June, 2012.
  16. D.Y. Ye, M.J. Zhang and D. Sutanto, "A hybrid multiagent framework with Q-learning for power grid systems restoration," IEEE Transactions on Power Systems, vol.26, pp. 2434-2441, 2011. https://doi.org/10.1109/TPWRS.2011.2157180
  17. J.S. Lin and K.T. Feng, "QoS-based adaptive contention/reservation medium access control protocols for wireless local area networks," IEEE Transactions on Mobile Computing, vol. 10, pp. 1785-1803, 2011. https://doi.org/10.1109/TMC.2010.235
  18. S. Zhao, W.X. Shi and S.S. Fan, et al, "A GRA-based network selection mechanism in heterogeneous wireless networks," in Proc. of International Conference on Computer, Mechatronics, Control and Electronic Engineering (CMCE), pp. 215-218, 2010.
  19. R. Ben Ali and S. Pierre, "On the impact of soft vertical handoff on optimal voice admission control in PCF-based WLANs loosely coupled to 3G networks," IEEE Transactions on Wireless Communications, vol. 8, no. 3, pp. 1356-1365, March, 2009. https://doi.org/10.1109/TWC.2008.071354
  20. K.S. Munasinghe and A. Jamalipour, "Interworked WiMAX-3G cellular data networks: an architecture for mobility management and performance evaluation," Transactions on Wireless Communications, vol. 8, no. 4, pp. 1847-1853, April, 2009. https://doi.org/10.1109/T-WC.2008.080094
  21. J.Y.K. Aulin and D. Jeremic, "Compressive sensing aided determination of WCDMA constrained capacity," in Proc. of IEEE International Conference on Communications (ICC), pp. 4072-4077, 2012.
  22. H.Q. Zhai, X. Chen and Y.G. Fang, "How well can the IEEE 802.11 wireless LAN support quality of service?" IEEE Transactions on Wireless Communications, vol. 4, no. 6, pp. 3084-3094, 2005. https://doi.org/10.1109/TWC.2005.857994
  23. C. Brouzioutis, V. Vitsas, P. Chatzimisios, "Studying the impact of data traffic on voice capacity in IEEE 802.11 WLANs," in Proc. of IEEE International Conference on Communications (ICC), pp. 1-6, 2010.
  24. T. Venkatesh, Y.V. Kiran and C.S.R. Murthy, "Joint path and wavelength selection using Q-learning in optical burst switching networks," in Proc. of IEEE International Conference on Communications, pp. 1-5, August, 2009.
  25. H.E. Kim and H.S. Ahn, "Convergence of multiagent Q-learning: multi action replay process approach," in Proc. of IEEE International Symposium on Intelligent Control, Part of IEEE Multi-Conference on Systems and Control, pp. 789-794, September, 2010.
  26. C.J.C.H. Watkins and P. Dayan, "Technical note Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992.
  27. C. Makaya, and S. Pierre, "An analytical framework for performance evaluation of ipv6-based mobility management protocols," IEEE Transactions on Wireless Communications, vol. 7, no. 3, March, 2008.
  28. Q.S. Guan, F.R. Yu, S.M. Jiang, et al, "Prediction-based topology control and routing in cognitive radio mobile Ad Hoc networks," IEEE Transactions on Vehicular Technology, vol. 59, no. 9, pp. 4443-4452, November, 2010. https://doi.org/10.1109/TVT.2010.2069105