ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Exploring reward efficacy in traffic management using deep reinforcement learning in intelligent transportation system

  • Paul, Ananya (Department of Computer Science and Technology, Indian Institute of Engineering Science and Technology) ;
  • Mitra, Sulata (Department of Computer Science and Technology, Indian Institute of Engineering Science and Technology)
  • Received : 2021.11.01
  • Accepted : 2022.02.20
  • Published : 2022.04.10
Download PDF
⟨ Previous Next ⟩

Abstract

In the last decade, substantial progress has been achieved in intelligent traffic control technologies to overcome consistent difficulties of traffic congestion and its adverse effect on smart cities. Edge computing is one such advanced progress facilitating real-time data transmission among vehicles and roadside units to mitigate congestion. An edge computing-based deep reinforcement learning system is demonstrated in this study that appropriately designs a multiobjective reward function for optimizing different objectives. The system seeks to overcome the challenge of evaluating actions with a simple numerical reward. The selection of reward functions has a significant impact on agents' ability to acquire the ideal behavior for managing multiple traffic signals in a large-scale road network. To ascertain effective reward functions, the agent is trained withusing the proximal policy optimization method in several deep neural network models, including the state-of-the-art transformer network. The system is verified using both hypothetical scenarios and real-world traffic maps. The comprehensive simulation outcomes demonstrate the potency of the suggested reward functions.

Keywords

References

  1. Y. Lin, X. Dai, L. Li, and F.-Y. Wang, An efficient deep reinforcement learning model for urban traffic control, 2018, arXiv preprint arXiv:1808.01876.
  2. Y. Huo, Q. Tao, and J. Hu, Cooperative control for multi-intersection traffic signal based on deep reinforcement learning and imitation learning, IEEE Access 8 (2020), 199573-199585. https://doi.org/10.1109/access.2020.3034419
  3. A. Paul and S. Mitra, Deep reinforcement learning based traffic signal optimization for multiple intersections in ITS, (IEEE International Conference on Advanced Networks and Telecommunications Systems, New Delhi, India), Dec. 2020, pp. 1-6.
  4. S. Touhbi, M. A. Babram, T. Nguyen-Huu, N. Marilleau, M. L. Hbid, C. Cambier, and S. Stinckwich, Adaptive traffic signal control: Exploring reward definition for reinforcement learning, Procedia Comput. Sci. 109 (2017), 513-520. https://doi.org/10.1016/j.procs.2017.05.327
  5. A. C. Egea, S. Howell, M. Knutins, and C. Connaughton, Assessment of reward functions for reinforcement learning traffic signal control under real-world limitations, (IEEE International Conference on Systems, Man, and Cybernetics, Toronto, Canada), Oct. 2020, pp. 965-972.
  6. D. Zhong and A. Boukerche, Traffic signal control using deep reinforcement learning with multiple resources of rewards, (PE-WASUN '19: Proceedings of the 16th ACM International Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous Networks, Miami Beach, FL, USA), 2019, pp. 23-28.
  7. A. R. M. Jamil, K. K. Ganguly, and N. Nower, Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning, IET Intell. Transp. Syst. 14 (2021), no. 14, 2030-2041.
  8. H. Wei, G. Zheng, H. Yao, and Z. Li, Intellilight: A reinforcement learning approach for intelligent traffic light control, (KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK), 2018, pp. 2496-2505.
  9. A. Paul and S. Mitra, Management of traffic signals using deep reinforcement learning in bidirectional recurrent neural network in ITS, (ISMSI 2021: 2021 5th International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence, Victoria, Seychelles), 2021, pp. 60-64.