The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Path Planning with Obstacle Avoidance Based on Double Deep Q Networks

이중 심층 Q 네트워크 기반 장애물 회피 경로 계획

  • 자오 용지앙 (전남대학교 대학원 컴퓨터공학과) ;
  • 첸센폰 (전남대학교 대학원 컴퓨터공학과) ;
  • 성승제 (전남대학교 대학원 컴퓨터공학과) ;
  • 허정규 (전남대학교 대학원 컴퓨터공학과) ;
  • 임창균 (전남대학교 컴퓨터공학과)
  • Received : 2023.01.28
  • Accepted : 2023.04.17
  • Published : 2023.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

It remains a challenge for robots to learn avoiding obstacles automatically in path planning using deep reinforcement learning (DRL). More and more researchers use DRL to train a robot in a simulated environment and verify the possibility of DRL to achieve automatic obstacle avoidance. Due to the influence factors of different environments robots and sensors, it is rare to realize automatic obstacle avoidance of robots in real scenarios. In order to learn automatic path planning by avoiding obstacles in the actual scene we designed a simple Testbed with the wall and the obstacle and had a camera on the robot. The robot's goal is to get from the start point to the end point without hitting the wall as soon as possible. For the robot to learn to avoid the wall and obstacle we propose to use the double deep Q networks (DDQN) to verify the possibility of DRL in automatic obstacle avoidance. In the experiment the robot used is Jetbot, and it can be applied to some robot task scenarios that require obstacle avoidance in automated path planning.

심층 강화 학습(Deep Reinforcement Learning)을 사용한 경로 계획에서 장애물을 자동으로 회피하기 위해 로봇을 학습시키는 일은 쉬운 일이 아니다. 많은 연구자가 DRL을 사용하여 주어진 환경에서 로봇 학습을 통해 장애물 회피하여 경로 계획을 수립하려는 가능성을 시도하였다. 그러나 다양한 환경에서 로봇과 장착된 센서의 오는 다양한 요인 때문에 주어진 시나리오에서 로봇이 모든 장애물을 완전히 회피하여 이동하는 것을 실현하는 일은 흔치 않다. 이러한 문제 해결의 가능성과 장애물을 회피 경로 계획 실험을 위해 테스트베드를 만들었고 로봇에 카메라를 장착하였다. 이 로봇의 목표는 가능한 한 빨리 벽과 장애물을 피해 시작점에서 끝점까지 도달하는 것이다. 본 논문에서는 벽과 장애물을 회피하기 위한 DRL의 가능성을 검증하기 위해 이중 심층 Q 네트워크(DDQN)를 제안하였다. 실험에 사용된 로봇은 Jetbot이며 자동화된 경로 계획에서 장애물 회피가 필요한 일부 로봇 작업 시나리오에 적용할 수 있을 것이다.

Keywords

Acknowledgement

This work was supported by the Technological Innovation R&D Program (S3264239) funded by the Ministry of SMEs and Startups, and the Technological Innovation R&D Program (S3154675) funded by the Ministry of SMEs and Startups(MSS, Korea).

References

  1. C. Kim, H. Cho, T. Yun, H. Shin and H. Park, "RFID-based Shortest Time Algorithm linetracer," J. of the Korea Institute of Electronic Communication Sciences, vol. 17, no. 6, Dec. 2022, pp. 1221-1228. 
  2. S. Zhou, X. Liu, Y. Xu and J. Guo, "A Deep Q-network (DQN) Based Path Planning Method for Mobile Robots," 2018 IEEE International Conference on Information and Automation (ICIA), Wuyishan, China, 2018. 
  3. Q. Liu, L. Shi, L. Sun, J. Li, M. Ding and F. Shu, "Path Planning for UAV-Mounted Mobile Edge Computing With Deep Reinforcement Learning," IEEE Transactions on Vehicular Technology, vol. 69, no. 5, May. 2020, pp. 5723-5728.  https://doi.org/10.1109/TVT.2020.2982508
  4. A. Villanueva and A. Fajardo, "Deep Reinforcement Learning with Noise Injection for UAV Path Planning," 2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS), Kuala Lumpur, Malaysia, 2019. 
  5. T. Ribeiro, F. Goncalves, I. Garcia, G. Lopes and A. F. Ribeiro, "Q-Learning for Autonomous Mobile Robot Obstacle Avoidance," 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Porto, Portugal, 2019. 
  6. C. C. Chang, Y. H. Juan, C. L. Huang and H. J. Chen, "Scenario Analysis for Road Following Using JetBot," 2020 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE), Yunlin, Taiwan, 2020. 
  7. S. Yong, H Park, Y. You and I. Moon, "Q-Learning Policy and Reward Design for Efficient Path Selection," Journal of Advanced Navigation Technology, vol. 26, no. 2, Apr. 2022, pp. 72-77.  https://doi.org/10.12673/JANT.2022.26.2.72
  8. L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba and P. Abbeel, "Asymmetric Actor Critic for Image-Based Robot Learning," arXiv:1710.06542v1, 2017. 
  9. A. Singla, S. Padakandla and S. Bhatnagar, "Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge," IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 1, Jan. 2021, pp. 107-118.  https://doi.org/10.1109/TITS.2019.2954952
  10. Y. F. Chen, M. Liu, M. Everett and J. P. How, "Decentralized noncommunicating multiagent collision avoidance with deep reinforcement learning," IEEE International Conference on Robotics and Automation, Singapore, 2017. 
  11. B. Q. Huang, G. Y. Cao and M. Guo, "Reinforcement Learning Neural Network to the Problem of Autonomous Mobile Robot Obstacle Avoidance," 2005 Int. Conf. Mach. Learn. Cybern, Guangzhou, China, 2005. 
  12. D. Kim, S. Park and D. Kim, "The Classification Scheme of ADHD for children based on the CNN Model," J. of the Korea Institute of Electronic Communication Sciences, vol. 17, no. 5, Oct. 2022, pp. 809-814.