Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Mapless Navigation Based on DQN Considering Moving Obstacles, and Training Time Reduction Algorithm

이동 장애물을 고려한 DQN 기반의 Mapless Navigation 및 학습 시간 단축 알고리즘

  • Yoon, Beomjin (Electrical System Integrated Team, Renault Technology Korea) ;
  • Yoo, Seungryeol (School of Mechanical Engineering, Korea University of Technology and Education)
  • Received : 2021.02.01
  • Accepted : 2021.02.18
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, in accordance with the 4th industrial revolution, The use of autonomous mobile robots for flexible logistics transfer is increasing in factories, the warehouses and the service areas, etc. In large factories, many manual work is required to use Simultaneous Localization and Mapping(SLAM), so the need for the improved mobile robot autonomous driving is emerging. Accordingly, in this paper, an algorithm for mapless navigation that travels in an optimal path avoiding fixed or moving obstacles is proposed. For mapless navigation, the robot is trained to avoid fixed or moving obstacles through Deep Q Network (DQN) and accuracy 90% and 93% are obtained for two types of obstacle avoidance, respectively. In addition, DQN requires a lot of learning time to meet the required performance before use. To shorten this, the target size change algorithm is proposed and confirmed the reduced learning time and performance of obstacle avoidance through simulation.

최근 4차 산업혁명에 따라 공장, 물류창고, 서비스영역에서 유연한 물류이송을 위한 자율 이동형 모바일 로봇의 사용이 증가하고 있다. 대규모 공장에서는 Simultaneous Localization and Mapping(SLAM)을 수행하기 위하여 많은 수작업이 필요하기 때문에 개선된 모바일 로봇 자율 주행에 대한 필요성이 대두되고 있다. 이에 따라 본 논문에서는 고정 및 이동 장애물을 피해 최적의 경로로 주행하는 Mapless Navigation에 대한 알고리즘을 제안하고자 한다. Mapless Navigation을 위하여 Deep Q Network(DQN)을 통해 고정 및 이동 장애물을 회피하도록 학습하였고 두 종류의 장애물 회피에 대하여 각각 정확도 90%, 93%를 얻었다. 또한 DQN은 많은 학습 시간을 필요로 하는데 이를 단축하기 위한 목표의 크기 변화 알고리즘을 제안하고 이를 시뮬레이션을 통하여 단축된 학습시간과 장애물 회피 성능을 확인하였다.

Keywords

References

  1. A. Li, J. Wang, M. Xu, and Z. Chen, "DP-SLAM: A visual SLAM with moving probability towards dynamic environments," Information Science, vol. 556, pp. 128-142, 2021. https://doi.org/10.1016/j.ins.2020.12.019
  2. A. Rohacza, S. WeiBenfels, and S. Strassburger, "Concept for the comparison of intralogistics designs with real factory layout using augmented reality SLAM and marker-based tracking," in Procedia CIRP, vol. 93, pp. 341-346, 2020. https://doi.org/10.1016/j.procir.2020.03.039
  3. J. Kim, S. Jung, T. Jeon, and S. Kim, "SLAM based on feature map for autonomous vehicle," Journal of the Korea Institute of Information and Communication Engineering, vol. 13, no. 7, pp. 1437-1443, 2009. https://doi.org/10.6109/JKIICE.2009.13.7.1437
  4. A. Cheong, E. Foo, M. Lau, J. Chen, and H. Gan, "Development of a Robotics Waiter System for the food and beverage industry," in Proceedings of The Third International Conference on Advances in Mechanical and Robotics Engineering, pp. 21-25, 2015.
  5. D. V. Lu, D. H. Hershberger, and W. D. Smart, "Layered costmaps for context-sensitive navigation," in IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), pp. 709-715, 2014.
  6. R. P. Monroe, S. A. Miller, and A. T. Bradley, "Behavioral mapless navigation using rings," in IEEE International Conference on Robotics and Automation, 2012.
  7. L. Tei, G. Paolo, and M. Liu, "Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation," in IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), pp. 23-30, 2017.
  8. K. J. Gil, ROS Machine Learning Package [Internet]. Available: https://github.com/ROBOTIS-GIT/emanual/blob/master/docs/en/platform/turtlebot3/machine_learning.md.
  9. L. Qiang, D. Nanxum, L. Huican, and W. Heng, "A model-free mpless navigation method for mobile robot using reinforcement learning," in Chinese Control And Decision Conference(CCDC), pp. 3410-3415, 2018.
  10. G. Hinton, N. Srivastava, and K. Swersky, Lecture 6a overview of mini-batch gradient descent, [Internet]. pp. 29. Available: http://www.cs.toronto.edu/-tijmen/csc321/.