Journal of the Korea Institute of Military Science and Technology (한국군사과학기술학회지)

The Korea Institute of Military Science and Technology (한국군사과학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Dynamic Window Approach with path-following for Unmanned Surface Vehicle based on Reinforcement Learning

무인수상정 경로점 추종을 위한 강화학습 기반 Dynamic Window Approach

  • Heo, Jinyeong (Department of Industrial Engineering, Ajou University) ;
  • Ha, Jeesoo (Unmanned Systems, LIG Nex1 Co., Ltd.) ;
  • Lee, Junsik (Unmanned Systems, LIG Nex1 Co., Ltd.) ;
  • Ryu, Jaekwan (Unmanned Systems, LIG Nex1 Co., Ltd.) ;
  • Kwon, Yongjin (Department of Industrial Engineering, Ajou University)
  • 허진영 (아주대학교 산업공학과) ;
  • 하지수 (LIG넥스원(주) 무인체계개발단) ;
  • 이준식 (LIG넥스원(주) 무인체계개발단) ;
  • 유재관 (LIG넥스원(주) 무인체계개발단) ;
  • 권용진 (아주대학교 산업공학과)
  • Received : 2020.10.06
  • Accepted : 2021.01.29
  • Published : 2021.02.05
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, autonomous navigation technology is actively being developed due to the increasing demand of an unmanned surface vehicle(USV). Local planning is essential for the USV to safely reach its destination along paths. the dynamic window approach(DWA) algorithm is a well-known navigation scheme as a local path planning. However, the existing DWA algorithm does not consider path line tracking, and the fixed weight coefficient of the evaluation function, which is a core part, cannot provide flexible path planning for all situations. Therefore, in this paper, we propose a new DWA algorithm that can follow path lines in all situations. Fixed weight coefficients were trained using reinforcement learning(RL) which has been actively studied recently. We implemented the simulation and compared the existing DWA algorithm with the DWA algorithm proposed in this paper. As a result, we confirmed the effectiveness of the proposed algorithm.

Keywords

References

  1. Hyogon Kim, Sung-Jo Yun, Young-Ho Choi, Jae-Kwan Ryu, Byong-Jae Won, Jin-Ho Suh, "Improved Dynamic Window Approach with Ellipse Equations for Autonomous Navigation of Unmanned Surface Vehicle," Journal of Institute of Control, Robotics and Systems, 26(8), pp. 624-629, 2020. https://doi.org/10.5302/J.ICROS.2020.20.0084
  2. Kwimi Kim, Jungmok Ma, "A Study on the Research Trends in Unmanned Surface Vehicle using Topic Modeling," Korea Academy Industrial Cooperation Society, 21(7), pp. 597-606, 2020.
  3. Jong-Gyu Ham, Joong-Tae Park, Jae-Bok Song, "Mobile Robot Navigation based on Global DWA with Optimal Waypoints," Journal of Institute of Control, Robotics and Systems, 13(7), pp. 624-630, 2007. https://doi.org/10.5302/J.ICROS.2007.13.7.624
  4. Kim Jee-Seon, "Local Collision Avoidance Algorithm in Dynamic Environment using Collision Probability," The Graduate School of Ewha Womans University, 2020.
  5. D. Fox, W. Burgard, S. Thrun, "The Dynamic Window Approach to Collision Avoidance," IEEE Robotics & Automation Magazine, Vol. 4, No. 1, pp. 23-33, 1997. https://doi.org/10.1109/100.580977
  6. Hyogon Kim, Sung-Jo Yun, Young-Ho Choi, Jung-Woo Lee, Jae-KWan Ryu, Byong-Jae Won, Jin-Ho Suh, "Improved Dynamic Window Approach With Path-Following for Unmanned Surface Vehicle," Journal of Embedded Systems and Applications, 12(5), pp. 295-301, 2017.
  7. Suyeong Jang et. al., "Research Trends on Deep Reinforcement Learning," Electronics and Telecommunications Trends, Vol. 34 No. 4, pp. 1-14, 2019.
  8. Dong-Ham Kim, Sung-Uk Lee, Jong-Ho Nam, Yoshitaka Furukawa, "Determination of Ship Collision Avoidance Path using Deep Deterministic Policy Gradient Algorithm," Journal of the Society of Naval Architects of Korea, 56(1), pp. 58-65, 2019. https://doi.org/10.3744/SNAK.2019.56.1.058
  9. Yi-Hong Liang, Sin-Jin Kang, Sung Hyun Cho, "A Study about the Usefulness of Reinforcement Learning in Business Simulation Games using PPO Algorithm," Journal of Korea Game Society, 19(6), pp. 61-70, 2019. https://doi.org/10.7583/JKGS.2019.19.6.61