• Journal of Electrical Engineering and Technology
• /
• v.12 no.2
• /
• pp.890-903
• /
• 2017

The problem of determining a smooth and collision-free path with maximum possible speed for a Mobile Robot (MR) which is chasing a moving target in a dynamic environment is addressed in this paper. Genetic Network Programming with Reinforcement Learning (GNP-RL) has several important features over other evolutionary algorithms such as it combines offline and online learning on the one hand, and it combines diversified and intensified search on the other hand, but it was used in solving the problem of MR navigation in static environment only. This paper presents GNP-RL based on predicting collision positions as a first attempt to apply it for MR navigation in dynamic environment. The combination between features of the proposed collision prediction and that of GNP-RL provides safe navigation (effective obstacle avoidance) in dynamic environment, smooth movement, and reducing the obstacle avoidance latency time. Simulation in dynamic environment is used to evaluate the performance of collision prediction based GNP-RL compared with that of two state-of-the art navigation approaches, namely, Q-Learning (QL) and Artificial Potential Field (APF). The simulation results show that the proposed GNP-RL outperforms both QL and APF in terms of smooth movement and safer navigation. In addition, it outperforms APF in terms of preserving maximum possible speed during obstacle avoidance.

• Journal of Positioning, Navigation, and Timing
• /
• v.11 no.4
• /
• pp.361-368
• /
• 2022

An indoor navigation system that utilizes long-term evolution (LTE) signals has the benefit of no additional infrastructure installation expenses and low base station database management costs. Among the LTE signal measurements, received signal strength (RSS) is particularly appealing because it can be easily obtained with mobile devices. Propagation channel models can be used to estimate the position of mobile devices with RSS. However, conventional channel models have a shortcoming in that they do not discriminate between line-of-sight (LOS) and non-line-of-sight (NLOS) conditions of the received signal. Accordingly, a previous study has suggested separated LOS and NLOS channel models. However, a method for determining LOS and NLOS conditions was not devised. In this study, a machine learning-based LOS/NLOS classification method using RSS measurements is developed. We suggest several machine-learning features and evaluate various machine-learning algorithms. As an indoor experimental result, up to 87.5% classification accuracy was achieved with an ensemble algorithm. Furthermore, the range estimation accuracy with an average error of 13.54 m was demonstrated, which is a 25.3% improvement over the conventional channel model.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2023.11a
• /
• pp.232-235
• /
• 2023

This study suggests how to build a training environment for the application of reinforcement learning techniques to USV, and Ihow to apply the training result to a real USV. The purpose of RL is to move USV from departure point to destination point autonomously using rudder.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.23 no.4
• /
• pp.399-406
• /
• 2020

In this paper, we propose a time-varying proportional navigation guidance law that determines the proportional navigation gain in real-time according to the operating situation. When intercepting a target, an unidentified evasion strategy causes a loss of optimality. To compensate for this problem, proper proportional navigation gain is derived at every time step by solving an optimal control problem with the inferred evader's strategy. Recently, deep reinforcement learning algorithms are introduced to deal with complex optimal control problem efficiently. We adapt the actor-critic method to build a proportional navigation gain network and the network is trained by the Proximal Policy Optimization(PPO) algorithm to learn an evasion strategy of the target. Numerical experiments show the effectiveness and optimality of the proposed method.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2004.08a
• /
• pp.129-138
• /
• 2004

Recently, maintenance of clean ocean is very important issue. One part of the issue strongly relates to the safe navigation of vessels. Most parts of safe navigation depend on the mariner's competencies. STCW was established to guarantee the sufficient mariner's competencies. However the code dose not indicate the each necessary competency clearly. Therefore, the understandings on STCW are not same among training institutes, and each training institute interprets them individually and executes them. As a result, it makes big differences among the institutes concerning training methods and contents and the assessment on mariner's competencies. The countries in EU have paid attention on this issue and the activities enhance the rational assessment of competencies through METNET. In order to execute the rational assessments, it is necessary to clarify the techniques being assessed. And it means the necessity to clarify the necessary techniques for safe navigation and to assess the competencies to achieve them, and then we can attain the objectives of STCW. In this paper, the necessary techniques for achievement of safe navigation are discussed and the methods of assessment of competencies are proposed. As we apply proposed assessment system, we can get the mariner's competency quantitatively and continuously through training period. Then we can know the trend of the competency that is the learning process. By clarifying the learning process of the techniques, we can decide the necessary training time to achieve the competencies. Furthermore, we can discuss the issues on conversion between training onboard and simulator used training by analysing the learning processes.

• Journal of Positioning, Navigation, and Timing
• /
• v.12 no.2
• /
• pp.149-155
• /
• 2023

In urban areas it can be difficult to utilize global navigation satellite systems (GNSS) due to signal reflections and blockages. It is thus crucial to detect reflected or blocked signals because they lead to significant degradation of GNSS positioning accuracy. In a previous study, a classifier for global positioning system (GPS) signal reception conditions was developed using three features and the support vector machine (SVM) algorithm. However, this classifier had limitations in its classification performance. Therefore, in this study, we developed an improved machine learning based method of classifying GPS signal reception conditions by including an additional feature with the existing features. Furthermore, we applied various machine learning classification algorithms. As a result, when tested with datasets collected in different environments than the training environment, the classification accuracy improved by nine percentage points compared to the existing method, reaching up to 58%.

• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.9
• /
• pp.281-290
• /
• 2020

The Vision-and-Language Navigation(VLN) task is a complex intelligence problem that requires both visual and language comprehension skills. In this paper, we propose a new learning model for visual-language navigation agents. The model adopts a hybrid learning that combines imitation learning based on demo data and reinforcement learning based on action reward. Therefore, this model can meet both problems of imitation learning that can be biased to the demo data and reinforcement learning with relatively low data efficiency. In addition, the proposed model uses a novel path-based reward function designed to solve the problem of existing goal-based reward functions. In this paper, we demonstrate the high performance of the proposed model through various experiments using both Matterport3D simulation environment and R2R benchmark dataset.

• Journal of Positioning, Navigation, and Timing
• /
• v.10 no.4
• /
• pp.297-305
• /
• 2021

Monocular depth estimation helps the robot to understand the surrounding environments in 3D. Especially, deep-learning-based monocular depth estimation has been widely researched, because it may overcome the scale ambiguity problem, which is a main issue in classical methods. Those learning based methods can be mainly divided into three parts: supervised learning, unsupervised learning, and semi-supervised learning. Supervised learning trains the network from dense ground-truth depth information, unsupervised one trains it from images sequences and semi-supervised one trains it from stereo images and sparse ground-truth depth. We describe the basics of each method, and then explain the recent research efforts to enhance the depth estimation performance.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2023.05a
• /
• pp.160-161
• /
• 2023

Currently, the ship's passage planning relies on the navigator officer's knowledge and empirical methods. However, as ship autonomous navigation technology has recently developed, automation technology for passage planning has been studied in various ways. In this study, we intend to generate an optimal route for a ship based on Q-learning, one of the reinforcement learning techniques. Reinforcement learning is applied in a way that trains experiences for various situations and makes optimal decisions based on them.

• Korean Journal of Radiology
• /
• v.20 no.5
• /
• pp.844-853
• /
• 2019

Objective: To evaluate the learning curve for C-arm cone-beam computed tomography (CBCT) virtual navigation-guided percutaneous transthoracic needle biopsy (PTNB) and to determine the amount of experience needed to develop appropriate skills for this procedure using cumulative summation (CUSUM). Materials and Methods: We retrospectively reviewed 2042 CBCT virtual navigation-guided PTNBs performed by 7 novice operators between March 2011 and December 2014. Learning curves for CBCT virtual navigation-guided PTNB with respect to its diagnostic performance and the occurrence of biopsy-related pneumothorax were analyzed using standard and risk-adjusted CUSUM (RA-CUSUM). Acceptable failure rates were determined as 0.06 for diagnostic failure and 0.25 for PTNB-related pneumothorax. Results: Standard CUSUM indicated that 6 of the 7 operators achieved an acceptable diagnostic failure rate after a median of 105 PTNB procedures (95% confidence interval [CI], 14-240), and 6 of the operators achieved acceptable pneumothorax occurrence rate after a median of 79 PTNB procedures (95% CI, 27-155). RA-CUSUM showed that 93 (95% CI, 39-142) and 80 (95% CI, 38-127) PTNB procedures were required to achieve acceptable diagnostic performance and pneumothorax occurrence, respectively. Conclusion: The novice operators' skills in performing CBCT virtual navigation-guided PTNBs improved with increasing experience over a wide range of learning periods.