• Journal of Platform Technology
• /
• v.11 no.6
• /
• pp.21-27
• /
• 2023

In this paper, we introduce a reinforcement learning-based algorithm for personalized tourist path recommendations. The algorithm employs a reinforcement learning agent to explore tourist regions and identify optimal paths that are expected to enhance tourism experiences. The concept of tourism experience is defined through points of interest (POI) located along tourist paths within the tourist area. These metrics are quantified through aggregated evaluation scores derived from reviews submitted by past visitors. In the experimental setup, the foundational learning model used to find tour paths is the Deep Q-Network (DQN). Despite the limited availability of historical tourist behavior data, the agent adeptly learns travel paths by incorporating preference scores of tourist POIs and spatial information of the travel area.

• Journal of Korean Association for Spatial Structures
• /
• v.23 no.2
• /
• pp.29-36
• /
• 2023

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

• Journal of Navigation and Port Research
• /
• v.47 no.2
• /
• pp.75-84
• /
• 2023

Ships need to ensure safety during their navigation, which makes route determination highly important. It must be accompanied by a route following controller that can accurately follow the route. This study proposes a method for automatically generating the ship route based on deep reinforcement learning algorithm and following it using a route following controller. To generate a ship route, under keel clearance was applied to secure the ship's safety and navigation chart information was used to apply ship navigation related regulations. For the experiment, a target ship with a draft of 8.23 m was designated. The target route in this study was to depart from Busan port and arrive at the pilot boarding place of the Ulsan port. As a route following controller, a velocity type fuzzy P ID controller that could compensate for the limitation of a linear controller was applied. As a result of using the deep Q network, a route with a total distance of 62.22 km and 81 waypoints was generated. To simplify the route, the Douglas-Peucker algorithm was introduced to reduce the total distance to 55.67 m and the number of way points to 3. After that, an experiment was conducted to follow the path generated by the target ship. Experiment results revealed that the velocity type fuzzy P ID controller had less overshoot and fast settling time. In addition, it had the advantage of reducing the energy loss of the ship because the change in rudder angle was smooth. This study can be used as a basic study of route automatic generation. It suggests a method of combining ship route generation with the route following control.

• Smart Structures and Systems
• /
• v.24 no.6
• /
• pp.745-757
• /
• 2019

Road surface deteriorations such as potholes have caused motorists heavy monetary damages every year. However, effective road condition monitoring has been a continuing challenge to road owners. Depth cameras have a small field of view and can be easily affected by vehicle bouncing. Traditional image processing methods based on algorithms such as segmentation cannot adapt to varying environmental and camera scenarios. In recent years, novel object detection methods based on deep learning algorithms have produced good results in detecting typical objects, such as faces, vehicles, structures and more, even in scenarios with changing object distances, camera angles, lighting conditions, etc. Therefore, in this study, a Deep Learning Pothole Detector (DLPD) based on the deep region-based convolutional neural network is proposed for autonomous detection of potholes from images. About 900 images with potholes and road surface conditions are collected and divided into training and testing data. Parameters of the network in the DLPD are calibrated based on sensitivity tests. Then, the calibrated DLPD is trained by the training data and applied to the 215 testing images to evaluate its performance. It is demonstrated that potholes can be automatically detected with high average precision over 93%. Potholes can be differentiated from manholes by training and applying a manhole-pothole classifier which is constructed using the convolutional neural network layers in DLPD. Repeated detection of the same potholes can be prevented through feature matching of the newly detected pothole with previously detected potholes within a small region.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.05a
• /
• pp.538-540
• /
• 2021

In this paper, we proposed a smart target detection system that detects and recognizes a designated target to provide relative motion information when performing a target detection mission of a drone. The proposed system focused on developing an algorithm that can secure adequate accuracy (i.e. mAP, IoU) and high real-time at the same time. The proposed system showed an accuracy of close to 1.0 after 100k learning of the Google Inception V2 deep learning model, and the inference speed was about 60-80[Hz] when using a high-performance laptop based on the real-time performance Nvidia GTX 2070 Max-Q. The proposed smart target detection system will be operated like a drone and will be helpful in successfully performing surveillance and reconnaissance missions by automatically recognizing the target using computer image processing and following the target.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.21 no.6
• /
• pp.163-171
• /
• 2021

In this paper, the energy transaction system was optimized by applying a resource allocation algorithm and deep reinforcement learning in the distributed power system. The power demand and supply environment were predicted by deep reinforcement learning. We propose a system that pursues common interests in power trading and increases the efficiency of long-term power transactions in the paradigm shift from conventional centralized to distributed power systems in the power trading system. For a realistic energy simulation model and environment, we construct the energy market by learning weather and monthly patterns adding Gaussian noise. In simulation results, we confirm that the proposed power trading systems are cooperative with each other, seek common interests, and increase profits in the prolonged energy transaction.

• Journal of Internet Computing and Services
• /
• v.23 no.3
• /
• pp.13-20
• /
• 2022

IEEE 802.11ay Wi-Fi is the next generation wireless technology and operates in mmWave band. It supports the MU-MIMO (Multiple User Multiple Input Multiple Output) transmission in which an AP (Access Point) can transmit multiple data streams simultaneously to multiple STAs (Stations). To this end, the AP should perform MU-MIMO beamforming training with the STAs. For efficient MU-MIMO beamforming training, it is important for the AP to estimate signal strength measured at each STA at which multiple beams are used simultaneously. Therefore, in the paper, we propose a deep learning-based link quality estimation scheme. Our proposed scheme estimates the signal strength with high accuracy by utilizing a deep learning model pre-trained for a certain indoor or outdoor propagation scenario. Specifically, to estimate the signal strength of the multiple concurrent beams, our scheme uses the signal strengths of the respective single beams, which can be obtained without additional signaling overhead, as the input of the deep learning model. For performance evaluation, we utilized a Q-D (Quasi-Deterministic) Channel Realization open source software and extensive channel measurement campaigns were conducted with NIST (National Institute of Standards and Technology) to implement the millimeter wave (mmWave) channel. Our simulation results demonstrate that our proposed scheme outperforms comparison schemes in terms of the accuracy of the signal strength estimation.

• International Journal of Computer Science & Network Security
• /
• v.24 no.4
• /
• pp.87-106
• /
• 2024

Vehicle-to-Home (V2H) and Home Centralized Photovoltaic (HCPV) systems can address various energy storage issues and enhance demand response programs. Renewable energy, such as solar energy and wind turbines, address the energy gap. However, no energy management system is currently available to regulate the uncertainty of renewable energy sources, electric vehicles, and appliance consumption within a smart microgrid. Therefore, this study investigated the impact of solar photovoltaic (PV) panels, electric vehicles, and Micro-Grid (MG) storage on maximum solar radiation hours. Several Deep Learning (DL) algorithms were applied to account for the uncertainty. Moreover, a Reinforcement Learning HCPV (RL-HCPV) algorithm was created for efficient real-time energy scheduling decisions. The proposed algorithm managed the energy demand between PV solar energy generation and vehicle energy storage. RL-HCPV was modeled according to several constraints to meet household electricity demands in sunny and cloudy weather. Simulations demonstrated how the proposed RL-HCPV system could efficiently handle the demand response and how V2H can help to smooth the appliance load profile and reduce power consumption costs with sustainable power generation. The results demonstrated the advantages of utilizing RL and V2H as potential storage technology for smart buildings.

• Smart Media Journal
• /
• v.9 no.2
• /
• pp.92-98
• /
• 2020

This paper proposes a vehicle license plate recognition system using light weight deep learning models without high-end server. The proposed license plate recognition system consists of 3 steps: [license plate detection]-[character area segmentation]-[character recognition]. SSD-Mobilenet was used for license plate detection, ResNet with localization was used for character area segmentation, ResNet was used for character recognition. Experiemnts using Samsung Galaxy S7 and LG Q9, accuracy showed 85.3% accuracy and around 1.1 second running time.

• Journal of Digital Convergence
• /
• v.19 no.12
• /
• pp.103-113
• /
• 2021

In this study, the Q method is used to investigate the types of perceptions of liberal arts education perceived by college students and to investigate the characteristics of each type. 33 Q samples were extracted from the Q population collected through literature research, open questionnaires, and deep interviews. Q classification was conducted for 27 students of A University located in B City. The data was analyzed using the QUANL program. In the research, the types of awareness of liberal arts education were derived as "pursuit of various experiences", "pursuit of practical studies", "pursuit of accident expansion", and "pursuit of social change". The results of this study re-establish the meaning of liberal arts education in university education and suggest that it is necessary to consider various educational contents and teaching learning methods.