• 한국멀티미디어학회논문지
• /
• 제21권10호
• /
• pp.1182-1194
• /
• 2018

The Dynamic Adaptive Streaming over HTTP(DASH) is envisioned to evolve to meet an increasing demand on providing seamless video streaming services in the near future. The DASH performance heavily depends on the client's adaptive quality selection algorithm that is not included in the standard. The existing conventional algorithms are basically based on a procedural algorithm that is not easy to capture and reflect all variations of dynamic network and traffic conditions in a variety of network environments. To solve this problem, this paper proposes a novel quality selection mechanism based on the Deep Q-Network(DQN) model, the DQN-based DASH Adaptive Bitrate(ABR) mechanism. The proposed mechanism adopts a new reward calculation method based on five major performance metrics to reflect the current conditions of networks and devices in real time. In addition, the size of the consecutive video segment to be downloaded is also considered as a major learning metric to reflect a variety of video encodings. Experimental results show that the proposed mechanism quickly selects a suitable video quality even in high error rate environments, significantly reducing frequency of quality changes compared to the existing algorithm and simultaneously improving average video quality during video playback.

• International Journal of Computer Science & Network Security
• /
• 제21권9호
• /
• pp.125-131
• /
• 2021

Detecting fraudulent insurance claims is difficult due to small and unbalanced data. Some research has been carried out to better cope with various types of fraudulent claims. Nowadays, technology for detecting fraudulent insurance claims has been increasingly utilized in insurance and technology fields, thanks to the use of artificial intelligence (AI) methods in addition to traditional statistical detection and rule-based methods. This study obtained meaningful results for a fraudulent insurance claim detection model based on machine learning (ML) and deep learning (DL) technologies, using fraudulent insurance claim data from previous research. In our search for a method to enhance the detection of fraudulent insurance claims, we investigated the reinforcement learning (RL) method. We examined how we could apply the RL method to the detection of fraudulent insurance claims. There are limited previous cases of applying the RL method. Thus, we first had to define the RL essential elements based on previous research on detecting anomalies. We applied the deep Q-network (DQN) and double deep Q-network (DDQN) in the learning fraudulent insurance claim detection model. By doing so, we confirmed that our model demonstrated better performance than previous machine learning models.

• 한국정보통신학회논문지
• /
• 제24권11호
• /
• pp.1500-1506
• /
• 2020

본 논문에서는 오버레이 단말 간 직접 (Device-to-Device : D2D) 통신 네트워크를 위한 강화학습 기반 스케쥴링 문제를 연구한다. 강화학습 모델 중 하나인 Q-learning을 이용한 D2D 통신 기술들이 연구되었지만, Q-learning은 상태와 행동의 개수가 증가함에 따라서 높은 복잡도를 유발한다. 이러한 문제를 해결하기 위하여 Deep Q Network (DQN) 기반 D2D 통신 기술들이 연구되었다. 본 논문에서는 무선 통신 시스템 특성을 고려한 DQN 모델을 디자인하고, 피드백 및 시그널링 오버헤드를 줄일 수 있는 DQN 기반 분산적 스케쥴링 방식을 제안한다. 제안 방식은 중앙집중식으로 변수들을 학습시키고, 최종 학습된 파라미터를 모든 단말들에게 전달한다. 모든 단말들은 최종 학습된 파라미터를 이용하여 각자의 행동을 개별적으로 결정한다. 제안 방식의 성능을 컴퓨터 시뮬레이션을 통하여 분석하고, 최적방식, 기회주의적 선택 방식, 전체 전송 방식과 비교한다.

• 한국항해항만학회지
• /
• 제47권2호
• /
• pp.75-84
• /
• 2023

Ships need to ensure safety during their navigation, which makes route determination highly important. It must be accompanied by a route following controller that can accurately follow the route. This study proposes a method for automatically generating the ship route based on deep reinforcement learning algorithm and following it using a route following controller. To generate a ship route, under keel clearance was applied to secure the ship's safety and navigation chart information was used to apply ship navigation related regulations. For the experiment, a target ship with a draft of 8.23 m was designated. The target route in this study was to depart from Busan port and arrive at the pilot boarding place of the Ulsan port. As a route following controller, a velocity type fuzzy P ID controller that could compensate for the limitation of a linear controller was applied. As a result of using the deep Q network, a route with a total distance of 62.22 km and 81 waypoints was generated. To simplify the route, the Douglas-Peucker algorithm was introduced to reduce the total distance to 55.67 m and the number of way points to 3. After that, an experiment was conducted to follow the path generated by the target ship. Experiment results revealed that the velocity type fuzzy P ID controller had less overshoot and fast settling time. In addition, it had the advantage of reducing the energy loss of the ship because the change in rudder angle was smooth. This study can be used as a basic study of route automatic generation. It suggests a method of combining ship route generation with the route following control.

• 대한임베디드공학회논문지
• /
• 제17권5호
• /
• pp.297-307
• /
• 2022

The simple procedural segment selection algorithm commonly used in Dynamic Adaptive Streaming over HTTP (DASH) reveals severe weakness to provide high-quality streaming services in the integrated mobile networks of various wired and wireless links. A major issue could be how to properly cope with dynamically changing underlying network conditions. The key to meet it should be to make the segment selection algorithm much more adaptive to fluctuation of network traffics. This paper presents a system architecture that replaces the existing procedural segment selection algorithm with a deep reinforcement learning algorithm based on the Asynchronous Advantage Actor-Critic (A3C). The distributed A3C-based deep learning server is designed and implemented to allow multiple clients in different network conditions to stream videos simultaneously, collect learning data quickly, and learn asynchronously, resulting in greatly improved learning speed as the number of video clients increases. The performance analysis shows that the proposed algorithm outperforms both the conventional DASH algorithm and the Deep Q-Network algorithm in terms of the user's quality of experience and the speed of deep learning.

• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2019년도 춘계학술발표대회
• /
• pp.493-495
• /
• 2019

제어 분야의 가장 기초적인 시스템인 Rotary Inverted Pendulum 을 제어하기 위하여, 본 논문에서는 강화학습에서 Deep Q-Network 과 함께 대표적인 알고리즘으로 알려진 Asynchronous Advantage Actor-Critic 을 활용하여 다중 디바이스 제어를 설계한다. Deep Q-Network 알고리즘을 활용한 기존 연구와 동일한 방식으로 실 세계의 물리 에이전트와 가상 환경을 맵핑시키며, 스위치를 통하여 로컬 에이전트와 글로벌 네트워크 간 통신을 구성한다. 본 논문에서는 분산 Asynchronous Advantage Actor-Critic 을 이용함으로써 실 세계의 다중 에이전트 제어를 위한 강화 학습의 활용 가능성을 조명한다.

• 한국산업융합학회 논문집
• /
• 제24권1호
• /
• pp.19-29
• /
• 2021

In the Port Logistics system, Container Stowage planning is an important issue for cost-effective efficiency improvements. At present, Planners are mainly carrying out Stowage planning by manual or semi-automatically. However, as the trend of super-large container ships continues, it is difficult to calculate an efficient Stowage plan with manpower. With the recent rapid development of artificial intelligence-related technologies, many studies have been conducted to apply enhanced learning to optimization problems. Accordingly, in this paper, we intend to develop and present a Deep Q-Learning Network model for the Master Stowage planning of Container ships.

• Smart Structures and Systems
• /
• 제24권6호
• /
• pp.745-757
• /
• 2019

Road surface deteriorations such as potholes have caused motorists heavy monetary damages every year. However, effective road condition monitoring has been a continuing challenge to road owners. Depth cameras have a small field of view and can be easily affected by vehicle bouncing. Traditional image processing methods based on algorithms such as segmentation cannot adapt to varying environmental and camera scenarios. In recent years, novel object detection methods based on deep learning algorithms have produced good results in detecting typical objects, such as faces, vehicles, structures and more, even in scenarios with changing object distances, camera angles, lighting conditions, etc. Therefore, in this study, a Deep Learning Pothole Detector (DLPD) based on the deep region-based convolutional neural network is proposed for autonomous detection of potholes from images. About 900 images with potholes and road surface conditions are collected and divided into training and testing data. Parameters of the network in the DLPD are calibrated based on sensitivity tests. Then, the calibrated DLPD is trained by the training data and applied to the 215 testing images to evaluate its performance. It is demonstrated that potholes can be automatically detected with high average precision over 93%. Potholes can be differentiated from manholes by training and applying a manhole-pothole classifier which is constructed using the convolutional neural network layers in DLPD. Repeated detection of the same potholes can be prevented through feature matching of the newly detected pothole with previously detected potholes within a small region.

• 한국정보통신학회논문지
• /
• 제25권1호
• /
• pp.50-55
• /
• 2021

다중경로 시스템은 유선망, LTE망, 위성망 등 다양한 망을 동시에 활용하여 데이터를 전송하는 시스템으로, 통신망의 전송속도, 신뢰도, 보안성 등을 높이기 위해 제안되었다. 본 논문에서는 이 시스템에서 각 망의 지연시간을 보상으로 하는 강화학습 기반 경로 선택 방안을 제안하고자 한다. 기존의 강화학습 모델과는 다르게, deep Q 학습을 이용하여 망의 변화하는 환경에 즉각적으로 대응하도록 알고리즘을 설계하였다. 네트워크 환경에서는 보상 정보를 일정 지연시간이 지나야 얻을 수 있으므로 이를 보정하는 방안 또한 함께 제안하였다. 성능을 평가하기 위해, 분산 데이터베이스와 텐서플로우 모듈 등을 포함한 테스트베드 학습 서버를 개발하였다. 시뮬레이션 결과, 제안 알고리즘이 RTT 감소 측면에서 최저 지연시간을 선택하는 방안보다 20% 가량 좋은 성능을 가지는 것을 확인하였다.

• International Journal of Computer Science & Network Security
• /
• 제22권5호
• /
• pp.73-78
• /
• 2022

The aim of this paper is to investigate the prevalence of Deep Learning in the literature on Fire & Rescue Service. It is found that deep learning techniques are only beginning to benefit the firefighters. The popular areas where deep learning techniques are making an impact are situational awareness, decision making, mental stress, injuries, well-being of the firefighter such as his sudden fall, inability to move and breathlessness, path planning by the firefighters while getting to an fire scene, wayfinding, tracking firefighters, firefighter physical fitness, employment, prediction of firefighter intervention, firefighter operations such as object recognition in smoky areas, firefighter efficacy, smart firefighting using edge computing, firefighting in teams, and firefighter clothing and safety. The techniques that were found applied in firefighting were Deep learning, Traditional K-Means clustering with engineered time and frequency domain features, Convolutional autoencoders, Long Short-Term Memory (LSTM), Deep Neural Networks, Simulation, VR, ANN, Deep Q Learning, Deep learning based on conditional generative adversarial networks, Decision Trees, Kalman Filters, Computational models, Partial Least Squares, Logistic Regression, Random Forest, Edge computing, C5 Decision Tree, Restricted Boltzmann Machine, Reinforcement Learning, and Recurrent LSTM. The literature review is centered on Firefighters/firemen not involved in wildland fires. The focus was also not on the fire itself. It must also be noted that several deep learning techniques such as CNN were mostly used in fire behavior, fire imaging and identification as well. Those papers that deal with fire behavior were also not part of this literature review.