• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.6
• /
• pp.61-67
• /
• 2023

The rise of 5G and the proliferation of smart devices have underscored the significance of multi-access edge computing (MEC). Amidst this trend, interest in effectively processing computation-intensive and latency-sensitive applications has increased. This study investigated a novel task offloading strategy considering the probabilistic MEC environment to address these challenges. Initially, we considered the frequency of dynamic task requests and the unstable conditions of wireless channels to propose a method for minimizing vehicle power consumption and latency. Subsequently, our research delved into a deep reinforcement learning (DRL) based offloading technique, offering a way to achieve equilibrium between local computation and offloading transmission power. We analyzed the power consumption and queuing latency of vehicles using the deep deterministic policy gradient (DDPG) and deep Q-network (DQN) techniques. Finally, we derived and validated the optimal performance enhancement strategy in a vehicle based MEC environment.

• KIPS Transactions on Software and Data Engineering
• /
• v.10 no.5
• /
• pp.179-186
• /
• 2021

Manipulation of complex objects with an anthropomorphic robot hand like a human hand is a challenge in the human-centric environment. In order to train the anthropomorphic robot hand which has a high degree of freedom (DoF), human demonstration augmented deep reinforcement learning policy optimization methods have been proposed. In this work, we first demonstrate augmentation of human demonstration in deep reinforcement learning (DRL) is effective for object manipulation by comparing the performance of the augmentation-free Natural Policy Gradient (NPG) and Demonstration Augmented NPG (DA-NPG). Then three DRL policy optimization methods, namely NPG, Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO), have been evaluated with DA (i.e., DA-NPG, DA-TRPO, and DA-PPO) and without DA by manipulating six objects such as apple, banana, bottle, light bulb, camera, and hammer. The results show that DA-NPG achieved the average success rate of 99.33% whereas NPG only achieved 60%. In addition, DA-NPG succeeded grasping all six objects while DA-TRPO and DA-PPO failed to grasp some objects and showed unstable performances.

• Electronics and Telecommunications Trends
• /
• v.35 no.4
• /
• pp.81-90
• /
• 2020

The use of machine learning technologies such as deep and reinforcement learning has proliferated in various domains with the advancement of deep neural network studies. To make the learning successful, both big data acquisition and fast processing are required. However, for some physical world applications such as autonomous drone flight, it is difficult to achieve efficient learning because learning with a premature A.I. is dangerous, cost-ineffective, and time-consuming. To solve these problems, simulation-based approaches can be considered. In this study, we analyze recent trends in drone simulation technologies and compare their features. Subsequently, we introduce Octopus, which is a highly precise and scalable drone simulator being developed by ETRI.

• The Journal of Korea Robotics Society
• /
• v.14 no.3
• /
• pp.157-162
• /
• 2019

The video game Tetris is one of most popular game and it is well known that its game rule can be modelled as MDP (Markov Decision Process). This paper presents a DQN (Deep Q-Network) based game agent for Tetris game. To this end, the state is defined as the captured image of the Tetris game board and the reward is designed as a function of cleared lines by the game agent. The action is defined as left, right, rotate, drop, and their finite number of combinations. In addition to this, PER (Prioritized Experience Replay) is employed in order to enhance learning performance. To train the network more than 500000 episodes are used. The game agent employs the trained network to make a decision. The performance of the developed algorithm is validated via not only simulation but also real Tetris robot agent which is made of a camera, two Arduinos, 4 servo motors, and artificial fingers by 3D printing.

• Journal of the Korea Computer Graphics Society
• /
• v.30 no.3
• /
• pp.109-123
• /
• 2024

Research into vision-based end-to-end autonomous driving systems utilizing deep learning and reinforcement learning has been steadily increasing. These systems typically encode continuous and high-dimensional vehicle states, such as location, velocity, orientation, and sensor data, into latent features, which are then decoded into a vehicular control policy. The complexity of urban driving environments necessitates the use of state representation learning through networks like Variational Autoencoders (VAEs) or Convolutional Neural Networks (CNNs). This paper analyzes the impact of different image state encoding methods on reinforcement learning performance in autonomous driving. Experiments were conducted in the CARLA simulator using RGB images and semantically segmented images captured by the vehicle's front camera. These images were encoded using VAE and Vision Transformer (ViT) networks. The study examines how these networks influence the agents' learning outcomes and experimentally demonstrates the role of each state representation technique in enhancing the learning efficiency and decision- making capabilities of autonomous driving systems.

• Journal of Broadcast Engineering
• /
• v.26 no.5
• /
• pp.599-607
• /
• 2021

Inactive region means a region filled with invalid pixel values to represent a specific image. Generally, inactive regions are occurred when the non-rectangular formatted images are converted to the rectangular shaped image, especially when 3D images are represented in 2D format. Because these inactive regions highly degrade the compression efficiency, filtering approaches are often applied to the boundaries between active and inactive regions. However, the image characteristics are not carefully considered during filtering. In the proposed method, inactive regions are padded through reinforcement learning that can consider the compression process and the image characteristics. Experimental results show that the proposed method performs an average of 3.4% better than the conventional padding method.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.8
• /
• pp.3821-3841
• /
• 2019

Cloud radio access networks (C-RANs) have been regarded in recent times as a promising concept in future 5G technologies where all DSP processors are moved into a central base band unit (BBU) pool in the cloud, and distributed remote radio heads (RRHs) compress and forward received radio signals from mobile users to the BBUs through radio links. In such dynamic environment, automatic decision-making approaches, such as artificial intelligence based deep reinforcement learning (DRL), become imperative in designing new solutions. In this paper, we propose a generic framework of autonomous cell activation and customized physical resource allocation schemes for energy consumption and QoS optimization in wireless networks. We formulate the problem as fractional power control with bandwidth adaptation and full power control and bandwidth allocation models and set up a Q-learning model to satisfy the QoS requirements of users and to achieve low energy consumption with the minimum number of active RRHs under varying traffic demand and network densities. Extensive simulations are conducted to show the effectiveness of our proposed solution compared to existing schemes.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.9
• /
• pp.3138-3150
• /
• 2021

Multimedia services on the Internet are continuously increasing. Accordingly, the demand for a technology for efficiently delivering multimedia traffic is also constantly increasing. The multicast technique, that delivers the same content to several destinations, is constantly being developed. This technique delivers a content from a source to all destinations through the multicast tree. The multicast tree with low cost increases the utilization of network resources. However, the finding of the optimal multicast tree that has the minimum link costs is very difficult and its calculation complexity is the same as the complexity of the Steiner tree calculation which is NP-complete. Therefore, we need an effective way to obtain a multicast tree with low cost and less calculation time on SDN-based smart network platforms. In this paper, we propose a new multicast tree generation algorithm which produces a multicast tree using an agent trained by model-based meta reinforcement learning. Experiments verified that the proposed algorithm generated multicast trees in less time compared with existing approximation algorithms. It produced multicast trees with low cost in a dynamic network environment compared with the previous DQN-based algorithm.

• Computers and Concrete
• /
• v.30 no.1
• /
• pp.43-74
• /
• 2022

Machine learning technique is recently opening new opportunities to identify the complex shear transfer mechanisms of reinforced concrete (RC) beam members. This study employed 1224 shear test specimens to train decision tree-based machine learning (ML) programs, by which strong correlations between shear capacity of RC beams and key input parameters were affirmed. In addition, shear contributions of concrete and shear reinforcement (the so-called V_c and V_s) were identified by establishing three independent ML models trained under different strategies with various combinations of datasets. Detailed parametric studies were then conducted by utilizing the well-trained ML models. It appeared that the presence of shear reinforcement can make the predicted shear contribution from concrete in RC beams larger than the pure shear contribution of concrete due to the intervention effect between shear reinforcement and concrete. On the other hand, the size effect also brought a significant impact on the shear contribution of concrete (V_c), whereas, the addition of shear reinforcements can effectively mitigate the size effect. It was also found that concrete tends to be the primary source of shear resistance when shear span-depth ratio a/d<1.0 while shear reinforcements become the primary source of shear resistance when a/d>2.0.

• Journal of the Korean Society of Industry Convergence
• /
• v.24 no.1
• /
• pp.19-29
• /
• 2021

In the Port Logistics system, Container Stowage planning is an important issue for cost-effective efficiency improvements. At present, Planners are mainly carrying out Stowage planning by manual or semi-automatically. However, as the trend of super-large container ships continues, it is difficult to calculate an efficient Stowage plan with manpower. With the recent rapid development of artificial intelligence-related technologies, many studies have been conducted to apply enhanced learning to optimization problems. Accordingly, in this paper, we intend to develop and present a Deep Q-Learning Network model for the Master Stowage planning of Container ships.