• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2001.01a
• /
• pp.65-70
• /
• 2001

We consider a hyper-redundant system that consists of many uniform units. The hyper-redundant system has many degrees of freedom and it can accomplish various tasks. Applysing the reinforcement learning to the hyper-redundant system is very attractive because it is possible to acquire various behaviors for various tasks automatically. In this paper we present a new reinforcement learning algorithm "Q-learning with propagation of motion". The algorithm is designed for the multi-agent systems that have strong connections. The proposed algorithm needs only one small Q-table even for a large scale system. So using the proposed algorithm, it is possible for the hyper-redundant system to learn the effective behavior. In this algorithm, only one leader agent learns the own behavior using its local information and the motion of the leader is propagated to another agents with time delay. The reward of the leader agent is given by using the whole system information. And the effective behavior of the leader is learned and the effective behavior of the system is acquired. We apply the proposed algorithm to a snake-like hyper-redundant robot. The necessary condition of the system to be Markov decision process is discussed. And the computer simulation of learning the locomotion is demonstrated. From the simulation results we find that the task of the locomotion of the robot to the desired point is learned and the winding motion is acquired. We can conclude that our proposed system and our analysis of the condition, that the system is Markov decision process, is valid.

• Journal of Digital Convergence
• /
• v.19 no.12
• /
• pp.347-352
• /
• 2021

In this paper, we propose a method for finding feature subsets that are effective for classification in an input dataset by using a multi-agent reinforcement learning method. In the field of machine learning, it is crucial to find features suitable for classification. A dataset may have numerous features; while some features may be effective for classification or prediction, others may have little or rather negative effects on results. In machine learning problems, feature selection for increasing classification or prediction accuracy is a critical problem. To solve this problem, we proposed a feature selection method based on reinforced learning. Each feature has one agent, which determines whether the feature is selected. After obtaining corresponding rewards for each feature that is selected, but not by the agents, the Q-value of each agent is updated by comparing the rewards. The reward comparison of the two subsets helps agents determine whether their actions were right. These processes are performed as many times as the number of episodes, and finally, features are selected. As a result of applying this method to the Wisconsin Breast Cancer, Spambase, Musk, and Colon Cancer datasets, accuracy improvements of 0.0385, 0.0904, 0.1252 and 0.2055 were shown, respectively, and finally, classification accuracies of 0.9789, 0.9311, 0.9691 and 0.9474 were achieved, respectively. It was proved that our proposed method could properly select features that were effective for classification and increase classification accuracy.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.11
• /
• pp.4244-4274
• /
• 2021

The utilization of UAVs in various fields has led to the development of flying ad hoc network (FANET) technology. In a network environment with highly dynamic topology and frequent link changes, the traditional routing technology of FANET cannot satisfy the new communication demands. Traditional routing algorithm, based on geographic location, can "fall" into a routing hole. In view of this problem, we propose a geolocation routing protocol based on multi-agent reinforcement learning, which decreases the packet loss rate and routing cost of the routing protocol. The protocol views each node as an intelligent agent and evaluates the value of its neighbor nodes through the local information. In the value function, nodes consider information such as link quality, residual energy and queue length, which reduces the possibility of a routing hole. The protocol uses global rewards to enable individual nodes to collaborate in transmitting data. The performance of the protocol is experimentally analyzed for UAVs under extreme conditions such as topology changes and energy constraints. Simulation results show that our proposed QLGR-S protocol has advantages in performance parameters such as throughput, end-to-end delay, and energy consumption compared with the traditional GPSR protocol. QLGR-S provides more reliable connectivity for UAV networking technology, safeguards the communication requirements between UAVs, and further promotes the development of UAV technology.

• Annual Conference of KIPS
• /
• 2021.05a
• /
• pp.447-450
• /
• 2021

시뮬레이션을 비롯한 많은 다중 에이전트 환경에서는 중앙 집중 훈련 및 분산 수행(centralized training with decentralized execution; CTDE) 방식이 활용되고 있다. CTDE 방식 하에서 중앙 집중 훈련 및 분산 수행 환경에서의 다중 에이전트 학습을 위한 상태 행동 가치 기반(state-action value; Q-value) 다중 에이전트 알고리즘들에 대한 많은 연구가 이루어졌다. 이러한 알고리즘들은 Independent Q-learning (IQL)이라는 강력한 벤치 마크 알고리즘에서 파생되어 다중 에이전트의 공동의 상태 행동 가치의 분해(Decomposition) 문제에 대해 집중적으로 연구되었다. 본 논문에서는 앞선 연구들에 관한 알고리즘들에 대한 분석과 실용적이고 일반적인 도메인에서의 실험 분석을 통해 검증한다.

• Proceedings of the Korean Information Science Society Conference
• /
• 2006.10b
• /
• pp.393-396
• /
• 2006

강화학습은 환경과 상호작용하는 과정을 통하여 목표를 이루기 위한 전략을 학습하는 방법으로써 에이전트의 학습방법으로 많이 사용한다. 독립적인 에이전트가 아닌 상호 의사소통이 가능한 다중 에이전트 환경에서 에이전트의 학습정보를 서로 검색 및 공유가 가능하다면 환경이 거대하더라도 기존의 강화학습 보다 빠르게 학습이 이루어질 것이다. 하지만 아직 다중 에이전트 환경에서 학습 방법에 대한 연구가 미흡하여 학습정보의 검색과 공유에 대해 다양한 방법들이 요구되고 있다. 본 논문에서는 대상 에이전트 학습 정보와 주변 에이전트들의 학습 정보 사이에 편집거리를 비교하여 유사한 에이전트를 찾고 그 에이전트 정보를 강화학습 사전정보로 사용함으로써 학습속도를 향상시킨 ED+Q-Learning 시스템을 제안한다.

• International Journal of Computer Science & Network Security
• /
• v.24 no.10
• /
• pp.71-80
• /
• 2024

The Internet of Things (IoT) is an emerging element that is becoming increasingly indispensable to the Internet and shaping our current understanding of the future of the Internet. IoT continues to extend deeper into the daily lives of people, offering distributed and critical services. In contrast with current Internet, IoT depends on a dynamic architecture where physical objects with embedded sensors will communicate via cloud to send and analyze data [1-3]. Its security troubles will surely impinge all aspects of civilization. Mobile agents are widely used in the context of the IoT and due to the possibility of transmitting their execution status from one device to another in an IoT network, they offer many advantages such as reducing network load, encapsulating protocols, exceeding network latency, etc. Also, cryptographic technologies, like PKI and Blockchain technology, and Artificial Intelligence are growing rapidly allowing the addition of an approved security layer in many areas. Security issues related to mobile agent migration can be resolved with the use of these technologies, thus allowing increased reliability and credibility and ensure information collecting, sharing, and processing in IoT environments, while ensuring maximum autonomy by relying on the AI to allow the agent to choose the most secure and optimal path between the nodes of an IoT environment. This paper aims to present a new model to secure mobile agents in the context of the Internet of Things based on Public Key Infrastructure (PKI), Ethereum Blockchain Technology and Q-learning. The proposed model provides a secure migration of mobile agents to ensure security and protect the IoT application against malevolent nodes that could infiltrate these IoT systems.

• Journal of KIISE:Software and Applications
• /
• v.35 no.3
• /
• pp.179-188
• /
• 2008

An important issue in multiagent reinforcement learning is how an agent should team its optimal policy through trial-and-error interactions in a dynamic environment where there exist other agents able to influence its own performance. Most previous works for multiagent reinforcement teaming tend to apply single-agent reinforcement learning techniques without any extensions or are based upon some unrealistic assumptions even though they build and use explicit models of other agents. In this paper, basic concepts that constitute the common foundation of multiagent reinforcement learning techniques are first formulated, and then, based on these concepts, previous works are compared in terms of characteristics and limitations. After that, a policy model of the opponent agent and a new multiagent reinforcement learning method using this model are introduced. Unlike previous works, the proposed multiagent reinforcement learning method utilize a policy model instead of the Q function model of the opponent agent. Moreover, this learning method can improve learning efficiency by using a simpler one than other richer but time-consuming policy models such as Finite State Machines(FSM) and Markov chains. In this paper. the Cat and Mouse game is introduced as an adversarial multiagent environment. And effectiveness of the proposed multiagent reinforcement learning method is analyzed through experiments using this game as testbed.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.21 no.3
• /
• pp.175-181
• /
• 2021

This paper analyzes the artificial intelligence-based approach for short-term energy consumption prediction. In this paper, we employ the reinforcement learning algorithms to improve the limitation of the supervised learning algorithms which usually utilize to the short-term energy consumption prediction technologies. The supervised learning algorithm-based approaches have high complexity because the approaches require contextual information as well as energy consumption data for sufficient performance. We propose a deep reinforcement learning algorithm based on multi-agent to predict energy consumption only with energy consumption data for improving the complexity of data and learning models. The proposed scheme is simulated using public energy consumption data and confirmed the performance. The proposed scheme can predict a similar value to the actual value except for the outlier data.

• Annual Conference of KIPS
• /
• 2002.11a
• /
• pp.369-372
• /
• 2002

다양한 특성들을 가지고 있는 멀티에이전트 시스템의 행동학습은 에이전트 설계에 많은 부담을 덜어준다. 특성들로부터 나오는 다양한 행동의 효과적인 학습은 에이전트들이 환경에 대한 자율성과 반응성을 높여준 수 있다. 행동학습은 model-based learning과 같은 교사학습보다는 각 상태를 바로 지각하여 학습하는 강화학습과 같은 비교사 학습이 효과적이다. 본 논문은 로봇축구환경에 에이전트들의 행동을 개선된 강화학습법인 Modular Q-learning을 적용하여 복잡한 상태공간을 효과적으로 나누어 에이전트들의 자율성과 반응성을 높일 수 있는 강화학습구조를 제안한다.

• Journal of Korean Association for Spatial Structures
• /
• v.23 no.3
• /
• pp.87-94
• /
• 2023

Chloride is one of the most common threats to reinforced concrete (RC) durability. Alkaline environment of concrete makes a passive layer on the surface of reinforcement bars that prevents the bar from corrosion. However, when the chloride concentration amount at the reinforcement bar reaches a certain level, deterioration of the passive protection layer occurs, causing corrosion and ultimately reducing the structure's safety and durability. Therefore, understanding the chloride diffusion and its prediction are important to evaluate the safety and durability of RC structure. In this study, the chloride diffusion coefficient is predicted by machine learning techniques. Various machine learning techniques such as multiple linear regression, decision tree, random forest, support vector machine, artificial neural networks, extreme gradient boosting annd k-nearest neighbor were used and accuracy of there models were compared. In order to evaluate the accuracy, root mean square error (RMSE), mean square error (MSE), mean absolute error (MAE) and coefficient of determination (R2) were used as prediction performance indices. The k-fold cross-validation procedure was used to estimate the performance of machine learning models when making predictions on data not used during training. Grid search was applied to hyperparameter optimization. It has been shown from numerical simulation that ensemble learning methods such as random forest and extreme gradient boosting successfully predicted the chloride diffusion coefficient and artificial neural networks also provided accurate result.