• Proceedings of the Korean Information Science Society Conference
• /
• 2001.10b
• /
• pp.55-57
• /
• 2001

강화학습은 한 에이전트가 자신이 놓여진 환경으로부터의 보상을 최대화할 수 있는 최적의 행동 전략을 학습하는 것이다. 따라서 강화학습은 입력(상태)과 출력(행동)의 쌍으로 명확한 훈련 예들이 제공되는 교사 학습과는 다르다. 특히 Q-학습과 같은 비 모델 기반(model-free)의 강화학습은 사전에 환경에 대한 별다른 모델을 설정하거나 학습할 필요가 없으며 다양한 상태와 행동들을 충분히 자주 경험할 수만 있으면 최적의 행동전략에 도달할 수 있어 다양한 응용분야에 적용되고 있다. 하지만 실제 응용분야에서 Q-학습과 같은 강화학습이 겪는 최대의 문제는 큰 상태 공간을 갖는 문제의 경우에는 적절한 시간 내에 각 상태와 행동들에 대한 최적의 Q값에 수렴할 수 없어 효과를 거두기 어렵다는 점이다. 이런 문제점을 고려하여 본 논문에서는 로봇 축구 시뮬레이션 환경에서 각 선수 에이전트의 동적 위치 결정을 위해 효과적인 새로운 Q-학습 방법을 제안한다. 이 방법은 원래 문제의 상태공간을 몇 개의 작은 모듈들로 나누고 이들의 개별적인 Q-학습 결과를 단순히 결합하는 종래의 모듈화 Q-학습(Modular Q-Learning)을 개선하여, 보상에 끼친 각 모듈의 기여도에 따라 모듈들의 학습결과를 적응적으로 결합하는 방법이다. 이와 같은 적응적 중재에 기초한 모듈화 Q-학습법(Adaptive Mediation based Modular Q-Learning, AMMQL)은 종래의 모듈화 Q-학습법의 장점과 마찬가지로 큰 상태공간의 문제를 해결할 수 있을 뿐 아니라 보다 동적인 환경변화에 유연하게 적응하여 새로운 행동 전략을 학습할 수 있다는 장점을 추가로 가질 수 있다. 이러한 특성을 지닌 AMMQL 학습법은 로봇축구와 같이 끊임없이 실시간적으로 변화가 일어나는 다중 에이전트 환경에서 특히 높은 효과를 볼 수 있다. 본 논문에서는 AMMQL 학습방법의 개념을 소개하고, 로봇축구 에이전트의 동적 위치 결정을 위한 학습에 어떻게 이 학습방법을 적용할 수 있는지 세부 설계를 제시한다.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2009.10a
• /
• pp.264-267
• /
• 2009

기존의 자율형 휴머노이드 로봇 축구승부차기에서는 거리센서와 시각센서를 모두 이용한다. 본 논문에서는 시각센서만을 사용하는 사람과 유사한 승부차기 시스템을 제안한다. 이를 위하여 시각센서가 유연하게 움직일 수 있는 적합한 로봇의 조립 형태와 지능적 3차원 공간분석을 채용한다. 지식표현과 추론은 자체 개발한 지식처리 시스템인 NEO를 사용하였고, 그 NEO 시스템에 지능적 처리를 위한 영상처리 라이브러리인 OpenCV를 탑재한 시스템 VisionNEO를 사용하였다.

• The Magazine of the IEIE
• /
• v.31 no.10
• /
• pp.38-52
• /
• 2004

• The KIPS Transactions:PartB
• /
• v.9B no.2
• /
• pp.139-146
• /
• 2002

The robot soccer simulation game is a dynamic multi-agent environment. In this paper we suggest a new reinforcement learning approach to each agent's dynamic positioning in such dynamic environment. Reinforcement learning is the machine learning in which an agent learns from indirect, delayed reward an optimal policy to choose sequences of actions that produce the greatest cumulative reward. Therefore the reinforcement learning is different from supervised learning in the sense that there is no presentation of input-output pairs as training examples. Furthermore, model-free reinforcement learning algorithms like Q-learning do not require defining or learning any models of the surrounding environment. Nevertheless these algorithms can learn the optimal policy if the agent can visit every state-action pair infinitely. However, the biggest problem of monolithic reinforcement learning is that its straightforward applications do not successfully scale up to more complex environments due to the intractable large space of states. In order to address this problem, we suggest Adaptive Mediation-based Modular Q-Learning (AMMQL) as an improvement of the existing Modular Q-Learning (MQL). While simple modular Q-learning combines the results from each learning module in a fixed way, AMMQL combines them in a more flexible way by assigning different weight to each module according to its contribution to rewards. Therefore in addition to resolving the problem of large state space effectively, AMMQL can show higher adaptability to environmental changes than pure MQL. In this paper we use the AMMQL algorithn as a learning method for dynamic positioning of the robot soccer agent, and implement a robot soccer agent system called Cogitoniks.

• Journal of the Institute of Convergence Signal Processing
• /
• v.10 no.1
• /
• pp.80-85
• /
• 2009

The performance of the robot is very limited in the conventional model-based control methods when the environments around a robot are not structured or are varying dynamically. The reason for that is the methods are based on the model of the environments which is very difficult to match with the real environments and on a path planning which is complex and time-consuming. On the other hand, the behavior-based control methods are not dependant on the model of the environments nor a complex planning. In those methods, a specific behavior is coupled with a specific sensor output, so the response of a robot is quite reactive and timely in dynamic and unstructured environments. In this thesis, we propose a situation dependant behavior based control architecture, in which a robot may behave differently to the same sensor output depending on various situations. We also show some experimental results to show the feasibility of the proposed control architecture.

• Proceedings of the Korean Society of Precision Engineering Conference
• /
• 2002.05a
• /
• pp.309-313
• /
• 2002

This paper presents an artificial intelligent model for a soccer robot. We classified soccer robot as artificial intelligent model into three elemental groups including instinct intuition and reason. Instinct is responsible for keeping the ball, walking or rushing toward the ball. This is very simple fundamental action without regard to associates and enemies. Intuition contributes to the faster/slower moving and simple basic turning to get near to the ball and to make a goal noticing associates and enemies. Reason is the most intelligent part, the law of reason is not simple relatively with instinct and intuition. We shall expect to design the best law of reason for a soccer robot some time. We also compared nerve system and muscles of human being model with controller and motors of a physical soccer robot model individually. We had designed several algorithms and made programs to investigate effects and control soccer robot.

• Proceedings of the Korean Society of Precision Engineering Conference
• /
• 2003.06a
• /
• pp.975-978
• /
• 2003

This paper presents an artificial intelligent model for a soccer robot. We classified soccer robot as artificial intelligent model into three elemental groups as instinct, intuition, reason. Instinct is responsible for keeping the ball, driving or rushing toward the ball. This is very simple fundamental action without regard to associates and enemies. Intuition contributes to the fast/slow moving and simple basic turing to get near to the ball and to make a goal noticing associates and enemies. Reason is the most intelligent part. The law of reason is not simple relatively with instinct and intuition. We also compared nerve system and muscles of human being model with controller and motor of physical soccer robot model individually. We had designed several algorithms and made programs th investigate effects and control soccer robot.

• Journal of the Korean Society for Precision Engineering
• /
• v.19 no.9
• /
• pp.142-150
• /
• 2002

This study describes a method of finding an optimal path of a soccer robot by using a concentric circle method with different radii of rotation. Comparing with conventional algorithms which try to find the shortest path length, the variable concentric circle method find the shortest moving time. The radius fur the shortest moving time for a given ball location depends on the relative location between a shooting robot and a ball. Practically it is difficult to find an analytical solution due to many unknowns. Assuming a radius of rotation within a possible range, total path moving time can be calculated by adding the times needed for straight path and circular path. Among these times the shortest time is obtained. In this paper, a graphical solution is presented such that the game ground is divided into 3 regions with a minimum, medium, and maximum radius of rotation.

• Proceedings of the Korean Society of Precision Engineering Conference
• /
• 1996.11a
• /
• pp.434-438
• /
• 1996

In this paper we present the multi-agent robot system and the vision system developed for participating in micro robot soccer tournament. The multi-agent robot system consists of micro robot, a vision system, a host computer and a communication module. Micro robot are equipped with two mini DC motors witf encoders and gearboxes, a R/F receiver, a CPU and infrared sensors for obstacle detection. A vision system is used to recognize the position of the ball and opponent robots, position and orientation of our robots. The vision system is composed of a color CCD camera and a vision processing unit(AISI vision computer). The vision algorithm is based on morphological method. And it takes about 90 msec to detect ball and 3-our robots and 3-opponent robots with reasonable accuracy

• Journal of the Korean Society for Precision Engineering
• /
• v.19 no.9
• /
• pp.133-141
• /
• 2002

In this paper, we propose a hierarchical motion controller with stand-alone vision system to enhance the flexibility of the robot soccer system. In addition, we simplified the model of dynamic environments of the robot using petri-net and simple state diagram. Based on the proposed model, we designed the robot soccer system with velocity and position controller that includes 4-level hierarchically structured controller. Some experimental results using the stand-alone vision system from host system show improvement of the controller performance by reducing processing time of vision algorithm.