• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2003.09a
• /
• pp.66-69
• /
• 2003

This paper considers a reinforcement learning(RL) which deals with real environments. Most reinforcement learning studies have been made by simulations because real-environment learning requires large computational cost and much time. Furthermore, it is more difficult to acquire many rewards efficiently in real environments than in virtual ones. The most important requirement to make real-environment learning successful is the appropriate construction of the state space. In this paper, to begin with, I show the basic overview of the reinforcement learning under real environments. Next, 1 introduce a state-space construction method under real environmental which is State Partition Method. Finally I apply this method to a robot navigation problem and compare it with conventional methods.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.7 no.1
• /
• pp.77-84
• /
• 2007

In this paper, Takagi-Sugeno (TS) fuzzy integral control is investigated to regulate the output voltage of an asymmetric half-bridge (AHB) DC/DC converter; First, we model the dynamic characteristics of the AHB DC/DC converter with state-space averaging method and small perturbation at an operating point. After introducing an additional integral state of the output regulation error, we obtain the $5^{th}$-order TS fuzzy model of the AHB DC/DC converter. Second, the concept of the parallel distributed compensation is applied to design the fuzzy integral controller, in which the state feedback gains are obtained by solving the linear matrix inequalities (LMIs). Finally, simulation results are presented to show the performance of the considered design method as the output voltage regulator and compared to the results for which the conventional loop gain method is used.

• Journal of Institute of Control, Robotics and Systems
• /
• v.20 no.1
• /
• pp.6-11
• /
• 2014

Repetitive control is a specialized control scheme to track and/or attenuate a periodic reference trajectory and/or disturbance. Most researches about repetitive control have been performed in the frequency domain. Recently, several approaches to deal with repetitive control systems in the state space are developed by representing a q filter as a state-space equation. This paper presents a design method of a repetitive control system in the state space to satisfy $H_{\infty}$ performance. The overall system is composed of a plant, a repetitive controller, and a state-feedback controller, which can be converted to a standard form used in $H_{\infty}$ control. A LMI (Linear Matrix Inequality)-based stability condition is derived for fixed state-feedback gains. Under a given q filter, another LMI condition is derived to improve $H_{\infty}$ performance and is employed to find state-feedback gains by solving an optimization problem. Finally, to verify the feasibility of the proposed method, a numerical example is demonstrated.

• Journal of Korea Multimedia Society
• /
• v.3 no.1
• /
• pp.77-90
• /
• 2000

Real-Time verification is a procedure that verifies the correctness of specification related to requirement in time as well as in logic. One serious problem encountered in the verification task is that the state space grows exponentially owing to the unboundedness of time, which is termed the state space explosion problem. In this paper, we propose a real-time verification technique checking the correctness of specification by showing that a system model described in timed automata is equivalent to the characteristic of system property specified in timed modal-mu calculus. For this, we propose a local model checking method based on the value of the formula in initial state with constructing product graph concerned to only the nodes needed for verification process. Since this method does not search for every state of system model, the state space is reduced drastically so that the proposed method can be applied effectively to real-time system verification.

• Structural Engineering and Mechanics
• /
• v.34 no.6
• /
• pp.739-753
• /
• 2010

Pointing to the design requirement of prestressed space grid structure being the target cable force, the pretension scheme decision analysis method is studied when there's great difference between structural actual state and the analytical model. Based on recursive formulation of cable forces, the simulative recursive system for pretension process is established from the systematic viewpoint, including four kinds of parameters, i.e., system initial value (structural initial state), system input value (tensioning control force scheme), system state parameters (influence matrix of cable forces), system output value (pretension accomplishment). The system controllability depends on the system state parameters. Based on cable force observation values, the influence matrix for system state parameters can be calculated, making the system controllable. Next, the pretension scheme decision method based on cable force observation values can be formed on the basis of iterative calculation for recursive system. In this way, the tensioning control force scheme that can meet the design requirement when next cyclic supplemental tension finished is obtained. Engineering example analysis results show that the proposed method in this paper can reduce a lot of cyclic tensioning work and meanwhile the design requirement can be met.

• Journal of Computing Science and Engineering
• /
• v.9 no.1
• /
• pp.9-19
• /
• 2015

The most important challenge in the region-based abstraction method as an approach to compute the state space of time Petri Nets (TPNs) for model checking is that the method results in a huge number of regions, causing a state explosion problem. Thus, region-based abstraction methods are not appropriate for use in developing practical tools. To address this limitation, this paper applies a modification to the basic region abstraction method to be used specially for computing the state space of TPN models, so that the number of regions becomes smaller than that of the situations in which the current methods are applied. The proposed approach is based on the special features of TPN that helps us to construct suitable and small region graphs that preserve the time properties of TPN. To achieve this, we use TPN-TCTL as a timed extension of CTL for specifying a subset of properties in TPN models. Then, for model checking TPN-TCTL properties on TPN models, CTL model checking is used on TPN models by translating TPN-TCTL to the equivalent CTL. Finally, we compare our proposed method with the current region-based abstraction methods proposed for TPN models in terms of the size of the resulting region graph.

• The Transactions of the Korea Information Processing Society
• /
• v.5 no.8
• /
• pp.2091-2101
• /
• 1998

Interoperability(lOP) is one of the major goal of Information Technology and Telecommunication fields. In this paper, we discuss dcvclOlJing an interoperabiiity testing(IOPT) method. As is easily gueOisecl from conformance testing and formal verification. state space explosion problem is the most serious problem we encounter in deriving interoperability test method and its test suite. A new state space reduction method to support interoperability testing is suggested based on Petri nets. The proposed test method can be applied to 1 to many communication protocols as well as 1 to 1 communication protocols efficiently

• 제어로봇시스템학회:학술대회논문집
• /
• 2000.10a
• /
• pp.174-174
• /
• 2000

Q-learning, based on discrete state and action space, is a most widely used reinforcement Learning. However, this requires a lot of memory and much time for learning all actions of each state when it is applied to a real mobile robot navigation using continuous state and action space Region-based Q-learning is a reinforcement learning method that estimates action values of real state by using triangular-type action distribution model and relationship with its neighboring state which was defined and learned before. This paper proposes a new Region-based Q-learning which uses a reward assigned only when the agent reached the target, and get out of the Local optimal path with adjustment of random action rate. If this is applied to mobile robot navigation, less memory can be used and robot can move smoothly, and optimal solution can be learned fast. To show the validity of our method, computer simulations are illusrated.

• Journal of the Korean housing association
• /
• v.15 no.3
• /
• pp.93-99
• /
• 2004

This study was conducted to estimate the solar energy, as an alternative energy evaluating an effect of solar radiation on indoor space of residential building. The basic data of solar radiation which is useful for architectural design was suggested using theoretical and experimental analysis. Accordingly, this study was carried out measuring the solar energy using Explicit Method. These results were compared with the results using steady state heat transfer method. The results of this study are summarized as follows; Based on the results using Explicit Method and steady state heat transfer on the indoor space of building, it was shown that an analysis on heat transfer using Explicit Method is more sensitive to the outdoor environmental changes. The results using Explicit Method to analysis and evaluate the solar radiation should be used for residential building design.

• Korean Chemical Engineering Research
• /
• v.45 no.1
• /
• pp.67-72
• /
• 2007

An efficient method for identification of MIMO state space model has been developed by combining partial least squares (PLS) regression, balanced realization, and balanced truncation. In the developed method, a MIMO system is decomposed into multiple MISO systems each of which is represented by a high-order ARX model and the parameters of the ARX models are estimated by PLS. Then, MISO state space models for respective MISO ARX transfer function are found through realization and combined to a MIMO state space model. Finally, a minimal balanced MIMO state space model is obtained through balanced realization and truncation. The proposed method was applied to the design of model predictive control for temperature control of a high pressure $CO_2$ solubility measurement system.