• Proceedings of the IEEK Conference
• /
• 2001.06c
• /
• pp.13-16
• /
• 2001

In reinforcement teaming, Q-learning converges quite slowly to a good policy. Its because searching for the goal state takes very long time in a large stochastic domain. So I propose the speedup method using the Q-value initialization for model-free reinforcement learning. In the speedup method, it learns a naive model of a domain and makes boundaries around the goal state. By using these boundaries, it assigns the initial Q-values to the state-action pairs and does Q-learning with the initial Q-values. The initial Q-values guide the agent to the goal state in the early states of learning, so that Q-teaming updates Q-values efficiently. Therefore it saves exploration time to search for the goal state and has better performance than Q-learning. 1 present Speedup Q-learning algorithm to implement the speedup method. This algorithm is evaluated. in a grid-world domain and compared to Q-teaming.

• Proceedings of the KIEE Conference
• /
• 2003.07b
• /
• pp.1114-1116
• /
• 2003

Voltage sags are known as a serious problem causing mal-operation of equipment like computers, process controllers and adjustable-speed drives. In this paper, characterization of voltage sag and an overview of methods used in the mitigation of voltage sags are presented. Moreover a fast detection method for voltage disturbances is explored. The algorithm is based on the theory that allows a set of three-phase voltages be converted the d-q value. The utility input voltages are sensed and then converted to some quantities in the d-q transformation. And the difference between reference value and input value are showed that some disturbances happened in the system.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2005.04a
• /
• pp.198-201
• /
• 2005

본 논문에서는 다수개의 로봇을 효율적으로 제어하기 위한 면적기반 Q-learning에 대해 논한다. 각 로봇은 $60^{\circ}$의 각을 이루도록 배치된 6개 센서를 가지고 있고 이를 통해 자신과 주변환경 사이의 거리를 센싱한다. 다음으로, 이 획득된 거리 데이터들로부터 6방향의 면적을 계산하여, 이후의 진행에 있어 보다 넓은 행동 반경을 보장해주는 영역으로 이동한다. 이 이동을 어떤 상태에서 다른 상태로의 전이로 간주, 이동 후 다시 6방향의 면적을 계산하여 이전 상태에서 현재 상태로의 행동에 대한 Q-Value를 업데이트 한다. 본 논문의 실험에서는 5개의 로봇을 이용해 장애물 사이에 숨어있는 물체를 찾아내는 것을 시도하였고, 3개의 서로 다른 제어 방법 - 랜덤 탐색, 면적 기반 탐색, 면적 기반 Q-learning 탐색 - 에 따른 결과를 나타내었다.

• Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
• /
• v.28 no.1
• /
• pp.96-104
• /
• 2014

The all pass filter generates a virtual q-axis voltage component with $90^{\circ}$ phase delay but the virtual q-axis voltage cannot detect the voltage depending on the phase of the grid voltage. To overcome the problem, q-axis voltage component is generated from difference between the current and previous value of d-axis voltage component in the stationary reference frame. However, the difference voltage between the current and previous value around the zero crossing voltage is not enough to detect the voltage sag. Therefore, the new detection scheme which can detect the sag around the zero crossing voltage is proposed.

• Transactions on Control, Automation and Systems Engineering
• /
• v.3 no.3
• /
• pp.170-175
• /
• 2001

Q-learning is a kind of reinforcement learning where the agent solves the given task based on rewards received from the environment. Most research done in the field of Q-learning has focused on discrete domains, although the environment with which the agent must interact is generally continuous. Thus we need to devise some methods that enable Q-learning to be applicable to the continuous problem domain. In this paper, an extended fuzzy rule is proposed so that it can incorporate Q-learning. The interpolation technique, which is widely used in memory-based learning, is adopted to represent the appropriate Q value for current state and action pair in each extended fuzzy rule. The resulting structure based on the fuzzy inference system has the capability of solving the continuous state about the environment. The effectiveness of the proposed structure is shown through simulation on the cart-pole system.

• Journal of IKEEE
• /
• v.23 no.4
• /
• pp.1150-1156
• /
• 2019

This paper explores a model-free value-based approach for solving survival gridworld problem. Survival gridworld problem opens up a challenge involving taking risks to gain better rewards. Classic value-based approach in model-free reinforcement learning assumes minimal risk decisions. The proposed method involves a hybrid on-policy and off-policy updates to experience roll-outs using a modified Q-based update equation that introduces a parametric linear rectifier and motivational discount. The significance of this approach is it allows model-free training of agents that take into account risk factors and motivated exploration to gain better path decisions. Experimentations suggest that the proposed method achieved better exploration and path selection resulting to higher episode scores than classic off-policy and on-policy Q-based updates.

• Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
• /
• v.28 no.12
• /
• pp.65-73
• /
• 2014

In this paper, the detection scheme of the voltage variation using a two difference voltage is proposed. The conventional sag detector is from a single-phase digital phase-locked loop (DPLL) that is based on a d-q transformation using an all-pass filter (APF). The APF generates a virtual q-axis voltage component with $90^{\circ}$ phase delay but the APF cannot generate the virtual q-axis voltage depending on the phase of the grid voltage. To overcome the problem, q-axis voltage component is generated from difference between the current and previous value of d-axis voltage component in the stationary reference frame. However, the difference voltage around the zero crossing is not enough to detect the voltage sag. Therefore, the new detection scheme using the two difference voltage which can detect the sag around the zero crossing voltage is proposed.

• Transactions on Electrical and Electronic Materials
• /
• v.17 no.4
• /
• pp.212-216
• /
• 2016

A reflective electronic display that uses negatively and positively charged particles has excellent bistability, a welldefined threshold voltage, and an extremely fast response time in comparison with other reflective displays. This type of display shows images through the movement of charged particles whose motion depends on the value of q/m (charge per mass for a particle). However, the ratio q/m can easily be changed by the forces acting on the charged particles in a cell of the panel and by friction that occurs after mixing oppositely charged particles and in the particle-insertion process. In this study, we propose a method to determine the appropriate range of q/m by using the reflectivity and response time of charged particles to modify q/m. In this manner, the electrical and optical properties of reflective displays are improved.

• Journal of Electrical Engineering and Technology
• /
• v.11 no.1
• /
• pp.157-166
• /
• 2016

In this paper, a new sag and peak voltage detector is proposed for a single-phase inverter using delta square operation. The conventional sag detector is from a single-phase digital phase-locked loop (DPLL) that is based on d-q transformations using an all-pass filter (APF). The d-q transformation is typically used in the three-phase coordinate system. The APF generates a virtual q-axis voltage component with a 90° phase delay, but this virtual phase cannot reflect a sudden change in the grid voltage at the instant the voltage sag occurs. As a result, the peak value is drastically distorted, and it settles down slowly. A modified APF generates the virtual q-axis voltage component from the difference between the current and the previous values of the d-axis voltage component in the stationary reference frame. However, the modified APF cannot detect the voltage sag and peak value when the sag occurs around the zero crossing points such as 0° and 180°, because the difference voltage is not sufficient to detect the voltage sag. The proposed algorithm detects the sag voltage through all regions including the zero crossing voltage. Moreover, the exact voltage drop can be acquired by calculating the q-axis component that is proportional to the d-axis component. To verify the feasibility of the proposed system, the conventional and proposed methods are compared using simulations and experimental results.

• Journal of the Korean Institute of Electrical and Electronic Material Engineers
• /
• v.20 no.12
• /
• pp.1049-1055
• /
• 2007

Lots of integration work has been done in order to miniaturize the devices for communication. To do this work, one of key work is to get miniaturized inductor with high Q factor for RF circuitry. However, it is not easy to get high Q inductor with silicon based substrate in the range of GHz. Although silicon is well known for its good electrical and mechanical characteristics, silicon has many losses due to small resistivity and high permittivity in the range of high frequency. MEMS technology is a key technology to fabricate miniaturized devices and LTCC is one of good substrate materials in the range of high frequency due to its characteristics of high resistivity and low permittivity. Therefore, we proposed and studied to fabricate and analyze the inductor on the LTCC substrate with MEMS fabrication technology as the one of solutions to overcome this problem. We succeeded in fabricating and characterizing the high Q inductor on the LTCC substrate and then compared and analyzed the results of this inductor with that on a silicon and a glass substrate. The inductor on the LTCC substrate has larger Q factor value and inductance value than that on a silicon and a glass substrate. The values of Q factor with the LTCC substrate are 12 at 3 GHz, 33 at 6 GHz, 51 at 7 GHz and the values of inductance is 1.8, 1.5, 0.6 nH in the range of 5 GHz on the silicon, glass, and LTCC substrate, respectively.