• 대한전기학회논문지:시스템및제어부문D
• /
• 제52권9호
• /
• pp.533-539
• /
• 2003

Reinforcement learning (RL) has been widely used as a learning mechanism of an artificial life system. However, RL usually suffers from slow convergence to the optimum state-action sequence or a sequence of stimulus-response (SR) behaviors, and may not correctly work in non-Markov processes. In this paper, first, to cope with slow-convergence problem, if some state-action pairs are considered as disturbance for optimum sequence, then they no to be eliminated in long-term memory (LTM), where such disturbances are found by a shortest path-finding algorithm. This process is shown to let the system get an enhanced learning speed. Second, to partly solve a non-Markov problem, if a stimulus is frequently met in a searching-process, then the stimulus will be classified as a sequential percept for a non-Markov hidden state. And thus, a correct behavior for a non-Markov hidden state can be learned as in a Markov environment. To show the validity of our proposed learning technologies, several simulation result j will be illustrated.

• 로봇학회논문지
• /
• 제16권2호
• /
• pp.137-146
• /
• 2021

When controlling manipulator, degree of freedom is lost in singularity so specific joint velocity does not propagate to the end effector. In addition, control problem occurs because jacobian inverse matrix can not be calculated. To avoid singularity, we apply Deep Deterministic Policy Gradient(DDPG), algorithm of reinforcement learning that rewards behavior according to actions then determines high-reward actions in simulation. DDPG uses off-policy that uses 𝝐-greedy policy for selecting action of current time step and greed policy for the next step. In the simulation, learning is given by negative reward when moving near singulairty, and positive reward when moving away from the singularity and moving to target point. The reward equation consists of distance to target point and singularity, manipulability, and arrival flag. Dual arm manipulators hold long rod at the same time and conduct experiments to avoid singularity by simulated path. In the learning process, if object to be avoided is set as a space rather than point, it is expected that avoidance of obstacles will be possible in future research.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제15권9호
• /
• pp.3120-3137
• /
• 2021

Small-sized IoT wireless sensing devices can be deployed with small aircraft such as drones, and the deployment of mobile IoT devices can be relocated to suit data collection with efficient relocation algorithms. However, the terrain may not be able to predict its shape. Mobile IoT devices suitable for these terrains are hopping devices that can move with jumps. So far, most hopping sensor relocation studies have made the unrealistic assumption that all hopping devices know the overall state of the entire network and each device's current state. Recent work has proposed the most realistic distributed network environment-based relocation algorithms that do not require sharing all information simultaneously. However, since the shortest path-based algorithm performs communication and movement requests with terminals, it is not suitable for an area where the distribution of obstacles is uneven. The proposed scheme applies a simple Monte Carlo method based on relay nodes selection random variables that reflect the obstacle distribution's characteristics to choose the best relay node as reinforcement learning, not specific relay nodes. Using the relay node selection random variable could significantly reduce the generation of additional messages that occur to select the shortest path. This paper's additional contribution is that the world's first distributed environment-based relocation protocol is proposed reflecting real-world physical devices' characteristics through the OMNeT++ simulator. We also reconstruct the three days-long disaster environment, and performance evaluation has been performed by applying the proposed protocol to the simulated real-world environment.

• 한국표면공학회지
• /
• 제36권4호
• /
• pp.347-355
• /
• 2003

Roughening of metal surfaces frequently enhances the adhesion strength of metals to polymers by mechanical interlocking. When a failure occurs at a roughened metal/polymer interface, the failure prone to be cohesive. In a previous work, an adhesion study on a roughened metal (oxidized copper-based leadframe)/polymer (Epoxy Molding Compound, EMC) interface was carried out, and the correlation between adhesion strength and failure path was investigated. In the present work, an attempt to interpret the failure path was made under the assumption that microvoids are formed in the EMC as well as near the roots of the CuO needles during compression-molding process. A simple adhesion model developed from the theory of fiber reinforcement of composite materials was introduced to explain the adhesion behavior of the oxidized copper-based leadframe/EMC interface and failure path. It is believed that this adhesion model can be used to explain the adhesion behavior of other similarly roughened metal/polymer interfaces.

• 한국전산구조공학회:학술대회논문집
• /
• 한국전산구조공학회 2006년도 정기 학술대회 논문집
• /
• pp.891-897
• /
• 2006

The volume control method which utilize a pressure node added into a finite shell element can overcome the drawbacks of conventional load control method and displacement control method. In this study, an improved volume control method is introduced for effective analysis of path-dependant behaviors of RC structures subjected to cyclic loading. RC shell structures including RC hollow columns are anlayized by discretizing the structures with layered shell elements and by applying in-plane two dimensional constitutive equations for concrete layers and reinforcement layers of the shell elements. The so-called path dependant volume control method is verified by comparing analysis results with other data including experimental results.

• 한국통신학회논문지
• /
• 제30권5B호
• /
• pp.304-309
• /
• 2005

본 논문에서는 TD(temporal differences) 기법을 사용한 localized QoS 라우팅 기법을 제안하였다. 이 기법은 이웃노드로부터 얻어지는 성공 기댓값을 통해 라우팅 정책을 결정하는 기법이다. 이에 본 논문에서는 라우팅 성공 기댓값을 기반으로 한 다양한 탐색기법으로 경로 선택 시 라우팅 성능을 비교 평가하였으며, 특히 Exploration Bonus를 적용한 탐색 기법이 다른 탐색 기법에 비해 더욱 우수한 성능을 보여주고 있는데, 이는 다른 탐색 기법에 비해 네트워크 상황에 더 적응적으로 경로를 선택할 수 있기 때문이다.

• 한국인터넷방송통신학회논문지
• /
• 제23권6호
• /
• pp.33-39
• /
• 2023

최근 비행 애드-훅 네트워크(Flying Ad-hoc Network) 환경에서 강화학습을 이용한 통신 성능 개선과 이동성 모델 설계에 관한 연구가 진행되고 있다. 무인항공기(UAV)에서의 이동성 모델은 움직임을 예측하고 제어하기 위한 핵심요소로 주목받고 있다. 본 논문에서는 무인항공기가 운용되는 3차원 가상 환경을 구현하고, 무인항공기의 경로 최적화를 위해 푸리에 기저 함수 근사를 적용한 Q-learning과 DQN 두 가지 강화학습 알고리즘을 적용하여 모델을 설계 및 성능을 분석하였다. 실험 결과를 통해 3차원 가상 환경에서 DQN 모델이 Q-learning 모델 대비 최적의 경로 탐색에 적합한 것을 확인하였다.

• Computers and Concrete
• /
• 제2권4호
• /
• pp.309-324
• /
• 2005

The moment-curvature relationship of reinforced concrete beams made of normal- and high-strength concrete experiencing complex load history is studied using a numerical method that employs the actual stress-strain curves of the constitutive materials and takes into account the stress-path dependence of the concrete and steel reinforcement. The load history considered includes loading, unloading and reloading. From the results obtained, it is found that the complete moment-curvature relationship, which is also path-dependent, is similar to the material stress-strain relationship with stress-path dependence. However, the unloading part of the moment-curvature relationship of the beam section is elastic but not perfectly linear, although the unloading of both concrete and steel is assumed to be linearly elastic. It is also observed that when unloading happens, the variation of neutral axis depth has different trends for under- and over-reinforced sections. Moreover, even when the section is fully unloaded, there are still residual curvature and stress in the section in some circumstances. Various issues related to the post-peak behavior of reinforced concrete beams are also discussed.

• Journal of Information Processing Systems
• /
• 제18권2호
• /
• pp.188-196
• /
• 2022

Recently, neural architecture search (NAS) has received increasing attention as it can replace human experts in designing the architecture of neural networks for different tasks and has achieved remarkable results in many challenging tasks. In this study, a path-based computation neural architecture encoder (PCE) was proposed. Our PCE first encodes the computation of information on each path in a neural network, and then aggregates the encodings on all paths together through an attention mechanism, simulating the process of information computation along paths in a neural network and encoding the computation on the neural network instead of the structure of the graph, which is more consistent with the computational properties of neural networks. We performed an extensive comparison with eight encoding methods on two commonly used NAS search spaces (NAS-Bench-101 and NAS-Bench-201), which included a comparison of the predictive capabilities of performance predictors and search capabilities based on two search strategies (reinforcement learning-based and Bayesian optimization-based) when equipped with different encoders. Experimental evaluation shows that PCE is an efficient encoding method that effectively ranks and predicts neural architecture performance, thereby improving the search efficiency of neural architectures.

• 한국통신학회논문지
• /
• 제31권3B호
• /
• pp.175-182
• /
• 2006

본 논문에서는 Localized Aptive QoS 라우팅을 위해 POMDP(Partially Observable Markov Decision Processes)와 Exploration Bonus 기법을 사용하는 방법을 제안하였다. 또한, POMDP 문제를 해결하기 위해 Dynamic Programming을 사용하여 최적의 행동을 찾는 연산이 매우 복잡하고 어렵기 때문에 CEA(Certainty Equivalency Approximation) 기법을 통한 기댓값 사용으로 문제를 단순하였으며, Exploration Bonus 방식을 사용해 현재 경로보다 나은 경로를 탐색하고자 하였다. 이를 위해 다중 경로 탐색 알고리즘(SEMA)을 제안했다. 더욱이 탐색의 횟수와 간격을 정의하기 위해 $\phi$와 k 성능 파라미터들을 사용하여 이들을 통해 탐색의 횟수 변화를 통한 서비스 성공률과 성공 시 사용된 평균 홉 수에 대한 성능을 살펴보았다. 결과적으로 $\phi$ 값이 증가함에 따라 현재의 경로보다 더 나은 경로를 찾게 되며, k 값이 증가할수록 탐색이 증가함을 볼 수 있다.